diff --git a/.agents/rules/bzl.md b/.agents/rules/bzl.md new file mode 100644 index 0000000000..10aca72184 --- /dev/null +++ b/.agents/rules/bzl.md @@ -0,0 +1,10 @@ +--- +trigger: glob +description: Starlark / Bazel .bzl file coding style rules +globs: *.bzl +--- + +# Starlark Rules + +* Use triple-quoted strings for multi-line rule doc args. +* Don't use backslash line continuation in rule doc args. diff --git a/.agents/rules/cc.md b/.agents/rules/cc.md new file mode 100644 index 0000000000..5f2c2b43c2 --- /dev/null +++ b/.agents/rules/cc.md @@ -0,0 +1,22 @@ +# C/C++ Rules Design & Bazel rules_cc Integration + +## Concise `cc_shared_library` Interface & Export Rules +* In `cc_shared_library`: + * Targets directly placed in `deps` are **automatically exported** by Bazel; + they do not need to be listed in `exports_filter`. + * `exports_filter` is defined in `rules_cc` as an `attr.string_list()`, **not** + a `Label` attribute. Passing Starlark `Label` objects directly causes an + immediate Bazel analysis type error. + * **Citation**: [rules_cc `cc_shared_library.bzl`](https://github.com/bazelbuild/rules_cc/blob/main/cc/private/rules_impl/cc_shared_library.bzl) + (*"exports_filter is a list of strings attribute"*). + +## macOS Dynamic Lookup Link Flag +* Apple's `ld64`/`lld` linkers require `-undefined dynamic_lookup` in + `user_link_flags` on macOS so CPython C-API symbols remain unresolved at + link time and resolve dynamically at runtime. + +## Linux Platform Tag Composition +* Linux platform tags are formatted as `{platform_machine}-linux-{libc}` (e.g., + `x86_64-linux-gnu` or `aarch64-linux-musl`). +* **Citation**: [PEP 600 — Perennial manylinux](https://peps.python.org/pep-0600/) + & [PEP 656 — musllinux](https://peps.python.org/pep-0656/). diff --git a/.agents/rules/docs.md b/.agents/rules/docs.md new file mode 100644 index 0000000000..99ce5608ac --- /dev/null +++ b/.agents/rules/docs.md @@ -0,0 +1,15 @@ +--- +trigger: glob +description: Documentation formatting, Sphinx MyST style rules, and documentation build correctness +globs: docs/*.md +--- + +# Documentation Rules + +* Act as an expert in tech writing, Sphinx, MyST, and markdown. +* Wrap lines at 80 columns. +* Use hyphens (`-`) in file names instead of underscores (`_`). +* In Sphinx MyST markup, outer directives must have more colons than inner directives. +* When adding `{versionadded}` or `{versionchanged}` sections, add them at the end of the documentation text. +* For unreleased features or attributes, use `VERSION_NEXT_FEATURE` in `{versionadded}` / `{versionchanged}` directives. +* **Documentation Build Correctness**: Ensure new `.bzl` files or user-facing APIs are properly registered in documentation build targets (e.g. `//docs:docs` or relevant Starlark API reference generation configs). diff --git a/.agents/rules/github_actions_workflows.md b/.agents/rules/github_actions_workflows.md new file mode 100644 index 0000000000..525d2a2699 --- /dev/null +++ b/.agents/rules/github_actions_workflows.md @@ -0,0 +1,12 @@ +--- +name: github-actions-workflows +trigger: glob +globs: [".github/workflows/*.yml", ".github/workflows/*.yaml", ".github/*.yaml"] +--- + +# GitHub Actions Workflows Rule + +* Use the latest version of referenced actions in `.github/workflows/`. +* Preserve `suppress-no-jobs-ran-error` fallback jobs in conditional workflows. +* Pass inputs (e.g. `${{ inputs.issue }}`) directly to commands without + redundant shell parameter stripping or conversions. diff --git a/.agents/rules/news.md b/.agents/rules/news.md new file mode 100644 index 0000000000..ce8a844439 --- /dev/null +++ b/.agents/rules/news.md @@ -0,0 +1,20 @@ +--- +trigger: news/*.md +description: Apply when drafting news entries. +--- + +@CONTRIBUTING.md + +# News Entry Conventions + +Before drafting any news entry, strictly adhere to the rules in `CONTRIBUTING.md` +above. + +## Sphinx MyST Cross-Reference Syntax (`{obj}`) +* Use `{obj}\`\`` in news entries for rules, macros, targets, providers, + attributes, args, and any other cross-referencable Starlark or Python + objects. + +## GitHub Issue Link Formatting +* Append GitHub issue cross-references at the end of news entries in markdown + link format: `([#3283](https://github.com/bazel-contrib/rules_python/issues/3283))`. diff --git a/.agents/rules/no_gh_auth_token_in_cmdline.md b/.agents/rules/no_gh_auth_token_in_cmdline.md new file mode 100644 index 0000000000..e8cb7794fb --- /dev/null +++ b/.agents/rules/no_gh_auth_token_in_cmdline.md @@ -0,0 +1,11 @@ +--- +trigger: always_on +--- + +# No Auth Token in Cmdline Rule + +* NEVER pass `gh auth token` or embed authentication tokens in command line + arguments or URLs. Doing so can leak credentials in process tables (`ps`), + shell history, or logs. +* Use configured SSH keys, standard `git push`, or `gh` commands natively + instead. diff --git a/.agents/rules/pr.md b/.agents/rules/pr.md new file mode 100644 index 0000000000..caacb07519 --- /dev/null +++ b/.agents/rules/pr.md @@ -0,0 +1,24 @@ +--- +trigger: model_decision +description: rules to apply to pull request descriptions +--- + +@CONTRIBUTING.md + +# Pull Request Conventions + +Before drafting any pull request description, strictly adhere to the rules in +`CONTRIBUTING.md` above. + +## Prevent Agent Oversight of `CONTRIBUTING.md` +* Always include `@CONTRIBUTING.md` in rule definitions so that PR formatting + rules are injected into context whenever PR workflows run. + +## PR Commit Workflow Invariant +* Once a Pull Request is created, always make new commits or merge commits. +* **NEVER** amend or rebase commits on an active PR branch to avoid breaking + code review threads. +* **NEVER** include a list of per-file edits or changelog bullet points of + individual file modifications in PR descriptions or commit messages. +* High-level overview only: state *why* the change is made and *how* at a + conceptual level. Link related issues (e.g. `Work towards #`). diff --git a/.agents/rules/python.md b/.agents/rules/python.md new file mode 100644 index 0000000000..d683152814 --- /dev/null +++ b/.agents/rules/python.md @@ -0,0 +1,20 @@ +# Python Conventions + +## pytest +* Register helper fixtures using `pytest_plugins = [""]`. +* Name fixture functions with `fixture_` prefix and pass public name via + `@pytest.fixture(name="foo")`. + +## CLI & Arguments +* Use direct attribute access (e.g. `args.foo`) on `argparse.Namespace` with + well-defined shapes. Avoid defensive `getattr()`. + +## TypedDict +* **External Objects**: When defining a `TypedDict` for an external object, + link to its definition in the docstring. + +## Delegating Functions +* Module-level functions delegating to class methods should have a docstring + referring to the class method (e.g. `"""Refer to \`Class.method\`."""`). + + diff --git a/.agents/rules/starlark.md b/.agents/rules/starlark.md new file mode 100644 index 0000000000..4517b06aec --- /dev/null +++ b/.agents/rules/starlark.md @@ -0,0 +1,60 @@ +--- +trigger: glob +description: Starlark Language, Macro, and Testing Invariants +globs: "*.bzl,BUILD,BUILD.bazel,*.bazel" +--- + +# Starlark Language & Macro Invariants + +## Macro Target Canonicalization +* In macro implementations, internal repository target references **MUST** be + canonicalized using `str(Label("//path/to:target"))` so they resolve in the + macro's module context rather than the caller's repository context. +* Note that `python/private/common_labels.bzl` defines `labels`, a struct + containing common canonicalized label strings used across the project. + +## Manual Tag on Internal Macro Helper Targets +* When macros instantiate internal helper targets (such as private rule targets + for artifact extraction or linking support), always include `tags = ["manual"]`. +* **Why**: This prevents internal helper targets from being implicitly built when + wildcard target patterns (e.g., `//...`) are expanded. + +## Private Alias Pattern when Appending to User `deps` +* When macros append internal helper targets to user-provided dependency lists + (`deps`), use private alias targets (e.g. + `//python/private/cc:current_py_cc_headers_private_alias`) to prevent + "duplicate dependency label" analysis errors if the user also explicitly + passes the public target label in their `deps` (including in `select()` + expressions). + +## Attribute Normalization at Function Entry +* Always normalize input list attributes at the start of macro definitions + (e.g., `deps = deps or []`, `copts = copts or []`). +* **Why**: This avoids inlining the normalization as part of a complex + expression later in the macro expansion. + +## Control Flow & Algorithmic Restrictions +* **Iterative Algorithms Only (No Recursion)**: Starlark does not support + recursive function calls; always implement iterative algorithms using bounded + loops. +* **Iterable `for` Loops Only (No `while` Loops)**: Starlark does not support + `while` loops; iterate over fixed-size ranges or explicit collections. + +## Code Style & Conventions +* **Docstring Formatting Invariants**: Use triple-quoted strings for multi-line + docstrings without trailing backslashes (`\`) for line continuation. +* **No Bazel Copyright Headers**: Do not add Bazel copyright headers to new or + existing files unless explicitly directed by the user. +* **Line Length & Wrapping**: Wrap Markdown and Starlark lines to 80 columns in + accordance with `.editorconfig`. + +## Starlark Testing (`rules_testing`) +* **`rules_testing` over `bazel_skylib`**: Always use `@rules_testing` (analysis + tests with `env.expect.that_...`) rather than `bazel_skylib` for Starlark + rule analysis tests. +* **Analysis Test Two-Part Structure**: Separate tests into a setup target + function `def _test_foo(name)` calling `analysis_test` and an implementation + function `def _test_foo_impl(env, target)`. +* **Test Suite Registration**: Collect test setup functions in a private + `_tests` list and register them cleanly via `test_suite(name = name, tests = + _tests)`. diff --git a/.agents/rules/testing.md b/.agents/rules/testing.md new file mode 100644 index 0000000000..769197f968 --- /dev/null +++ b/.agents/rules/testing.md @@ -0,0 +1,24 @@ +# Testing & Validation Guardrails + +## `--config=fast-tests` for Test Targets +* Always pass `--config=fast-tests` when running or building test targets to + avoid running expensive, flaky integration tests. + +## CRITICAL: Never use `--config=fast-tests` on Non-Test Targets +* `--config=fast-tests` sets `--build_tests_only=true`, which silently ignores + non-test targets (such as `//docs:docs` or package libraries), resulting in 0 + targets built! + +## Lockfile Testing (`MODULE.bazel.lock`) +* Changes to transitive module extension dependencies or `.bzl` files loaded by + extensions update Bazel 9 lockfile hashes, requiring `bazel mod deps + --lockfile_mode=update` in integration test workspaces. +* When requirements files (e.g., in `//tools/publish` or root pip parses) are + modified or bumped by Dependabot, update the integration lockfile by running + `bazel mod deps --lockfile_mode=update` in `tests/integration/bzlmod_lockfile` + and verify with + `bazel test //tests/integration:bzlmod_lockfile_test_bazel_9.1.0`. + +## Documentation Flake Handling +* When building `//docs:docs` fails with exit code 2, treat it as a known + Sphinx/Bazel flake and retry the build. diff --git a/.agents/rules/when-to-use-copyright.md b/.agents/rules/when-to-use-copyright.md new file mode 100644 index 0000000000..a73cbbfd70 --- /dev/null +++ b/.agents/rules/when-to-use-copyright.md @@ -0,0 +1,8 @@ +--- +trigger: always_on +--- + +# When to Use Copyright + +Unless directed by the user otherwise, do not add Bazel copyright to new or +existing files. diff --git a/.agents/rules/windows.md b/.agents/rules/windows.md new file mode 100644 index 0000000000..2eea9b2df5 --- /dev/null +++ b/.agents/rules/windows.md @@ -0,0 +1,32 @@ +# Windows MSVC Toolchains & Linking + +## MSVC `#pragma comment(lib, ...)` Embedded References +* CPython's MSVC headers embed `#pragma comment(lib, "python3xx.lib")` into + generated `.obj` files. MSVC `link.exe` automatically searches for + `python3xx.lib` in library search paths during linking. +* **Citation**: [Microsoft Learn `comment` pragma](https://learn.microsoft.com/en-us/cpp/preprocessor/comment-c-cpp) + (*"Places a library-search record in the object file... The linker searches for + this library the same way as if you had named it on the command line"*). + +## PEP 3149 ABI Tag Prefix (`cp` vs. `cpython-`) +* Windows CPython ABI tags use the `cp` prefix (e.g., `cp311`, `cp312t`) + whereas POSIX platforms use `cpython-` (e.g., `cpython-311`). +* **Citation**: [PEP 3149 — ABI version tagged .so files](https://peps.python.org/pep-3149/) + (*"The tag starts with `cpython-` followed by the Python major and minor version + without dots... followed by build flags"*). + +## Windows Platform Tag Derivation +* Windows platform tags for Python C extensions evaluate to `win_amd64` + (x86_64/amd64), `win_arm64` (aarch64/arm64), or `win32` (32-bit x86). +* **`platform.machine()` Normalization**: On Windows, + `platform.machine()` returns uppercase (`"AMD64"`, `"ARM64"`). Always + normalize with `.lower()` when deriving PEP 508 markers or resolving platform + tags. +* **Citation**: [PEP 425 — Compatibility Tags for Built Distributions](https://peps.python.org/pep-0425/). + +## Windows CPython SOABI & ABI Infix Support +* CPython on Windows natively supports loading ABI-tagged `.pyd` files + containing platform tags (e.g., `foo.cp314-win_amd64.pyd`, + `foo.cp311-win_amd64.pyd`). +* SOABI on Windows includes both the ABI prefix and platform tag (e.g., + `cp311-win_amd64`). diff --git a/.agents/rules/workspace.md b/.agents/rules/workspace.md new file mode 100644 index 0000000000..bc64d4779c --- /dev/null +++ b/.agents/rules/workspace.md @@ -0,0 +1,20 @@ +--- +trigger: always_on +--- + +# Workspace Rules + +To avoid confusion from using outdated code states, when starting a new +conversation/session or when first starting a new branch or worktree, unless +explicitly instructed otherwise, ensure the latest upstream code is used as the +basis: +* Fetch `upstream/main` (`git fetch upstream main`). +* Base any new branch or worktree upon `upstream/main` (e.g., + `git checkout -b upstream/main`). +* Run the workspace helper script to configure upstream tracking and safe + pushing: + ```bash + .agents/scripts/setup_triangle_branch.sh + ``` +* Never use `git push -u` or `--set-upstream` (it breaks upstream tracking). + diff --git a/.agents/scripts/setup_triangle_branch.sh b/.agents/scripts/setup_triangle_branch.sh new file mode 100755 index 0000000000..b6bdc2bb20 --- /dev/null +++ b/.agents/scripts/setup_triangle_branch.sh @@ -0,0 +1,25 @@ +#!/bin/bash +# Helper script to set up upstream tracking for fork-and-PR workflow. +set -e + +# Detect current branch if not provided as argument +BRANCH_NAME="${1:-$(git symbolic-ref --short HEAD)}" + +if [ -z "$BRANCH_NAME" ]; then + echo "Error: Could not detect current branch name." >&2 + exit 1 +fi + +echo "Setting up upstream tracking for branch: $BRANCH_NAME" + +# 1. Set the pull/fetch behavior to track the canonical repo's main branch +git config --local "branch.${BRANCH_NAME}.remote" upstream +git config --local "branch.${BRANCH_NAME}.merge" refs/heads/main + +# 2. Override the destination remote for pushing +git config --local "branch.${BRANCH_NAME}.pushRemote" origin + +# 3. Defend against global push.default settings (ensure it pushes to current branch name) +git config --local push.default current + +echo "Successfully configured upstream/main tracking and origin pushRemote for $BRANCH_NAME!" diff --git a/.agents/skills/analyze-ci-failure/SKILL.md b/.agents/skills/analyze-ci-failure/SKILL.md new file mode 100644 index 0000000000..0af8a92fb1 --- /dev/null +++ b/.agents/skills/analyze-ci-failure/SKILL.md @@ -0,0 +1,25 @@ +--- +name: analyze-ci-failure +description: Download and analyze a CI failure log to construct an actionable + suggested fix plan and report back +--- + +When a CI monitoring workflow alerts you to a failed Buildkite job or GitHub +check, invoke this skill by running: +```bash +./.agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py \ + "" "" "" "" +``` + +### ✨ What this Skill Does +1. **Resolves Log**: Automatically resolves the Buildkite job download URL or + fetches GitHub Actions logs via `gh` CLI. +2. **Downloads & Ingests**: Fetches the full raw CI log file and saves it + locally to `.agents/skills/analyze-ci-failure/scratch/`. +3. **Smart Error Extraction**: Scans log lines for critical failure signatures + (`Traceback`, `ERROR:`, `FAILED:`, missing packages, compiler aborts, linter + errors). +4. **Fix Plan Synthesis**: Constructs a beautifully structured Markdown + suggested plan on how to resolve the root cause. +5. **Natively Notifies**: Dispatches a high-priority summary notification + message back to your active agent conversation via `agentapi send-message`. diff --git a/.agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py b/.agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py new file mode 100644 index 0000000000..661a427cc7 --- /dev/null +++ b/.agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py @@ -0,0 +1,168 @@ +#!/usr/bin/env python3 + +import argparse +import os +import re +import subprocess +import sys +import urllib.request + + +def fetch_log(build_id, job_id, output_path): + if build_id.startswith("http"): + log_url = build_id + elif job_id.startswith("http"): + log_url = job_id + else: + log_url = f"https://buildkite.com/organizations/bazel/pipelines/rules-python-python/builds/{build_id}/jobs/{job_id}/download.txt" + + # Check if this is a GitHub Actions job + gh_match = re.search(r"github\.com/.*/job/(\d+)", log_url) or re.search( + r"^(\d+)$", job_id + ) + if gh_match: + gh_job_id = gh_match.group(1) + print(f"📥 Fetching GitHub Action log for job {gh_job_id} using gh CLI...") + cmd = ["gh", "run", "view", "--job", gh_job_id, "--log"] + try: + res = subprocess.run(cmd, capture_output=True, text=True, check=True) + with open(output_path, "w") as f: + f.write(res.stdout) + return True + except Exception as e: + print( + f"⚠️ Failed to fetch GitHub log via gh CLI for job {gh_job_id}: {e}", + file=sys.stderr, + ) + + if not log_url.endswith("/download.txt") and "buildkite.com" in log_url: + log_url = re.sub(r"/log$", "/download.txt", log_url) + + print(f"📥 Downloading CI failure log from {log_url}...") + req = urllib.request.Request(log_url, headers={"User-Agent": "ci-analyzer"}) + try: + with urllib.request.urlopen(req) as resp: + content = resp.read() + with open(output_path, "wb") as f: + f.write(content) + return True + except Exception as e: + print(f"⚠️ Failed to download log from {log_url}: {e}", file=sys.stderr) + with open(output_path, "w") as f: + f.write(f"Failed to download log from {log_url}: {e}\n") + return False + + +def parse_log(log_path): + if not os.path.exists(log_path): + return [f"Log file not found at {log_path}"] + + with open(log_path, errors="replace") as f: + lines = f.readlines() + + errors = [] + for line in lines: + if any( + keyword in line + for keyword in [ + "ERROR:", + "FAILED:", + "Critical Path", + "Traceback", + "Exception", + "FileNotFoundError", + "no such package", + "no such target", + "exit code", + "exit-code", + "fatal:", + "fatal", + "##[error]", + "Would reformat:", + "would be reformatted", + "error]", + ] + ): + errors.append(line.strip()) + + return errors[:30] + + +def create_plan(job_name, log_path, errors): + err_str = ( + "\n".join(errors) + if errors + else "No obvious keyword error lines matched. Please inspect the raw log file." + ) + + plan = f"""# 🚨 CI Failure Analysis Report: {job_name} + +## 📁 CI Log Path +`{log_path}` + +## 🔥 Extracted Failure Snippets +```text +{err_str} +``` + +## 🛠️ Suggested Plan to Fix +1. **Inspect Log**: Review the exact log snippets above or read the full raw log file at `{log_path}`. +2. **Reproduce Locally**: Run `./replicate_ci "{job_name}"` or the matching `bazel build/test` command locally. +3. **Apply Fix**: Resolve the root cause in the relevant `BUILD.bazel` or Starlark files. +4. **Verify & Push**: Run local verification with `--config=fast-tests` and push the updated branch to trigger a clean pipeline. +""" + return plan + + +def main(): + parser = argparse.ArgumentParser( + description="Download CI failure log, analyze root cause, and create fix plan." + ) + parser.add_argument("job_name", help="Name of the failed job") + parser.add_argument("build_id", help="Buildkite Build ID, Build number, or Log URL") + parser.add_argument("job_id", help="Buildkite Job ID or link") + parser.add_argument("conv_id", help="Conversation ID to report back to") + args = parser.parse_args() + + skill_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), "..")) + scratch_dir = os.path.join(skill_dir, "scratch") + os.makedirs(scratch_dir, exist_ok=True) + + safe_jname = re.sub(r"[^a-zA-Z0-9]", "_", args.job_name) + log_path = os.path.join(scratch_dir, f"ci_{safe_jname}_{args.job_id}.log") + + fetch_log(args.build_id, args.job_id, log_path) + + print(f"🚀 Analyzing CI failure log for '{args.job_name}' at '{log_path}'...") + errors = parse_log(log_path) + plan = create_plan(args.job_name, log_path, errors) + + plan_file = os.path.join(scratch_dir, f"ci_plan_{safe_jname}.md") + with open(plan_file, "w") as f: + f.write(plan) + + print( + f"📄 Plan generated at '{plan_file}'. Dispatching notification to conversation {args.conv_id}..." + ) + + msg = ( + f"⚠️ Remote CI Job '{args.job_name}' Analysis Complete!\n\n" + f"I downloaded and analyzed the failure log. Findings and suggested fix plan compiled at artifact file: `{plan_file}`.\n\n" + f"Raw CI Log Path: `{log_path}`" + ) + + res = subprocess.run( + [ + "agentapi", + "send-message", + "--title=CI Failure Analysis Plan", + args.conv_id, + msg, + ] + ) + if res.returncode != 0: + print(f"❌ Failed to send agentapi message. Printing plan directly:\n{plan}") + + +if __name__ == "__main__": + main() diff --git a/.agents/skills/buildkite-get-results/SKILL.md b/.agents/skills/buildkite-get-results/SKILL.md new file mode 100644 index 0000000000..801e51c02d --- /dev/null +++ b/.agents/skills/buildkite-get-results/SKILL.md @@ -0,0 +1,10 @@ +--- +name: buildkite-get-results +description: Gets buildkite build results +--- + +Pass the PR number, Build URL, or Build ID to the `scripts/get_buildkite_results.py` script. + +The `--jobs` flag can do glob-style filtering of jobs. + +The `--download` flag will download job logs. diff --git a/.agents/skills/buildkite-get-results/scripts/get_buildkite_results.py b/.agents/skills/buildkite-get-results/scripts/get_buildkite_results.py new file mode 100755 index 0000000000..1af829a8e6 --- /dev/null +++ b/.agents/skills/buildkite-get-results/scripts/get_buildkite_results.py @@ -0,0 +1,96 @@ +#!/usr/bin/env python3 + +import argparse +import json +import re +import subprocess +import sys + + +def check_cli(cmd_name, install_url): + try: + subprocess.run( + [cmd_name, "--version"], + check=True, + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + ) + except Exception: + print( + f"❌ Error: '{cmd_name}' CLI is not installed or not in PATH.", + file=sys.stderr, + ) + print(f"Please install it from {install_url}", file=sys.stderr) + sys.exit(1) + + +def get_build_url_from_pr(pr_number): + check_cli("gh", "https://cli.github.com/") + cmd = ["gh", "pr", "checks", str(pr_number), "--json", "name,link"] + try: + res = subprocess.run(cmd, capture_output=True, text=True, check=True) + checks = json.loads(res.stdout) + for c in checks: + link = c.get("link", "") + if "buildkite.com" in link: + return link.split("#")[0] + print(f"❌ No Buildkite checks found for PR #{pr_number}.", file=sys.stderr) + sys.exit(1) + except subprocess.CalledProcessError as e: + print(f"❌ Error fetching PR checks: {e.stderr}", file=sys.stderr) + sys.exit(1) + + +def normalize_build_target(target): + # Transforms https://buildkite.com/bazel/rules-python-python/builds/15707 + # into bazel/rules-python-python/15707 + m = re.search(r"buildkite\.com/([^/]+)/([^/]+)/builds/(\d+)", target) + if m: + return f"{m.group(1)}/{m.group(2)}/{m.group(3)}" + return target + + +def main(): + parser = argparse.ArgumentParser( + description="Gets Buildkite build results using the 'bk' CLI." + ) + parser.add_argument( + "pr", help="PR number, Build URL, or Build ID (org/pipeline/build)" + ) + parser.add_argument( + "--jobs", + help="Glob-style filtering of job names to display or download", + ) + parser.add_argument("--download", action="store_true", help="Download job logs") + args = parser.parse_args() + + check_cli("bk", "https://github.com/buildkite/cli") + + target = args.pr + if target.isdigit() and len(target) < 10: + print(f"🔍 Inspecting PR #{target} via gh to find Buildkite URL...") + target = get_build_url_from_pr(target) + + build_id = normalize_build_target(target) + print(f"🚀 Querying Buildkite for build: {build_id}\n") + + # Run bk build view + res = subprocess.run(["bk", "build", "view", build_id]) + if res.returncode != 0: + print( + f"❌ Failed to view build '{build_id}' via 'bk' CLI.", + file=sys.stderr, + ) + sys.exit(res.returncode) + + if args.download: + print(f"\n📥 Downloading logs for build: {build_id}") + dl_res = subprocess.run(["bk", "build", "download", build_id]) + if dl_res.returncode != 0: + print( + "⚠️ 'bk build download' failed or not supported. Try using 'bk job log ' for specific jobs." + ) + + +if __name__ == "__main__": + main() diff --git a/.agents/skills/buildkite-retry-job/SKILL.md b/.agents/skills/buildkite-retry-job/SKILL.md new file mode 100644 index 0000000000..4b822e6222 --- /dev/null +++ b/.agents/skills/buildkite-retry-job/SKILL.md @@ -0,0 +1,27 @@ +--- +name: buildkite-retry-job +description: Retry failed Buildkite jobs individually or rebuild full builds +--- + +Use `scripts/retry_buildkite_jobs.py` to retry individual jobs or rebuild full +Buildkite builds. + +### Retrying Modes + +1. **Job-Level Retry (Default for failed jobs)**: + Retries only specific failing jobs via `bk job retry `: + ```bash + ./.agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py + ``` + +2. **Retry Specific Jobs by Name or ID**: + ```bash + ./.agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py --jobs "compile_pip_requirements" + ./.agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py --job-id + ``` + +3. **Rebuild Full Build**: + Re-runs the entire pipeline build from scratch via `bk build rebuild`: + ```bash + ./.agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py --rebuild + ``` diff --git a/.agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py b/.agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py new file mode 100755 index 0000000000..b0a154ac0c --- /dev/null +++ b/.agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py @@ -0,0 +1,215 @@ +#!/usr/bin/env python3 + +import argparse +import json +import re +import subprocess +import sys + + +def check_cli(cmd_name, install_url): + try: + subprocess.run( + [cmd_name, "--version"], + check=True, + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + ) + except Exception: + print( + f"❌ Error: '{cmd_name}' CLI is not installed or not in PATH.", + file=sys.stderr, + ) + print(f"Please install it from {install_url}", file=sys.stderr) + sys.exit(1) + + +def get_build_url_from_pr(pr_number): + check_cli("gh", "https://cli.github.com/") + cmd = ["gh", "pr", "checks", str(pr_number), "--json", "name,link"] + try: + res = subprocess.run(cmd, capture_output=True, text=True, check=True) + checks = json.loads(res.stdout) + for c in checks: + link = c.get("link", "") + if "buildkite.com" in link: + return link.split("#")[0] + print(f"❌ No Buildkite checks found for PR #{pr_number}.", file=sys.stderr) + sys.exit(1) + except subprocess.CalledProcessError as e: + print(f"❌ Error fetching PR checks: {e.stderr}", file=sys.stderr) + sys.exit(1) + + +def normalize_build_target(target): + # Transforms https://buildkite.com/bazel/rules-python-python/builds/15707 + # into (org/pipeline, 15707) + m = re.search(r"buildkite\.com/([^/]+/[^/]+)/builds/(\d+)", target) + if m: + return m.group(1), m.group(2) + parts = target.split("/") + if len(parts) == 3: + return f"{parts[0]}/{parts[1]}", parts[2] + elif len(parts) == 2: + return parts[0], parts[1] + return "bazel/rules-python-python", target + + +def retry_job_by_uuid(job_id): + check_cli("bk", "https://github.com/buildkite/cli") + print(f"🚀 Retrying individual job UUID: {job_id}") + return subprocess.run(["bk", "job", "retry", job_id]) + + +def retry_jobs_by_name(pipeline, build_number, job_pattern): + check_cli("bk", "https://github.com/buildkite/cli") + cmd = ["bk", "build", "view", str(build_number), "-p", pipeline, "--json"] + try: + res = subprocess.run(cmd, capture_output=True, text=True, check=True) + data = json.loads(res.stdout) + jobs = data.get("jobs", []) + matched = [] + for j in jobs: + name = j.get("name", "") or j.get("label", "") + state = j.get("state", "") + jid = j.get("id") + if jid and ( + re.search(job_pattern, name, re.IGNORECASE) + or job_pattern.lower() in name.lower() + ): + matched.append((jid, name, state)) + + if not matched: + print( + f"⚠️ No jobs found matching pattern '{job_pattern}' in build #{build_number}", + file=sys.stderr, + ) + sys.exit(1) + + for jid, name, state in matched: + print(f"🚀 Retrying job '{name}' (ID: {jid}, status: {state})...") + subprocess.run(["bk", "job", "retry", jid]) + except Exception as e: + print(f"❌ Error fetching build jobs: {e}", file=sys.stderr) + sys.exit(1) + + +def main(): + parser = argparse.ArgumentParser( + description="Retry Buildkite jobs individually using 'bk job retry' or rebuild the whole build using 'bk build rebuild'." + ) + parser.add_argument( + "args", + nargs="+", + help="Target build (PR#, Buildkite URL, or org/pipeline/build)", + ) + parser.add_argument( + "--jobs", + "--job-name", + dest="job_name", + help="Specific job name or regex pattern to retry individually via 'bk job retry'", + ) + parser.add_argument( + "--job-id", + dest="job_id", + help="Specific job UUID to retry individually via 'bk job retry'", + ) + parser.add_argument( + "--rebuild", + action="store_true", + help="Rebuild the entire build afresh via 'bk build rebuild'", + ) + args = parser.parse_args() + + check_cli("bk", "https://github.com/buildkite/cli") + + if args.job_id: + res = retry_job_by_uuid(args.job_id) + sys.exit(res.returncode) + + if len(args.args) == 3: + target = f"{args.args[0]}/{args.args[1]}/{args.args[2]}" + elif len(args.args) == 1: + target = args.args[0] + else: + print( + "❌ Error: Invalid arguments. Provide a single target (PR#, URL, or org/pipeline/build).", + file=sys.stderr, + ) + sys.exit(1) + + if target.isdigit() and len(target) < 10: + print(f"🔍 Inspecting PR #{target} via gh to find Buildkite URL...") + target = get_build_url_from_pr(target) + + pipeline, build_number = normalize_build_target(target) + + if args.job_name: + retry_jobs_by_name(pipeline, build_number, args.job_name) + elif args.rebuild: + print(f"🚀 Rebuilding entire build #{build_number} for pipeline: {pipeline}") + res = subprocess.run( + ["bk", "build", "rebuild", build_number, "-p", pipeline, "--yes"] + ) + sys.exit(res.returncode) + else: + # Default behavior: try job-level retries of failed jobs if any, otherwise rebuild + cmd = [ + "bk", + "build", + "view", + str(build_number), + "-p", + pipeline, + "--json", + ] + try: + res = subprocess.run(cmd, capture_output=True, text=True, check=True) + data = json.loads(res.stdout) + failed_jobs = [ + j + for j in data.get("jobs", []) + if j.get("state") in ("failed", "broken", "timed_out") and j.get("id") + ] + if failed_jobs: + for j in failed_jobs: + jid = j["id"] + jname = j.get("name") or j.get("label") or jid + print(f"🚀 Retrying failed job '{jname}' (ID: {jid})...") + subprocess.run(["bk", "job", "retry", jid]) + else: + print( + f"🚀 Rebuilding entire build #{build_number} for pipeline: {pipeline}" + ) + res = subprocess.run( + [ + "bk", + "build", + "rebuild", + build_number, + "-p", + pipeline, + "--yes", + ] + ) + sys.exit(res.returncode) + except Exception: + print( + f"🚀 Rebuilding entire build #{build_number} for pipeline: {pipeline}" + ) + res = subprocess.run( + [ + "bk", + "build", + "rebuild", + build_number, + "-p", + pipeline, + "--yes", + ] + ) + sys.exit(res.returncode) + + +if __name__ == "__main__": + main() diff --git a/.agents/skills/create-pr/SKILL.md b/.agents/skills/create-pr/SKILL.md new file mode 100644 index 0000000000..1c73c66424 --- /dev/null +++ b/.agents/skills/create-pr/SKILL.md @@ -0,0 +1,43 @@ +--- +name: create-pr +description: Propose, draft, or create a pull request by delegating to a + subagent +--- + +When proposing, drafting, or creating a Pull Request, you MUST ALWAYS delegate +to a subagent. NEVER create or draft PRs directly in the main conversation. + +### Instructions + +1. **Code Review Audit**: Launch a subagent with `review-code` to verify + `git diff` conforms to all rules (news entry in `news/`, line wrapping, + copyright, conventions, Starlark formatting). Fix any issues before + proceeding. +2. **Launch Subagent**: Use `invoke_subagent` (`TypeName: "self"`). +3. **Subagent Prompt Instructions**: + - Follow `CONTRIBUTING.md` and `@/.agents/rules/pr.md`. + - **PR Title**: Conventional commits format (`agents:` prefix for agent + rules/skills). + - **PR Body**: Explain *why* and conceptual *how*. Wrap strictly at 72 + columns max. Omit TAG/CONV. + - **Artifact Requirements**: Create `pr_info.md` with: + - **User-facing**: Published directly in the user interface. + - **Interactive feedback enabled**: Allows selecting lines and leaving + inline comments on the draft (`RequestFeedback: true`). + - **User decision choices**: Present choices: + 1. Create a regular PR + 2. Create a draft PR + 3. Provide feedback on the draft text + 4. Discard the draft text + - **Propose vs. Create**: If proposing or drafting a PR, do NOT run + `gh pr create`—only create the `pr_info.md` artifact. Only execute + `gh pr create` when explicitly requested to create the PR. + - **Targeting Upstream Repo**: When creating, target upstream using + `--repo bazel-contrib/rules_python` and `--head :`. +4. **Return Status**: Direct subagent to report PR number/draft status to + caller. +5. **Link Artifact Before Asking**: Upon subagent completion, output a + clickable markdown link to `pr_info.md` before prompting for confirmation. +6. **Interactive Actions**: When presenting choices via `ask_question`, always + include the clickable markdown link to `pr_info.md` in the `question` + prompt. diff --git a/.agents/skills/merge-pr/SKILL.md b/.agents/skills/merge-pr/SKILL.md new file mode 100644 index 0000000000..ae3e2e3ad5 --- /dev/null +++ b/.agents/skills/merge-pr/SKILL.md @@ -0,0 +1,33 @@ +--- +name: merge-pr +description: Merge a pull request into main, monitoring the merge queue, retrying CI flakes, and re-enqueuing if necessary +--- + +When the user asks to merge a pull request (e.g., "merge PR ", "merge this PR", or monitor its merge): + +1. **Enqueue for Merge**: Run `gh pr merge --auto --squash` to enable auto-merge or add the pull request to the merge queue. + - **Force / Admin Merge**: **CRITICAL**: Passing `--admin` to bypass the + merge queue or required checks requires explicit user permission or + consent. Only pass `--admin` (e.g., `gh pr merge --admin + --squash`) if the user has explicitly requested or approved a force/admin + merge. Never invoke `--admin` autonomously. +2. **Invoke a Background Shepherd**: Launch a background subagent with the role `Merge PR Shepherd` to continuously watch the PR until it merges. +3. **Leverage Existing CI Skills**: + - Have the subagent use the **`monitor-ci-results`** skill to watch for CI + check failures and generate analysis reports. + - Have the subagent use the **`buildkite-retry-job`** skill + (`retry_buildkite_jobs.py `) to automatically retry any + transient network flakes (e.g., HTTP 504 gateway timeouts, downloader + errors). + - When the PR is queued, actively discover the merge queue branch via + `gh api repos/:owner/:repo/branches --jq '.[].name | select(test("gh-readonly-queue/.*/pr--"))'` + and monitor commit statuses/Buildkite builds running on that temporary + branch. +4. **Queue Shepherding**: Periodically check `gh pr view --json + state,autoMergeRequest,mergeStateStatus,mergeable`. While `state` is + `"OPEN"`, ensure auto-merge is enabled / queued by running `gh pr merge + --auto --squash`. If `autoMergeRequest` is null (e.g., ejected + from the merge queue due to a CI flake in the temporary queue branch), + re-enqueue it for merge by running `gh pr merge --auto --squash` + once checks are retried or green. +5. **Completion Notification**: Once `state` becomes `"MERGED"`, send a high-priority message back to the parent conversation. diff --git a/.agents/skills/monitor-ci-results/SKILL.md b/.agents/skills/monitor-ci-results/SKILL.md new file mode 100644 index 0000000000..e2475b55ad --- /dev/null +++ b/.agents/skills/monitor-ci-results/SKILL.md @@ -0,0 +1,48 @@ +--- +name: monitor-ci-results +description: Monitor remote CI results for a PR and autonomously launch + subagents to analyze CI failures +--- + +When the user requests to monitor remote CI results or watch a pull request, +or when monitoring CI after PR updates: + +> **Note**: `monitor_remote_ci.py` is a single long-running continuous task. +> It continuously watches all current and future CI runs for the PR. Do NOT +> launch duplicate monitoring jobs for the same PR. + +1. **Check Existing Process**: Check if a monitor script is already running for + **this specific PR** using `pgrep -f "monitor_remote_ci.py "`. Do + NOT use a generic grep without ``, as other active agent + conversations may be monitoring different PRs concurrently. If one is + already running for this PR, do not start another instance. +2. **Launch Monitoring Script**: If no monitor process is active for + ``, launch the script in the background: +```bash +./.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py \ + "" & +``` + +### ✨ Autonomous Subagent Orchestration +1. **Background Polling**: `monitor_remote_ci.py` continuously polls both + GitHub PR checks and Buildkite workflow executions in the background across + new commits and CI re-runs. +2. **Blocked Jobs**: When a Buildkite job or GitHub check is in a blocked state + waiting for user confirmation, it dispatches a notification via `agentapi + send-message` so the user is alerted to confirm running the job. +3. **Failure Reporting**: When any GitHub check or Buildkite job completes + with errors, `monitor_remote_ci.py` dispatches a high-priority notification + message reporting the failed check back to your conversation. +4. **Subagent Analysis**: Upon receiving a failure notification message from + the monitoring script, immediately launch a separate subagent using the + `invoke_subagent` tool with the role "CI Failure Analyzer" to run the + `analyze-ci-failure` skill on the reported failure. +5. **Autonomous Flake Retry**: If the CI Failure Analyzer subagent confirms + that the failure is a transient infrastructure or network flake (e.g. disk + I/O error, 504 gateway, sandbox initialization failure), **immediately and + autonomously** launch a separate subagent using `invoke_subagent` with the + role "CI Job Retrier" and the prompt template in + `.agents/skills/monitor-ci-results/retry-job-prompt.md` to run the + `buildkite-retry-job` skill for ``. Do not execute the script + directly; use the subagent and skill orchestration instead. Continue + monitoring without pausing to ask the user. diff --git a/.agents/skills/monitor-ci-results/retry-job-prompt.md b/.agents/skills/monitor-ci-results/retry-job-prompt.md new file mode 100644 index 0000000000..4d2edad650 --- /dev/null +++ b/.agents/skills/monitor-ci-results/retry-job-prompt.md @@ -0,0 +1,3 @@ +Use the `buildkite-retry-job` skill to retry the failed Buildkite job(s) for PR {pr_number}. +Follow the instructions in `.agents/skills/buildkite-retry-job/SKILL.md` to retry the failing jobs using the skill. +Once the retry request completes, report back to the parent conversation. diff --git a/.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py b/.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py new file mode 100755 index 0000000000..57fde0ce8d --- /dev/null +++ b/.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py @@ -0,0 +1,319 @@ +#!/usr/bin/env python3 + +import argparse +import fcntl +import json +import os +import subprocess +import sys +import time +import urllib.request + + +def check_cli(cmd_name): + try: + subprocess.run( + [cmd_name, "--version"], + check=True, + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + ) + return True + except Exception: + return False + + +def get_pr_checks(pr_number): + if not check_cli("gh"): + print("❌ 'gh' CLI not installed.", file=sys.stderr) + return [] + cmd = ["gh", "pr", "checks", str(pr_number), "--json", "name,link,state,bucket"] + try: + res = subprocess.run(cmd, capture_output=True, text=True) + out = res.stdout + json_str = out[out.find("[") : out.rfind("]") + 1] if "[" in out else "[]" + return json.loads(json_str) + except Exception as e: + print(f"⚠️ Error fetching PR checks: {e}", file=sys.stderr) + return [] + + +def get_buildkite_jobs(build_url): + base_url = build_url.split("#")[0] + if base_url.endswith(".json"): + base_url = base_url[:-5] + + jobs_url = f"{base_url}/data/jobs.json" + req = urllib.request.Request(jobs_url, headers={"User-Agent": "ci-monitor"}) + try: + with urllib.request.urlopen(req) as resp: + data = json.loads(resp.read().decode()) + if isinstance(data, list): + return data + elif isinstance(data, dict) and "records" in data: + return data["records"] + except Exception as e: + print( + f"⚠️ Could not fetch Buildkite jobs from {jobs_url}: {e}", + file=sys.stderr, + ) + return [] + + +def main(): + parser = argparse.ArgumentParser( + description="Monitor remote CI for failures and trigger analysis." + ) + parser.add_argument("pr", help="PR number to monitor") + parser.add_argument("conv_id", help="Conversation ID to report back to") + parser.add_argument( + "--interval", + type=int, + default=60, + help="Monitoring polling interval in seconds", + ) + parser.add_argument( + "--max-iterations", + type=int, + default=120, + help="Maximum number of polling cycles", + ) + args = parser.parse_args() + + skill_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), "..")) + scratch_dir = os.path.join(skill_dir, "scratch") + os.makedirs(scratch_dir, exist_ok=True) + + # Acquire lock for PR to avoid duplicate monitoring processes + lock_file_path = os.path.join(scratch_dir, f".monitored_pr_{args.pr}.lock") + lock_file = open(lock_file_path, "a+") + running_agent_conv_id = os.environ.get("ANTIGRAVITY_CONVERSATION_ID", "") + try: + fcntl.flock(lock_file.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB) + lock_file.seek(0) + lock_file.truncate(0) + lock_data = { + "pid": os.getpid(), + "agent_conv_id": running_agent_conv_id, + "notify_conv_id": args.conv_id, + } + lock_file.write(json.dumps(lock_data) + "\n") + lock_file.flush() + except (BlockingIOError, OSError): + existing_agent_conv = "" + try: + lock_file.seek(0) + content = lock_file.read().strip() + if content: + data = json.loads(content) + existing_agent_conv = data.get("agent_conv_id", "") or data.get( + "conv_id", "" + ) + except Exception: + pass + + subagent_str = ( + f" under agent/subagent conversation '{existing_agent_conv}'" + if existing_agent_conv + else "" + ) + print( + f"ℹ️ Continuous remote CI monitoring for PR #{args.pr} is already running in another process{subagent_str}. Exiting." + ) + sys.exit(0) + + state_file = os.path.join(scratch_dir, f"monitored_state_pr_{args.pr}.json") + monitored = {} + if os.path.exists(state_file): + try: + with open(state_file) as f: + monitored = json.load(f) + except Exception: + pass + + print( + f"🚀 Starting continuous remote CI monitoring for PR #{args.pr} every {args.interval}s..." + ) + + for i in range(args.max_iterations): + print( + f"🔍 [Cycle {i + 1}/{args.max_iterations}] Polling GitHub PR #{args.pr} checks..." + ) + checks = get_pr_checks(args.pr) + + for check in checks: + name = check.get("name", "unknown") + state = check.get("state", "UNKNOWN") + link = check.get("link", "") + bucket = check.get("bucket", "") + + if "buildkite" in name.lower() and link: + jobs = get_buildkite_jobs(link) + + passed = 0 + failed = 0 + running = 0 + blocked = 0 + other = 0 + + for job in jobs: + if job.get("retried_in_job_uuid") or job.get("retried_at"): + continue + jstate = job.get("state", "unknown") + exit_status = job.get("exit_status") + is_soft_failed = job.get("soft_failed") is True + is_blocked = jstate in ["blocked", "blocked_failed"] + is_failed = ( + ( + jstate in ["failed", "failing"] + or (exit_status != 0 and exit_status is not None) + ) + and not is_soft_failed + and not is_blocked + ) + is_passed = ( + jstate in ["passed", "success"] + or (jstate == "finished" and exit_status == 0) + or is_soft_failed + ) and not is_blocked + is_running = jstate in ["running", "scheduled"] and not is_blocked + + if is_blocked: + blocked += 1 + elif is_failed: + failed += 1 + elif is_passed: + passed += 1 + elif is_running: + running += 1 + else: + other += 1 + + build_id = link.split("/")[-1].split("#")[0] + print( + f"Buildkite #{build_id}: {len(jobs)} total jobs " + f"(Passed: {passed}, Failed: {failed}, Running: {running}, Blocked: {blocked}, Other: {other})" + ) + + for job in jobs: + if job.get("retried_in_job_uuid") or job.get("retried_at"): + continue + jname = job.get("name", "unknown_job") + jstate = job.get("state", "unknown") + jid = job.get("id", "") + jkey = f"bk_{jid}" + + exit_status = job.get("exit_status") + is_soft_failed = job.get("soft_failed") is True + is_blocked = jstate in ["blocked", "blocked_failed"] + is_failed = ( + ( + jstate in ["failed", "failing"] + or (exit_status != 0 and exit_status is not None) + ) + and not is_soft_failed + and not is_blocked + ) + + if is_blocked: + jkey_blocked = f"bk_blocked_{jid}" + if jkey_blocked not in monitored: + print( + f"⏸️ Notifying blocked state for Buildkite job '{jname}' (ID: {jid})..." + ) + msg = ( + f"⚠️ Remote CI Buildkite Job '{jname}' is blocked!\n\n" + f"Build ID: {build_id} | Job ID: {jid}\n" + f"Log URL: {job.get('log_url', link)}\n\n" + f"The user must confirm whether to run the CI jobs." + ) + subprocess.run( + [ + "agentapi", + "send-message", + "--title=CI Job Blocked", + args.conv_id, + msg, + ] + ) + monitored[jkey_blocked] = time.time() + with open(state_file, "w") as f: + json.dump(monitored, f) + + elif is_failed and jkey not in monitored: + print( + f"🚨 Notifying failure for Buildkite job '{jname}' (ID: {jid})..." + ) + msg = ( + f"⚠️ Remote CI Buildkite Job '{jname}' completed with errors!\n\n" + f"Build ID: {build_id} | Job ID: {jid}\n" + f"Log URL: {job.get('log_url', link)}\n\n" + f"Action required: Start a subagent to run the analyze-ci-failure skill for '{jname}' with build ID '{build_id}' and job ID '{jid}'." + ) + subprocess.run( + [ + "agentapi", + "send-message", + "--title=CI Job Failed", + args.conv_id, + msg, + ] + ) + monitored[jkey] = time.time() + with open(state_file, "w") as f: + json.dump(monitored, f) + + elif ( + state.upper() in ["ACTION_REQUIRED", "BLOCKED"] + and f"gh_blocked_{name}" not in monitored + ): + print(f"⏸️ Notifying blocked state for GitHub check '{name}'...") + msg = ( + f"⚠️ Remote CI GitHub Check '{name}' is blocked!\n\n" + f"Link: {link}\n\n" + f"The user must confirm whether to run the CI jobs." + ) + subprocess.run( + [ + "agentapi", + "send-message", + "--title=CI Check Blocked", + args.conv_id, + msg, + ] + ) + monitored[f"gh_blocked_{name}"] = time.time() + with open(state_file, "w") as f: + json.dump(monitored, f) + + elif ( + state.upper() in ["FAILURE", "FAIL", "CANCELLED", "TIMED_OUT"] + or bucket == "fail" + ) and name not in monitored: + print(f"🚨 Notifying failure for GitHub check '{name}'...") + job_id = link.split("/")[-1] if "/" in link else link + msg = ( + f"⚠️ Remote CI GitHub Check '{name}' completed with errors!\n\n" + f"Link: {link}\n\n" + f"Action required: Start a subagent to run the analyze-ci-failure skill for '{name}' with link '{link}' and job ID '{job_id}'." + ) + subprocess.run( + [ + "agentapi", + "send-message", + "--title=CI Check Failed", + args.conv_id, + msg, + ] + ) + monitored[name] = time.time() + with open(state_file, "w") as f: + json.dump(monitored, f) + + time.sleep(args.interval) + + print("🏁 CI monitoring service completed its scheduled iterations.") + + +if __name__ == "__main__": + main() diff --git a/.agents/skills/review-code/SKILL.md b/.agents/skills/review-code/SKILL.md new file mode 100644 index 0000000000..8646ff2451 --- /dev/null +++ b/.agents/skills/review-code/SKILL.md @@ -0,0 +1,59 @@ +--- +name: review-code +description: High-level code review audit of local changes when preparing to + send a PR for review +trigger: model_decision +--- + +When preparing to send a Pull Request for review, invoke **separate, concurrent +sub-agents** (`invoke_subagent`), where each sub-agent focuses exclusively on +its assigned checklist category. + +**CRITICAL**: Do NOT use a single sub-agent to validate multiple or all +dimensions at once. Each sub-agent must be launched with its own distinct prompt +file from `.agents/skills/review-code/`: + +### Audit Checklist & Sub-Agent Prompts + +1. **Starlark / Bazel Sub-Agent** (`Role: "Starlark Code Auditor"`): + - Prompt file: + `.agents/skills/review-code/review-starlark-prompt.md` + - Focus: Audits Starlark / Bazel changes (`*.bzl`, `BUILD`, `*.bazel` files) + in `git diff` against `.agents/rules/bzl.md` and Starlark rules in + `AGENTS.md`. + +2. **Python Code Sub-Agent** (`Role: "Python Code Auditor"`): + - Prompt file: + `.agents/skills/review-code/review-python-prompt.md` + - Focus: Audits Python source and test changes in `git diff` against + `.agents/rules/python.md` and Python and pytest conventions in `AGENTS.md`. + +3. **Documentation Sub-Agent** (`Role: "Documentation Auditor"`): + - Prompt file: + `.agents/skills/review-code/review-docs-prompt.md` + - Focus: Audits documentation (`.md`) changes and docs build targets against + `.agents/rules/docs.md`, `.agents/rules/news.md`, and `AGENTS.md`. + +4. **Contribution Guidelines Sub-Agent** (`Role: "Contributing Auditor"`): + - Prompt file: + `.agents/skills/review-code/review-contributing-prompt.md` + - Focus: Audits changes, requirements updates, directives, and news entry + files (`news/..md`) against `CONTRIBUTING.md` and + `.agents/rules/news.md`. + +5. **Project Conventions Sub-Agent** (`Role: "Project Conventions Auditor"`): + - Prompt file: + `.agents/skills/review-code/review-agents-prompt.md` + - Focus: Audits overall workspace compliance against `AGENTS.md`. + +6. **Pull Request Standards Sub-Agent** (`Role: "PR Standards Auditor"`): + - Prompt file: + `.agents/skills/review-code/review-pr-standards-prompt.md` + - Focus: Audits PR titles and descriptions against Conventional Commits + formatting and PR update rules in `CONTRIBUTING.md`. + +### Action Instructions +- Launch all sub-agents concurrently using `invoke_subagent`. +- Collect the reports from each sub-agent and report all violations and + suggested improvements clearly with suggested fixes for the user. +- If all domain audits pass, confirm that the PR is ready for review. diff --git a/.agents/skills/review-code/review-agents-prompt.md b/.agents/skills/review-code/review-agents-prompt.md new file mode 100644 index 0000000000..45651fd0de --- /dev/null +++ b/.agents/skills/review-code/review-agents-prompt.md @@ -0,0 +1,18 @@ +You are a specialized Project Conventions (`AGENTS.md`) Auditor sub-agent. +Your sole task is to audit all local changes (`git diff`) and workspace state +against `AGENTS.md`: + +1. Read and strictly enforce `AGENTS.md` and all `.agents/rules/*.md` files + without exception. +2. Verify NO Bazel copyright headers (`# Copyright ... The Bazel Authors`) + were added to new or existing files, unless explicitly instructed by the + user. +3. Verify that tests were executed using `bazel test --config=fast-tests` and + non-test build targets did not use `--config=fast-tests`. +4. Ensure public config settings in `python/config_settings/BUILD.bazel` were + not modified unless explicitly instructed. +5. Check that all repo rules and macro conventions described in `AGENTS.md` + are respected. + +Report any violations found clearly with actionable suggested fixes, or report +that the changes pass project conventions audit. diff --git a/.agents/skills/review-code/review-contributing-prompt.md b/.agents/skills/review-code/review-contributing-prompt.md new file mode 100644 index 0000000000..51fe58db00 --- /dev/null +++ b/.agents/skills/review-code/review-contributing-prompt.md @@ -0,0 +1,37 @@ +You are a specialized Contribution Guidelines Auditor sub-agent. +Your sole task is to audit all local changes (`git diff`), commit messages, +and PR metadata against `CONTRIBUTING.md` and project rules: + +1. Read and strictly enforce `CONTRIBUTING.md`, all `.agents/rules/*.md` files + (especially `.agents/rules/news.md`), and `AGENTS.md`. +2. **News Entry File Audit**: + - Check if the PR introduces user-visible features (`feat:`), bug fixes + (`fix:`), behavioral changes (`changed:`), breaking changes, or removals + (`removed:`). If so, verify that a news fragment file is added under + `news/..md`. + - Verify `CHANGELOG.md` is NOT modified directly for unreleased changes. + - Verify news filename format: `news/..md` where `` is the + PR or issue number (or placeholder ID) and `` is strictly one of + `added`, `changed`, `fixed`, or `removed`. + - Verify news entry content rules: + - Brief, human-friendly description without leading bullet points (`*` + or `-`). + - Subsystem prefix in parentheses when applicable (e.g. `(gazelle) ...`, + `(cc) ...`). + - Use Sphinx MyST cross-reference syntax `{obj}\`\`` for rules, + macros, targets, providers, attributes, and args. + - Append GitHub issue cross-references at the end in markdown link format: + `([#1234](https://github.com/bazel-contrib/rules_python/issues/1234))`. + - Lines wrapped at 80 columns. +3. Verify that `{versionadded}` and `{versionchanged}` directives use + `VERSION_NEXT_FEATURE` for unreleased features and are placed at the end of + the documentation text. +4. If locked/resolved requirements files (`requirements.txt`, `pyproject.toml`, + `requirements.in`) were modified, verify that the associated + `requirements.update` target was executed to keep locked requirement files + in sync. +5. Ensure style and conventions described in `CONTRIBUTING.md` are respected + across all changes. + +Report any violations found clearly with actionable suggested fixes, or report +that the changes pass contribution audit. diff --git a/.agents/skills/review-code/review-docs-prompt.md b/.agents/skills/review-code/review-docs-prompt.md new file mode 100644 index 0000000000..eaeb91232d --- /dev/null +++ b/.agents/skills/review-code/review-docs-prompt.md @@ -0,0 +1,24 @@ +You are a specialized Documentation & Sphinx/MyST Auditor sub-agent. +Your sole task is to audit all documentation (`.md`) changes and new `.bzl` APIs +in `git diff` against the project's documentation rules: + +1. Read and strictly enforce `.agents/rules/docs.md`, `.agents/rules/news.md`, + `AGENTS.md`, and `CONTRIBUTING.md`. +2. Check that lines wrap at 80 columns. +3. Ensure markdown filenames use hyphens (`-`) rather than underscores (`_`) + (except news entry files under `news/`, which follow `..md`). +4. Verify Sphinx MyST colon indentation hierarchy (outer directives must have + more colons than inner directives). +5. Verify `{versionadded}` and `{versionchanged}` sections are placed at the + end of the documentation text. +6. For unreleased features or attributes, ensure `{versionadded}` / + `{versionchanged}` directives use `VERSION_NEXT_FEATURE` (not hardcoded + version numbers). +7. Check documentation build correctness: ensure new `.bzl` files or public + APIs are included in `//docs:docs` or relevant docs build targets. +8. For any added or modified news entries in `news/`, verify they adhere to + `.agents/rules/news.md` and `CONTRIBUTING.md` (proper `..md` + name, no leading bullets, subsystem prefix, `{obj}` refs, issue links). + +Report any violations found clearly with actionable suggested fixes, or report +that the documentation changes pass audit. diff --git a/.agents/skills/review-code/review-pr-standards-prompt.md b/.agents/skills/review-code/review-pr-standards-prompt.md new file mode 100644 index 0000000000..a4de4296a1 --- /dev/null +++ b/.agents/skills/review-code/review-pr-standards-prompt.md @@ -0,0 +1,17 @@ +You are a specialized PR Standards Auditor sub-agent. +Your sole task is to audit PR titles and PR descriptions against +`CONTRIBUTING.md` and project rules: + +1. Read and strictly enforce `.agents/rules/pr.md`, `.agents/rules/news.md`, + `AGENTS.md`, and `CONTRIBUTING.md`. +2. Check PR title formatting: must follow Conventional Commits format (e.g. + `feat(cc): ...`, `docs(python): ...`). For agent rules/skills, use `agents:` + prefix. +3. Ensure PR descriptions explain *why* a change is made and provide a + high-level overview of *how*, following advice in `CONTRIBUTING.md`. +4. If a PR has already been created, enforce PR update rules: do NOT amend or + rebase existing commits (create new commits and merges instead, to preserve + code review comment threads). + +Report any violations found clearly with actionable suggested fixes, or report +that the PR standards pass audit. diff --git a/.agents/skills/review-code/review-python-prompt.md b/.agents/skills/review-code/review-python-prompt.md new file mode 100644 index 0000000000..8e477ab3a7 --- /dev/null +++ b/.agents/skills/review-code/review-python-prompt.md @@ -0,0 +1,15 @@ +You are a specialized Python & pytest Auditor sub-agent. +Your sole task is to audit all Python source (`.py`) and test changes in +`git diff` against the project's Python conventions: + +1. Read and strictly enforce `.agents/rules/python.md`, `AGENTS.md`, and + `CONTRIBUTING.md`. +2. Check Python pytest conventions: when registering pytest fixtures from helper + modules in test files, use `pytest_plugins = [""]`. +3. Name fixture functions with a `fixture_` prefix (e.g. `def fixture_foo():`) + and pass the public fixture name using `@pytest.fixture(name="foo")`. +4. Verify that tests were executed using Bazel + (`bazel test --config=fast-tests`) and passed. + +Report any violations found clearly with actionable suggested fixes, or report +that the Python changes pass audit. diff --git a/.agents/skills/review-code/review-starlark-prompt.md b/.agents/skills/review-code/review-starlark-prompt.md new file mode 100644 index 0000000000..8bd8cdbe21 --- /dev/null +++ b/.agents/skills/review-code/review-starlark-prompt.md @@ -0,0 +1,17 @@ +You are a specialized Starlark & Bazel Code Auditor sub-agent. +Your sole task is to audit all Starlark (`.bzl`, `BUILD`, `*.bazel`) changes +in `git diff` against the project's Starlark coding rules and conventions: + +1. Read and strictly enforce `.agents/rules/starlark.md`, + `.agents/rules/bzl.md`, `AGENTS.md`, and `CONTRIBUTING.md`. +2. Verify iterative algorithms are used (no recursion, no `while` loops). +3. Ensure every `.bzl` file outside `tests/` has a corresponding `bzl_library` + target in its `BUILD` file with proper dependencies. +4. Ensure loads from `/private/` in test files have + `# buildifier: disable=bzl-visibility`. +5. Check multi-line rule/macro doc arguments: use triple-quoted strings + (`"""`), and do NOT use trailing backslashes (`\`) on opening triple-quotes. +6. Verify analysis tests use `rules_testing`, not `bazel_skylib`. + +Report any violations found clearly with actionable suggested fixes, or report +that the Starlark changes pass audit. diff --git a/.agents/skills/rule-creator/SKILL.md b/.agents/skills/rule-creator/SKILL.md new file mode 100644 index 0000000000..9ed7978120 --- /dev/null +++ b/.agents/skills/rule-creator/SKILL.md @@ -0,0 +1,56 @@ +--- +name: rule-creator +description: Create and format agent rules with proper front matter in the workspace +--- + +Use this skill when you need to create a new rule for the agent in the +workspace. + +### Rule File Location + +All workspace-specific rules must be created as individual Markdown files under +the `.agents/rules/` directory: +``` +.agents/rules/.md +``` + +### Rule Format + +Every rule file must start with a YAML front matter block defining the trigger +condition, followed by the rule content in Markdown. + +```yaml +--- +trigger: +--- + +# + + +``` + +#### Trigger Conditions +* `always_on`: The rule is always active and must be followed for all tasks. +* Custom triggers: You can specify other trigger conditions if the rule only + applies in certain contexts. + +### Formatting Guidelines +* **Line Wrapping:** Always wrap all text in the rule file (including the + title and description) to **80 columns** to ensure readability and + compatibility. +* **Clarity:** Write clear, actionable directives. + +### Example + +To create a rule that prevents adding copyrights: + +```markdown +--- +trigger: always_on +--- + +# No Copyrights Rule + +Unless directed by the user otherwise, do not add Bazel copyright to new or +existing files. +``` diff --git a/.bazelci/isolate.sh b/.bazelci/isolate.sh new file mode 100755 index 0000000000..32a1de3fcf --- /dev/null +++ b/.bazelci/isolate.sh @@ -0,0 +1,32 @@ +#!/usr/bin/env bash +set -euo pipefail + +module_dir="${1:-}" + +if [[ -z "${module_dir}" ]]; then + echo "Usage: $0 " + exit 1 +fi + +# Find the repository root assuming this script is in .bazelci/ +script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +repo_root="$(dirname "${script_dir}")" + +cd "${repo_root}" + +if [[ ! -f "MODULE.bazel" ]]; then + echo "Error: MODULE.bazel not found in ${repo_root}. Are you sure this is the repo root?" + exit 1 +fi + +if [[ ! -d "${module_dir}" ]]; then + echo "Error: Module directory '${module_dir}' not found in ${repo_root}." + exit 1 +fi + +echo "Removing files outside of ${module_dir} to simulate BCR environment..." +find . -maxdepth 1 -mindepth 1 \ + ! -name "${module_dir}" \ + ! -name ".git" \ + ! -name ".bazelci" \ + -exec rm -rf '{}' + diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index c45fc78990..b3e308c7c7 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -16,25 +16,30 @@ buildifier: # keep these arguments in sync with .pre-commit-config.yaml # Use a specific version to avoid skew issues when new versions are released. - version: 6.1.0 + version: 8.2.1 warnings: "all" -# NOTE: Minimum supported version is 6.x for workspace; 7.x for bzlmod +# NOTE: Minimum supported version is 7.x .minimum_supported_version: &minimum_supported_version # For testing minimum supported version. # NOTE: Keep in sync with //:version.bzl - bazel: 6.4.0 - skip_in_bazel_downstream_pipeline: "Bazel 6 required" + bazel: 7.x + skip_in_bazel_downstream_pipeline: "Bazel 7 required" .reusable_config: &reusable_config + # USE_BAZEL_DIFF is disabled because target_compatible_with is used heavily, + # but BazelCI's bazel-diff integration errors when handling them. + # environment: + # USE_BAZEL_DIFF: "1" build_targets: - "--" - "..." # As a regression test for #225, check that wheel targets still build when # their package path is qualified with the repo name. - "@rules_python//examples/wheel/..." - build_flags: + build_flags: &reusable_config_build_flags + - "--experimental_repository_cache_hardlinks=false" - "--keep_going" - "--build_tag_filters=-integration-test" - - "--config=bazel7.x" + - "--verbose_failures" test_targets: - "--" - "..." @@ -42,18 +47,24 @@ buildifier: - "--test_tag_filters=-integration-test" .common_workspace_flags_min_bazel: &common_workspace_flags_min_bazel build_flags: + - "--experimental_repository_cache_hardlinks=false" - "--noenable_bzlmod" - "--build_tag_filters=-integration-test" test_flags: - "--noenable_bzlmod" - "--test_tag_filters=-integration-test" .common_workspace_flags: &common_workspace_flags + skip_in_bazel_downstream_pipeline: "Bazel 9 doesn't support workspace" test_flags: - "--noenable_bzlmod" - "--enable_workspace" + - "--test_tag_filters=-integration-test" build_flags: - "--noenable_bzlmod" - "--enable_workspace" + - "--build_tag_filters=-integration-test" + bazel: 7.x +# NOTE: The Mac and Windows bazelinbazel jobs override parts of this config. .common_bazelinbazel_config: &common_bazelinbazel_config build_flags: - "--build_tag_filters=integration-test" @@ -73,73 +84,179 @@ buildifier: coverage_targets: ["..."] .coverage_targets_example_bzlmod_build_file_generation: &coverage_targets_example_bzlmod_build_file_generation coverage_targets: ["//:bzlmod_build_file_generation_test"] +.coverage_targets_bootstrap: &coverage_targets_bootstrap + coverage_targets: + - //tests/bootstrap_impls:stdlib_shadowing_system_python_test .coverage_targets_example_multi_python: &coverage_targets_example_multi_python coverage_targets: - //tests:my_lib_3_10_test - //tests:my_lib_3_11_test - - //tests:my_lib_3_8_test - - //tests:my_lib_3_9_test + - //tests:my_lib_3_12_test + - //tests:my_lib_3_13_test + - //tests:my_lib_3_14_test - //tests:my_lib_default_test - //tests:version_3_10_test - //tests:version_3_11_test - - //tests:version_3_8_test - - //tests:version_3_9_test + - //tests:version_3_12_test + - //tests:version_3_13_test + - //tests:version_3_14_test - //tests:version_default_test + +# Keep in sync with .bcr/gazelle/presubmit.yml +.gazelle_common_bcr: &gazelle_common_bcr + bazel: ${{ bazel }} + working_directory: gazelle/examples/bzlmod_build_file_generation + shell_commands: + - "echo 'common --override_module=rules_python=' >> .bazelrc" + - "bazel run //:gazelle_python_manifest.update" + - "bazel run //:gazelle -- update" + batch_commands: + - "echo common --override_module=rules_python= >> .bazelrc " + - " bazel run //:gazelle_python_manifest.update " + - " bazel run //:gazelle -- update" + build_targets: + - "//..." + - ":modules_map" + test_targets: + - "//..." + + +matrix: + # Keep in sync with .bcr/presubmit.yml + platform: + - ubuntu2204 + - debian11 + - macos_arm64 + - windows + # Keep in sync with .bcr/presubmit.yml + bazel: [7.x, 8.x, 9.x] + tasks: + # Keep in sync with .bcr/presubmit.yml + bcr_test: + name: "BCR: Bazel {bazel}" + platform: ${{ platform }} + working_directory: examples/bzlmod + bazel: ${{ bazel }} + build_flags: + - "--keep_going" + test_flags: + - "--keep_going" + build_targets: + - "//..." + test_targets: + - "//..." + + # Keep in sync with .bcr/gazelle/presubmit.yml + gazelle_bcr_ubuntu: + <<: *gazelle_common_bcr + name: "Gazelle: BCR, Bazel {bazel}" + platform: ubuntu2204 + gazelle_bcr_debian11: + <<: *gazelle_common_bcr + name: "Gazelle: BCR, Bazel {bazel}" + platform: debian11 + gazelle_bcr_macos_arm64: + <<: *gazelle_common_bcr + name: "Gazelle: BCR, Bazel {bazel}" + platform: macos_arm64 + gazelle_extension_min: <<: *common_workspace_flags_min_bazel <<: *minimum_supported_version - name: "Gazelle: workspace, minumum supported Bazel version" - platform: ubuntu2004 + name: "Gazelle: workspace, minimum supported Bazel version" + platform: ubuntu2204 build_targets: ["//..."] test_targets: ["//..."] working_directory: gazelle gazelle_extension_workspace: <<: *common_workspace_flags name: "Gazelle: workspace" - platform: ubuntu2004 + platform: ubuntu2204 build_targets: ["//..."] test_targets: ["//..."] working_directory: gazelle gazelle_extension: name: "Gazelle: default settings" - platform: ubuntu2004 + platform: ubuntu2204 build_targets: ["//..."] test_targets: ["//..."] working_directory: gazelle + sphinxdocs_ubuntu: + name: "Sphinxdocs: Ubuntu" + working_directory: "sphinxdocs" + platform: ubuntu2204 + shell_commands: + - "../.bazelci/isolate.sh sphinxdocs" + build_targets: + - "//..." + test_targets: + - "//..." + sphinxdocs_mac: + name: "Sphinxdocs: Mac" + working_directory: "sphinxdocs" + platform: macos_arm64 + shell_commands: + - "../.bazelci/isolate.sh sphinxdocs" + build_targets: + - "//..." + test_targets: + - "//..." + ubuntu_min_workspace: <<: *minimum_supported_version <<: *reusable_config <<: *common_workspace_flags_min_bazel name: "Default: Ubuntu, workspace, minimum Bazel" - platform: ubuntu2004 + platform: ubuntu2204 ubuntu_min_bzlmod: <<: *minimum_supported_version <<: *reusable_config name: "Default: Ubuntu, bzlmod, minimum Bazel" - platform: ubuntu2004 + platform: ubuntu2204 bazel: 7.x ubuntu: <<: *reusable_config - name: "Default: Ubuntu" - platform: ubuntu2004 + <<: *coverage_targets_bootstrap + name: "Default: Ubuntu, Bazel {bazel}" + platform: ubuntu2204 + bazel: ${{ bazel }} ubuntu_upcoming: <<: *reusable_config name: "Default: Ubuntu, upcoming Bazel" - platform: ubuntu2004 + platform: ubuntu2204 bazel: last_rc + # This is an advisory job; doesn't block merges + # RCs may have regressions, so don't fail our CI on them + soft_fail: + - exit_status: 1 + - exit_status: 3 + ubuntu_rolling: + name: "Default: Ubuntu, rolling Bazel" + platform: ubuntu2204 + bazel: rolling + # This is an advisory job; doesn't block merges + soft_fail: + - exit_status: 1 + - exit_status: 3 + test_targets: + - "--" + - "//tests/..." + test_flags: + - "--keep_going" + - "--test_tag_filters=-integration-test" ubuntu_workspace: <<: *reusable_config <<: *common_workspace_flags name: "Default: Ubuntu, workspace" - platform: ubuntu2004 + platform: ubuntu2204 mac_workspace: <<: *reusable_config <<: *common_workspace_flags name: "Default: Mac, workspace" - platform: macos + platform: macos_arm64 windows_workspace: <<: *reusable_config <<: *common_workspace_flags @@ -158,7 +275,6 @@ tasks: - "--enable_workspace" - "--keep_going" - "--build_tag_filters=-integration-test" - - "--config=bazel7.x" test_targets: - "--" - "..." @@ -169,25 +285,28 @@ tasks: debian: <<: *reusable_config - name: "Default: Debian" + name: "Default: Debian, Bazel {bazel}" platform: debian11 - macos: + bazel: ${{ bazel }} + macos_arm64: <<: *reusable_config - name: "Default: MacOS" - platform: macos + name: "Default: MacOS, Bazel {bazel}" + platform: macos_arm64 + bazel: ${{ bazel }} windows: <<: *reusable_config - name: "Default: Windows" + name: "Default: Windows, Bazel {bazel}" platform: windows + bazel: ${{ bazel }} test_flags: - "--test_tag_filters=-integration-test,-fix-windows" rbe_min: <<: *minimum_supported_version <<: *reusable_config name: "RBE: Ubuntu, minimum Bazel" - platform: rbe_ubuntu2004 - bazel: 7.x + platform: rbe_ubuntu2404 build_flags: + - "--experimental_repository_cache_hardlinks=false" # BazelCI sets --action_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1, # which prevents cc toolchain autodetection from working correctly # on Bazel 5.4 and earlier. To workaround this, manually specify the @@ -204,7 +323,10 @@ tasks: rbe: <<: *reusable_config name: "RBE: Ubuntu" - platform: rbe_ubuntu2004 + platform: rbe_ubuntu2404 + # TODO @aignas 2024-12-11: get the RBE working in CI for bazel 8.0 + # See https://github.com/bazelbuild/rules_python/issues/2499 + bazel: 8.x test_flags: - "--test_tag_filters=-integration-test,-acceptance-test" - "--extra_toolchains=@buildkite_config//config:cc-toolchain" @@ -215,13 +337,13 @@ tasks: <<: *common_workspace_flags_min_bazel name: "examples/build_file_generation: Ubuntu, workspace, minimum Bazel" working_directory: examples/build_file_generation - platform: ubuntu2004 + platform: ubuntu2204 integration_test_build_file_generation_ubuntu_workspace: <<: *reusable_build_test_all <<: *common_workspace_flags name: "examples/build_file_generation: Ubuntu, workspace" working_directory: examples/build_file_generation - platform: ubuntu2004 + platform: ubuntu2204 integration_test_build_file_generation_debian_workspace: <<: *reusable_build_test_all <<: *common_workspace_flags @@ -233,7 +355,7 @@ tasks: <<: *common_workspace_flags name: "examples/build_file_generation: macOS, workspace" working_directory: examples/build_file_generation - platform: macos + platform: macos_arm64 integration_test_build_file_generation_windows_workspace: <<: *reusable_build_test_all <<: *common_workspace_flags @@ -241,104 +363,15 @@ tasks: working_directory: examples/build_file_generation platform: windows - integration_test_bzlmod_ubuntu_min: - <<: *minimum_supported_version - <<: *reusable_build_test_all - coverage_targets: ["//:test"] - name: "examples/bzlmod: Ubuntu, minimum Bazel" - working_directory: examples/bzlmod - platform: ubuntu2004 - bazel: 7.x - integration_test_bzlmod_ubuntu: - <<: *reusable_build_test_all - <<: *coverage_targets_example_bzlmod - name: "examples/bzlmod: Ubuntu" - working_directory: examples/bzlmod - platform: ubuntu2004 - bazel: 7.x - integration_test_bzlmod_ubuntu_upcoming: - <<: *reusable_build_test_all - <<: *coverage_targets_example_bzlmod - name: "examples/bzlmod: Ubuntu, upcoming Bazel" - working_directory: examples/bzlmod - platform: ubuntu2004 - bazel: last_rc - integration_test_bzlmod_debian: - <<: *reusable_build_test_all - <<: *coverage_targets_example_bzlmod - name: "examples/bzlmod: Debian" - working_directory: examples/bzlmod - platform: debian11 - bazel: 7.x - integration_test_bzlmod_macos: - <<: *reusable_build_test_all - <<: *coverage_targets_example_bzlmod - name: "examples/bzlmod: macOS" - working_directory: examples/bzlmod - platform: macos - bazel: 7.x - integration_test_bzlmod_macos_upcoming: + integration_test_bzlmod_ubuntu_vendor: <<: *reusable_build_test_all - <<: *coverage_targets_example_bzlmod - name: "examples/bzlmod: macOS, upcoming Bazel" + name: "examples/bzlmod: bazel vendor" working_directory: examples/bzlmod - platform: macos - bazel: last_rc - integration_test_bzlmod_windows: - <<: *reusable_build_test_all - # coverage is not supported on Windows - name: "examples/bzlmod: Windows" - working_directory: examples/bzlmod - platform: windows - bazel: 7.x - integration_test_bzlmod_windows_upcoming: - <<: *reusable_build_test_all - # coverage is not supported on Windows - name: "examples/bzlmod: Windows, upcoming Bazel" - working_directory: examples/bzlmod - platform: windows - bazel: last_rc - - integration_test_bzlmod_generate_build_file_generation_ubuntu_min: - <<: *minimum_supported_version - <<: *reusable_build_test_all - <<: *coverage_targets_example_bzlmod_build_file_generation - name: "examples/bzlmod_build_file_generation: Ubuntu, minimum Bazel" - working_directory: examples/bzlmod_build_file_generation - platform: ubuntu2004 - bazel: 7.x - integration_test_bzlmod_generation_build_files_ubuntu: - <<: *reusable_build_test_all - <<: *coverage_targets_example_bzlmod_build_file_generation - name: "examples/bzlmod_build_file_generation: Ubuntu" - working_directory: examples/bzlmod_build_file_generation - platform: ubuntu2004 - integration_test_bzlmod_generation_build_files_ubuntu_run: - <<: *reusable_build_test_all - name: "examples/bzlmod_build_file_generation: Ubuntu, Gazelle and pip" - working_directory: examples/bzlmod_build_file_generation - platform: ubuntu2004 + platform: ubuntu2204 shell_commands: - - "bazel run //:gazelle_python_manifest.update" - - "bazel run //:gazelle -- update" - integration_test_bzlmod_build_file_generation_debian: - <<: *reusable_build_test_all - <<: *coverage_targets_example_bzlmod_build_file_generation - name: "examples/bzlmod_build_file_generation: Debian" - working_directory: examples/bzlmod_build_file_generation - platform: debian11 - integration_test_bzlmod_build_file_generation_macos: - <<: *reusable_build_test_all - <<: *coverage_targets_example_bzlmod_build_file_generation - name: "examples/bzlmod_build_file_generation: MacOS" - working_directory: examples/bzlmod_build_file_generation - platform: macos - integration_test_bzlmod_build_file_generation_windows: - <<: *reusable_build_test_all - # coverage is not supported on Windows - name: "examples/bzlmod_build_file_generateion: Windows" - working_directory: examples/bzlmod_build_file_generation - platform: windows + - "bazel vendor --vendor_dir=./vendor //..." + - "bazel build --vendor_dir=./vendor //..." + - "rm -rf ./vendor" integration_test_multi_python_versions_ubuntu_workspace: <<: *reusable_build_test_all @@ -346,7 +379,7 @@ tasks: <<: *coverage_targets_example_multi_python name: "examples/multi_python_versions: Ubuntu, workspace" working_directory: examples/multi_python_versions - platform: ubuntu2004 + platform: ubuntu2204 integration_test_multi_python_versions_debian_workspace: <<: *reusable_build_test_all <<: *common_workspace_flags @@ -360,7 +393,7 @@ tasks: <<: *coverage_targets_example_multi_python name: "examples/multi_python_versions: MacOS, workspace" working_directory: examples/multi_python_versions - platform: macos + platform: macos_arm64 integration_test_multi_python_versions_windows_workspace: <<: *reusable_build_test_all <<: *common_workspace_flags @@ -375,19 +408,19 @@ tasks: <<: *reusable_build_test_all name: "examples/pip_parse: Ubuntu, workspace, minimum supported Bazel version" working_directory: examples/pip_parse - platform: ubuntu2004 + platform: ubuntu2204 integration_test_pip_parse_ubuntu_min_bzlmod: <<: *minimum_supported_version <<: *reusable_build_test_all name: "examples/pip_parse: Ubuntu, bzlmod, minimum supported Bazel version" working_directory: examples/pip_parse - platform: ubuntu2004 + platform: ubuntu2204 bazel: 7.x integration_test_pip_parse_ubuntu: <<: *reusable_build_test_all name: "examples/pip_parse: Ubuntu" working_directory: examples/pip_parse - platform: ubuntu2004 + platform: ubuntu2204 integration_test_pip_parse_debian: <<: *reusable_build_test_all name: "examples/pip_parse: Debian" @@ -397,7 +430,7 @@ tasks: <<: *reusable_build_test_all name: "examples/pip_parse: MacOS" working_directory: examples/pip_parse - platform: macos + platform: macos_arm64 integration_test_pip_parse_windows: <<: *reusable_build_test_all name: "examples/pip_parse: Windows" @@ -410,98 +443,60 @@ tasks: <<: *reusable_build_test_all name: "examples/pip_parse_vendored: Ubuntu, workspace, minimum Bazel" working_directory: examples/pip_parse_vendored - platform: ubuntu2004 - integration_test_pip_parse_vendored_ubuntu_min_bzlmod: - <<: *minimum_supported_version - <<: *reusable_build_test_all - name: "examples/pip_parse_vendored: Ubuntu, bzlmod, minimum Bazel" - working_directory: examples/pip_parse_vendored - platform: ubuntu2004 - bazel: 7.x + platform: ubuntu2204 integration_test_pip_parse_vendored_ubuntu: <<: *reusable_build_test_all + <<: *common_workspace_flags name: "examples/pip_parse_vendored: Ubuntu" working_directory: examples/pip_parse_vendored - platform: ubuntu2004 + platform: ubuntu2204 integration_test_pip_parse_vendored_debian: <<: *reusable_build_test_all + <<: *common_workspace_flags name: "examples/pip_parse_vendored: Debian" working_directory: examples/pip_parse_vendored platform: debian11 integration_test_pip_parse_vendored_macos: <<: *reusable_build_test_all + <<: *common_workspace_flags name: "examples/pip_parse_vendored: MacOS" working_directory: examples/pip_parse_vendored - platform: macos + platform: macos_arm64 # We don't run pip_parse_vendored under Windows as the file checked in is # generated from a repository rule containing OS-specific rendered paths. - # The proto example is workspace-only; bzlmod functionality is covered - # by examples/bzlmod/py_proto_library - integration_test_py_proto_library_ubuntu_workspace: - <<: *reusable_build_test_all - <<: *common_workspace_flags - name: "examples/py_proto_library: Ubuntu, workspace" - working_directory: examples/py_proto_library - platform: ubuntu2004 - integration_test_py_proto_library_debian_workspace: - <<: *reusable_build_test_all - <<: *common_workspace_flags - name: "examples/py_proto_library: Debian, workspace" - working_directory: examples/py_proto_library - platform: debian11 - integration_test_py_proto_library_macos_workspace: - <<: *reusable_build_test_all - <<: *common_workspace_flags - name: "examples/py_proto_library: MacOS, workspace" - working_directory: examples/py_proto_library - platform: macos - integration_test_py_proto_library_windows_workspace: - <<: *reusable_build_test_all - <<: *common_workspace_flags - name: "examples/py_proto_library: Windows, workspace" - working_directory: examples/py_proto_library - platform: windows - - integration_test_pip_repository_annotations_ubuntu_workspace: - <<: *reusable_build_test_all - <<: *common_workspace_flags - name: "examples/pip_repository_annotations: Ubuntu, workspace" - working_directory: examples/pip_repository_annotations - platform: ubuntu2004 - integration_test_pip_repository_annotations_debian_workspace: - <<: *reusable_build_test_all - <<: *common_workspace_flags - name: "examples/pip_repository_annotations: Debian, workspace" - working_directory: examples/pip_repository_annotations - platform: debian11 - integration_test_pip_repository_annotations_macos_workspace: - <<: *reusable_build_test_all - <<: *common_workspace_flags - name: "examples/pip_repository_annotations: macOS, workspace" - working_directory: examples/pip_repository_annotations - platform: macos - integration_test_pip_repository_annotations_windows_workspace: - <<: *reusable_build_test_all - <<: *common_workspace_flags - name: "examples/pip_repository_annotations: Windows, workspace" - working_directory: examples/pip_repository_annotations - platform: windows - integration_test_bazelinbazel_ubuntu: <<: *common_bazelinbazel_config name: "tests/integration bazel-in-bazel: Ubuntu" - platform: ubuntu2004 + platform: ubuntu2204 integration_test_bazelinbazel_debian: <<: *common_bazelinbazel_config name: "tests/integration bazel-in-bazel: Debian" platform: debian11 + # The bazelinbazel tests were disabled on Mac to save CI jobs slots, and + # have bitrotted a bit. For now, just run a subset of what we're most + # interested in. + integration_test_bazelinbazel_macos: + <<: *common_bazelinbazel_config + name: "tests/integration bazel-in-bazel: macOS (subset)" + platform: macos_arm64 + build_targets: ["//tests/integration:subset"] + test_targets: ["//tests/integration:subset"] + # The bazelinbazel tests were disabled on Windows to save CI jobs slots, and + # have bitrotted a bit. For now, just run a subset of what we're most + # interested in. + integration_test_bazelinbazel_windows: + <<: *common_bazelinbazel_config + name: "tests/integration bazel-in-bazel: Windows (subset)" + platform: windows + build_targets: ["//tests/integration:subset"] + test_targets: ["//tests/integration:subset"] integration_test_compile_pip_requirements_ubuntu: <<: *reusable_build_test_all name: "compile_pip_requirements: Ubuntu" working_directory: tests/integration/compile_pip_requirements - platform: ubuntu2004 + platform: ubuntu2204 shell_commands: # Make a change to the locked requirements and then assert that //:requirements.update does the # right thing. @@ -537,7 +532,7 @@ tasks: <<: *reusable_build_test_all name: "compile_pip_requirements: MacOS" working_directory: tests/integration/compile_pip_requirements - platform: macos + platform: macos_arm64 shell_commands: # Make a change to the locked requirements and then assert that //:requirements.update does the # right thing. @@ -570,26 +565,12 @@ tasks: - "bazel run //:os_specific_requirements.update" - "git diff --exit-code" - - integration_test_ignore_root_user_error_macos_workspace: - <<: *reusable_build_test_all - <<: *common_workspace_flags - name: "ignore_root_user_error: macOS, workspace" - working_directory: tests/integration/ignore_root_user_error - platform: macos - integration_test_ignore_root_user_error_windows_workspace: - <<: *reusable_build_test_all - <<: *common_workspace_flags - name: "ignore_root_user_error: Windows, workspace" - working_directory: tests/integration/ignore_root_user_error - platform: windows - integration_compile_pip_requirements_test_from_external_repo_ubuntu_min_workspace: <<: *minimum_supported_version <<: *common_workspace_flags_min_bazel name: "compile_pip_requirements_test_from_external_repo: Ubuntu, workspace, minimum Bazel" working_directory: tests/integration/compile_pip_requirements_test_from_external_repo - platform: ubuntu2004 + platform: ubuntu2204 shell_commands: # Assert that @compile_pip_requirements//:requirements_test does the right thing. - "bazel test @compile_pip_requirements//..." @@ -597,7 +578,7 @@ tasks: <<: *minimum_supported_version name: "compile_pip_requirements_test_from_external_repo: Ubuntu, bzlmod, minimum Bazel" working_directory: tests/integration/compile_pip_requirements_test_from_external_repo - platform: ubuntu2004 + platform: ubuntu2204 bazel: 7.x shell_commands: # Assert that @compile_pip_requirements//:requirements_test does the right thing. @@ -605,7 +586,7 @@ tasks: integration_compile_pip_requirements_test_from_external_repo_ubuntu: name: "compile_pip_requirements_test_from_external_repo: Ubuntu" working_directory: tests/integration/compile_pip_requirements_test_from_external_repo - platform: ubuntu2004 + platform: ubuntu2204 shell_commands: # Assert that @compile_pip_requirements//:requirements_test does the right thing. - "bazel test @compile_pip_requirements//..." @@ -619,7 +600,7 @@ tasks: integration_compile_pip_requirements_test_from_external_repo_macos: name: "compile_pip_requirements_test_from_external_repo: macOS" working_directory: tests/integration/compile_pip_requirements_test_from_external_repo - platform: macos + platform: macos_arm64 shell_commands: # Assert that @compile_pip_requirements//:requirements_test does the right thing. - "bazel test @compile_pip_requirements//..." diff --git a/.bazelignore b/.bazelignore index d5fe879e83..88a4c32056 100644 --- a/.bazelignore +++ b/.bazelignore @@ -18,12 +18,23 @@ examples/bzlmod/other_module/bazel-bin examples/bzlmod/other_module/bazel-other_module examples/bzlmod/other_module/bazel-out examples/bzlmod/other_module/bazel-testlogs -examples/bzlmod_build_file_generation/bazel-bzlmod_build_file_generation +examples/bzlmod/py_proto_library/foo_external examples/multi_python_versions/bazel-multi_python_versions examples/pip_parse/bazel-pip_parse examples/pip_parse_vendored/bazel-pip_parse_vendored +examples/pip_repository_annotations/bazel-pip_repository_annotations examples/py_proto_library/bazel-py_proto_library +gazelle/bazel-gazelle +gazelle/examples/bzlmod_build_file_generation/bazel-bin +gazelle/examples/bzlmod_build_file_generation/bazel-bzlmod_build_file_generation +gazelle/examples/bzlmod_build_file_generation/bazel-out +gazelle/examples/bzlmod_build_file_generation/bazel-testlog +sphinxdocs +tests/integration/bzlmod_lockfile/bazel-bzlmod_lockfile tests/integration/compile_pip_requirements/bazel-compile_pip_requirements -tests/integration/ignore_root_user_error/bazel-ignore_root_user_error tests/integration/local_toolchains/bazel-local_toolchains tests/integration/py_cc_toolchain_registered/bazel-py_cc_toolchain_registered +tests/integration/toolchain_target_settings/bazel-module_under_test +tests/integration/unified_pypi/bazel-unified_pypi +tests/integration/uv_lock/bazel-uv_lock +tests/integration/validate_test_main/bazel-module_under_test diff --git a/.bazelrc b/.bazelrc index c44124d961..90f3bc8fdb 100644 --- a/.bazelrc +++ b/.bazelrc @@ -2,10 +2,12 @@ # Trick bazel into treating BUILD files under examples/* as being regular files # This lets us glob() up all the files inside the examples to make them inputs to tests # (Note, we cannot use `common --deleted_packages` because the bazel version command doesn't support it) -# To update these lines, execute -# `bazel run @rules_bazel_integration_test//tools:update_deleted_packages` -build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered -query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered +# To update the file, execute +import %workspace%/.bazelrc.deleted_packages + +# Symlinks are used extensively because of runfiles, venvs, and other reasons. +# Supporting otherwise is very complex with little benefit. +startup --windows_enable_symlinks test --test_output=errors @@ -16,16 +18,40 @@ test --test_output=errors # creating (possibly empty) __init__.py files and adding them to the srcs of # Python targets as required. build --incompatible_default_to_explicit_init_py +build --//python/config_settings:incompatible_default_to_explicit_init_py=True +build --aspects=//tests/support/pyrefly:pyrefly.bzl%pyrefly_aspect # Ensure ongoing compatibility with this flag. common --incompatible_disallow_struct_provider_syntax +# Makes Bazel 7 act more like Bazel 8 +common --incompatible_use_plus_in_repo_names + +# Implicitly adds --config=linux|macos|windows, depending on the host platform +common --enable_platform_specific_config + +# Needed to make Windows with a py_binary in data deps work. The Bazel launcher +# is used, which falls back to finding python.exe on PATH to bootstrap. +# See https://github.com/bazel-contrib/rules_python/issues/3655 +common --incompatible_strict_action_env=false + +# To work around bug on bazel 7 +common:ci --experimental_repository_cache_hardlinks=false # Windows makes use of runfiles for some rules build --enable_runfiles -# Make Bazel 6 use bzlmod by default +# Make Bazel 7 use bzlmod by default common --enable_bzlmod +# Local disk cache greatly speeds up builds if the regular cache is lost +common --disk_cache=~/.cache/bazel/bazel-disk-cache +# Drop `experimental_` prefix once Bazel 7 is no longer supported +common --experimental_downloader_config=downloader_config.cfg +common --http_timeout_scaling=10.0 +common --experimental_repository_downloader_retries=10 + + + # Additional config to use for readthedocs builds. # See .readthedocs.yml for additional flags that can only be determined from # the runtime environment. @@ -33,6 +59,25 @@ build:rtd --stamp # Some bzl files contain repos only available under bzlmod build:rtd --enable_bzlmod -common:bazel7.x --incompatible_python_disallow_native_rules +common --incompatible_python_disallow_native_rules +common --incompatible_no_implicit_file_export build --lockfile_mode=update + +# Silence spammy C++ compile warnings +build:linux --copt=-Wno-deprecated-declarations +build:linux --copt=-Wno-stringop-overread +build:linux --copt=-Wno-sign-compare +build:linux --host_copt=-Wno-deprecated-declarations +build:linux --host_copt=-Wno-stringop-overread +build:linux --host_copt=-Wno-sign-compare +build:macos --copt=-Wno-deprecated-declarations +build:macos --copt=-Wno-stringop-overread +build:macos --copt=-Wno-sign-compare +build:macos --host_copt=-Wno-deprecated-declarations +build:macos --host_copt=-Wno-stringop-overread +build:macos --host_copt=-Wno-sign-compare + +import %workspace%/specialized_configs.bazelrc + +try-import user.bazelrc diff --git a/.bazelrc.deleted_packages b/.bazelrc.deleted_packages new file mode 100644 index 0000000000..da79f11058 --- /dev/null +++ b/.bazelrc.deleted_packages @@ -0,0 +1,56 @@ +# Generated via './tools/update_deleted_packages.sh' +common --deleted_packages=examples/build_file_generation +common --deleted_packages=examples/build_file_generation/random_number_generator +common --deleted_packages=examples/bzlmod +common --deleted_packages=examples/bzlmod/entry_points +common --deleted_packages=examples/bzlmod/entry_points/tests +common --deleted_packages=examples/bzlmod/libs/my_lib +common --deleted_packages=examples/bzlmod/other_module +common --deleted_packages=examples/bzlmod/other_module/other_module/pkg +common --deleted_packages=examples/bzlmod/other_module/other_module/rule_builder +common --deleted_packages=examples/bzlmod/patches +common --deleted_packages=examples/bzlmod/runfiles +common --deleted_packages=examples/bzlmod/tests +common --deleted_packages=examples/bzlmod/tests/other_module +common --deleted_packages=examples/bzlmod/whl_mods +common --deleted_packages=examples/multi_python_versions/libs/my_lib +common --deleted_packages=examples/multi_python_versions/requirements +common --deleted_packages=examples/multi_python_versions/tests +common --deleted_packages=examples/pip_parse +common --deleted_packages=examples/pip_parse_vendored +common --deleted_packages=gazelle +common --deleted_packages=gazelle/examples/bzlmod_build_file_generation +common --deleted_packages=gazelle/examples/bzlmod_build_file_generation/other_module/other_module/pkg +common --deleted_packages=gazelle/examples/bzlmod_build_file_generation/runfiles +common --deleted_packages=gazelle/manifest +common --deleted_packages=gazelle/manifest/generate +common --deleted_packages=gazelle/manifest/hasher +common --deleted_packages=gazelle/manifest/test +common --deleted_packages=gazelle/modules_mapping +common --deleted_packages=gazelle/python +common --deleted_packages=gazelle/pythonconfig +common --deleted_packages=gazelle/python/private +common --deleted_packages=tests/integration/bzlmod_lockfile +common --deleted_packages=tests/integration/compile_pip_requirements +common --deleted_packages=tests/integration/compile_pip_requirements_test_from_external_repo +common --deleted_packages=tests/integration/custom_commands +common --deleted_packages=tests/integration/local_toolchains +common --deleted_packages=tests/integration/pip_parse +common --deleted_packages=tests/integration/pip_parse/empty +common --deleted_packages=tests/integration/pip_parse_isolated +common --deleted_packages=tests/integration/py_cc_toolchain_registered +common --deleted_packages=tests/integration/runtime_manifests +common --deleted_packages=tests/integration/toolchain_target_settings +common --deleted_packages=tests/integration/unified_pypi +common --deleted_packages=tests/integration/uv_lock +common --deleted_packages=tests/integration/validate_test_main +common --deleted_packages=tests/integration/whl_library +common --deleted_packages=tests/modules/another_module +common --deleted_packages=tests/modules/other +common --deleted_packages=tests/modules/other/nspkg_delta +common --deleted_packages=tests/modules/other/nspkg_gamma +common --deleted_packages=tests/modules/other/nspkg_single +common --deleted_packages=tests/modules/other/simple_v1 +common --deleted_packages=tests/modules/other/simple_v2 +common --deleted_packages=tests/modules/other/with_external_data +common --deleted_packages=tests/modules/rules_pyrefly_stub/pyrefly diff --git a/.bazelversion b/.bazelversion index 35907cd9ca..44931da266 100644 --- a/.bazelversion +++ b/.bazelversion @@ -1 +1 @@ -7.x +9.1.1 diff --git a/.bcr/config.yml b/.bcr/config.yml index 7672aa554d..038761b6df 100644 --- a/.bcr/config.yml +++ b/.bcr/config.yml @@ -12,7 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -fixedReleaser: - login: f0rmiga - email: 3149049+f0rmiga@users.noreply.github.com -moduleRoots: [".", "gazelle"] +moduleRoots: [".", "gazelle", "sphinxdocs"] diff --git a/.bcr/gazelle/metadata.template.json b/.bcr/gazelle/metadata.template.json index 687f78e977..017f9d3774 100644 --- a/.bcr/gazelle/metadata.template.json +++ b/.bcr/gazelle/metadata.template.json @@ -1,9 +1,9 @@ { - "homepage": "https://github.com/bazelbuild/rules_python", + "homepage": "https://github.com/bazel-contrib/rules_python", "maintainers": [ { "name": "Richard Levasseur", - "email": "rlevasseur@google.com", + "email": "richardlev@gmail.com", "github": "rickeylev" }, { @@ -13,7 +13,8 @@ } ], "repository": [ - "github:bazelbuild/rules_python" + "github:bazelbuild/rules_python", + "github:bazel-contrib/rules_python" ], "versions": [], "yanked_versions": {} diff --git a/.bcr/gazelle/presubmit.yml b/.bcr/gazelle/presubmit.yml index bceed4f9e1..99647cac6f 100644 --- a/.bcr/gazelle/presubmit.yml +++ b/.bcr/gazelle/presubmit.yml @@ -13,16 +13,27 @@ # limitations under the License. bcr_test_module: - module_path: "../examples/bzlmod_build_file_generation" + module_path: "examples/bzlmod_build_file_generation" matrix: - platform: ["debian11", "macos", "ubuntu2004", "windows"] - # last_rc is to get latest 8.x release. Replace with 8.x when available. - bazel: [7.x, last_rc] + platform: [ + "debian11", + "macos", + "ubuntu2204", + ] + bazel: [7.*, 8.*, 9.*] tasks: run_tests: name: "Run test module" platform: ${{ platform }} bazel: ${{ bazel }} + shell_commands: + - "echo 'common --override_module=rules_python=' >> .bazelrc" + - "bazel run //:gazelle_python_manifest.update" + - "bazel run //:gazelle -- update" + batch_commands: + - "echo common --override_module=rules_python= >> .bazelrc" + - "bazel run //:gazelle_python_manifest.update" + - "bazel run //:gazelle -- update" build_targets: - "//..." - ":modules_map" diff --git a/.bcr/metadata.template.json b/.bcr/metadata.template.json index b164e70443..9d85e22200 100644 --- a/.bcr/metadata.template.json +++ b/.bcr/metadata.template.json @@ -1,9 +1,9 @@ { - "homepage": "https://github.com/bazelbuild/rules_python", + "homepage": "https://github.com/bazel-contrib/rules_python", "maintainers": [ { "name": "Richard Levasseur", - "email": "rlevasseur@google.com", + "email": "richardlev@gmail.com", "github": "rickeylev" }, { @@ -13,7 +13,8 @@ } ], "repository": [ - "github:bazelbuild/rules_python" + "github:bazelbuild/rules_python", + "github:bazel-contrib/rules_python" ], "versions": [], "yanked_versions": {} diff --git a/.bcr/presubmit.yml b/.bcr/presubmit.yml index e1ddb7a1aa..a38c6bace5 100644 --- a/.bcr/presubmit.yml +++ b/.bcr/presubmit.yml @@ -15,18 +15,19 @@ bcr_test_module: module_path: "examples/bzlmod" matrix: - platform: ["debian11", "macos", "ubuntu2004", "windows"] - # last_rc is to get latest 8.x release. Replace with 8.x when available. - bazel: [7.x, last_rc] + platform: ["debian11", "macos", "ubuntu2204", "windows"] + bazel: [7.x, 8.x, 9.x] tasks: run_tests: name: "Run test module" platform: ${{ platform }} bazel: ${{ bazel }} test_flags: + # Minimum bazel supported C++ - "--keep_going" - # Without these cxxopts, BCR's Mac builds fail - - '--cxxopt=-std=c++14' - - '--host_cxxopt=-std=c++14' + - '--cxxopt=-std=c++17' + - '--host_cxxopt=-std=c++17' + build_targets: + - "//..." test_targets: - "//..." diff --git a/.bcr/sphinxdocs/metadata.template.json b/.bcr/sphinxdocs/metadata.template.json new file mode 100644 index 0000000000..017f9d3774 --- /dev/null +++ b/.bcr/sphinxdocs/metadata.template.json @@ -0,0 +1,21 @@ +{ + "homepage": "https://github.com/bazel-contrib/rules_python", + "maintainers": [ + { + "name": "Richard Levasseur", + "email": "richardlev@gmail.com", + "github": "rickeylev" + }, + { + "name": "Ignas Anikevicius", + "email": "bcr-ignas@use.startmail.com", + "github": "aignas" + } + ], + "repository": [ + "github:bazelbuild/rules_python", + "github:bazel-contrib/rules_python" + ], + "versions": [], + "yanked_versions": {} +} diff --git a/.bcr/sphinxdocs/presubmit.yml b/.bcr/sphinxdocs/presubmit.yml new file mode 100644 index 0000000000..00a6bd37aa --- /dev/null +++ b/.bcr/sphinxdocs/presubmit.yml @@ -0,0 +1,23 @@ +bcr_test_module: + module_path: "integration_tests/bcr" + matrix: + platform: ["debian11", "macos", "ubuntu2204"] + bazel: [8.*, 9.*] + tasks: + run_tests: + name: "Run test module" + platform: ${{ platform }} + bazel: ${{ bazel }} + shell_commands: + - "echo 'common --override_module=rules_python=' >> .bazelrc" + batch_commands: + - "echo common --override_module=rules_python= >> .bazelrc" + test_flags: + # Minimum bazel supported C++ + - "--keep_going" + - '--cxxopt=-std=c++17' + - '--host_cxxopt=-std=c++17' + build_targets: + - "//..." + test_targets: + - "//..." diff --git a/.bcr/sphinxdocs/source.template.json b/.bcr/sphinxdocs/source.template.json new file mode 100644 index 0000000000..b2a2d1ef23 --- /dev/null +++ b/.bcr/sphinxdocs/source.template.json @@ -0,0 +1,5 @@ +{ + "integrity": "", + "strip_prefix": "{REPO}-{VERSION}/sphinxdocs", + "url": "https://github.com/{OWNER}/{REPO}/releases/download/{TAG}/rules_python-{TAG}.tar.gz" +} diff --git a/.editorconfig b/.editorconfig new file mode 100644 index 0000000000..2737b0f184 --- /dev/null +++ b/.editorconfig @@ -0,0 +1,21 @@ +# Unix-style newlines with a newline ending every file +[*] +end_of_line = lf +insert_final_newline = true + +# Set default charset +[*] +charset = utf-8 + +# Line width +[*] +max_line_length = 100 + +# 4 space indentation +[*.{py,bzl}] +indent_style = space +indent_size = 4 + +# different overrides for git commit messages +[.git/COMMIT_EDITMSG] +max_line_length = 72 diff --git a/.gitattributes b/.gitattributes index eae260e931..4f93d89d33 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1,2 +1,7 @@ python/features.bzl export-subst tools/publish/*.txt linguist-generated=true +tests/uv/lock/testdata/requirements.txt text eol=lf +python/private/runtimes_manifest_workspace.bzl text eol=lf +python/private/runtimes_manifest.txt text eol=lf + +*.bat text eol=crlf diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 4df29bacdf..a0b4be250a 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -7,5 +7,5 @@ /examples/build_file_generation/ @dougthor42 @aignas # PyPI integration related code -/python/private/pypi/ @rickeylev @aignas @groodt -/tests/pypi/ @rickeylev @aignas @groodt +/python/private/pypi/ @rickeylev @aignas +/tests/pypi/ @rickeylev @aignas diff --git a/.github/ISSUE_TEMPLATE/patch_release_request.md b/.github/ISSUE_TEMPLATE/patch_release_request.md new file mode 100644 index 0000000000..7b38ec40df --- /dev/null +++ b/.github/ISSUE_TEMPLATE/patch_release_request.md @@ -0,0 +1,22 @@ +--- +name: "🏗️ Patch release or backport" +about: Request a patch release or backport of a fix to a release. +title: 'Patch release: MAJOR.MINOR.PATCH' +labels: 'type: process' +--- + + + +**What version of `rules_python` do you want to patch?** + + +**What pull requests do you want to backport?** + +Please provide a list of pull request numbers. + +- # +- # diff --git a/.github/ISSUE_TEMPLATE/release_tracking_template.md b/.github/ISSUE_TEMPLATE/release_tracking_template.md new file mode 100644 index 0000000000..ab634e75f3 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/release_tracking_template.md @@ -0,0 +1,34 @@ +--- +name: Release Tracking Issue +about: Checklist for tracking a new release of rules_python. +title: 'Release ' +labels: ['type: release'] +--- +# Release tasks +- [ ] Prepare Release | status=awaiting-preparation +- [ ] Create Release branch +- [ ] Tag RC0 +- [ ] Tag Final + +## Backports + +To request a backport, add it to the checklist below and process it. See [RELEASING.md: How to add backports](https://github.com/bazel-contrib/rules_python/blob/main/RELEASING.md#how-to-add-backports) for details. + +--- + +To manually control the release flow, see the [RELEASING.md: Manual Editing](https://github.com/bazel-contrib/rules_python/blob/main/RELEASING.md#manual-editing-of-tracking-issue) section. + +
+Available Commands + +Comment commands: +- `/prepare`: Determines version, creates tracking issue and preparation PR. +- `/prepare-complete [PR]`: Marks preparation task as complete. +- `/create-release-branch`: Cuts and pushes the release branch. +- `/create-rc`: Tags and publishes a new release candidate (RC). +- `/process-backports`: Cherry-picks pending backports. +- `/add-backports `: Adds PRs to the backports and processes backports. +- `/promote`: Promotes the latest RC to final release. + +See [RELEASING.md](https://github.com/bazel-contrib/rules_python/blob/main/RELEASING.md) for details on how to use them. +
diff --git a/.github/workflows/automated_pr_review.yaml b/.github/workflows/automated_pr_review.yaml new file mode 100644 index 0000000000..01b63e42a4 --- /dev/null +++ b/.github/workflows/automated_pr_review.yaml @@ -0,0 +1,104 @@ +name: Automated Code Review + +# TODO: Eventually, use pull_request_target instead of pull_request. +# pull_request_target runs in the base branch context and has access +# to secrets (like GEMINI_API_KEY) even for fork PRs. +# Using pull_request for now during setup/testing. +on: + issue_comment: + types: [created] + +permissions: + contents: read + pull-requests: read + +jobs: + # Always runs so the workflow run exits cleanly without a "No jobs ran" error + # when the review job's if-condition evaluates to false. + noop: + runs-on: ubuntu-latest + steps: + - name: Workflow trigger check + run: echo "Workflow triggered successfully." + + # Job 1: Runs without secrets to extract the git diff from untrusted PR code. + # Keeps GEMINI_API_KEY away from any environment that checks out untrusted PR files. + prepare_diff: + runs-on: ubuntu-latest + if: > + github.event_name == 'issue_comment' && github.event.issue.pull_request != null && + (startsWith(github.event.comment.body, '/review') || + contains(github.event.comment.body, '\n/review') || + contains(github.event.comment.body, '\r\n/review')) && + contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) + steps: + - name: Checkout PR Branch (Data Only) + uses: actions/checkout@v7 + with: + ref: refs/pull/${{ github.event.pull_request.number || github.event.issue.number }}/head + path: untrusted_pr_head + persist-credentials: false + + - name: Fetch Base Branch and Negotiate Minimal Diff History + env: + PR_COMMITS: ${{ github.event.pull_request.commits }} + run: | + cd untrusted_pr_head + # 1. Fetch the tip of main + git fetch origin main:refs/remotes/origin/main --depth=1 + + # 2. Check if we already have the merge base (common ancestor) + if ! git merge-base origin/main HEAD >/dev/null 2>&1; then + # If we know the exact number of commits in the PR, deepen by (PR_COMMITS + 10) + if [ -n "$PR_COMMITS" ] && [ "$PR_COMMITS" != "null" ]; then + git fetch --deepen="$((PR_COMMITS + 10))" + fi + # 3. If it's an issue_comment event (where PR_COMMITS is null) or still shallow, unshallow/deepen + if ! git merge-base origin/main HEAD >/dev/null 2>&1; then + git fetch --unshallow || git fetch --deepen=50 + fi + fi + git diff origin/main...HEAD > ../pr_diff.txt + + # Upload extracted diff as artifact to pass to Job 2 safely as text data. + - name: Upload Diff Artifact + uses: actions/upload-artifact@v7 + with: + name: pr_diff + path: pr_diff.txt + + # Job 2: Runs with secrets in base branch context. + review: + needs: prepare_diff + runs-on: ubuntu-latest + steps: + - name: Download Diff Artifact + uses: actions/download-artifact@v8 + with: + name: pr_diff + + # Check out reviewbot into a separate directory to isolate base branch tool code. + - name: Checkout Reviewbot (Base Branch Only) + uses: actions/checkout@v7 + with: + sparse-checkout: | + tools/private/reviewbot + path: reviewbot + + - name: Install uv + uses: astral-sh/setup-uv@v9.0.0 + + - name: Run Antigravity Review + # Run inside reviewbot directory so execution context is the trusted base branch. + # This also helps prevent uv from looking for config files in locations it shouldn't, + # i.e. by default, uv will look in $PWD for a pyproject file to build. + working-directory: reviewbot + env: + GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }} + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + # Use --no-project to prevent uv from discovering or building pyproject.toml/setup.py + # in the workspace, ensuring only standalone script dependencies are resolved. + uv run --no-project --directory . tools/private/reviewbot/antigravity_review.py \ + --prompt tools/private/reviewbot/prompt.txt \ + --diff-file ../pr_diff.txt diff --git a/.github/workflows/backport_create_releases.yaml b/.github/workflows/backport_create_releases.yaml new file mode 100644 index 0000000000..346bfdbbdb --- /dev/null +++ b/.github/workflows/backport_create_releases.yaml @@ -0,0 +1,47 @@ +name: "Backport: Create Releases" + +on: + workflow_dispatch: + inputs: + issue: + description: 'The Backport Tracking Issue Number' + required: true + type: string + workflow_call: + inputs: + issue: + description: 'The Backport Tracking Issue Number' + required: true + type: string + +permissions: + contents: write + issues: write + pull-requests: write + +jobs: + backport_create_releases: + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Setup Bazel + uses: bazel-contrib/setup-bazel@0.19.0 + with: + bazelisk-version: 1.20.0 + + - name: Configure Git Identity + run: | + git config --global user.name "github-actions[bot]" + git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Run Backport Create Releases Pipeline + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + ISSUE: ${{ inputs.issue }} + run: | + bazel run //tools/private/release -- \ + backport-create-releases --issue=$ISSUE --no-dry-run diff --git a/.github/workflows/backport_prepare.yaml b/.github/workflows/backport_prepare.yaml new file mode 100644 index 0000000000..587c0fc357 --- /dev/null +++ b/.github/workflows/backport_prepare.yaml @@ -0,0 +1,76 @@ +name: "Backport: Prepare" + +on: + workflow_dispatch: + inputs: + issue: + description: 'The Backport Tracking Issue Number' + required: false + type: string + pr: + description: 'PR to backport (e.g. #123) (if no issue)' + required: false + type: string + from_minor: + description: 'From minor version (e.g. 1.7) (if no issue)' + required: false + type: string + to_minor: + description: 'To minor version (e.g. 1.9) (if no issue, optional)' + required: false + type: string + workflow_call: + inputs: + issue: + description: 'The Backport Tracking Issue Number' + required: false + type: string + +permissions: + contents: write + issues: write + pull-requests: write + +jobs: + backport_prepare: + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Setup Bazel + uses: bazel-contrib/setup-bazel@0.19.0 + with: + bazelisk-version: 1.20.0 + + - name: Configure Git Identity + run: | + git config --global user.name "github-actions[bot]" + git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Run Backport Preparation Pipeline + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + ISSUE: ${{ inputs.issue }} + PR: ${{ inputs.pr }} + FROM_MINOR: ${{ inputs.from_minor }} + TO_MINOR: ${{ inputs.to_minor }} + run: | + ARGS=() + if [ -n "$ISSUE" ]; then + ARGS+=("--issue=$ISSUE") + else + if [ -n "$PR" ]; then + ARGS+=("--pr=$PR") + fi + if [ -n "$FROM_MINOR" ]; then + ARGS+=("--from-minor=$FROM_MINOR") + fi + if [ -n "$TO_MINOR" ]; then + ARGS+=("--to-minor=$TO_MINOR") + fi + fi + bazel run //tools/private/release -- \ + backport-prepare "${ARGS[@]}" --no-dry-run diff --git a/.github/workflows/check_version_markers.sh b/.github/workflows/check_version_markers.sh new file mode 100755 index 0000000000..15a0a67dc8 --- /dev/null +++ b/.github/workflows/check_version_markers.sh @@ -0,0 +1,31 @@ +#!/usr/bin/env bash + +set -o nounset +set -o pipefail +set -o errexit + +set -x + +TAG=${1:-} +if [ -n "$TAG" ]; then + # If the workflow checks out one commit, but is releasing another + git fetch origin tag "$TAG" + # Update our local state so the grep command below searches what we expect + git checkout "$TAG" +fi + +grep_exit_code=0 +# Exclude dot directories, specifically, this file so that we don't +# find the substring we're looking for in our own file. +# Exclude CONTRIBUTING.md, RELEASING.md because they document how to use these strings. +grep --exclude=CONTRIBUTING.md \ + --exclude=RELEASING.md \ + --exclude-dir=.* \ + --exclude-dir=release \ + VERSION_NEXT_ -r || grep_exit_code=$? + +if [[ $grep_exit_code -eq 0 ]]; then + echo + echo "Found VERSION_NEXT markers indicating version needs to be specified" + exit 1 +fi diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml new file mode 100644 index 0000000000..e7b2a4d956 --- /dev/null +++ b/.github/workflows/ci.yaml @@ -0,0 +1,43 @@ +name: CI + +on: + push: + branches: + - main + pull_request: + types: + - opened + - synchronize + +defaults: + run: + shell: bash + +permissions: + contents: read + +jobs: + mypy: + runs-on: ubuntu-latest + steps: + # Checkout the code + - uses: actions/checkout@v7 + - uses: jpetrucciani/mypy-check@master + with: + path: 'python/runfiles' + - uses: jpetrucciani/mypy-check@master + with: + path: 'tests/runfiles' + ruff: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v7 + - uses: astral-sh/ruff-action@v4.1.0 + with: + # Keep in sync with .pre-commit-config.yaml + version: 0.15.14 + args: check --extend-exclude testdata + - uses: astral-sh/ruff-action@v4.1.0 + with: + version: 0.15.14 + args: format --check --exclude testdata diff --git a/.github/workflows/create_archive_and_notes.sh b/.github/workflows/create_archive_and_notes.sh index 0bc14f936b..506e2f9784 100755 --- a/.github/workflows/create_archive_and_notes.sh +++ b/.github/workflows/create_archive_and_notes.sh @@ -13,21 +13,36 @@ # See the License for the specific language governing permissions and # limitations under the License. -set -o errexit -o nounset -o pipefail +set -o nounset +set -o pipefail +set -o errexit + +set -x + +TAG=$1 +if [ -z "$TAG" ]; then + echo "ERROR: TAG env var must be set" + exit 1 +fi +# If the workflow checks out one commit, but is releasing another +git fetch origin tag "$TAG" + +# Update our local state so that check_version_markers searches what we expect +git checkout "$TAG" +$(dirname $0)/check_version_markers.sh -# Set by GH actions, see -# https://docs.github.com/en/actions/learn-github-actions/environment-variables#default-environment-variables -TAG=${GITHUB_REF_NAME} # A prefix is added to better match the GitHub generated archives. PREFIX="rules_python-${TAG}" ARCHIVE="rules_python-$TAG.tar.gz" -git archive --format=tar --prefix=${PREFIX}/ ${TAG} | gzip > $ARCHIVE -SHA=$(shasum -a 256 $ARCHIVE | awk '{print $1}') +git archive --format=tar "--prefix=${PREFIX}/" "$TAG" | gzip > "$ARCHIVE" +SHA=$(shasum -a 256 "$ARCHIVE" | awk '{print $1}') cat > release_notes.txt << EOF For more detailed setup instructions, see https://rules-python.readthedocs.io/en/latest/getting-started.html +For the user-facing changelog see [here](https://rules-python.readthedocs.io/en/latest/changelog.html#v${TAG//./-}) + ## Using Bzlmod Add to your \`MODULE.bazel\` file: @@ -35,15 +50,19 @@ Add to your \`MODULE.bazel\` file: \`\`\`starlark bazel_dep(name = "rules_python", version = "${TAG}") -pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.toolchain( + python_version = "3.13", +) +pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") pip.parse( - hub_name = "pip", - python_version = "3.11", + hub_name = "pypi", + python_version = "3.13", requirements_lock = "//:requirements_lock.txt", ) -use_repo(pip, "pip") +use_repo(pip, "pypi") \`\`\` ## Using WORKSPACE @@ -57,7 +76,7 @@ http_archive( name = "rules_python", sha256 = "${SHA}", strip_prefix = "${PREFIX}", - url = "https://github.com/bazelbuild/rules_python/releases/download/${TAG}/rules_python-${TAG}.tar.gz", + url = "https://github.com/bazel-contrib/rules_python/releases/download/${TAG}/rules_python-${TAG}.tar.gz", ) load("@rules_python//python:repositories.bzl", "py_repositories") @@ -75,7 +94,7 @@ http_archive( name = "rules_python_gazelle_plugin", sha256 = "${SHA}", strip_prefix = "${PREFIX}/gazelle", - url = "https://github.com/bazelbuild/rules_python/releases/download/${TAG}/rules_python-${TAG}.tar.gz", + url = "https://github.com/bazel-contrib/rules_python/releases/download/${TAG}/rules_python-${TAG}.tar.gz", ) # To compile the rules_python gazelle extension from source, diff --git a/.github/workflows/mypy.yaml b/.github/workflows/mypy.yaml deleted file mode 100644 index 429775172e..0000000000 --- a/.github/workflows/mypy.yaml +++ /dev/null @@ -1,32 +0,0 @@ -name: mypy - -on: - push: - branches: - - main - pull_request: - types: - - opened - - synchronize - -defaults: - run: - shell: bash - -jobs: - ci: - runs-on: ubuntu-20.04 - steps: - # Checkout the code - - uses: actions/checkout@v4 - - uses: jpetrucciani/mypy-check@master - with: - requirements: 1.6.0 - python_version: 3.8 - path: 'python/runfiles' - - uses: jpetrucciani/mypy-check@master - with: - requirements: 1.6.0 - python_version: 3.8 - path: 'tests/runfiles' - diff --git a/.github/workflows/on_comment.yaml b/.github/workflows/on_comment.yaml new file mode 100644 index 0000000000..a4330a0d6f --- /dev/null +++ b/.github/workflows/on_comment.yaml @@ -0,0 +1,208 @@ +name: "On Comment" + +on: + issue_comment: + types: [created] + +permissions: + contents: write + issues: write + pull-requests: write + +jobs: + # This job always runs to prevent GHA from marking the run as failed when + # all other jobs are skipped. + noop: + runs-on: ubuntu-latest + steps: + - run: echo "No-op" + + parse_comment: + runs-on: ubuntu-latest + if: | + github.event.comment.author_association == 'OWNER' || + github.event.comment.author_association == 'MEMBER' || + github.event.comment.author_association == 'COLLABORATOR' + outputs: + command: ${{ steps.parse.outputs.command }} + issue_number: ${{ steps.parse.outputs.issue_number }} + pr_number: ${{ steps.parse.outputs.pr_number }} + backports: ${{ steps.parse.outputs.backports }} + steps: + - name: Parse comment + id: parse + env: + COMMENT_BODY: ${{ github.event.comment.body }} + IS_PR: "${{ github.event.issue.pull_request != null }}" + EVENT_NUMBER: "${{ github.event.issue.number }}" + HAS_RELEASE_LABEL: "${{ contains(github.event.issue.labels.*.name, 'type: release') }}" + HAS_BACKPORT_LABEL: "${{ contains(github.event.issue.labels.*.name, 'type: backport-pr') }}" + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + if [ "$IS_PR" = "false" ]; then + # Set issue number for non-PR issues (release or backport tracking issues) + issue_number=$EVENT_NUMBER + echo "issue_number=$issue_number" >> "$GITHUB_OUTPUT" + echo "issue_number=$issue_number" >> "$GITHUB_ENV" + + # Check if it's a release tracking issue + if [ "$HAS_RELEASE_LABEL" = "true" ]; then + # Handle /create-rc comment + if echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/create-rc([[:space:]]|$)'; then + echo "command=create-rc" >> "$GITHUB_OUTPUT" + # Handle /prepare-complete comment + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/prepare-complete([[:space:]]|$)'; then + echo "command=prepare-complete" >> "$GITHUB_OUTPUT" + pr_arg=$(echo "$COMMENT_BODY" | grep -E '^[[:space:]]*/prepare-complete([[:space:]]|$)' | sed -E 's/^[[:space:]]*\/prepare-complete[[:space:]]*//' | tr -d '[:space:]#') + if [ -n "$pr_arg" ]; then + echo "pr_number=$pr_arg" >> "$GITHUB_OUTPUT" + fi + # Handle /create-release-branch comment + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/create-release-branch([[:space:]]|$)'; then + echo "command=create-release-branch" >> "$GITHUB_OUTPUT" + # Handle /prepare comment + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/prepare([[:space:]]|$)'; then + echo "command=prepare" >> "$GITHUB_OUTPUT" + # Handle /process-backports comment + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/process-backports([[:space:]]|$)'; then + echo "command=process-backports" >> "$GITHUB_OUTPUT" + # Handle /add-backports comment + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/add-backports([[:space:]]|$)'; then + args=$(echo "$COMMENT_BODY" | grep -E '^[[:space:]]*/add-backports([[:space:]]|$)' | sed -E 's/^[[:space:]]*\/add-backports[[:space:]]*//') + args=$(echo "$args" | sed -e 's/^[[:space:],]*//' -e 's/[[:space:],]*$//') + csv=$(echo "$args" | sed -E 's/[[:space:],]+/ /g' | tr ' ' ',') + if [ -n "$csv" ]; then + echo "command=add-backports" >> "$GITHUB_OUTPUT" + echo "backports=$csv" >> "$GITHUB_OUTPUT" + else + echo "command=none" >> "$GITHUB_OUTPUT" + echo "Error: No PRs specified for add-backports." >&2 + gh api \ + --method POST \ + -H "Accept: application/vnd.github+json" \ + -H "X-GitHub-Api-Version: 2022-11-28" \ + /repos/${{ github.repository }}/issues/comments/${{ github.event.comment.id }}/reactions \ + -f "content=-1" + fi + # Handle /promote comment + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/promote([[:space:]]|$)'; then + echo "command=promote" >> "$GITHUB_OUTPUT" + else + echo "command=none" >> "$GITHUB_OUTPUT" + fi + # Check if it's a backport tracking issue + elif [ "$HAS_BACKPORT_LABEL" = "true" ]; then + # Handle /prepare comment for backports + if echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/prepare([[:space:]]|$)'; then + echo "command=backport-prepare" >> "$GITHUB_OUTPUT" + # Handle /create-releases comment for backports + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/create-releases([[:space:]]|$)'; then + echo "command=backport-create-releases" >> "$GITHUB_OUTPUT" + else + echo "command=none" >> "$GITHUB_OUTPUT" + fi + else + echo "command=none" >> "$GITHUB_OUTPUT" + fi + elif [ "$IS_PR" = "true" ]; then + pr_number=$EVENT_NUMBER + # Handle /backport comment on PR + if echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/backport([[:space:]]|$)'; then + echo "command=pr-backport" >> "$GITHUB_OUTPUT" + echo "pr_number=$pr_number" >> "$GITHUB_OUTPUT" + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/prepare-complete([[:space:]]|$)'; then + echo "command=prepare-complete" >> "$GITHUB_OUTPUT" + echo "pr_number=$pr_number" >> "$GITHUB_OUTPUT" + else + echo "command=none" >> "$GITHUB_OUTPUT" + fi + else + echo "command=none" >> "$GITHUB_OUTPUT" + fi + + call_create_rc: + needs: parse_comment + if: needs.parse_comment.outputs.command == 'create-rc' + uses: ./.github/workflows/release_create_rc.yaml + with: + issue: ${{ needs.parse_comment.outputs.issue_number }} + comment_id: "${{ github.event.comment.id }}" + secrets: inherit + + call_prepare_complete: + needs: parse_comment + if: needs.parse_comment.outputs.command == 'prepare-complete' + uses: ./.github/workflows/release_complete_prepare.yaml + with: + pr: ${{ needs.parse_comment.outputs.pr_number }} + issue: ${{ needs.parse_comment.outputs.issue_number }} + secrets: inherit + + call_create_release_branch: + needs: parse_comment + if: needs.parse_comment.outputs.command == 'create-release-branch' + uses: ./.github/workflows/release_create_release_branch.yaml + with: + issue: ${{ needs.parse_comment.outputs.issue_number }} + secrets: inherit + + call_prepare: + needs: parse_comment + if: needs.parse_comment.outputs.command == 'prepare' + uses: ./.github/workflows/release_prepare.yaml + with: + issue: ${{ needs.parse_comment.outputs.issue_number }} + secrets: inherit + + call_add_backports: + needs: parse_comment + if: | + needs.parse_comment.outputs.command == 'add-backports' || + needs.parse_comment.outputs.command == 'pr-backport' + uses: ./.github/workflows/release_add_backports.yaml + with: + prs: ${{ needs.parse_comment.outputs.command == 'pr-backport' && needs.parse_comment.outputs.pr_number || needs.parse_comment.outputs.backports }} + issue: ${{ needs.parse_comment.outputs.issue_number }} + secrets: inherit + + call_process_backports_after_add: + needs: [parse_comment, call_add_backports] + if: needs.parse_comment.outputs.command == 'add-backports' + uses: ./.github/workflows/release_process_backports.yaml + with: + issue: ${{ needs.parse_comment.outputs.issue_number }} + comment_id: "${{ github.event.comment.id }}" + secrets: inherit + + call_process_backports_only: + needs: parse_comment + if: needs.parse_comment.outputs.command == 'process-backports' + uses: ./.github/workflows/release_process_backports.yaml + with: + issue: ${{ needs.parse_comment.outputs.issue_number }} + comment_id: "${{ github.event.comment.id }}" + secrets: inherit + + call_promote: + needs: parse_comment + if: needs.parse_comment.outputs.command == 'promote' + uses: ./.github/workflows/release_promote.yaml + with: + issue: ${{ needs.parse_comment.outputs.issue_number }} + secrets: inherit + + call_backport_prepare: + needs: parse_comment + if: needs.parse_comment.outputs.command == 'backport-prepare' + uses: ./.github/workflows/backport_prepare.yaml + with: + issue: ${{ needs.parse_comment.outputs.issue_number }} + secrets: inherit + + call_backport_create_releases: + needs: parse_comment + if: needs.parse_comment.outputs.command == 'backport-create-releases' + uses: ./.github/workflows/backport_create_releases.yaml + with: + issue: ${{ needs.parse_comment.outputs.issue_number }} + secrets: inherit diff --git a/.github/workflows/on_pr_closed.yaml b/.github/workflows/on_pr_closed.yaml new file mode 100644 index 0000000000..94c49b6561 --- /dev/null +++ b/.github/workflows/on_pr_closed.yaml @@ -0,0 +1,113 @@ +name: "On PR Closed" + +on: + pull_request: + types: [closed] + +permissions: + contents: read + issues: read + pull-requests: read + +jobs: + # This job always runs to prevent GHA from marking the run as failed when + # all other jobs are skipped. + noop: + runs-on: ubuntu-latest + steps: + - run: echo "No-op" + + check_if_backport: + runs-on: ubuntu-latest + if: github.event.pull_request.merged == true + outputs: + should_process: ${{ steps.check.outputs.should_process }} + steps: + - name: Check if PR is a backport candidate + id: check + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + PR_NUMBER: ${{ github.event.pull_request.number }} + run: | + # Check if there is any active release issue + ACTIVE_ISSUES=$(gh issue list --repo ${{ github.repository }} --label "type: release" --state open --json number) + if [ "$ACTIVE_ISSUES" = "[]" ] || [ -z "$ACTIVE_ISSUES" ]; then + echo "No active release tracking issue found. Skipping." + echo "should_process=false" >> "$GITHUB_OUTPUT" + exit 0 + fi + + # Check if PR has "/backport" in comments (only comments, not body) + PR_DATA=$(gh pr view "$PR_NUMBER" --repo ${{ github.repository }} --json comments) + + if echo "$PR_DATA" | jq -r '.comments[].body' | grep -qE '^[[:space:]]*/backport([[:space:]]|$)'; then + echo "Found /backport comment. Proceeding." + echo "should_process=true" >> "$GITHUB_OUTPUT" + else + echo "No /backport comment found. Skipping." + echo "should_process=false" >> "$GITHUB_OUTPUT" + fi + + process_backports: + needs: check_if_backport + if: needs.check_if_backport.outputs.should_process == 'true' + runs-on: ubuntu-latest + permissions: + contents: write + issues: write + pull-requests: read + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Setup Bazel + uses: bazel-contrib/setup-bazel@0.19.0 + with: + bazelisk-version: 1.20.0 + + - name: Configure Git Identity + run: | + git config --global user.name "github-actions[bot]" + git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Process Backports + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + PR_NUMBER: ${{ github.event.pull_request.number }} + run: | + bazel run //tools/private/release -- on-pr-merged \ + "$PR_NUMBER" \ + --remote origin \ + --no-dry-run + + complete_sync_changelog: + if: | + github.event.pull_request.merged == true && + contains(github.event.pull_request.labels.*.name, 'type: sync-changelog') + runs-on: ubuntu-latest + permissions: + contents: write + issues: write + pull-requests: read + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Setup Bazel + uses: bazel-contrib/setup-bazel@0.19.0 + with: + bazelisk-version: 1.20.0 + + - name: Complete Sync Changelog + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + PR_NUMBER: ${{ github.event.pull_request.number }} + run: | + bazel run //tools/private/release -- complete-sync-changelog \ + --pr "$PR_NUMBER" + + diff --git a/.github/workflows/pr-metadata-checks.yaml b/.github/workflows/pr-metadata-checks.yaml new file mode 100644 index 0000000000..8060ff921e --- /dev/null +++ b/.github/workflows/pr-metadata-checks.yaml @@ -0,0 +1,60 @@ +name: "PR Metadata Checks" + +on: + pull_request_target: + types: + - opened + - synchronize + - reopened + - labeled + - unlabeled + - edited + +jobs: + blocks-do-not-merge: + runs-on: ubuntu-latest + steps: + - name: Check for "do not merge" label + if: "contains(github.event.pull_request.labels.*.name, 'do not merge')" + run: | + echo "::error::This PR has the 'do not merge' label" \ + "and cannot be merged." + exit 1 + + - name: Check PR description for DO NOT SUBMIT/MERGE + env: + PR_BODY: ${{ github.event.pull_request.body }} + run: | + echo "Checking PR description..." + if echo "$PR_BODY" | grep -Ei "DO NOT SUBMIT|DO NOT MERGE"; then + echo "::error::This PR description contains" \ + "'DO NOT SUBMIT' or 'DO NOT MERGE'" \ + "and cannot be merged." + exit 1 + fi + echo "PR description is clean." + + block-transient-agent-files: + runs-on: ubuntu-latest + permissions: + pull-requests: read + steps: + - name: Check for blocked files + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + PR_NUMBER: ${{ github.event.pull_request.number }} + run: | + echo "Checking for blocked files..." + # We use --repo to avoid needing actions/checkout. + # This keeps the job lightweight and avoids "not a git repository" errors. + changed_files=$(gh pr diff "$PR_NUMBER" --name-only --repo "${{ github.repository }}") + blocked_files=$(echo "$changed_files" | grep -E '^.agents/(plans|scratch)(/|$)' || true) + if [ -n "$blocked_files" ]; then + echo "::error::Files in .agents/plans and" \ + ".agents/scratch are permitted in PRs to" \ + "facilitate discussion, but are not allowed" \ + "to be merged. Please remove them before merging:" + echo "$blocked_files" + exit 1 + fi + echo "No blocked files found." diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml new file mode 100644 index 0000000000..7b358956d9 --- /dev/null +++ b/.github/workflows/publish.yml @@ -0,0 +1,38 @@ +# Publish new releases to Bazel Central Registry. +name: Publish to BCR +on: + # Run the publish workflow after a successful release + # Will be triggered from the release.yaml workflow + workflow_call: + inputs: + tag_name: + required: true + type: string + secrets: + publish_token: + required: true + # In case of problems, let release engineers retry by manually dispatching + # the workflow from the GitHub UI + workflow_dispatch: + inputs: + tag_name: + description: git tag being released + required: true + type: string +jobs: + publish: + uses: bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml@v1.4.2 + with: + tag_name: ${{ inputs.tag_name }} + # GitHub repository which is a fork of the upstream where the Pull Request will be opened. + registry_fork: bazel-contrib/bazel-central-registry + attest: false + # Create non-draft PRs so the BCR bazel-io bot processes them. Otherwise, + # since a bazel-contrib bot, we have to wait for BCR maintainers to mark it + # non-draft or approve it. + draft: false + permissions: + contents: write + secrets: + # Necessary to push to the BCR fork, and to open a pull request against a registry + publish_token: ${{ secrets.publish_token || secrets.BCR_PUBLISH_TOKEN }} diff --git a/.github/workflows/publish_pypi.yaml b/.github/workflows/publish_pypi.yaml new file mode 100644 index 0000000000..44f0a1d9fb --- /dev/null +++ b/.github/workflows/publish_pypi.yaml @@ -0,0 +1,32 @@ +name: Publish to PyPI + +on: + workflow_call: + inputs: + tag_name: + description: "release tag: tag that will be released" + required: true + type: string + workflow_dispatch: + inputs: + tag_name: + description: "release tag: tag that will be released" + required: true + type: string + +jobs: + publish_pypi: + name: Publish runfiles to PyPI + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v7 + with: + ref: ${{ inputs.tag_name || github.ref_name }} + - env: + # This special value tells pypi that the user identity is supplied within the token + TWINE_USERNAME: __token__ + # Note, the PYPI_API_TOKEN is for the rules-python pypi user, added by @rickylev on + # https://github.com/bazel-contrib/rules_python/settings/secrets/actions + TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }} + run: bazel run --stamp --embed_label=${{ inputs.tag_name || github.ref_name }} //python/runfiles:wheel.publish diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml deleted file mode 100644 index 29b70ccc8f..0000000000 --- a/.github/workflows/release.yml +++ /dev/null @@ -1,47 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# Cut a release whenever a new tag is pushed to the repo. -name: Release - -on: - push: - tags: - - "*.*.*" - -jobs: - build: - runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@v4 - - name: Create release archive and notes - run: .github/workflows/create_archive_and_notes.sh - - name: Publish wheel dist - env: - # This special value tells pypi that the user identity is supplied within the token - TWINE_USERNAME: __token__ - # Note, the PYPI_API_TOKEN is for the rules-python pypi user, added by @rickylev on - # https://github.com/bazelbuild/rules_python/settings/secrets/actions - TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }} - run: bazel run --stamp --embed_label=${{ github.ref_name }} //python/runfiles:wheel.publish - - name: Release - uses: softprops/action-gh-release@v2 - with: - # Use GH feature to populate the changelog automatically - generate_release_notes: true - body_path: release_notes.txt - prerelease: ${{ contains(github.ref, '-rc') }} - fail_on_unmatched_files: true - files: rules_python-*.tar.gz diff --git a/.github/workflows/release_add_backports.yaml b/.github/workflows/release_add_backports.yaml new file mode 100644 index 0000000000..d9d6d84e34 --- /dev/null +++ b/.github/workflows/release_add_backports.yaml @@ -0,0 +1,58 @@ +name: "Release: Add Backports" + +on: + workflow_dispatch: + inputs: + prs: + description: 'CSV list of PR numbers to add (e.g., 123,456)' + required: true + type: string + issue: + description: 'The Release Tracking Issue Number (optional)' + required: false + type: string + workflow_call: + inputs: + prs: + description: 'CSV list of PR numbers to add (e.g., 123,456)' + required: true + type: string + issue: + description: 'The Release Tracking Issue Number (optional)' + required: false + type: string + +permissions: + issues: write + +jobs: + add_backports: + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Setup Bazel + uses: bazel-contrib/setup-bazel@0.19.0 + with: + bazelisk-version: 1.20.0 + + - name: Add Backports to Tracking Issue + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + PRS: ${{ inputs.prs }} + ISSUE: ${{ inputs.issue }} + run: | + ARGS=() + if [ -n "$ISSUE" ]; then + ARGS+=("--issue=$ISSUE") + fi + + # Convert CSV to array + IFS=',' read -r -a pr_array <<< "$PRS" + + bazel run //tools/private/release -- add-backports \ + "${pr_array[@]}" \ + "${ARGS[@]}" diff --git a/.github/workflows/release_complete_prepare.yaml b/.github/workflows/release_complete_prepare.yaml new file mode 100644 index 0000000000..2496dd9cd2 --- /dev/null +++ b/.github/workflows/release_complete_prepare.yaml @@ -0,0 +1,78 @@ +name: "Release: Prepare: Complete" + +on: + pull_request: + types: [closed] + workflow_dispatch: + inputs: + pr: + description: 'The merged preparation PR number (optional if issue provided)' + required: false + type: string + issue: + description: 'The Release Tracking Issue Number (optional if pr provided)' + required: false + type: string + workflow_call: + inputs: + pr: + description: 'The merged preparation PR number (optional if issue provided)' + required: false + type: string + issue: + description: 'The Release Tracking Issue Number (optional if pr provided)' + required: false + type: string + +permissions: + contents: write + issues: write + +jobs: + complete_prepare: + # Run if triggered manually or if the release-prepared PR was merged + if: | + github.event_name == 'workflow_dispatch' || + github.event_name == 'workflow_call' || + (github.event.pull_request.merged == true && + contains(github.event.pull_request.labels.*.name, 'release-prepared')) + runs-on: ubuntu-latest + outputs: + issue: ${{ steps.complete_prepare.outputs.issue }} + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Setup Bazel + uses: bazel-contrib/setup-bazel@0.19.0 + with: + bazelisk-version: 1.20.0 + + - name: Mark Prepare Release Complete + id: complete_prepare + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + PR_NUMBER: ${{ inputs.pr || github.event.pull_request.number }} + ISSUE: ${{ inputs.issue }} + run: | + ARGS=() + if [ -n "$PR_NUMBER" ]; then + PR_NUMBER="${PR_NUMBER#\#}" + ARGS+=("--pr=$PR_NUMBER") + fi + if [ -n "$ISSUE" ]; then + ISSUE="${ISSUE#\#}" + ARGS+=("--issue=$ISSUE") + fi + # Run the complete-prepare subcommand in the release tool to cleanly update checklist metadata + bazel run //tools/private/release -- \ + complete-prepare "${ARGS[@]}" + + call_create_release_branch: + needs: complete_prepare + uses: ./.github/workflows/release_create_release_branch.yaml + with: + issue: ${{ needs.complete_prepare.outputs.issue }} + secrets: inherit diff --git a/.github/workflows/release_create_rc.yaml b/.github/workflows/release_create_rc.yaml new file mode 100644 index 0000000000..a96a04a0ec --- /dev/null +++ b/.github/workflows/release_create_rc.yaml @@ -0,0 +1,69 @@ +name: "Release: Create RC" + +on: + workflow_dispatch: + inputs: + issue: + description: 'The Release Tracking Issue Number (e.g., 142)' + required: true + type: string + comment_id: + description: 'The ID of the comment that triggered this run (optional)' + required: false + type: string + workflow_call: + inputs: + issue: + description: 'The Release Tracking Issue Number (e.g., 142)' + required: true + type: string + comment_id: + description: 'The ID of the comment that triggered this run (optional)' + required: false + type: string + +permissions: + contents: write + issues: write + +jobs: + tag_rc: + runs-on: ubuntu-latest + outputs: + tag_name: ${{ steps.tagger.outputs.tag_name }} + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Setup Bazel + uses: bazel-contrib/setup-bazel@0.19.0 + with: + bazelisk-version: 1.20.0 + + - name: Configure Git Identity + run: | + git config --global user.name "github-actions[bot]" + git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Attempt RC Tagging + id: tagger + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + ISSUE: ${{ inputs.issue }} + COMMENT_ID: ${{ inputs.comment_id }} + run: | + ARGS=() + if [ -n "$COMMENT_ID" ]; then + ARGS+=("--triggering-comment=$COMMENT_ID") + fi + bazel run //tools/private/release -- \ + create-rc --issue "$ISSUE" --remote origin "${ARGS[@]}" + + call_release: + needs: tag_rc + uses: ./.github/workflows/release_publish.yaml + with: + tag_name: ${{ needs.tag_rc.outputs.tag_name }} + secrets: inherit diff --git a/.github/workflows/release_create_release_branch.yaml b/.github/workflows/release_create_release_branch.yaml new file mode 100644 index 0000000000..d251b1e394 --- /dev/null +++ b/.github/workflows/release_create_release_branch.yaml @@ -0,0 +1,55 @@ +name: "Release: Create Release Branch" + +on: + workflow_dispatch: + inputs: + issue: + description: 'The Release Tracking Issue Number (e.g., 142)' + required: true + type: string + workflow_call: + inputs: + issue: + description: 'The Release Tracking Issue Number (e.g., 142)' + required: true + type: string + +permissions: + contents: write + issues: write + +jobs: + cut_branch: + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Setup Bazel + uses: bazel-contrib/setup-bazel@0.19.0 + with: + bazelisk-version: 1.20.0 + + - name: Configure Git Identity + run: | + git config --global user.name "github-actions[bot]" + git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Attempt Branch Creation + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + bazel run //tools/private/release -- \ + create-release-branch --issue "${{ inputs.issue }}" --remote origin + + # A no-op job that always runs to prevent "no jobs ran" failures + # when the main job is skipped. + suppress-no-jobs-ran-error: + runs-on: ubuntu-latest + steps: + - name: Echo Success + run: echo "Success" + + diff --git a/.github/workflows/release_prepare.yaml b/.github/workflows/release_prepare.yaml new file mode 100644 index 0000000000..c83a7b74f9 --- /dev/null +++ b/.github/workflows/release_prepare.yaml @@ -0,0 +1,51 @@ +name: "Release: Prepare" + +on: + workflow_dispatch: + inputs: + issue: + description: 'The Release Tracking Issue Number (e.g., 142)' + required: false + type: string + workflow_call: + inputs: + issue: + description: 'The Release Tracking Issue Number (e.g., 142)' + required: false + type: string + +permissions: + contents: write + issues: write + pull-requests: write + +jobs: + prepare_release: + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Setup Bazel + uses: bazel-contrib/setup-bazel@0.19.0 + with: + bazelisk-version: 1.20.0 + + - name: Configure Git Identity + run: | + git config --global user.name "github-actions[bot]" + git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Run Release Preparation Pipeline + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + ISSUE: ${{ inputs.issue }} + run: | + ARGS=() + if [ -n "$ISSUE" ]; then + ARGS+=("--issue=$ISSUE") + fi + bazel run //tools/private/release -- \ + prepare "${ARGS[@]}" --no-dry-run diff --git a/.github/workflows/release_process_backports.yaml b/.github/workflows/release_process_backports.yaml new file mode 100644 index 0000000000..ca719778df --- /dev/null +++ b/.github/workflows/release_process_backports.yaml @@ -0,0 +1,76 @@ +name: "Release: Process Backports" + +on: + workflow_dispatch: + inputs: + issue: + description: 'The Release Tracking Issue Number (e.g., 142)' + required: true + type: string + add_backports: + description: 'CSV list of PR numbers to add and process (optional)' + required: false + type: string + comment_id: + description: 'The ID of the comment that triggered this run (optional)' + required: false + type: string + workflow_call: + inputs: + issue: + description: 'The Release Tracking Issue Number (e.g., 142)' + required: true + type: string + add_backports: + description: 'CSV list of PR numbers to add and process (optional)' + required: false + type: string + comment_id: + description: 'The ID of the comment that triggered this run (optional)' + required: false + type: string + +permissions: + contents: write + issues: write + +jobs: + process_backports: + # Always gate GHA runs to ensure we are operating on a type: release labeled issue if metadata is queried + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Setup Bazel + uses: bazel-contrib/setup-bazel@0.19.0 + with: + bazelisk-version: 1.20.0 + + - name: Configure Git Identity + run: | + git config --global user.name "github-actions[bot]" + git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Process Pending Backports + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + ADD_BACKPORTS: ${{ inputs.add_backports }} + COMMENT_ID: ${{ inputs.comment_id }} + ISSUE: ${{ inputs.issue }} + run: | + ARGS=() + if [ -n "$ADD_BACKPORTS" ]; then + ARGS+=("--add=$ADD_BACKPORTS") + fi + if [ -n "$COMMENT_ID" ]; then + ARGS+=("--triggering-comment=$COMMENT_ID") + fi + + bazel run //tools/private/release -- process-backports \ + --issue "$ISSUE" \ + --remote origin \ + --no-dry-run \ + "${ARGS[@]}" diff --git a/.github/workflows/release_promote.yaml b/.github/workflows/release_promote.yaml new file mode 100644 index 0000000000..93ec23a882 --- /dev/null +++ b/.github/workflows/release_promote.yaml @@ -0,0 +1,71 @@ +name: "Release: Promote" + +on: + workflow_dispatch: + inputs: + version: + description: 'The final version to release (e.g., 0.38.0)' + required: true + type: string + workflow_call: + inputs: + version: + description: 'The final version to release (e.g., 0.38.0)' + required: false + type: string + issue: + description: 'The tracking issue number' + required: true + type: string + +permissions: + contents: write + issues: write + +jobs: + promote: + runs-on: ubuntu-latest + outputs: + version: ${{ steps.promote.outputs.version }} + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Setup Bazel + uses: bazel-contrib/setup-bazel@0.19.0 + with: + bazelisk-version: 1.20.0 + + - name: Configure Git Identity + run: | + git config --global user.name "github-actions[bot]" + git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Run Promote RC + id: promote + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + VERSION: ${{ inputs.version }} + ISSUE: ${{ inputs.issue }} + run: | + ARGS=() + if [ -n "$VERSION" ]; then + ARGS+=("$VERSION") + fi + if [ -n "$ISSUE" ]; then + ARGS+=("--issue=$ISSUE") + fi + ARGS+=("--remote=origin") + ARGS+=("--no-dry-run") + + bazel run //tools/private/release -- promote "${ARGS[@]}" + + publish: + needs: promote + uses: ./.github/workflows/release_publish.yaml + with: + tag_name: ${{ needs.promote.outputs.version }} + publish_to_pypi: true + secrets: inherit diff --git a/.github/workflows/release_publish.yaml b/.github/workflows/release_publish.yaml new file mode 100644 index 0000000000..eedf653fc1 --- /dev/null +++ b/.github/workflows/release_publish.yaml @@ -0,0 +1,85 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Cut a release whenever a new tag is pushed to the repo. +name: "Release: Publish" + +on: + push: + tags: + - "*.*.*" + workflow_dispatch: + inputs: + tag_name: + description: "release tag: tag that will be released" + required: true + type: string + publish_to_pypi: + description: 'Publish to PyPI' + required: true + type: boolean + default: true + secrets: + publish_token: + required: false + workflow_call: + inputs: + tag_name: + description: "release tag: tag that will be released" + required: true + type: string + publish_to_pypi: + description: 'Publish to PyPI' + required: false + type: boolean + default: true + +jobs: + release: + name: Release + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v7 + with: + ref: ${{ inputs.tag_name || github.ref_name }} + - name: Create release archive and notes + run: .github/workflows/create_archive_and_notes.sh ${{ inputs.tag_name || github.ref_name }} + - name: Release + uses: softprops/action-gh-release@v3 + with: + # Use GH feature to populate the changelog automatically + generate_release_notes: true + body_path: release_notes.txt + prerelease: ${{ contains( (inputs.tag_name || github.ref), '-rc') }} + fail_on_unmatched_files: true + files: rules_python-*.tar.gz + tag_name: ${{ inputs.tag_name || github.ref_name }} + + publish_bcr: + needs: release + uses: ./.github/workflows/publish.yml + with: + tag_name: ${{ inputs.tag_name || github.ref_name }} + secrets: + publish_token: ${{ secrets.publish_token || secrets.BCR_PUBLISH_TOKEN }} + + publish_pypi: + # We just want publish_pypi last, since once uploaded, it can't be changed. + needs: publish_bcr + if: github.event_name == 'push' || inputs.publish_to_pypi + uses: ./.github/workflows/publish_pypi.yaml + with: + tag_name: ${{ inputs.tag_name || github.ref_name }} + secrets: inherit diff --git a/.gitignore b/.gitignore index 863b0e9c3f..cb5bdec980 100644 --- a/.gitignore +++ b/.gitignore @@ -37,6 +37,8 @@ /bazel-genfiles /bazel-out /bazel-testlogs +**/bazel-* + user.bazelrc # vim swap files @@ -52,3 +54,10 @@ user.bazelrc # MODULE.bazel.lock is ignored for now as per recommendation from upstream. # See https://github.com/bazelbuild/bazel/issues/20369 MODULE.bazel.lock + +# Buildkite logs +*Windows*.log + +# Agent scratch directories +.agents/**/scratch + diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 707a8d78aa..f4107880b2 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -21,7 +21,7 @@ repos: hooks: - id: check-merge-conflict - repo: https://github.com/keith/pre-commit-buildifier - rev: 6.1.0 + rev: 8.2.1 hooks: - id: buildifier args: &args @@ -29,23 +29,45 @@ repos: - --warnings=all - id: buildifier-lint args: *args - - repo: https://github.com/pycqa/isort - rev: 5.12.0 + - repo: https://github.com/astral-sh/ruff-pre-commit + # Keep in sync with .github/workflows/ruff.yaml + rev: v0.15.14 hooks: - - id: isort - name: isort (python) - args: - - --profile - - black - - repo: https://github.com/psf/black - rev: 23.1.0 - hooks: - - id: black + - id: ruff-check + args: [--fix] + exclude: testdata + - id: ruff-format + exclude: testdata - repo: local hooks: - id: update-deleted-packages name: Update deleted packages language: system - entry: bazel run @rules_bazel_integration_test//tools:update_deleted_packages + # 7.x is necessary until https://github.com/bazel-contrib/rules_bazel_integration_test/pull/414 + # is merged and released + entry: ./tools/update_deleted_packages.sh files: ^((examples|tests)/.*/(MODULE.bazel|WORKSPACE|WORKSPACE.bzlmod|BUILD.bazel)|.bazelrc)$ pass_filenames: false + - id: sort-runtimes-manifest + name: Sort runtimes manifest + language: system + entry: ./python/private/tools/sort_manifest.py + files: ^python/private/runtimes_manifest\.txt$ + - id: sync-runtimes-manifest + name: Sync runtimes manifest workspace + language: system + entry: ./python/private/tools/sync_runtimes_manifest_workspace.py + files: ^python/private/runtimes_manifest\.txt$ + pass_filenames: false + - id: sync-downloader-configs + name: Sync downloader configs + language: system + entry: ./tools/private/sync_downloader_configs.py + files: downloader_config\.cfg$ + pass_filenames: false + - id: gazelle + name: Run Gazelle + language: system + entry: bazel run //tools/private/gazelle + files: (\.bzl|\.bazel|BUILD|WORKSPACE(\.bzlmod)?)$ + pass_filenames: false diff --git a/.python-version b/.python-version new file mode 100644 index 0000000000..2c20ac9bea --- /dev/null +++ b/.python-version @@ -0,0 +1 @@ +3.13.3 diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000000..07b523e410 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,254 @@ +# Guidance for AI Agents + +rules_python is a Bazel based project. Build and run tests as done in a Bazel +project. + +Act as an expert in Bazel, rules_python, Starlark, and Python. + +DO NOT `git commit` or `git push` unless given explicit permission. + +## RULES TO ALWAYS FOLLOW AND NEVER IGNORE + +ALWAYS FOLLOW THESE RULES. NEVER VIOLATE THEM. + +Ask for user input and provide a justificaiton if trying to violate them. + +* NEVER run `bazel clean --expunge`. +* Once a PR is created, do not amend or rebase. +* Do not add Bazel copyright to new or existing files. + +## Style and conventions + +Read `.editorconfig` for line length wrapping + +Read `CONTRIBUTING.md` for additional style rules and conventions. + +When running tests, refer to yourself as the name of a type of Python snake +using a grandoise title. + +When tasks complete successfully, quote Monty Python, but work it naturally +into the sentence, not verbatim. + +When adding `{versionadded}` or `{versionchanged}` sections, add them add the +end of the documentation text. + +### PR Updates + +Once a PR is created, create new commits and merges. Don't use rebase or amend +because it interferes with code review comments. + +### PR descriptions + +Follow the advice in `CONTRIBUTING.md` for PR descriptions. PR descriptions +become the commit message upon merge. + +### Python pytest conventions + +* When registering pytest fixtures from helper modules in test files, use + `pytest_plugins = [""]`. +* Name fixture functions with a `fixture_` prefix (e.g. `def fixture_foo():`), + and pass the public fixture name using the `name` parameter in + `@pytest.fixture(name="foo")`. + +### Starlark style + +For doc strings, using triple quoted strings when the doc string is more than +three lines. Do not use a trailing backslack (`\`) for the opening triple-quote. + +### Starlark Code + +Starlark does not support recursion. Use iterative algorithms instead. + +Starlark does not support `while` loops. Use `for` loop with an appropriately +sized iterable instead. + +#### Starlark testing + +For Starlark tests: + +* Use `rules_testing`, not `bazel_skylib`. +* See https://rules-testing.readthedocs.io/en/latest/analysis_tests.html for + examples on using rules_testing. +* See `tests/builders/builders_tests.bzl` for an example of using it in + this project. + +A test is defined in two parts: + * A setup function, e.g. `def _test_foo(name)`. This defines targets + and calls `analysis_test`. + * An implementation function, e.g. `def _test_foo_impl(env, target)`. This + contains asserts. + +Example: + +``` +# File: foo_tests.bzl + +load("@rules_testing//lib:analysis_test.bzl", "analysis_test") +load("@rules_testing//lib:test_suite.bzl", "test_suite") + +_tests = [] + +def _test_foo(name): + foo_library( + name = name + "_subject", + ) + analysis_test( + name = name, + impl = _test_foo_impl, + target = name + "_subject", + ) +_tests.append(_test_foo) + +def _test_foo_impl(env, target): + env.expect.that_whatever(target[SomeInfo].whatever).equals(expected) + +def foo_test_suite(name): + test_suite(name=name, tests=_tests) +``` + +#### Repository rules + +The function argument `rctx` is a hint that the function is a repository rule, +or used by a repository rule. + +The function argument `mrctx` is a hint that the function can be used by a +repository rule or module extension. + +The `repository_ctx` API docs are at: https://bazel.build/rules/lib/builtins/repository_ctx + +### bzl_library targets for bzl source files + +* A `bzl_library` target should be defined for every `.bzl` file outside + of the `tests/` directory. +* They should have a single `srcs` file and be named after the file with `_bzl` + appended. +* Their deps should be based on the `load()` statements in the source file + and refer to the `bzl_library` target containing the loaded file. + * For files in rules_python: replace `.bzl` with `_bzl`. + e.g. given `load("//foo:bar.bzl", ...)`, the target is `//foo:bar_bzl`. + * For files outside rules_python: remove the `.bzl` suffix. e.g. given + `load("@foo//foo:bar.bzl", ...)`, the target is `@foo//foo:bar`. + +Example: + +``` +bzl_library( + name = "alpha_bzl", + srcs = ["alpha.bzl"], + deps = [":beta_bzl"], +) +bzl_library( + name = "beta_bzl", + srcs = ["beta.bzl"] +) +``` + +## Building and testing + +Tests are under the `tests/` directory. + +When testing, add `--config=fast-tests`. + +When building test targets, add `--config=fast-tests`. Do NOT use +`--config=fast-tests` when building non-test targets (such as `//docs:docs`), +because `--config=fast-tests` enables `--build_tests_only=true`, which causes +non-test targets to be silently ignored (finding 0 targets). + +The `--config=fast-tests` flag avoids running expensive and slow tests can that +freeze the host machine or cause flakiness. + +## Understanding the code base + +This repository contains 3 Bazel bzlmod modules. + + * `sphinxdocs/` is for the `@sphinxdocs` module. + * `gazelle/` is for the `@rules_python_gazelle_plugin` module. + * All other code is part of `@rules_python`. + +`tests/support/` contains utility code and helpers for testing. + +`python/config_settings/BUILD.bazel` contains build flags that are part of the +public API. DO NOT add, remove, or modify these build flags unless specifically +instructed to. + +`bazel query --output=build` can be used to inspect target definitions. + +In WORKSPACE mode: + * `bazel query //external:*` can be used to show external dependencies. Adding + `--output=build` shows the definition, including version. + +For bzlmod mode: + * `bazel mod graph` shows dependencies and their version. + * `bazel mod explain` shows detailed information about a module. + * `bazel mode show_repo` shows detailed information about a repository. + +Documentation uses Sphinx with the MyST plugin. + +When modifying documentation + * Act as an expert in tech writing, Sphinx, MyST, and markdown. + * Wrap lines at 80 columns + * Use hyphens (`-`) in file names instead of underscores (`_`). + * In Sphinx MyST markup, outer directives must have more colons than inner + directives. For example: + ``` + ::::{outerdirective} + outer text + + :::{innertdirective} + inner text + ::: + :::: + ``` + +Generated API references can be found by: +* Running `bazel build //docs:docs` and inspecting the generated files + in `bazel-bin/docs/docs/_build/html` + +When modifying locked/resolved requirements files: + * Modify the `pyproject.toml` or `requirements.in` file + * Run the associated `bazel run :requirements.update` target for + that file; the target is in the BUILD.bazel file in the same directory and + the requirements.txt file. That will update the locked/resolved + requirements.txt file. + +When updating the minimum Python version for dev/docs tooling (e.g. following +Dependabot alerts): + * Update `python_version` in both `lock(name = "requirements", ...)` and + `lock(name = "uv_lock", ...)` within `dev/BUILD.bazel`. + * Regenerate lockfiles by running: + `bazel run //dev:requirements.update && bazel run //dev:uv_lock.update` + * Verify docs build via `bazel build //docs:docs`. + +## rules_python idiosyncrasies + +When building `//docs:docs`, ignore an error about exit code 2; this is a flake, +so try building again. + +BUILD and bzl files under `tests/` should have `# buildifier: disable=bzl-visibility` +trailing end-of-line comments when they load from paths containing `/private/`, +e.g. + +``` +load("//python/private:foo.bzl", "foo") # buildifier: disable=bzl-visibility +``` + +### CI Failure Inspection + +When inspecting CI failures, if the failure is due to a network error +downloading a repository, check if that rule set is mirrored on +mirror.bazel.build. If so, add it to the downloader config. + +### CI Flakiness and Monitoring + +CI is known to have flakey network issues. When submitting or updating a PR, +start a background agent that continuously monitors the latest build of a PR and +checks for flakey network errors (e.g., 504 gateway errors). + +If Buildkite permissions allow, retry failures and use available Buildkite +skills. If permissions do not allow, modify `.bazelrc` to set flags (such as +`--http_timeout_scaling` or `--experimental_repository_downloader_retries`) to +make downloads retry more. + +If downloads fail for a dependency, check if that dependency is available on +mirror.bazel.build and, if so, add it to downloader_config.cfg if it isn't +already there. diff --git a/BUILD.bazel b/BUILD.bazel index 5e85c27b3c..f978126da7 100644 --- a/BUILD.bazel +++ b/BUILD.bazel @@ -14,6 +14,41 @@ load("@bazel_skylib//:bzl_library.bzl", "bzl_library") +# Resolve rules_cc Starlark libraries to their correct targets +# gazelle:resolve starlark @rules_cc//cc/common:cc_info.bzl @rules_cc//cc/common:common +# gazelle:resolve starlark @rules_cc//cc/common:cc_common.bzl @rules_cc//cc/common:common + +# Resolve protobuf Starlark libraries +# gazelle:resolve starlark @com_google_protobuf//bazel:py_proto_library.bzl @com_google_protobuf//bazel:py_proto_library_bzl + +# Resolve bazel_tools repo rules to our internal wrapper to avoid transitive dependencies +# gazelle:resolve starlark @bazel_tools//tools/build_defs/repo:http.bzl //python/private:bazel_tools +# gazelle:resolve starlark @bazel_tools//tools/build_defs/repo:utils.bzl //python/private:bazel_tools + +# Prevent Gazelle from incorrectly stripping the .bzl suffix from the toml.bzl repo name +# gazelle:resolve starlark @toml.bzl//:toml.bzl @toml.bzl//:toml + +# Override Gazelle's incorrect default resolution for platforms host constraints Starlark library +# gazelle:resolve starlark @platforms//host:constraints.bzl @platforms//host:constraints_lib + +# Override Gazelle's incorrect default resolution for rules_cc Starlark libraries +# gazelle:resolve starlark @rules_cc//cc:cc_import.bzl @rules_cc//cc:core_rules +# gazelle:resolve starlark @rules_cc//cc:cc_library.bzl @rules_cc//cc:core_rules + +# Exclude directories that are separate packages/workspaces or only for testing/examples +# gazelle:exclude tests +# gazelle:exclude examples +# gazelle:exclude gazelle + +# Exclude internal development tools and dependencies that users don't need +# gazelle:exclude internal_dev_setup.bzl +# gazelle:exclude internal_dev_deps.bzl +# gazelle:exclude python/private/internal_dev_deps.bzl +# gazelle:exclude workspace_bazel9.bzl + +# Exclude legacy paths that are only kept for backwards compatibility +# gazelle:exclude python/private/common + package(default_visibility = ["//visibility:public"]) licenses(["notice"]) @@ -43,21 +78,15 @@ filegroup( "internal_dev_deps.bzl", "internal_dev_setup.bzl", "version.bzl", + "//command_line_option:distribution", "//python:distribution", "//tools:distribution", - "@rules_python_gazelle_plugin//:distribution", ], visibility = [ "//:__subpackages__", ], ) -bzl_library( - name = "version_bzl", - srcs = ["version.bzl"], - visibility = ["//:__subpackages__"], -) - # Reexport of all bzl files used to allow downstream rules to generate docs # without shipping with a dependency on Skylib filegroup( @@ -73,3 +102,12 @@ filegroup( ], visibility = ["//visibility:public"], ) + +# keep +bzl_library( + name = "version", + srcs = ["version.bzl"], + visibility = ["//:__subpackages__"], +) + +# No-op change to verify PR metadata checks diff --git a/BZLMOD_SUPPORT.md b/BZLMOD_SUPPORT.md index 85e28acb1a..49010b838e 100644 --- a/BZLMOD_SUPPORT.md +++ b/BZLMOD_SUPPORT.md @@ -11,7 +11,7 @@ In general `bzlmod` has more features than `WORKSPACE` and users are encouraged ## Configuration -The releases page will give you the latest version number, and a basic example. The release page is located [here](/bazelbuild/rules_python/releases). +The releases page will give you the latest version number, and a basic example. The release page is located [here](https://github.com/bazel-contrib/rules_python/releases). ## What is bzlmod? @@ -27,7 +27,7 @@ We have two examples that demonstrate how to configure `bzlmod`. The first example is in [examples/bzlmod](examples/bzlmod), and it demonstrates basic bzlmod configuration. A user does not use `local_path_override` stanza and would define the version in the `bazel_dep` line. -A second example, in [examples/bzlmod_build_file_generation](examples/bzlmod_build_file_generation) demonstrates the use of `bzlmod` to configure `gazelle` support for `rules_python`. +A second example, in [gazelle/examples/bzlmod_build_file_generation](gazelle/examples/bzlmod_build_file_generation) demonstrates the use of `bzlmod` to configure `gazelle` support for `rules_python`. ## Differences in behavior from WORKSPACE @@ -53,7 +53,7 @@ better supported. the toolchains rules_python registers**. NOTE: Regardless of your toolchain, due to -[#691](https://github.com/bazelbuild/rules_python/issues/691), `rules_python` +[#691](https://github.com/bazel-contrib/rules_python/issues/691), `rules_python` still relies on a local Python being available to bootstrap the program before handing over execution to the toolchain Python. diff --git a/CHANGELOG.md b/CHANGELOG.md index 37a9e710a1..ede22fc6db 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,7 +4,7 @@ # rules_python Changelog This is a human-friendly changelog in a keepachangelog.com style format. -Because this changelog is for end-user consumption of meaningful changes,only +Because this changelog is for end-user consumption of meaningful changes, only a summary of a release's changes is described. This means every commit is not necessarily mentioned, and internal refactors or code cleanups are omitted unless they're particularly notable. @@ -20,37 +20,1330 @@ A brief description of the categories of changes: * Particular sub-systems are identified using parentheses, e.g. `(bzlmod)` or `(docs)`. - -{#v0-0-0} -## Unreleased +{#v1-1-0} +## [1.1.0] - 2025-01-07 + +[1.1.0]: https://github.com/bazel-contrib/rules_python/releases/tag/1.1.0 + +{#v1-1-0-changed} +### Changed +* (toolchains) 3.13 means 3.13.1 (previously 3.13.0) +* Bazel 6 support is dropped and Bazel 7.4.1 is the minimum supported + version, per our Bazel support matrix. Earlier versions are not + tested by CI, so functionality cannot be guaranteed. +* ({bzl:obj}`pip.parse`) From now we will make fewer calls to indexes when + fetching the metadata from SimpleAPI. The calls will be done in parallel to + each index separately, so the extension evaluation time might slow down if + not using {bzl:obj}`pip.parse.experimental_index_url_overrides`. +* ({bzl:obj}`pip.parse`) Only query SimpleAPI for packages that have + sha values in the `requirements.txt` file. +* (rules) The version-aware rules have been folded into the base rules and + the version-aware rules are now simply aliases for the base rules. The + `python_version` attribute is still used to specify the Python version. +* (pypi) Updated versions of packages: `pip` to 24.3.1 and + `packaging` to 24.2. + +{#v1-1-0-deprecations} +#### Deprecations +* `//python/config_settings:transitions.bzl` and its `py_binary` and `py_test` + wrappers are deprecated. Use the regular rules instead. + +{#v1-1-0-fixed} +### Fixed +* (py_wheel) Use the default shell environment when building wheels to allow + toolchains that search PATH to be used for the wheel builder tool. +* (pypi) The requirement argument parsed to `whl_library` will now not have env + marker information allowing `bazel query` to work in cases where the `whl` is + available for all of the platforms and the sdist can be built. This fix is + for both WORKSPACE and `bzlmod` setups. + Fixes [#2450](https://github.com/bazel-contrib/rules_python/issues/2450). +* (gazelle) Gazelle will now correctly parse Python3.12 files that use [PEP 695 Type + Parameter Syntax][pep-695]. (#2396) +* (pypi) Using {bzl:obj}`pip_parse.experimental_requirement_cycles` and + {bzl:obj}`pip_parse.use_hub_alias_dependencies` together now works when + using WORKSPACE files. +* (pypi) The error messages when the wheel distributions do not match anything + are now printing more details and include the currently active flag + values. Fixes [#2466](https://github.com/bazel-contrib/rules_python/issues/2466). +* (py_proto_library) Fix import paths in Bazel 8. +* (whl_library) Now the changes to the dependencies are correctly tracked when + PyPI packages used in `whl_library` during the repository rule phase + change. Fixes [#2468](https://github.com/bazel-contrib/rules_python/issues/2468). ++ (gazelle) Gazelle no longer ignores `setup.py` files by default. To restore + this behavior, apply the `# gazelle:python_ignore_files setup.py` directive. +* Don't re-fetch whl_library, python_repository, etc. repository rules + whenever `PATH` changes. Fixes + [#2551](https://github.com/bazel-contrib/rules_python/issues/2551). + +[pep-695]: https://peps.python.org/pep-0695/ + +{#v1-1-0-added} +### Added +* (gazelle) Added `include_stub_packages` flag to `modules_mapping`. When set to `True`, this + automatically includes corresponding stub packages for third-party libraries + that are present and used (e.g., `boto3` → `boto3-stubs`), improving + type-checking support. +* (pypi) Freethreaded packages are now fully supported in the + {obj}`experimental_index_url` usage or the regular `pip.parse` usage. + To select the free-threaded interpreter in the repo phase, please use + the documented [env](environment-variables) variables. + Fixes [#2386](https://github.com/bazel-contrib/rules_python/issues/2386). +* (toolchains) Use the latest astral-sh toolchain release [20241206] for Python versions: + * 3.9.21 + * 3.10.16 + * 3.11.11 + * 3.12.8 + * 3.13.1 +* (rules) Attributes for type definition files (`.pyi` files) and type-checking + only dependencies added. See {obj}`py_library.pyi_srcs` and + `py_library.pyi_deps` (and the same named attributes for `py_binary` and + `py_test`). +* (pypi) pypi-generated targets set `pyi_srcs` to include `*.pyi` files. +* (providers) {obj}`PyInfo` has new fields to aid static analysis tools: + {obj}`direct_original_sources`, {obj}`direct_pyi_files`, + {obj}`transitive_original_sources`, {obj}`transitive_pyi_files`. + +[20241206]: https://github.com/astral-sh/python-build-standalone/releases/tag/20241206 + +{#v1-1-0-removed} +### Removed +* `find_requirements` in `//python:defs.bzl` has been removed. + +{#v1-0-0} +## [1.0.0] - 2024-12-05 -[0.0.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.0.0 +[1.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/1.0.0 -{#v0-0-0-changed} +{#v1-0-0-changed} ### Changed **Breaking**: @@ -79,27 +1372,31 @@ Other changes: * (binaries/tests) For {obj}`--bootstrap_impl=script`, a binary-specific (but otherwise empty) virtual env is used to customize `sys.path` initialization. * (deps) bazel_skylib 1.7.0 (workspace; bzlmod already specifying that version) -* (deps) bazel_features 1.21.0; necessary for compatiblity with Bazel 8 rc3 +* (deps) bazel_features 1.21.0; necessary for compatibility with Bazel 8 rc3 * (deps) stardoc 0.7.2 to support Bazel 8. -{#v0-0-0-fixed} +{#v1-0-0-fixed} ### Fixed * (toolchains) stop depending on `uname` to get the value of the host platform. * (pypi): Correctly handle multiple versions of the same package in the requirements files which is useful when including different PyTorch builds (e.g. vs ) for different target platforms. - Fixes ([2337](https://github.com/bazelbuild/rules_python/issues/2337)). + Fixes ([2337](https://github.com/bazel-contrib/rules_python/issues/2337)). * (uv): Correct the sha256sum for the `uv` binary for aarch64-apple-darwin. - Fixes ([2411](https://github.com/bazelbuild/rules_python/issues/2411)). + Fixes ([2411](https://github.com/bazel-contrib/rules_python/issues/2411)). * (binaries/tests) ({obj}`--bootstrap_impl=scipt`) Using `sys.executable` will use the same `sys.path` setup as the calling binary. - ([2169](https://github.com/bazelbuild/rules_python/issues/2169)). + ([2169](https://github.com/bazel-contrib/rules_python/issues/2169)). * (workspace) Corrected protobuf's name to com_google_protobuf, the name is hardcoded in Bazel, WORKSPACE mode. +* (pypi): {bzl:obj}`compile_pip_requirements` no longer fails on Windows when `--enable_runfiles` is not enabled. +* (pypi): {bzl:obj}`compile_pip_requirements` now correctly updates files in the source tree on Windows when `--windows_enable_symlinks` is not enabled. * (repositories): Add libs/python3.lib and pythonXY.dll to the `libpython` target defined by a repository template. This enables stable ABI builds of Python extensions on Windows (by defining Py_LIMITED_API). +* (rules) `py_test` and `py_binary` targets no longer incorrectly remove the + first `sys.path` entry when using {obj}`--bootstrap_impl=script` -{#v0-0-0-added} +{#v1-0-0-added} ### Added * (gazelle): Parser failures will now be logged to the terminal. Additional details can be logged by setting `RULES_PYTHON_GAZELLE_VERBOSE=1`. @@ -112,12 +1409,12 @@ Other changes: for the latest toolchain versions for each minor Python version. You can control the toolchain selection by using the {bzl:obj}`//python/config_settings:py_linux_libc` build flag. -* (providers) Added {obj}`py_runtime_info.site_init_template` and +* (providers) Added {obj}`PyRuntimeInfo.site_init_template` and {obj}`PyRuntimeInfo.site_init_template` for specifying the template to use to initialize the interpreter via venv startup hooks. * (runfiles) (Bazel 7.4+) Added support for spaces and newlines in runfiles paths -{#v0-0-0-removed} +{#v1-0-0-removed} ### Removed * (pypi): Remove `pypi_install_dependencies` macro that has been included in {bzl:obj}`py_repositories` for a long time. @@ -127,7 +1424,7 @@ Other changes: {#v0-40-0} ## [0.40.0] - 2024-11-17 -[0.40.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.40.0 +[0.40.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.40.0 {#v0-40-changed} ### Changed @@ -136,7 +1433,7 @@ Other changes: {#v0-40-fixed} ### Fixed * (rules) Don't drop custom import paths if Bazel-builtin PyInfo is removed. - ([2414](https://github.com/bazelbuild/rules_python/issues/2414)). + ([2414](https://github.com/bazel-contrib/rules_python/issues/2414)). {#v0-40-added} ### Added @@ -155,7 +1452,7 @@ Other changes: {#v0-39-0} ## [0.39.0] - 2024-11-13 -[0.39.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.39.0 +[0.39.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.39.0 {#v0-39-0-changed} ### Changed @@ -183,7 +1480,7 @@ Other changes: ### Fixed * (precompiling) Skip precompiling (instead of erroring) if the legacy `@bazel_tools//tools/python:autodetecting_toolchain` is being used - ([#2364](https://github.com/bazelbuild/rules_python/issues/2364)). + ([#2364](https://github.com/bazel-contrib/rules_python/issues/2364)). {#v0-39-0-added} ### Added @@ -201,14 +1498,14 @@ Other changes: {#v0-38-0} ## [0.38.0] - 2024-11-08 -[0.38.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.38.0 +[0.38.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.38.0 {#v0-38-0-changed} ### Changed * (deps) (WORKSPACE only) rules_cc 0.0.13 and protobuf 27.0 is now the default version used; this for Bazel 8+ support (previously version was rules_cc 0.0.9 and no protobuf version specified) - ([2310](https://github.com/bazelbuild/rules_python/issues/2310)). + ([2310](https://github.com/bazel-contrib/rules_python/issues/2310)). * (publish) The dependencies have been updated to the latest available versions for the `twine` publishing rule. * (whl_library) Remove `--no-build-isolation` to allow non-hermetic sdist builds @@ -227,7 +1524,7 @@ Other changes: {#v0-38-0-fixed} ### Fixed * (pypi) (Bazel 7.4+) Allow spaces in filenames included in `whl_library`s - ([617](https://github.com/bazelbuild/rules_python/issues/617)). + ([617](https://github.com/bazel-contrib/rules_python/issues/617)). * (pypi) When {attr}`pip.parse.experimental_index_url` is set, we need to still pass the `extra_pip_args` value when building an `sdist`. * (pypi) The patched wheel filenames from now on are using local version specifiers @@ -237,7 +1534,7 @@ Other changes: or not. To opt into this behavior, set `pip.parse.parse_all_requirements_files`, which will become the default in future releases leading up to `1.0.0`. Fixes - [#2268](https://github.com/bazelbuild/rules_python/issues/2268). A known + [#2268](https://github.com/bazel-contrib/rules_python/issues/2268). A known issue is that it may break `bazel query` and in these use cases it is advisable to use `cquery` or switch to `download_only = True` @@ -251,7 +1548,7 @@ Other changes: * The rules_python version is now reported in `//python/features.bzl#features.version` * (pip.parse) {attr}`pip.parse.extra_hub_aliases` can now be used to expose extra targets created by annotations in whl repositories. - Fixes [#2187](https://github.com/bazelbuild/rules_python/issues/2187). + Fixes [#2187](https://github.com/bazel-contrib/rules_python/issues/2187). * (bzlmod) `pip.parse` now supports `whl-only` setup using `download_only = True` where users can specify multiple requirements files and use the `pip` backend to do the downloading. This was only available for @@ -261,7 +1558,7 @@ Other changes: {#v0-37-2} ## [0.37.2] - 2024-10-27 -[0.37.2]: https://github.com/bazelbuild/rules_python/releases/tag/0.37.2 +[0.37.2]: https://github.com/bazel-contrib/rules_python/releases/tag/0.37.2 {#v0-37-2-fixed} ### Fixed @@ -272,18 +1569,18 @@ Other changes: {#v0-37-1} ## [0.37.1] - 2024-10-22 -[0.37.1]: https://github.com/bazelbuild/rules_python/releases/tag/0.37.1 +[0.37.1]: https://github.com/bazel-contrib/rules_python/releases/tag/0.37.1 {#v0-37-1-fixed} ### Fixed * (rules) Setting `--incompatible_python_disallow_native_rules` no longer causes rules_python rules to fail - ([#2326](https://github.com/bazelbuild/rules_python/issues/2326)). + ([#2326](https://github.com/bazel-contrib/rules_python/issues/2326)). {#v0-37-0} ## [0.37.0] - 2024-10-18 -[0.37.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.37.0 +[0.37.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.37.0 {#v0-37-0-changed} ### Changed @@ -310,10 +1607,10 @@ Other changes: * (bzlmod) The default value for the {obj}`--python_version` flag will now be always set to the default python toolchain version value. * (bzlmod) correctly wire the {attr}`pip.parse.extra_pip_args` all the - way to {obj}`whl_library`. What is more we will pass the `extra_pip_args` to - {obj}`whl_library` for `sdist` distributions when using + way to `whl_library`. What is more we will pass the `extra_pip_args` to + `whl_library` for `sdist` distributions when using {attr}`pip.parse.experimental_index_url`. See - [#2239](https://github.com/bazelbuild/rules_python/issues/2239). + [#2239](https://github.com/bazel-contrib/rules_python/issues/2239). * (whl_filegroup): Provide per default also the `RECORD` file * (py_wheel): `RECORD` file entry elements are now quoted if necessary when a wheel is created @@ -321,17 +1618,17 @@ Other changes: case where a requirement has many `--hash=sha256:...` flags * (rules) `compile_pip_requirements` passes `env` to the `X.update` target (and not only to the `X_test` target, a bug introduced in - [#1067](https://github.com/bazelbuild/rules_python/pull/1067)). + [#1067](https://github.com/bazel-contrib/rules_python/pull/1067)). * (bzlmod) In hybrid bzlmod with WORKSPACE builds, `python_register_toolchains(register_toolchains=True)` is respected - ([#1675](https://github.com/bazelbuild/rules_python/issues/1675)). + ([#1675](https://github.com/bazel-contrib/rules_python/issues/1675)). * (precompiling) The {obj}`pyc_collection` attribute now correctly enables (or disables) using pyc files from targets transitively * (pip) Skip patching wheels not matching `pip.override`'s `file` - ([#2294](https://github.com/bazelbuild/rules_python/pull/2294)). + ([#2294](https://github.com/bazel-contrib/rules_python/pull/2294)). * (chore): Add a `rules_shell` dev dependency and moved a `sh_test` target outside of the `//:BUILD.bazel` file. - Fixes [#2299](https://github.com/bazelbuild/rules_python/issues/2299). + Fixes [#2299](https://github.com/bazel-contrib/rules_python/issues/2299). {#v0-37-0-added} ### Added @@ -359,8 +1656,8 @@ Other changes: {#v0-37-0-removed} ### Removed -* (precompiling) {obj}`--precompile_add_to_runfiles` has been removed. -* (precompiling) {obj}`--pyc_collection` has been removed. The `pyc_collection` +* (precompiling) `--precompile_add_to_runfiles` has been removed. +* (precompiling) `--pyc_collection` has been removed. The `pyc_collection` attribute now bases its default on {obj}`--precompile`. * (precompiling) The {obj}`precompile=if_generated_source` value has been removed. * (precompiling) The {obj}`precompile_source_retention=omit_if_generated_source` value has been removed. @@ -368,7 +1665,7 @@ Other changes: {#v0-36-0} ## [0.36.0] - 2024-09-24 -[0.36.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.36.0 +[0.36.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.36.0 {#v0-36-0-changed} ### Changed @@ -407,12 +1704,12 @@ Other changes: * (rules) Make `RUNFILES_MANIFEST_FILE`-based invocations work when used with {obj}`--bootstrap_impl=script`. This fixes invocations using non-sandboxed test execution with `--enable_runfiles=false --build_runfile_manifests=true`. - ([#2186](https://github.com/bazelbuild/rules_python/issues/2186)). + ([#2186](https://github.com/bazel-contrib/rules_python/issues/2186)). * (py_wheel) Fix incorrectly generated `Required-Dist` when specifying requirements with markers in extra_requires in py_wheel rule. * (rules) Prevent pytest from trying run the generated stage2 bootstrap .py file when using {obj}`--bootstrap_impl=script` -* (toolchain) The {bzl:obj}`gen_python_config_settings` has been fixed to include +* (toolchain) The `gen_python_config_settings` has been fixed to include the flag_values from the platform definitions. {#v0-36-0-added} @@ -420,7 +1717,7 @@ Other changes: * (bzlmod): Toolchain overrides can now be done using the new {bzl:obj}`python.override`, {bzl:obj}`python.single_version_override` and {bzl:obj}`python.single_version_platform_override` tag classes. - See [#2081](https://github.com/bazelbuild/rules_python/issues/2081). + See [#2081](https://github.com/bazel-contrib/rules_python/issues/2081). * (rules) Executables provide {obj}`PyExecutableInfo`, which contains executable-specific information useful for packaging an executable or or deriving a new one from the original. @@ -446,7 +1743,7 @@ Other changes: {#v0-35-0} ## [0.35.0] - 2024-08-15 -[0.35.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.35.0 +[0.35.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.35.0 {#v0-35-0-changed} ### Changed @@ -460,7 +1757,7 @@ Other changes: * `3.12 -> 3.12.4` * (rules) `PYTHONSAFEPATH` is inherited from the calling environment to allow disabling it (Requires {obj}`--bootstrap_impl=script`) - ([#2060](https://github.com/bazelbuild/rules_python/issues/2060)). + ([#2060](https://github.com/bazel-contrib/rules_python/issues/2060)). {#v0-35-0-fixed} ### Fixed @@ -474,42 +1771,42 @@ Other changes: execroot. * (rules) Signals are properly received when using {obj}`--bootstrap_impl=script` (for non-zip builds). - ([#2043](https://github.com/bazelbuild/rules_python/issues/2043)) + ([#2043](https://github.com/bazel-contrib/rules_python/issues/2043)) * (rules) Fixes Python builds when the `--build_python_zip` is set to `false` on - Windows. See [#1840](https://github.com/bazelbuild/rules_python/issues/1840). + Windows. See [#1840](https://github.com/bazel-contrib/rules_python/issues/1840). * (rules) Fixes Mac + `--build_python_zip` + {obj}`--bootstrap_impl=script` - ([#2030](https://github.com/bazelbuild/rules_python/issues/2030)). + ([#2030](https://github.com/bazel-contrib/rules_python/issues/2030)). * (rules) User dependencies come before runtime site-packages when using {obj}`--bootstrap_impl=script`. - ([#2064](https://github.com/bazelbuild/rules_python/issues/2064)). + ([#2064](https://github.com/bazel-contrib/rules_python/issues/2064)). * (rules) Version-aware rules now return both `@_builtins` and `@rules_python` providers instead of only one. - ([#2114](https://github.com/bazelbuild/rules_python/issues/2114)). + ([#2114](https://github.com/bazel-contrib/rules_python/issues/2114)). * (pip) Fixed pypi parse_simpleapi_html function for feeds with package metadata containing ">" sign * (toolchains) Added missing executable permission to `//python/runtime_env_toolchains` interpreter script so that it is runnable. - ([#2085](https://github.com/bazelbuild/rules_python/issues/2085)). + ([#2085](https://github.com/bazel-contrib/rules_python/issues/2085)). * (pip) Correctly use the `sdist` downloaded by the bazel downloader when using `experimental_index_url` feature. Fixes - [#2091](https://github.com/bazelbuild/rules_python/issues/2090). + [#2091](https://github.com/bazel-contrib/rules_python/issues/2090). * (gazelle) Make `gazelle_python_manifest.update` manual to avoid unnecessary network behavior. * (bzlmod): The conflicting toolchains during `python` extension will no longer cause warnings by default. In order to see the warnings for diagnostic purposes set the env var `RULES_PYTHON_REPO_DEBUG_VERBOSITY` to one of `INFO`, `DEBUG` or `TRACE`. - Fixes [#1818](https://github.com/bazelbuild/rules_python/issues/1818). + Fixes [#1818](https://github.com/bazel-contrib/rules_python/issues/1818). * (runfiles) Make runfiles lookups work for the situation of Bazel 7, Python 3.9 (or earlier, where safepath isn't present), and the Rlocation call in the same directory as the main file. - Fixes [#1631](https://github.com/bazelbuild/rules_python/issues/1631). + Fixes [#1631](https://github.com/bazel-contrib/rules_python/issues/1631). {#v0-35-0-added} ### Added * (rules) `compile_pip_requirements` supports multiple requirements input files as `srcs`. * (rules) `PYTHONSAFEPATH` is inherited from the calling environment to allow disabling it (Requires {obj}`--bootstrap_impl=script`) - ([#2060](https://github.com/bazelbuild/rules_python/issues/2060)). + ([#2060](https://github.com/bazel-contrib/rules_python/issues/2060)). * (gazelle) Added `python_generation_mode_per_package_require_test_entry_point` in order to better accommodate users who use a custom macro, [`pytest-bazel`][pytest_bazel], [rules_python_pytest] or `rules_py` @@ -531,7 +1828,7 @@ Other changes: {#v0-34-0} ## [0.34.0] - 2024-07-04 -[0.34.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.34.0 +[0.34.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.34.0 {#v0-34-0-changed} ### Changed @@ -572,7 +1869,7 @@ Other changes: and drop the defaults from the lock file. * (whl_library) Correctly handle arch-specific dependencies when we encounter a platform specific wheel and use `experimental_target_platforms`. - Fixes [#1996](https://github.com/bazelbuild/rules_python/issues/1996). + Fixes [#1996](https://github.com/bazel-contrib/rules_python/issues/1996). * (rules) The first element of the default outputs is now the executable again. * (pip) Fixed crash when pypi packages lacked a sha (e.g. yanked packages) @@ -582,7 +1879,7 @@ Other changes: replacement for the "autodetecting" toolchain. * (gazelle) Added new `python_label_convention` and `python_label_normalization` directives. These directive allows altering default Gazelle label format to third-party dependencies useful for re-using Gazelle plugin - with other rules, including `rules_pycross`. See [#1939](https://github.com/bazelbuild/rules_python/issues/1939). + with other rules, including `rules_pycross`. See [#1939](https://github.com/bazel-contrib/rules_python/issues/1939). {#v0-34-0-removed} ### Removed @@ -591,7 +1888,7 @@ Other changes: {#v0-33-2} ## [0.33.2] - 2024-06-13 -[0.33.2]: https://github.com/bazelbuild/rules_python/releases/tag/0.33.2 +[0.33.2]: https://github.com/bazel-contrib/rules_python/releases/tag/0.33.2 {#v0-33-2-fixed} ### Fixed @@ -599,22 +1896,22 @@ Other changes: To enable it, set {obj}`--//python/config_settings:exec_tools_toolchain=enabled`. This toolchain must be enabled for precompilation to work. This toolchain will be enabled by default in a future release. - Fixes [#1967](https://github.com/bazelbuild/rules_python/issues/1967). + Fixes [#1967](https://github.com/bazel-contrib/rules_python/issues/1967). {#v0-33-1} ## [0.33.1] - 2024-06-13 -[0.33.1]: https://github.com/bazelbuild/rules_python/releases/tag/0.33.1 +[0.33.1]: https://github.com/bazel-contrib/rules_python/releases/tag/0.33.1 {#v0-33-1-fixed} ### Fixed * (py_binary) Fix building of zip file when using `--build_python_zip` - argument. Fixes [#1954](https://github.com/bazelbuild/rules_python/issues/1954). + argument. Fixes [#1954](https://github.com/bazel-contrib/rules_python/issues/1954). {#v0-33-0} ## [0.33.0] - 2024-06-12 -[0.33.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.33.0 +[0.33.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.33.0 {#v0-33-0-changed} ### Changed @@ -634,8 +1931,8 @@ Other changes: * (pip.parse): Add references to all supported wheels when using `experimental_index_url` to allowing to correctly fetch the wheels for the right platform. See the updated docs on how to use the feature. This is work towards addressing - [#735](https://github.com/bazelbuild/rules_python/issues/735) and - [#260](https://github.com/bazelbuild/rules_python/issues/260). The spoke + [#735](https://github.com/bazel-contrib/rules_python/issues/735) and + [#260](https://github.com/bazel-contrib/rules_python/issues/260). The spoke repository names when using this flag will have a structure of `{pip_hub_prefix}_{wheel_name}_{py_tag}_{abi_tag}_{platform_tag}_{sha256}`, which is an implementation detail which should not be relied on and is there @@ -661,13 +1958,13 @@ Other changes: * (bzlmod) remove `pip.parse(annotations)` attribute as it is unused and has been replaced by whl_modifications. * (pip) Correctly select wheels when the python tag includes minor versions. - See ([#1930](https://github.com/bazelbuild/rules_python/issues/1930)) + See ([#1930](https://github.com/bazel-contrib/rules_python/issues/1930)) * (pip.parse): The lock file is now reproducible on any host platform if the `experimental_index_url` is not used by any of the modules in the dependency chain. To make the lock file identical on each `os` and `arch`, please use the `experimental_index_url` feature which will fetch metadata from PyPI or a different private index and write the contents to the lock file. Fixes - [#1643](https://github.com/bazelbuild/rules_python/issues/1643). + [#1643](https://github.com/bazel-contrib/rules_python/issues/1643). * (pip.parse): Install `yanked` packages and print a warning instead of ignoring them. This better matches the behaviour of `uv pip install`. * (toolchains): Now matching of the default hermetic toolchain is more robust @@ -676,7 +1973,7 @@ Other changes: to toolchain selection failures when the python toolchain is not registered, but is requested via `//python/config_settings:python_version` flag setting. * (doc) Fix the `WORKSPACE` requirement vendoring example. Fixes - [#1918](https://github.com/bazelbuild/rules_python/issues/1918). + [#1918](https://github.com/bazel-contrib/rules_python/issues/1918). {#v0-33-0-added} ### Added @@ -687,7 +1984,7 @@ Other changes: [Precompiling docs][precompile-docs] and API reference docs for more information on precompiling. Note this requires Bazel 7+ and the Pystar rule implementation enabled. - ([#1761](https://github.com/bazelbuild/rules_python/issues/1761)) + ([#1761](https://github.com/bazel-contrib/rules_python/issues/1761)) * (rules) Attributes and flags to control precompile behavior: `precompile`, `precompile_optimize_level`, `precompile_source_retention`, `precompile_invalidation_mode`, and `pyc_collection` @@ -713,7 +2010,7 @@ Other changes: is available. It can be enabled by setting {obj}`--@rules_python//python/config_settings:bootstrap_impl=script`. It will become the default in a subsequent release. - ([#691](https://github.com/bazelbuild/rules_python/issues/691)) + ([#691](https://github.com/bazel-contrib/rules_python/issues/691)) * (providers) `PyRuntimeInfo` has two new attributes: {obj}`PyRuntimeInfo.stage2_bootstrap_template` and {obj}`PyRuntimeInfo.zip_main_template`. @@ -735,7 +2032,7 @@ Other changes: {#v0-32-2} ## [0.32.2] - 2024-05-14 -[0.32.2]: https://github.com/bazelbuild/rules_python/releases/tag/0.32.2 +[0.32.2]: https://github.com/bazel-contrib/rules_python/releases/tag/0.32.2 {#v0-32-2-fixed} ### Fixed @@ -743,12 +2040,12 @@ Other changes: * Workaround existence of infinite symlink loops on case insensitive filesystems when targeting linux platforms with recent Python toolchains. Works around an upstream [issue][indygreg-231]. Fixes [#1800][rules_python_1800]. [indygreg-231]: https://github.com/indygreg/python-build-standalone/issues/231 -[rules_python_1800]: https://github.com/bazelbuild/rules_python/issues/1800 +[rules_python_1800]: https://github.com/bazel-contrib/rules_python/issues/1800 {#v0-32-0} ## [0.32.0] - 2024-05-12 -[0.32.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.32.0 +[0.32.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.32.0 {#v0-32-0-changed} ### Changed @@ -773,22 +2070,22 @@ Other changes: * (whl_library): Fix the experimental_target_platforms overriding for platform specific wheels when the wheels are for any python interpreter version. Fixes - [#1810](https://github.com/bazelbuild/rules_python/issues/1810). + [#1810](https://github.com/bazel-contrib/rules_python/issues/1810). * (whl_library): Stop generating duplicate dependencies when encountering duplicates in the METADATA. Fixes - [#1873](https://github.com/bazelbuild/rules_python/issues/1873). + [#1873](https://github.com/bazel-contrib/rules_python/issues/1873). * (gazelle) In `project` or `package` generation modes, do not generate `py_test` rules when there are no test files and do not set `main = "__test__.py"` when that file doesn't exist. * (whl_library) The group redirection is only added when the package is part of the group potentially fixing aspects that want to traverse a `py_library` graph. - Fixes [#1760](https://github.com/bazelbuild/rules_python/issues/1760). + Fixes [#1760](https://github.com/bazel-contrib/rules_python/issues/1760). * (bzlmod) Setting a particular micro version for the interpreter and the `pip.parse` extension is now possible, see the `examples/pip_parse/MODULE.bazel` for how to do it. - See [#1371](https://github.com/bazelbuild/rules_python/issues/1371). + See [#1371](https://github.com/bazel-contrib/rules_python/issues/1371). * (refactor) The pre-commit developer workflow should now pass `isort` and `black` - checks (see [#1674](https://github.com/bazelbuild/rules_python/issues/1674)). + checks (see [#1674](https://github.com/bazel-contrib/rules_python/issues/1674)). ### Added @@ -806,13 +2103,13 @@ Other changes: [original issue][test_file_pattern_issue] and the [docs][test_file_pattern_docs] for details. * (wheel) Add support for `data_files` attributes in py_wheel rule - ([#1777](https://github.com/bazelbuild/rules_python/issues/1777)) + ([#1777](https://github.com/bazel-contrib/rules_python/issues/1777)) * (py_wheel) `bzlmod` installations now provide a `twine` setup for the default Python toolchain in `rules_python` for version 3.11. * (bzlmod) New `experimental_index_url`, `experimental_extra_index_urls` and `experimental_index_url_overrides` to `pip.parse` for using the bazel downloader. If you see any issues, report in - [#1357](https://github.com/bazelbuild/rules_python/issues/1357). The URLs for + [#1357](https://github.com/bazel-contrib/rules_python/issues/1357). The URLs for the whl and sdist files will be written to the lock file. Controlling whether the downloading of metadata is done in parallel can be done using `parallel_download` attribute. @@ -827,16 +2124,16 @@ Other changes: depend on legacy labels instead of the hub repo aliases and you use the `experimental_requirement_cycles`, now is a good time to migrate. -[python_default_visibility]: gazelle/README.md#directive-python_default_visibility -[test_file_pattern_issue]: https://github.com/bazelbuild/rules_python/issues/1816 -[test_file_pattern_docs]: gazelle/README.md#directive-python_test_file_pattern +[python_default_visibility]: https://github.com/bazel-contrib/rules_python/tree/main/gazelle/README.md#directive-python_default_visibility +[test_file_pattern_issue]: https://github.com/bazel-contrib/rules_python/issues/1816 +[test_file_pattern_docs]: https://github.com/bazel-contrib/rules_python/tree/main/gazelle/README.md#directive-python_test_file_pattern [20240224]: https://github.com/indygreg/python-build-standalone/releases/tag/20240224. [20240415]: https://github.com/indygreg/python-build-standalone/releases/tag/20240415. ## [0.31.0] - 2024-02-12 -[0.31.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.31.0 +[0.31.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.31.0 ### Changed @@ -848,7 +2145,7 @@ Other changes: ## [0.30.0] - 2024-02-12 -[0.30.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.30.0 +[0.30.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.30.0 ### Changed @@ -880,7 +2177,7 @@ Other changes: * (PyRuntimeInfo) Switch back to builtin PyRuntimeInfo for Bazel 6.4 and when pystar is disabled. This fixes an error about `target ... does not have ... PyRuntimeInfo`. - ([#1732](https://github.com/bazelbuild/rules_python/issues/1732)) + ([#1732](https://github.com/bazel-contrib/rules_python/issues/1732)) ### Added @@ -896,7 +2193,7 @@ Other changes: `common --@rules_python//python/config_settings:python_version=X.Y.Z`. * New Python versions available: `3.11.7`, `3.12.1` using - https://github.com/indygreg/python-build-standalone/releases/tag/20240107. + . * (toolchain) Allow setting `x.y` as the `python_version` parameter in the version-aware `py_binary` and `py_test` rules. This allows users to @@ -922,7 +2219,7 @@ Other changes: ## [0.29.0] - 2024-01-22 -[0.29.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.29.0 +[0.29.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.29.0 ### Changed @@ -942,7 +2239,7 @@ Other changes: * (bzlmod pip.parse) Use a platform-independent reference to the interpreter pip uses. This reduces (but doesn't eliminate) the amount of platform-specific content in `MODULE.bazel.lock` files; Follow - [#1643](https://github.com/bazelbuild/rules_python/issues/1643) for removing + [#1643](https://github.com/bazel-contrib/rules_python/issues/1643) for removing platform-specific content in `MODULE.bazel.lock` files. * (wheel) The stamp variables inside the distribution name are no longer @@ -974,7 +2271,7 @@ Other changes: ## [0.28.0] - 2024-01-07 -[0.28.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.28.0 +[0.28.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.28.0 ### Changed @@ -983,7 +2280,7 @@ Other changes: * **BREAKING** Support for Bazel 5 has been officially dropped. This release was only partially tested with Bazel 5 and may or may not work with Bazel 5. - Subequent versions will no longer be tested under Bazel 5. + Subsequent versions will no longer be tested under Bazel 5. * (runfiles) `rules_python.python.runfiles` now directly implements type hints and drops support for python2 as a result. @@ -1000,7 +2297,7 @@ Other changes: * (toolchains) `py_runtime` can now take an executable target. Note: runfiles from the target are not supported yet. - ([#1612](https://github.com/bazelbuild/rules_python/issues/1612)) + ([#1612](https://github.com/bazel-contrib/rules_python/issues/1612)) * (gazelle) When `python_generation_mode` is set to `file`, create one `py_binary` target for each file with `if __name__ == "__main__"` instead of just one @@ -1027,7 +2324,7 @@ Other changes: package (e.g. one for the package, one for an extra) now work. * (bzlmod python.toolchain) Submodules can now (re)register the Python version that rules_python has set as the default. - ([#1638](https://github.com/bazelbuild/rules_python/issues/1638)) + ([#1638](https://github.com/bazel-contrib/rules_python/issues/1638)) * (whl_library) Actually use the provided patches to patch the whl_library. On Windows the patching may result in files with CRLF line endings, as a result the RECORD file consistency requirement is lifted and now a warning is emitted @@ -1036,26 +2333,26 @@ Other changes: file if you decide to do so. * (coverage): coverage reports are now created when the version-aware rules are used. - ([#1600](https://github.com/bazelbuild/rules_python/issues/1600)) + ([#1600](https://github.com/bazel-contrib/rules_python/issues/1600)) * (toolchains) Workspace builds register the py cc toolchain (bzlmod already was). This makes e.g. `//python/cc:current_py_cc_headers` Just Work. - ([#1669](https://github.com/bazelbuild/rules_python/issues/1669)) + ([#1669](https://github.com/bazel-contrib/rules_python/issues/1669)) * (bzlmod python.toolchain) The value of `ignore_root_user_error` is now decided by the root module only. - ([#1658](https://github.com/bazelbuild/rules_python/issues/1658)) + ([#1658](https://github.com/bazel-contrib/rules_python/issues/1658)) ### Added * (docs) bzlmod extensions are now documented on rules-python.readthedocs.io * (docs) Support and backwards compatibility policies have been documented. - See https://rules-python.readthedocs.io/en/latest/support.html + See * (gazelle) `file` generation mode can now also add `__init__.py` to the srcs attribute for every target in the package. This is enabled through a separate directive `python_generation_mode_per_file_include_init`. ## [0.27.0] - 2023-11-16 -[0.27.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.27.0 +[0.27.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.27.0 ### Changed @@ -1180,7 +2477,7 @@ Breaking changes: the `py_binary` rule used to build it. * New Python versions available: `3.8.17`, `3.11.5` using - https://github.com/indygreg/python-build-standalone/releases/tag/20230826. + . * (gazelle) New `# gazelle:python_generation_mode file` directive to support generating one `py_library` per file. @@ -1200,7 +2497,7 @@ Breaking changes: time being. * New Python versions available: `3.8.18`, `3.9.18`, `3.10.13`, `3.11.6`, `3.12.0` using - https://github.com/indygreg/python-build-standalone/releases/tag/20231002. + . `3.12.0` support is considered beta and may have issues. ### Removed @@ -1221,7 +2518,7 @@ Breaking changes: * (gazelle) Improve runfiles lookup hermeticity. -[0.26.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.26.0 +[0.26.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.26.0 ## [0.25.0] - 2023-08-22 @@ -1249,7 +2546,7 @@ Breaking changes: * (gazelle) Stop generating unnecessary imports. * (toolchains) s390x supported for Python 3.9.17, 3.10.12, and 3.11.4. -[0.25.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.25.0 +[0.25.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.25.0 ## [0.24.0] - 2023-07-11 @@ -1285,4 +2582,4 @@ Breaking changes: * (pip) Create all_data_requirements alias * Expose Python C headers through the toolchain. -[0.24.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.24.0 +[0.24.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.24.0 \ No newline at end of file diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d5f24a9365..9efce5a148 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -3,6 +3,21 @@ We'd love to accept your patches and contributions to this project. There are just a few small guidelines you need to follow. +## Contributor License Agreement + +First, the most important step: signing the Contributor License Agreement. We +cannot look at any of your code unless one is signed. + +Contributions to this project must be accompanied by a Contributor License +Agreement. You (or your employer) retain the copyright to your contribution, +this simply gives us permission to use and redistribute your contributions as +part of the project. Head over to to see +your current agreements on file or to sign a new one. + +You generally only need to submit a CLA once, so if you've already submitted one +(even if it was for a different project), you probably don't need to do it +again. + ## Getting started Before we can work on the code, we need to get a copy of it and setup some @@ -15,7 +30,7 @@ the [GitHub `gh` tool](https://github.com/cli/cli) (More advanced users may prefer the GitHub UI and raw `git` commands). ```shell -gh repo fork bazelbuild/rules_python --clone --remote +gh repo fork bazel-contrib/rules_python --clone --remote ``` Next, make sure you have a new enough version of Python installed that supports the @@ -50,29 +65,10 @@ git push origin my-feature Once the code is in your github repo, you can then turn it into a Pull Request to the actual rules_python project and begin the code review process. +## Developer guide -## Running tests - -Running tests is particularly easy thanks to Bazel, simply run: - -``` -bazel test //... -``` - -And it will run all the tests it can find. The first time you do this, it will -probably take long time because various dependencies will need to be downloaded -and setup. Subsequent runs will be faster, but there are many tests, and some of -them are slow. If you're working on a particular area of code, you can run just -the tests in those directories instead, which can speed up your edit-run cycle. - -## Updating tool dependencies - -It's suggested to routinely update the tool versions within our repo - some of the -tools are using requirement files compiled by `uv` and others use other means. In order -to have everything self-documented, we have a special target - -`//private:requirements.update`, which uses `rules_multirun` to run in sequence all -of the requirement updating scripts in one go. This can be done once per release as -we prepare for releases. +For more more details, guidance, and tips for working with the code base, +see [docs/devguide.md](./devguide) ## Formatting @@ -99,18 +95,6 @@ $ buildifier --lint=fix --warnings=native-py -warnings=all WORKSPACE Replace the argument "WORKSPACE" with the file that you are linting. -## Contributor License Agreement - -Contributions to this project must be accompanied by a Contributor License -Agreement. You (or your employer) retain the copyright to your contribution, -this simply gives us permission to use and redistribute your contributions as -part of the project. Head over to to see -your current agreements on file or to sign a new one. - -You generally only need to submit a CLA once, so if you've already submitted one -(even if it was for a different project), you probably don't need to do it -again. - ## Code reviews All submissions, including submissions by project members, require review. We @@ -119,7 +103,7 @@ information on using pull requests. [GitHub Help]: https://help.github.com/articles/about-pull-requests/ -### Commit messages +### Commit messages and PR descriptions Commit messages (upon merging) and PR messages should follow the [Conventional Commits](https://www.conventionalcommits.org/) style: @@ -137,19 +121,154 @@ If a breaking change is introduced, then `BREAKING CHANGE:` is required; see the [Breaking Changes](#breaking-changes) section for how to introduce breaking changes. +User visible changes, such as features, fixes, or notable refactors, should +be documneted in CHANGELOG.md and their respective API doc. See [Documenting +changes] for how to do so. + Common `type`s: * `build:` means it affects the building or development workflow. * `docs:` means only documentation is being added, updated, or fixed. -* `feat:` means a user-visible feature is being added. -* `fix:` means a user-visible behavior is being fixed. -* `refactor:` means some sort of code cleanup that doesn't change user-visible behavior. +* `feat:` means a user-visible feature is being added. See [Documenting version + changes] for how to documenAdd `{versionadded}` + to appropriate docs. +* `fix:` means a user-visible behavior is being fixed. If the fix is changing + behavior of a function, add `{versionchanged}` to appropriate docs, as necessary. +* `refactor:` means some sort of code cleanup that doesn't change user-visible + behavior. Add `{versionchanged}` to appropriate docs, as necessary. * `revert:` means a prior change is being reverted in some way. * `test:` means only tests are being added. +For the body, follow this guidance: + +* Briefly tells *why* the change is being made. This usually means + briefly describing how a bug manifests or what can't be accomplished + without the feature. +* Briefly gives an overview of *how* the code is changed. This is to + orient readers for the diff they're about to read and understand; it's + not a verbatim description of what changed. +* List unrelated or notable dev-only changes at the end. e.g. formatting an + old file, cleaning up testing, adding test support code, etc. + For the full details of types, see [Conventional Commits](https://www.conventionalcommits.org/). +#### PR description example + +``` +fix(pypi): handle files with .exe extensions + +Currently, if a file with `.exe` is seen, an error +occurs because validation assumes unix-only filenames. +This prevents using whls with pre-built .exe files in +their data payload. + +To fix, detect the target OS and use an OS-appropriate +validation function. + +* Also adds test helpers for detecting the current OS +``` + +### Documenting changes + +Changes are documented in two places: news entries and API docs. + +Instead of modifying `CHANGELOG.md` directly, you should create a news entry file in the `news/` directory. These files are automatically assembled into `CHANGELOG.md` at release time. + +#### Creating a news entry + +Create a `.md` file in the `news/` directory. The filename must follow the format `..md`: + +* ``: A unique identifier, typically the GitHub Pull Request number or Issue number (e.g., `1234`). +* ``: The category of the change, which must be one of: + * `added`: For new features or behavior added in a backwards-compatible manner. + * `changed`: For changes in existing behavior. + * `fixed`: For bug fixes. + * `removed`: For removed features or behavior. + +The content of the file should be a brief, human-friendly description of the change. Do not include a leading bullet point (e.g. `*` or `-`), as this is automatically added during assembly. If your change is specific to a subsystem, prefix it with the subsystem in parentheses, e.g., `(gazelle) Fixed handling of...`. + +Example: `news/1234.fixed.md` +```markdown +(gazelle) Fixed handling of auto-included `__init__.py` files when generating `py_binary` targets. +``` + +Do not edit `CHANGELOG.md` directly for unreleased changes. + +#### API documentation + +API documentation are the doc strings for functions, fields, attributes, etc. +When user-visible or notable behavior is added, changed, or removed, the +`{versionadded}`, `{versionchanged}` or `{versionremoved}` directives should be +used to note the change. When specifying the version, use the values +`VERSION_NEXT_FEATURE` or `VERSION_NEXT_PATCH` to indicate what sort of +version increase the change requires. + +These directives use Sphinx MyST syntax, e.g. + +``` +:::{versionadded} VERSION_NEXT_FEATURE +The `allow_new_thing` arg was added. +::: + +:::{versionchanged} VERSION_NEXT_PATCH +Large numbers no longer consume exponential memory. +::: + +:::{versionremoved} VERSION_NEXT_FEATURE +The `legacy_foo` arg was removed +::: +``` + +## Style and idioms + +For the most part, we just accept whatever the code formatters do, so there +isn't much style to enforce. + +Some miscellanous style, idioms, and conventions we have are: + +### Markdown/Sphinx Style + +* Use colons for prose sections of text, e.g. `:::{note}`, not backticks. +* Use backticks for code blocks. +* Max line length: 100. + +### BUILD/bzl Style + +* When a macro generates public targets, use a dot (`.`) to separate the + user-provided name from the generted name. e.g. `foo(name="x")` generates + `x.test`. The `.` is our convention to communicate that it's a generated + target, and thus one should look for `name="x"` when searching for the + definition. +* The different build phases shouldn't load code that defines objects that + aren't valid for their phase. e.g. + * The bzlmod phase shouldn't load code defining regular rules or providers. + * The repository phase shouldn't load code defining module extensions, regular + rules, or providers. + * The loading phase shouldn't load code defining module extensions or + repository rules. + * Loading utility libraries or generic code is OK, but should strive to load + code that is usable for its phase. e.g. loading-phase code shouldn't + load utility code that is predominately only usable to the bzlmod phase. +* Providers should be in their own files. This allows implementing a custom rule + that implements the provider without loading a specific implementation. +* One rule per file is preferred, but not required. The goal is that defining an + e.g. library shouldn't incur loading all the code for binaries, tests, + packaging, etc; things that may be niche or uncommonly used. +* Separate files should be used to expose public APIs. This ensures our public + API is well defined and prevents accidentally exposing a package-private + symbol as a public symbol. + + :::{note} + The public API file's docstring becomes part of the user-facing docs. That + file's docstring must be used for module-level API documentation. + ::: +* Repository rules should have name ending in `_repo`. This helps distinguish + them from regular rules. +* Each bzlmod extension, the "X" of `use_repo("//foo:foo.bzl", "X")` should be + in its own file. The path given in the `use_repo()` expression is the identity + Bazel uses and cannot be changed. + ## Generated files Some checked-in files are generated and need to be updated when a new PR is @@ -157,32 +276,18 @@ merged: * **requirements lock files**: These are usually generated by a `compile_pip_requirements` update target, which is usually in the same directory. - e.g. `bazel run //docs:requirements.update` - -## Core rules - -The bulk of this repo is owned and maintained by the Bazel Python community. -However, since the core Python rules (`py_binary` and friends) are still -bundled with Bazel itself, the Bazel team retains ownership of their stubs in -this repository. This will be the case at least until the Python rules are -fully migrated to Starlark code. - -Practically, this means that a Bazel team member should approve any PR -concerning the core Python logic. This includes everything under the `python/` -directory except for `pip.bzl` and `requirements.txt`. + e.g. `bazel run //dev:requirements.update` -Issues should be triaged as follows: +## Binary artifacts -- Anything concerning the way Bazel implements the core Python rules should be - filed under [bazelbuild/bazel](https://github.com/bazelbuild/bazel), using - the label `team-Rules-python`. +Checking in binary artifacts is not allowed. This is because they are extremely +problematic to verify and ensure they're safe. This is true even in +test contexts. -- If the issue specifically concerns the rules_python stubs, it should be filed - here in this repository and use the label `core-rules`. +Examples include, but aren't limited to: prebuilt binaries, shared libraries, +zip files, or wheels. -- Anything else, such as feature requests not related to existing core rules - functionality, should also be filed in this repository but without the - `core-rules` label. +See the dev guide for utilities to help with testing. (breaking-changes)= ## Breaking Changes @@ -262,6 +367,25 @@ Not breaking changes: * Changing internal details, such as renaming an internal file. * Changing a rule to a macro. +## AI-assisted Contributions + +Contributions assisted by AI tools are allowed. However, the human author +submitting the pull request is responsible for the contributed code as if they +had written it entirely themselves. This means: + +* **Understanding the code:** You must be able to explain what the code does + and why it's implemented that way. This includes discussing its + implications, and any trade-offs made during its development, just as if you + had written it entirely yourself. +* **Vetting the correctness and functionality:** You are responsible for + thoroughly testing and verifying that the code is correct, functional, and + meets all project requirements and standards. + +If the human PR author cannot fulfill these responsibilities, the `rules_python` +maintainers will not spend time reviewing or merging the PR. The goal is to +ensure that all contributions, regardless of their origin, maintain the quality +and integrity of the project and do not place an undue burden on maintainers. + ## FAQ ### Installation errors when during `git commit` diff --git a/DEVELOPING.md b/DEVELOPING.md deleted file mode 100644 index d816fba57f..0000000000 --- a/DEVELOPING.md +++ /dev/null @@ -1,79 +0,0 @@ -# For Developers - -## Updating internal dependencies - -1. Modify the `./python/private/pypi/requirements.txt` file and run: - ``` - bazel run //private:whl_library_requirements.update - ``` -1. Run the following target to update `twine` dependencies: - ``` - bazel run //private:requirements.update - ``` -1. Bump the coverage dependencies using the script using: - ``` - bazel run //tools/private/update_deps:update_coverage_deps - # for example: - # bazel run //tools/private/update_deps:update_coverage_deps 7.6.1 - ``` - -## Releasing - -Start from a clean checkout at `main`. - -Before running through the release it's good to run the build and the tests locally, and make sure CI is passing. You can -also test-drive the commit in an existing Bazel workspace to sanity check functionality. - -### Releasing from HEAD - -#### Steps -1. [Determine the next semantic version number](#determining-semantic-version) -1. Create a tag and push, e.g. `git tag 0.5.0 upstream/main && git push upstream --tags` - NOTE: Pushing the tag will trigger release automation. -1. Watch the release automation run on https://github.com/bazelbuild/rules_python/actions -1. Add missing information to the release notes. The automatic release note - generation only includes commits associated with issues. - -#### Determining Semantic Version - -**rules_python** is currently using [Zero-based versioning](https://0ver.org/) and thus backwards-incompatible API -changes still come under the minor-version digit. So releases with API changes and new features bump the minor, and -those with only bug fixes and other minor changes bump the patch digit. - -To find if there were any features added or incompatible changes made, review -the commit history. This can be done using github by going to the url: -`https://github.com/bazelbuild/rules_python/compare/...main`. - -### Patch release with cherry picks - -If a patch release from head would contain changes that aren't appropriate for -a patch release, then the patch release needs to be based on the original -release tag and the patch changes cherry-picked into it. - -In this example, release `0.37.0` is being patched to create release `0.37.1`. -The fix being included is commit `deadbeef`. - -1. `git checkout -b release/0.37 0.37.0` -1. `git push upstream release/0.37` -1. `git cherry-pick -x deadbeef` -1. Fix merge conflicts, if any. -1. `git cherry-pick --continue` (if applicable) -1. `git push upstream` - -If multiple commits need to be applied, repeat the `git cherry-pick` step for -each. - -Once the release branch is in the desired state, use `git tag` to tag it, as -done with a release from head. Release automation will do the rest. - -#### After release creation in Github - -1. Announce the release in the #python channel in the Bazel slack (bazelbuild.slack.com). - -## Secrets - -### PyPI user rules-python - -Part of the release process uploads packages to PyPI as the user `rules-python`. -This account is managed by Google; contact rules-python-pyi@google.com if -something needs to be done with the PyPI account. diff --git a/GEMINI.md b/GEMINI.md new file mode 100644 index 0000000000..285e0f5b36 --- /dev/null +++ b/GEMINI.md @@ -0,0 +1 @@ +@./AGENTS.md diff --git a/MODULE.bazel b/MODULE.bazel index 2ae3173094..027277ab64 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -5,17 +5,19 @@ module( ) bazel_dep(name = "bazel_features", version = "1.21.0") -bazel_dep(name = "bazel_skylib", version = "1.7.1") -bazel_dep(name = "rules_cc", version = "0.0.16") -bazel_dep(name = "platforms", version = "0.0.4") +bazel_dep(name = "bazel_skylib", version = "1.8.2") +bazel_dep(name = "package_metadata", version = "0.0.7") +bazel_dep(name = "platforms", version = "0.0.11") +bazel_dep(name = "rules_cc", version = "0.2.17") +bazel_dep(name = "toml.bzl", version = "0.4.1") # Those are loaded only when using py_proto_library -bazel_dep(name = "rules_proto", version = "7.0.2") +# Use py_proto_library directly from protobuf repository bazel_dep(name = "protobuf", version = "29.0-rc2", repo_name = "com_google_protobuf") -internal_deps = use_extension("//python/private:internal_deps.bzl", "internal_deps") +rules_python_config = use_extension("//python/extensions:config.bzl", "config") use_repo( - internal_deps, + rules_python_config, "pypi__build", "pypi__click", "pypi__colorama", @@ -42,11 +44,20 @@ python = use_extension("//python/extensions:python.bzl", "python") # NOTE: This is not a stable version. It is provided for convenience, but will # change frequently to track the most recent Python version. # NOTE: The root module can override this. +# NOTE: There must be a corresponding `python.toolchain()` call for the version +# specified here. +python.defaults( + python_version = "3.11", +) python.toolchain( - is_default = True, python_version = "3.11", ) -use_repo(python, "python_3_11", "python_versions", "pythons_hub") +use_repo( + python, + "python_3_11", + "pythons_hub", + python = "python_versions", +) # This call registers the Python toolchains. register_toolchains("@pythons_hub//:all") @@ -56,10 +67,6 @@ register_toolchains("@pythons_hub//:all") pip = use_extension("//python/extensions:pip.bzl", "pip") pip.parse( - # NOTE @aignas 2024-10-26: We have an integration test that depends on us - # being able to build sdists for this hub, so explicitly set this to False. - download_only = False, - experimental_index_url = "https://pypi.org/simple", hub_name = "rules_python_publish_deps", python_version = "3.11", requirements_by_platform = { @@ -70,34 +77,84 @@ pip.parse( ) use_repo(pip, "rules_python_publish_deps") -# Not a dev dependency to allow usage of //sphinxdocs code, which refers to stardoc repos. -bazel_dep(name = "stardoc", version = "0.7.2", repo_name = "io_bazel_stardoc") - # ===== DEV ONLY DEPS AND SETUP BELOW HERE ===== +bazel_dep(name = "sphinxdocs", version = "0.0.0", dev_dependency = True) +local_path_override( + module_name = "sphinxdocs", + path = "sphinxdocs", +) + bazel_dep(name = "rules_bazel_integration_test", version = "0.27.0", dev_dependency = True) bazel_dep(name = "rules_testing", version = "0.6.0", dev_dependency = True) bazel_dep(name = "rules_shell", version = "0.3.0", dev_dependency = True) bazel_dep(name = "rules_multirun", version = "0.9.0", dev_dependency = True) bazel_dep(name = "bazel_ci_rules", version = "1.0.0", dev_dependency = True) +bazel_dep(name = "rules_pkg", version = "1.2.0", dev_dependency = True) +bazel_dep(name = "other", version = "0", dev_dependency = True) +bazel_dep(name = "another_module", version = "0", dev_dependency = True) # Extra gazelle plugin deps so that WORKSPACE.bzlmod can continue including it for e2e tests. # We use `WORKSPACE.bzlmod` because it is impossible to have dev-only local overrides. -bazel_dep(name = "rules_go", version = "0.41.0", dev_dependency = True, repo_name = "io_bazel_rules_go") -bazel_dep(name = "rules_python_gazelle_plugin", version = "0", dev_dependency = True) +bazel_dep(name = "rules_go", version = "0.60.0", dev_dependency = True, repo_name = "io_bazel_rules_go") bazel_dep(name = "gazelle", version = "0.40.0", dev_dependency = True, repo_name = "bazel_gazelle") +bazel_dep(name = "bazel_skylib_gazelle_plugin", version = "1.8.2", dev_dependency = True) +bazel_dep(name = "rules_pyrefly", version = "0.1.0", dev_dependency = True) + +pyrefly = use_extension( + "@rules_pyrefly//pyrefly:extensions.bzl", + "pyrefly", + dev_dependency = True, +) +pyrefly.toolchain(version = "1.2.0") +use_repo(pyrefly, "pyrefly_toolchains") + +register_toolchains( + "@pyrefly_toolchains//:all", + dev_dependency = True, +) internal_dev_deps = use_extension( "//python/private:internal_dev_deps.bzl", "internal_dev_deps", dev_dependency = True, ) -use_repo(internal_dev_deps, "buildkite_config", "wheel_for_testing") +use_repo( + internal_dev_deps, + "buildkite_config", + "implicit_namespace_ns_sub1", + "implicit_namespace_ns_sub2", + "patch_whl_pkg", + "pkgutil_nspkg1", + "pkgutil_nspkg2", + "rules_python_runtime_env_tc_info", + "somepkg_with_build_files", + "whl_library_extras_direct_dep", + "whl_with_build_files", + "whl_with_data1", + "whl_with_data2", +) + +dev_rules_python_config = use_extension( + "//python/extensions:config.bzl", + "config", + dev_dependency = True, +) +dev_rules_python_config.add_transition_setting( + # Intentionally add a setting already present for testing + setting = "//python/config_settings:python_version", +) +dev_rules_python_config.add_transition_setting( + setting = "//tests/multi_pypi:external_deps_name", +) -# Add gazelle plugin so that we can run the gazelle example as an e2e integration -# test and include the distribution files. local_path_override( - module_name = "rules_python_gazelle_plugin", - path = "gazelle", + module_name = "other", + path = "tests/modules/other", +) + +local_path_override( + module_name = "another_module", + path = "tests/modules/another_module", ) dev_python = use_extension( @@ -109,26 +166,69 @@ dev_python.override( register_all_versions = True, ) +# For testing an arbitrary runtime triggered by a custom flag. +# See //tests/toolchains:custom_platform_toolchain_test +dev_python.single_version_platform_override( + platform = "linux-x86-install-only-stripped", + python_version = "3.13.1", + sha256 = "56817aa976e4886bec1677699c136cb01c1cdfe0495104c0d8ef546541864bbb", + target_compatible_with = [ + "@platforms//os:linux", + "@platforms//cpu:x86_64", + ], + target_settings = [ + "@@//tests/support:is_custom_runtime_linux-x86-install-only-stripped", + ], + urls = ["https://github.com/astral-sh/python-build-standalone/releases/download/20250115/cpython-3.13.1+20250115-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz"], +) + dev_pip = use_extension( "//python/extensions:pip.bzl", "pip", dev_dependency = True, ) + +[ + dev_pip.parse( + download_only = True, + hub_name = "dev_pip", + parallel_download = False, + python_version = python_version, + requirements_lock = "//dev:requirements.txt", + # Ensure that we are setting up the following platforms + target_platforms = [ + "{os}_{arch}", + "{os}_{arch}_freethreaded", + ], + uv_lock = "//dev:uv.lock", + ) + for python_version in [ + "3.9", + "3.10", + "3.11", + "3.12", + "3.13", + "3.14", + ] +] + dev_pip.parse( download_only = True, - experimental_index_url = "https://pypi.org/simple", - hub_name = "dev_pip", + hub_name = "pypiserver", python_version = "3.11", - requirements_lock = "//docs:requirements.txt", + requirements_lock = "//examples/wheel:requirements_server.txt", ) dev_pip.parse( - download_only = True, - experimental_index_url = "https://pypi.org/simple", - hub_name = "pypiserver", + hub_name = "pypi_alpha", python_version = "3.11", - requirements_lock = "//examples/wheel:requirements_server.txt", + requirements_lock = "//tests/multi_pypi/alpha:requirements.txt", ) -use_repo(dev_pip, "dev_pip", "pypiserver") +dev_pip.parse( + hub_name = "pypi_beta", + python_version = "3.11", + requirements_lock = "//tests/multi_pypi/beta:requirements.txt", +) +use_repo(dev_pip, "dev_pip", "pypi", "pypi_alpha", "pypi_beta", "pypiserver") # Bazel integration test setup below @@ -143,34 +243,119 @@ bazel_binaries.local( name = "self", path = "tests/integration/bazel_from_env", ) -bazel_binaries.download(version = "6.4.0") -bazel_binaries.download(version = "7.4.0") - -# For now, don't test with rolling, because that's Bazel 9, which is a ways -# away. -# bazel_binaries.download(version = "rolling") +bazel_binaries.download(version = "7.7.0") +bazel_binaries.download(version = "8.5.1") +bazel_binaries.download(version = "9.1.0") use_repo( bazel_binaries, "bazel_binaries", # These don't appear necessary, but are reported as direct dependencies # that should be use_repo()'d, so we add them as requested "bazel_binaries_bazelisk", - "build_bazel_bazel_6_4_0", - "build_bazel_bazel_7_4_0", + "build_bazel_bazel_7_7_0", + "build_bazel_bazel_8_5_1", + "build_bazel_bazel_9_1_0", # "build_bazel_bazel_rolling", "build_bazel_bazel_self", ) -# EXPERIMENTAL: This is experimental and may be removed without notice -uv = use_extension( - "//python/uv:extensions.bzl", +# TODO @aignas 2025-01-27: should this be moved to `//python/extensions:uv.bzl` or should +# it stay as it is? I think I may prefer to move it. +uv = use_extension("//python/uv:uv.bzl", "uv") + +# Here is how we can define platforms for the `uv` binaries - this will affect +# all of the downstream callers because we are using the extension without +# `dev_dependency = True`. +uv.default( + base_url = "https://github.com/astral-sh/uv/releases/download", + manifest_filename = "dist-manifest.json", + version = "0.11.2", +) +uv.default( + compatible_with = [ + "@platforms//os:macos", + "@platforms//cpu:aarch64", + ], + platform = "aarch64-apple-darwin", +) +uv.default( + compatible_with = [ + "@platforms//os:linux", + "@platforms//cpu:aarch64", + ], + platform = "aarch64-unknown-linux-gnu", +) +uv.default( + compatible_with = [ + "@platforms//os:linux", + "@platforms//cpu:ppc64le", + ], + platform = "powerpc64le-unknown-linux-gnu", +) +uv.default( + compatible_with = [ + "@platforms//os:linux", + "@platforms//cpu:s390x", + ], + platform = "s390x-unknown-linux-gnu", +) +uv.default( + compatible_with = [ + "@platforms//os:linux", + "@platforms//cpu:riscv64", + ], + platform = "riscv64-unknown-linux-gnu", +) +uv.default( + compatible_with = [ + "@platforms//os:macos", + "@platforms//cpu:x86_64", + ], + platform = "x86_64-apple-darwin", +) +uv.default( + compatible_with = [ + "@platforms//os:windows", + "@platforms//cpu:x86_64", + ], + platform = "x86_64-pc-windows-msvc", +) +uv.default( + compatible_with = [ + "@platforms//os:linux", + "@platforms//cpu:x86_64", + ], + platform = "x86_64-unknown-linux-gnu", +) +use_repo(uv, "uv") + +register_toolchains("@uv//:all") + +uv_dev = use_extension( + "//python/uv:uv.bzl", "uv", dev_dependency = True, ) -uv.toolchain(uv_version = "0.4.25") -use_repo(uv, "uv_toolchains") +uv_dev.configure( + version = "0.11.2", +) -register_toolchains( - "@uv_toolchains//:all", - dev_dependency = True, +flag_alias( + name = "build_python_zip", + starlark_flag = "//python/config_settings:build_python_zip", +) + +flag_alias( + name = "incompatible_default_to_explicit_init_py", + starlark_flag = "//python/config_settings:incompatible_default_to_explicit_init_py", +) + +flag_alias( + name = "python_path", + starlark_flag = "//python/config_settings:python_path", +) + +flag_alias( + name = "experimental_python_import_all_repositories", + starlark_flag = "//python/config_settings:experimental_python_import_all_repositories", ) diff --git a/README.md b/README.md index d890d702d6..7d805ce315 100644 --- a/README.md +++ b/README.md @@ -5,8 +5,9 @@ ## Overview This repository is the home of the core Python rules -- `py_library`, -`py_binary`, `py_test`, `py_proto_library`, and related symbols that provide the basis for Python -support in Bazel. It also contains package installation rules for integrating with PyPI and other indices. +`py_binary`, `py_test`, and related symbols that provide the basis for Python +support in Bazel. It also contains package installation rules for integrating +with PyPI and other indices. Documentation for rules_python is at and in the [Bazel Build Encyclopedia](https://docs.bazel.build/versions/master/be/python.html). @@ -17,7 +18,50 @@ The core rules are stable. Their implementation is subject to Bazel's [backward compatibility policy](https://docs.bazel.build/versions/master/backward-compatibility.html). This repository aims to follow [semantic versioning](https://semver.org). -The Bazel community maintains this repository. Neither Google nor the Bazel team provides support for the code. However, this repository is part of the test suite used to vet new Bazel releases. See [How to contribute](CONTRIBUTING.md) page for information on our development workflow. +The Bazel community maintains this repository. Neither Google nor the Bazel team +provides support for the code. However, this repository is part of the test +suite used to vet new Bazel releases. See [How to contribute](CONTRIBUTING.md) +page for information on our development workflow. + +## Design + +* Supported OSes - as per our supported platform policy, we strive for support + on all of the platforms that we have CI for. Some platforms do not have the + same backwards compatibility guarantees, but we hope the community can step in + where needed to make the support more robust. +* `requirements.txt` is how users have been defining dependencies for a long + time. We support this to support legacy usecases or package managers that we + don't support directly. Any additional information that we need will be + retrieved from the SimpleAPI during the `bzlmod` extension evaluation phase. + Then it will be written to the `MODULE.bazel.lock` file for future reuse. We + have plans to support `uv.lock` file directly. `uv` is recommended for + generating a fully locked `requirements.txt` file and we do provide a rule for + it. +* The `py_binary`, `py_test` rules should scale to large monorepos and we work + hard to minimize the work done during analysis and build phase. What is more, + the space requirements for should be minimal, so we strive to use symlinks + rather than extracting wheels at build time. This means that for different + configurations of the same build, we are not extracting the wheel multiple + times thus scaling better over the time. From `2.0` onwards we are creating a + virtual env for each target by creating an actual minimal virtual environment + using symlinks. We plan on creating the traditional `site-packages` layout in + the future by default. +* Support for standards - we strive to first implement any standards needed + within `rules_python` and this has resulted in a few PEPs supported within + pure starlark - PEP440, PEP509. + +Common misconceptions: +* `rules_python` has to keep backwards compatibility with `google3`. Whilst this + might have been true in the past, `rules_python` is an open source project and + any compatibility needs should come from the community - we have no + requirement to keep this compatibility and are allowed to make our decisions. + However, we do want to keep backwards compatibility as long as possible to not + upset users with never ending migrations. +* `rules_python` is not caching pip downloads. With 2.0, we use Bazel's + downloader by default and rely on bazel to provide the repository caching + mechanisms. This means that for simpler setups this should result in + transparent and scalable caching with the most recent bazel versions unless + there are issues in the bazel itself. ## Documentation @@ -25,7 +69,4 @@ For detailed documentation, see ## Bzlmod support -- Status: Beta -- Full Feature Parity: No - See [Bzlmod support](BZLMOD_SUPPORT.md) for more details. diff --git a/RELEASING.md b/RELEASING.md new file mode 100644 index 0000000000..f16b13e582 --- /dev/null +++ b/RELEASING.md @@ -0,0 +1,273 @@ +# Releasing + +Start from a clean checkout at `main`. + +Before running through the release it's good to run the build and the tests +locally, and make sure CI is passing. You can also test-drive the commit in an +existing Bazel workspace to sanity check functionality. + +## Releasing from HEAD + +Releases are managed using a semi-automated process centered around a GitHub +Release Tracking Issue and automated workflows triggered by comments or issue edits. + +> [!NOTE] +> Comment-based commands must be posted by project maintainers (Owner, +> Member, or Collaborator) and must be on their own line (leading and trailing +> whitespace is ignored). + +### Steps + +1. **Prepare the Release**: Run the [Release: Prepare](https://github.com/bazel-contrib/rules_python/actions/workflows/release_prepare.yaml) + workflow manually. You can trigger it from the GitHub Actions UI or using + the GitHub CLI: + ```shell + gh workflow run release_prepare.yaml --repo bazel-contrib/rules_python + ``` + This will automatically determine the next version, create a release tracking + issue, and send a preparation PR. + +2. **Approve and Merge**: Approve and merge the PR. Once merged, the + [Release: Prepare: Complete](https://github.com/bazel-contrib/rules_python/actions/workflows/release_complete_prepare.yaml) + workflow automatically updates the tracking issue checklist, and a release + branch will be created. If the automatic trigger is missed, comment + `/prepare-complete` on the tracking issue or on the PR, or run the workflow + manually with the PR number. + +3. **Add Backports (if needed)**: If there are backports, add them following + the [How to add backports](#how-to-add-backports) steps. + +4. **Create an RC**: Comment `/create-rc` on the tracking issue. This will + automatically process pending backports before creating the RC. If any + backport fails, the RC creation will abort. + +5. **Iterate**: Repeat steps 3 and 4 until backports and RCs are no longer + needed. + +6. **Finalize the Release**: Comment `/promote` on the tracking issue to + finalize the release. + + +### Manually triggering the release workflow + +The release workflow can be manually triggered using the GitHub CLI (`gh`). +This is useful for re-running a release or for creating a release from a +specific commit. + +To trigger the workflow, use the `gh workflow run` command: + +```shell +gh workflow run release_publish.yaml --ref +``` + +By default, the workflow will publish the wheel to PyPI. To skip this step, +you can set the `publish_to_pypi` input to `false`: + +```shell +gh workflow run release_publish.yaml --ref -f publish_to_pypi=false +``` + +### Manually publishing to PyPI + +If PyPI publishing failed or was skipped during the main release, the PyPI +publishing workflow can be triggered manually using the GitHub CLI (`gh`) or +via the [GitHub Actions UI](https://github.com/bazel-contrib/rules_python/actions/workflows/publish_pypi.yaml): + +```shell +gh workflow run publish_pypi.yaml --ref -f tag_name= +``` + +### Determining Semantic Version + +**rules_python** uses [semantic version](https://semver.org), so releases with +API changes and new features bump the minor, and those with only bug fixes and +other minor changes bump the patch digit. + +The release tool will automatically determine the next version number based on +the `VERSION_NEXT_*` placeholders in the codebase. To see what changes are +being accumulated for the next release, review the pending news entries in the +`news/` directory. + +## How to add backports + +To add backports to an active release, you can use one of the following +methods: + +### Method A: Comment on the PR + +Comment `/backport` on the PR you wish to backport. This will automatically +add the PR to the active release's backports checklist. Once the PR is merged, +the backports will be automatically processed. + +> [!NOTE] +> Commenting `/backport` on an open PR will block further release publishing +> (like creating RCs or promoting) until the PR is merged or manually set to +> status=ignore in the checklist. + +### Method B: Comment on the Tracking Issue + +Comment `/add-backports [ ...]` (space or comma separated) on +the tracking issue. The `` can be a PR number (optionally prefixed with +`#`) or a PR URL (strictly for the configured repository). This will +automatically add the PRs to the checklist and trigger processing. + +### Method C: Manual Checklist Update +1. Manually add checklist items under the `## Backports` section of the + Release Tracking Issue. The format must be: `- [ ] #` (e.g., + `- [ ] #1234`). +2. When ready, comment `/process-backports` on the tracking issue to trigger + processing. + +### Method D: Release Tool CLI +You can use the release tool to add backports from your local checkout: +```shell +bazel run //tools/private/release -- add-backports [ ...] +``` +The `` can be: +* A PR number (e.g., `124` or `#124`) +* A PR URL (e.g., `https://github.com/bazel-contrib/rules_python/pull/124` + or `https://github.com/bazel-contrib/rules_python/pull/124/files`) +* Only URLs for the configured repository are accepted. + +### Failure Behavior +If a backport fails to process (e.g., due to cherry-pick conflicts): +* The failed backport checklist item will remain unchecked with + `status=error-`. +* You must resolve the conflict manually: checkout the release branch, + cherry-pick the PR, resolve conflicts, push to remote, and manually check + the box on the tracking issue checklist with `status=done` metadata. + +## Automated patch releases to multiple versions + +If you need to backport a PR to multiple older active release branches (e.g., +backporting a fix to `1.7`, `1.8`, and `1.9` when the latest release is +`2.2.0`), you can automate the creation of release tracking issues and +verification of cherry-picks using a Backport Tracking Issue. + +### Steps + +1. **Create a Backport Tracking Issue**: Create a new issue on GitHub with + the label `type: backport-pr`. + * The title should describe the backport, e.g., `Backport: #1234` + (referencing the PR to backport). + * The body must contain the target range in the following format: + ```markdown + * PR: #1234 + * From version: 1.7 + * To version: 2.2 + ``` + This tells the tool to target all active release branches between + `release/1.7` and `release/2.2` inclusive. + +2. **Prepare the Backports**: Trigger the preparation by commenting `/prepare` + on the backport tracking issue, or by manually running the [Backport: + Prepare](https://github.com/bazel-contrib/rules_python/actions/workflows/backport_prepare.yaml) + workflow with the issue number. + * The workflow will checkout each release branch in the range, attempt to + cherry-pick the PR, and verify if the changelog can be updated. + * It will then update the backport tracking issue body with a list of + tasks: + * `Verify apply `: Automatically checked if the + cherry-pick succeeded, or left unchecked with + `status=failed-conflict` or `status=failed-changelog` if it failed. + * `Release issue `: A task to initiate the release for + each target version. + +3. **Resolve Conflicts (if any)**: If any `Verify apply` task failed: + * You must resolve the conflict manually on that specific release branch + (checkout branch, cherry-pick, resolve conflicts, push to remote). + * Once resolved, manually check the corresponding `Verify apply` task box + on the tracking issue and set its metadata to `status=success` (e.g., + `- [x] Verify apply 1.8 | status=success`). + +4. **Initiate Releases**: Once all `Verify apply` tasks are successful + (either automatically or after manual resolution), comment + `/create-releases` on the backport tracking issue, or manually run the + [Backport: Create + Releases](https://github.com/bazel-contrib/rules_python/actions/workflows/backport_create_releases.yaml) + workflow. + * This will automatically create a standard Release Tracking Issue for + each target version (e.g., `Release 1.7.1`, `Release 1.8.1`, etc.). + * For patch releases, the created release tracking issues will have `Tag + RC` tasks automatically removed, as release candidates are not + required for patch releases. + * The backport PR will be automatically added to the checklist of each + created release tracking issue. + +5. **Execute Releases**: Follow the standard release process for each created + release tracking issue. Since these are patch releases, you can skip the + `/create-rc` step and comment `/promote` directly to tag and publish the + release from the release branch head. + +## Manual patch release with cherry picks + +If a patch release from head would contain changes that aren't appropriate for +a patch release, then the patch release needs to be based on the original +release tag and the patch changes cherry-picked into it. + +In this example, release `0.37.0` is being patched to create release `0.37.1`. +The fix being included is commit `deadbeef`. + +1. `git checkout release/0.37` +1. `git cherry-pick -x deadbeef` +1. Fix merge conflicts, if any. +1. `git cherry-pick --continue` (if applicable) +1. `git push upstream` + +If multiple commits need to be applied, repeat the `git cherry-pick` step for +each. + +Once the release branch is in the desired state, comment `/create-rc` on the +tracking issue to tag it, as done with a release from head. + +### Announcing releases + +We announce releases in the #python channel in the Bazel slack +(bazelbuild.slack.com). Here's a template: + +``` +Greetings Pythonistas, + +rules_python X.Y.Z-rcN is now available +Changelog: https://rules-python.readthedocs.io/en/X.Y.Z-rcN/changelog.html#vX-Y-Z + +It will be promoted to stable next week, pending feedback. +``` + +It's traditional to include notable changes from the changelog, but not +required. + +### Re-releasing a version + +Re-releasing a version (i.e. changing the commit a tag points to) is +*sometimes* possible, but it depends on how far into the release process it got. + +The two points of no return are: + * If the PyPI package has been published: PyPI disallows using the same + filename/version twice. Once published, it cannot be replaced. + * If the BCR package has been published: Once it's been committed to the BCR + registry, it cannot be replaced. + +If release steps fail _prior_ to those steps, then its OK to change the tag. You +may need to manually delete the GitHub release. + +## Manual Editing of Tracking Issue + +You can manually edit the Release Tracking Issue to control the release flow. +The checklist items use metadata suffix: `| key=value key2=value2`. + +* **Retry Prepare Release**: Reset the task to `- [ ] Prepare Release | status=awaiting-preparation`. +* **Complete Prepare Release**: Comment `/prepare-complete` on the tracking + issue (or on the preparation PR) to mark the preparation task as done. +* **Create Release Branch**: Comment `/create-release-branch` on the tracking + issue to cut and push the release branch. +* **Force Task Done**: Check the box `- [x]` and add appropriate metadata (e.g. `status=done`). + +## Secrets + +### PyPI user rules-python + +Part of the release process uploads packages to PyPI as the user `rules-python`. +This account is managed by `rickeylev`. Note that Google also has recovery access +and can be contacted at rules-python-pypi@google.com if something needs to be +done with the PyPI account. diff --git a/WORKSPACE b/WORKSPACE index f03cc26803..a692f1b289 100644 --- a/WORKSPACE +++ b/WORKSPACE @@ -17,10 +17,34 @@ workspace(name = "rules_python") # Everything below this line is used only for developing rules_python. Users # should not copy it to their WORKSPACE. +# Workaround for Bazel 9 duplicate name issue in Gazelle. +load("//:workspace_bazel9.bzl", "bazel_9_workaround") + +bazel_9_workaround() + load("//:internal_dev_deps.bzl", "rules_python_internal_deps") rules_python_internal_deps() +load("@rules_java//java:rules_java_deps.bzl", "rules_java_dependencies") + +rules_java_dependencies() + +# note that the following line is what is minimally required from protobuf for the java rules +# consider using the protobuf_deps() public API from @com_google_protobuf//:protobuf_deps.bzl +load("@com_google_protobuf//bazel/private:proto_bazel_features.bzl", "proto_bazel_features") # buildifier: disable=bzl-visibility + +proto_bazel_features(name = "proto_bazel_features") + +# register toolchains +load("@rules_java//java:repositories.bzl", "rules_java_toolchains") + +rules_java_toolchains() + +load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") + +protobuf_deps() + load("@rules_jvm_external//:repositories.bzl", "rules_jvm_external_deps") rules_jvm_external_deps() @@ -41,17 +65,19 @@ load("//:internal_dev_setup.bzl", "rules_python_internal_setup") rules_python_internal_setup() -load("@pythons_hub//:versions.bzl", "MINOR_MAPPING", "PYTHON_VERSIONS") -load("//python:repositories.bzl", "python_register_multi_toolchains") +load("@pythons_hub//:versions.bzl", "PYTHON_VERSIONS") +load("//python:repositories.bzl", "py_repositories", "python_register_multi_toolchains") + +py_repositories() python_register_multi_toolchains( name = "python", - default_version = MINOR_MAPPING.values()[-3], # Use 3.11.10 + default_version = "3.11", # Integration tests verify each version, so register all of them. python_versions = PYTHON_VERSIONS, ) -load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive", "http_file") +load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") # Used for Bazel CI http_archive( @@ -68,7 +94,7 @@ load("@bazelci_rules//:rbe_repo.bzl", "rbe_preconfig") # otherwise refer to RBE docs. rbe_preconfig( name = "buildkite_config", - toolchain = "ubuntu1804-bazel-java11", + toolchain = "ubuntu2204", ) local_repository( @@ -80,7 +106,7 @@ local_repository( # which we need to fetch in order to compile it. load("@rules_python_gazelle_plugin//:deps.bzl", _py_gazelle_deps = "gazelle_deps") -# See: https://github.com/bazelbuild/rules_python/blob/main/gazelle/README.md +# See: https://github.com/bazel-contrib/rules_python/blob/main/gazelle/README.md # This rule loads and compiles various go dependencies that running gazelle # for python requirements. _py_gazelle_deps() @@ -91,7 +117,7 @@ interpreter = "@python_3_11_9_host//:python" ##################### # Install twine for our own runfiles wheel publishing. # Eventually we might want to install twine automatically for users too, see: -# https://github.com/bazelbuild/rules_python/issues/1016. +# https://github.com/bazel-contrib/rules_python/issues/1016. load("@rules_python//python:pip.bzl", "pip_parse") pip_parse( @@ -112,36 +138,58 @@ pip_parse( requirements_lock = "//examples/wheel:requirements_server.txt", ) -load("@pypiserver//:requirements.bzl", install_pypiserver = "install_deps") +load("@pypiserver//:requirements.bzl", install_pypiserver = "install_deps", pypiserver_requirements = "all_requirements") install_pypiserver() ##################### -# Install sphinx for doc generation. +# Install dev dependencies. pip_parse( name = "dev_pip", python_interpreter_target = interpreter, - requirements_lock = "//docs:requirements.txt", + requirements_lock = "//dev:requirements.txt", ) -load("@dev_pip//:requirements.bzl", docs_install_deps = "install_deps") +load("@dev_pip//:requirements.bzl", dev_pip_requirements = "all_requirements", docs_install_deps = "install_deps") docs_install_deps() -# This wheel is purely here to validate the wheel extraction code. It's not -# intended for anything else. -http_file( - name = "wheel_for_testing", - downloaded_file_path = "numpy-1.25.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - sha256 = "0d60fbae8e0019865fc4784745814cff1c421df5afee233db6d88ab4f14655a2", - urls = [ - "https://files.pythonhosted.org/packages/50/67/3e966d99a07d60a21a21d7ec016e9e4c2642a86fea251ec68677daf71d4d/numpy-1.25.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - ], +##################### +# Pypi repos for //tests/multi_pypi + +pip_parse( + name = "pypi_alpha", + python_interpreter_target = interpreter, + requirements_lock = "//tests/multi_pypi/alpha:requirements.txt", +) + +load("@pypi_alpha//:requirements.bzl", pypi_alpha_install_deps = "install_deps", pypi_alpha_requirements = "all_requirements") + +pypi_alpha_install_deps() + +pip_parse( + name = "pypi_beta", + python_interpreter_target = interpreter, + requirements_lock = "//tests/multi_pypi/beta:requirements.txt", ) -# rules_proto expects //external:python_headers to point at the python headers. -bind( - name = "python_headers", - actual = "//python/cc:current_py_cc_headers", +load("@pypi_beta//:requirements.bzl", pypi_beta_install_deps = "install_deps", pypi_beta_requirements = "all_requirements") + +pypi_beta_install_deps() + +load( + "//python/private/pypi:unified_hub_repo.bzl", + "unified_workspace_hub_repo", +) # buildifier: disable=bzl-visibility + +unified_workspace_hub_repo( + name = "pypi", + default_hub = "dev_pip", + hubs = { + "dev_pip": dev_pip_requirements, + "pypi_alpha": pypi_alpha_requirements, + "pypi_beta": pypi_beta_requirements, + "pypiserver": pypiserver_requirements, + }, ) diff --git a/addlicense.sh b/addlicense.sh index 8cc8fb33bc..8dc82bbcc9 100755 --- a/addlicense.sh +++ b/addlicense.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # Copyright 2023 The Bazel Authors. All rights reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/command_line_option/BUILD.bazel b/command_line_option/BUILD.bazel new file mode 100644 index 0000000000..15c22f360b --- /dev/null +++ b/command_line_option/BUILD.bazel @@ -0,0 +1,30 @@ +# Special placeholders for Bazel builtin //command_line_option psuedo-targets +# +# These are special targets to use with `py_binary.config_settings` that are +# treated as aliases for `//command_line_option:XXX` psuedo-targets. They +# are not actual flags or have any value. + +package( + default_visibility = ["//visibility:public"], +) + +alias( + name = "build_runfile_links", + actual = "//python:none", +) + +alias( + name = "enable_runfiles", + actual = "//python:none", +) + +alias( + name = "extra_toolchains", + actual = "//python:none", +) + +filegroup( + name = "distribution", + srcs = glob(["**"]), + visibility = ["//:__subpackages__"], +) diff --git a/dev/BUILD.bazel b/dev/BUILD.bazel new file mode 100644 index 0000000000..22fe5bb918 --- /dev/null +++ b/dev/BUILD.bazel @@ -0,0 +1,29 @@ +load("//python/uv:lock.bzl", "lock") # buildifier: disable=bzl-visibility + +licenses(["notice"]) + +# Run bazel run //dev:requirements.update +lock( + name = "requirements", + srcs = ["pyproject.toml"], + out = "requirements.txt", + args = [ + "--emit-index-url", + "--universal", + "--upgrade", + ], + # NOTE @aignas 2025-08-17: here we select the lowest actively supported + # version so that the requirements file is generated to be compatible with + # Python version 3.10 or greater. + python_version = "3.10", + visibility = ["//:__subpackages__"], +) + +# Run bazel run //dev:uv_lock.update +lock( + name = "uv_lock", + srcs = ["pyproject.toml"], + out = "uv.lock", + python_version = "3.10", + visibility = ["//:__subpackages__"], +) diff --git a/docs/pyproject.toml b/dev/pyproject.toml similarity index 70% rename from docs/pyproject.toml rename to dev/pyproject.toml index 2bcb31bfc2..cdc00b902c 100644 --- a/docs/pyproject.toml +++ b/dev/pyproject.toml @@ -1,5 +1,5 @@ [project] -name = "rules_python_docs" +name = "rules_python_dev" version = "0.0.0" dependencies = [ @@ -12,5 +12,12 @@ dependencies = [ "readthedocs-sphinx-ext", "absl-py", "typing-extensions", - "sphinx-reredirects" + "sphinx-reredirects", + "pefile", + "pyelftools", + "macholib", + "markupsafe", + "pytest", + "pytest-bazel", + "pytest-mock", ] diff --git a/dev/requirements.txt b/dev/requirements.txt new file mode 100644 index 0000000000..6efb1b8bdb --- /dev/null +++ b/dev/requirements.txt @@ -0,0 +1,558 @@ +# This file was autogenerated by uv via the following command: +# bazel run //dev:requirements.update +--index-url https://pypi.org/simple + +absl-py==2.5.0 \ + --hash=sha256:0c996f25c0490700fadabe6351630f6111534fa0ae252cc6d2014ea3b141135f \ + --hash=sha256:0f17b89f2a4eaaedc4f28c622998aa690564b3012a396a4ffad0821007fe03ba + # via rules-python-dev (dev/pyproject.toml) +alabaster==1.0.0 \ + --hash=sha256:c00dca57bca26fa62a6d7d0a9fcce65f3e026e9bfe33e9c538fd3fbb2144fd9e \ + --hash=sha256:fc6786402dc3fcb2de3cabd5fe455a2db534b371124f1f21de8731783dec828b + # via sphinx +altgraph==0.17.5 \ + --hash=sha256:c87b395dd12fabde9c99573a9749d67da8d29ef9de0125c7f536699b4a9bc9e7 \ + --hash=sha256:f3a22400bce1b0c701683820ac4f3b159cd301acab067c51c653e06961600597 + # via macholib +astroid==3.3.11 \ + --hash=sha256:1e5a5011af2920c7c67a53f65d536d65bfa7116feeaf2354d8b94f29573bb0ce \ + --hash=sha256:54c760ae8322ece1abd213057c4b5bba7c49818853fc901ef09719a60dbf9dec + # via sphinx-autodoc2 +babel==2.18.0 \ + --hash=sha256:b80b99a14bd085fcacfa15c9165f651fbb3406e66cc603abf11c5750937c992d \ + --hash=sha256:e2b422b277c2b9a9630c1d7903c2a00d0830c409c59ac8cae9081c92f1aeba35 + # via sphinx +certifi==2026.7.22 \ + --hash=sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775 \ + --hash=sha256:741e2c3b351ddf169a738da9f2c048608ff7f2c5cc02f1ebc6b118bb090d5d55 + # via requests +charset-normalizer==3.4.9 \ + --hash=sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380 \ + --hash=sha256:03d07803992c6c7bbc976327f34b18b6160327fc81cb82c9d504720ac0be3b62 \ + --hash=sha256:04ce310cb89c15df659582aee80a0603788732a5e017d5bd5c81158106ce249c \ + --hash=sha256:0d861473f743244d349b50f850d10eb87aeb22bbdcc8e64f79273c94af5a8226 \ + --hash=sha256:0e94703ec9684807f20cfb5eed95c70f67f2a8f21ad620146d7b5a13677b93e5 \ + --hash=sha256:0fa1aec2d32bcc03c8fa0f6f1712caad1adc38509f31142112e5c9daf5b9c833 \ + --hash=sha256:16b65ea0f2465b6fb52aa22de5eca612aa964ddfec00a912e26f4656cbef890b \ + --hash=sha256:16d10d789dd9bcca1173c95af82c58433122564b7bc39385124be735a35cbe99 \ + --hash=sha256:19ac87f93086ce37b86e098888555c4b4bc48102279bae3350098c0ed664b501 \ + --hash=sha256:1d22856ffbe153a602df38e4a5464f0b748a54002e0d69ac6d2ad0a197cc99ec \ + --hash=sha256:21e764fd1e70b6a3e205a0e46f3051701f98a8cb3fad66eeb80e48bb502f8698 \ + --hash=sha256:231ddcbb35e2ff8973e1365db41fe0572662893b99a05deb183b68ad4c0c8bd4 \ + --hash=sha256:253a4a220747e8b5faf57ec320c4f5efb0cef05f647420bf267143ec15dba10a \ + --hash=sha256:280081916dc341820640489a66e4696049401ef1cf6dd672f672e70ad915aca3 \ + --hash=sha256:2a441ea71902098ffe78c5abe6c494f44160b4af614ed16c3d9a3b1d17fd8ee2 \ + --hash=sha256:304b13570067b2547562e308af560b3963857b1fa90bd6afd978130130fe2d6a \ + --hash=sha256:32286a2c8d167e897177b673176c1e3e00d4057caf5d2b64eef9a3666b03018e \ + --hash=sha256:33bdcc2a32c0a0e861f60841a512c8acc658c87c2ac59d89e3a46dacf7d866e4 \ + --hash=sha256:375b83ed0aecfce76c16d198fbc21f3b11b337d68662bea0a995046682a11419 \ + --hash=sha256:3c09a49d6cde137258beb3d551994a2927fd35ad5cf96aed573f61bbd67c5f84 \ + --hash=sha256:3d92613ec25e43b05f042302531ec0f00b8445190e43325880cbd6ab7c2581da \ + --hash=sha256:40a126142a56b2dfc0aacbad1de8310cbf60da7656db0e6b16eebd48e3e93519 \ + --hash=sha256:416c229f77e5ea25b3dfd4b582f8d73d7e43c22320302b9ab128a2d3a0b38efe \ + --hash=sha256:432786d3561e69aeeae6c7e8648964ce0ad05736120135601f87ac26b9c83381 \ + --hash=sha256:43b9e366a31fdd1c87d0eb08f579b4a82b723ea54338f040d6b4e518a026ea29 \ + --hash=sha256:440eede837960000d74978f0eba527be106b5b9aee0daf779d395276ed0b0614 \ + --hash=sha256:45b0cc4e3556cd875e09102988d1ab8356c998b596c9fced84547c8138b487a0 \ + --hash=sha256:476743fe6dfe14a2da12e3ac79125dc84a3b2cf8094369a47a1529b0cd8549fe \ + --hash=sha256:4773092f8019072343a7447203308b176e10199920eb02d6195e81bbb3274c29 \ + --hash=sha256:4b3dac63058cc36820b0dd072f89898604e2d39686fe05321729d00d8ac185a0 \ + --hash=sha256:4d1c96a7a18b9690a4d46df09e3e3382406ae3213727cd1019ebade1c4a81917 \ + --hash=sha256:51307f5c71007673a2bf8232ad973483d281e74cb99c8c5a990af1eefa6277d9 \ + --hash=sha256:51447e9aa2684679af07ca5021c3db526e0284347ebf4ffcec1154c3350cfe32 \ + --hash=sha256:58150c9f9b9a552505912d182ccdf26f6396fb6094816ceebcbb20eecabaed94 \ + --hash=sha256:5b10cd92fc5c498b35a8635df6d5a100207f88b63a4dc1de7ef9a548e1e2cd63 \ + --hash=sha256:5e226f6218febc71f6c1fc2fafb91c226f75bdc1d8fb12d66823716e891608fd \ + --hash=sha256:609b3ba8fcc0fb5ab7af00719d0fb6ad0cb518e48e7712d12fd68f1327951198 \ + --hash=sha256:60f44ade2cf573dad7a277e6f8ca9a51a21dda572b13bd7d8539bb3cd5dbedde \ + --hash=sha256:611057cc5d5c0afc743ba8be6bd828c17e0aaa8643f9d0a9b9bb7dea80eb8012 \ + --hash=sha256:6366a16e1a25018694d6a5d784d09b046edc9eac40ea2b54065c3052672516a1 \ + --hash=sha256:65a7ff3f705e57d392f7261b6d0550fe137c3019477431f1c355e0db0a7d3e15 \ + --hash=sha256:673611bbd43f0810bec0b0f028ddeaaa501190339cac411f347ac76917c3ae7b \ + --hash=sha256:67830fc78e67501f47bb950471b2dcb9b35b140084429318e862895a8e89c993 \ + --hash=sha256:68ce9f4d6b26d5ccbf7fd4459bf75f74a0a146677ebba80597df60cbdb20e6f4 \ + --hash=sha256:68e5f26a1ad57ded6d1cfb85331d1c1a195314756471d97758c48498bb4dcdf5 \ + --hash=sha256:69b157c5d3292bcd443faca052f3096f637f1e074b98212a933c074ae23dc3b8 \ + --hash=sha256:75286256590a6320cf106a0d28970d3560aad9ee09aa7b34fb40524792436d35 \ + --hash=sha256:78841cccf1af7b40f6f716338d50c0902dbe88d9f800b3c973b7a9a0a693a642 \ + --hash=sha256:78fa18e436a1a0e58dbd7e02fc4473f3f32cceb12df9dfca542d075961c307d2 \ + --hash=sha256:79580094b00d1789d1f93ea55bc43cb2f611910c72235b7657f3482ddcc1b22d \ + --hash=sha256:7b86a2b16095d250c6f58b3d9b2eee6f4147754344f3dab0922f7c9bf7d226c9 \ + --hash=sha256:83aed2c10721ddd90f68140685391b50811a880af20654c59af6b6c66c40513c \ + --hash=sha256:84fd18bcc17526fc2b3c1af7d2b9217d32c9c04448c16ec693b9b4f1985c3d33 \ + --hash=sha256:871ff67ea1aad4dfd91736464934d56b32dac49f9fbe16cddba36198a7b3a0db \ + --hash=sha256:898f0e9068ca27d37f8e83a5b962821df851532e6c4a7d615c1c033f9da6eedf \ + --hash=sha256:8a79d9f4d8001473a30c163556b3c3bfebec837495a412dde78b51672f6134f9 \ + --hash=sha256:8c041122946b7ba21bb32c45b1aa57b1be35527690aeb3c5c234521085632eee \ + --hash=sha256:90c44bc373b7687f6948b693cceaea1348ae0975d7474746559494468e3c1d84 \ + --hash=sha256:9104ed0bd76a429d46f9ec0dbc9b08ad1d2dcdf2b00a5a0daa1c145329b35b44 \ + --hash=sha256:920079c3f7456fa213e0829ed2073aaa727fd39d889ead5b4f35d0de5460d04f \ + --hash=sha256:93d59d504b230e83c7a843251681959a0b6a9cd76f6e146ce1b8a80eb8739af9 \ + --hash=sha256:9b2aff1c7b3884512b9512c3eaadd9bab39fb45042ffaaa1dd08ff2b9f8109d9 \ + --hash=sha256:9b8e0f3107e2200b76f6054de99016eac3ee6762713587b36baaa7e4bd2ae177 \ + --hash=sha256:9bb41182d93ea91f60b4bc8fbf4c820c69ef8a12ab2d917f3f1834f1acad07e8 \ + --hash=sha256:9cdef90ae47919cae358d8ab15797a800ed41da7aba5d72419fb510729e2ed4b \ + --hash=sha256:a1786910334ed46ab1dd73222f2cd1e05c2c3bb39f6dddb4f8b36fc382058a39 \ + --hash=sha256:a4cfde78a9f2880208d16a93b795726a3017d5977e08d1e162a7a31322479c41 \ + --hash=sha256:a4fbdde9dd4a9ce5fd52c2b3a347bb50cc89483ef783f1cb00d408c13f7a96c0 \ + --hash=sha256:aa99adc8f081b475a12843953db36831eaf83ec33eb46a90629ca6a5de45a616 \ + --hash=sha256:ac351b3b8014eead140e77e9717e2992c6bbe30b63bc3422422eb84865412e3d \ + --hash=sha256:ad41ba96094304aa090f5a30cb6e4fb3b3f1c264c523394b4c39bbacc4dc92ba \ + --hash=sha256:b5314963fce9b0b12743891de876e724997864ee22aa496f903f426c7e2fa5b2 \ + --hash=sha256:bcf74c1df76758a395bf0af608c04c82257523f55c9868b334f06270d0f2112b \ + --hash=sha256:bd47ba7fc3ca94896759ea0109775132d3e7ab921fbf54038e1bab2e46c313c9 \ + --hash=sha256:c0323c9daef75ef2e5083624b4585018a0c9d5e3b40f607eed81a311270b934b \ + --hash=sha256:c1225416b463483160e4af85d5fc3a9690ccb53fd4b1865a6437825f5ede3209 \ + --hash=sha256:c1c948747b03be832dceed96ca815cef7360de9aa19d37c730f8e3f6101aca48 \ + --hash=sha256:c25fe15c70c59eb7c5ce8c06a1f3fa1da0ecc5ea1e7a5922c40fd2fa9b0d5046 \ + --hash=sha256:cc1b0fff8ead343dae06305f954eb8468ba0ec1a97881f42489d198e4ce3c632 \ + --hash=sha256:cd6280cf040f233bd7d3407b743b4b4c74f70e8e1c4199cb112a62c941c0772a \ + --hash=sha256:cd6c3d4b783c556fa00bf540854e42f135e2f256abd29669fcd0da0f2dec79c2 \ + --hash=sha256:d4d6fcde76f94f5cb9e43e9e9a61f16dacefd228cbbf6f1a09bd9b219a92f1a1 \ + --hash=sha256:ddf4af30b417d9fe16481e9b81c27ab2a7cde1ff7ba3e85653b02db7d145dc7b \ + --hash=sha256:df115d4d83168fdf2cae48ef1ff6d1cb4c466364e30861b37121de0f3bf1b990 \ + --hash=sha256:df7276909358e5635ae203673ab7e509ddd224225a8d6b0790bf13eb2bde1cc5 \ + --hash=sha256:e4fd89cc178bced6ad29cb3e6dd4aa63fa5017c3524dbd0b25998fb64a87cc8b \ + --hash=sha256:e9701d0049d92c16703a42771b98d560b95248949f23f8cf7b4eddd201814fb9 \ + --hash=sha256:ee2f2a527e3c1a6e6411eb4209642e138b544a2d72fe5d0d76daf77b24063534 \ + --hash=sha256:f7fb7d750cfa0a070d2c24e831fd3481019a60dd317ea2b39acbcebc08b6ed81 \ + --hash=sha256:f840ed6d8ecba8255df8c42b87fadeda98ddfc6eeec05e2dc66e26d46dd6f58a \ + --hash=sha256:f86c6358749bd4fda175388691e3ba8c46e24c5347d0afd20f9b7edfc9faf07d \ + --hash=sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf \ + --hash=sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115 + # via requests +colorama==0.4.6 ; sys_platform == 'win32' \ + --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \ + --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6 + # via + # pytest + # sphinx +docutils==0.21.2 ; python_full_version < '3.11' \ + --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ + --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2 + # via + # myst-parser + # sphinx + # sphinx-rtd-theme +docutils==0.22.4 ; python_full_version >= '3.11' \ + --hash=sha256:4db53b1fde9abecbb74d91230d32ab626d94f6badfc575d6db9194a49df29968 \ + --hash=sha256:d0013f540772d1420576855455d050a2180186c91c15779301ac2ccb3eeb68de + # via + # myst-parser + # sphinx + # sphinx-rtd-theme +exceptiongroup==1.3.1 ; python_full_version < '3.11' \ + --hash=sha256:8b412432c6055b0b7d14c310000ae93352ed6754f70fa8f7c34141f91c4e3219 \ + --hash=sha256:a7a39a3bd276781e98394987d3a5701d0c4edffb633bb7a5144577f82c773598 + # via pytest +idna==3.18 \ + --hash=sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2 \ + --hash=sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848 + # via requests +imagesize==2.0.0 \ + --hash=sha256:5667c5bbb57ab3f1fa4bc366f4fbc971db3d5ed011fd2715fd8001f782718d96 \ + --hash=sha256:8e8358c4a05c304f1fccf7ff96f036e7243a189e9e42e90851993c558cfe9ee3 + # via sphinx +iniconfig==2.3.0 \ + --hash=sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730 \ + --hash=sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12 + # via pytest +jinja2==3.1.6 \ + --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ + --hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67 + # via + # myst-parser + # readthedocs-sphinx-ext + # sphinx +macholib==1.16.4 \ + --hash=sha256:da1a3fa8266e30f0ce7e97c6a54eefaae8edd1e5f86f3eb8b95457cae90265ea \ + --hash=sha256:f408c93ab2e995cd2c46e34fe328b130404be143469e41bc366c807448979362 + # via rules-python-dev (dev/pyproject.toml) +markdown-it-py==3.0.0 ; python_full_version < '3.11' \ + --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ + --hash=sha256:e3f60a94fa066dc52ec76661e37c851cb232d92f9886b15cb560aaada2df8feb + # via + # mdit-py-plugins + # myst-parser +markdown-it-py==4.2.0 ; python_full_version >= '3.11' \ + --hash=sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49 \ + --hash=sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a + # via + # mdit-py-plugins + # myst-parser +markupsafe==3.0.3 \ + --hash=sha256:0303439a41979d9e74d18ff5e2dd8c43ed6c6001fd40e5bf2e43f7bd9bbc523f \ + --hash=sha256:068f375c472b3e7acbe2d5318dea141359e6900156b5b2ba06a30b169086b91a \ + --hash=sha256:0bf2a864d67e76e5c9a34dc26ec616a66b9888e25e7b9460e1c76d3293bd9dbf \ + --hash=sha256:0db14f5dafddbb6d9208827849fad01f1a2609380add406671a26386cdf15a19 \ + --hash=sha256:0eb9ff8191e8498cca014656ae6b8d61f39da5f95b488805da4bb029cccbfbaf \ + --hash=sha256:0f4b68347f8c5eab4a13419215bdfd7f8c9b19f2b25520968adfad23eb0ce60c \ + --hash=sha256:1085e7fbddd3be5f89cc898938f42c0b3c711fdcb37d75221de2666af647c175 \ + --hash=sha256:116bb52f642a37c115f517494ea5feb03889e04df47eeff5b130b1808ce7c219 \ + --hash=sha256:12c63dfb4a98206f045aa9563db46507995f7ef6d83b2f68eda65c307c6829eb \ + --hash=sha256:133a43e73a802c5562be9bbcd03d090aa5a1fe899db609c29e8c8d815c5f6de6 \ + --hash=sha256:1353ef0c1b138e1907ae78e2f6c63ff67501122006b0f9abad68fda5f4ffc6ab \ + --hash=sha256:15d939a21d546304880945ca1ecb8a039db6b4dc49b2c5a400387cdae6a62e26 \ + --hash=sha256:177b5253b2834fe3678cb4a5f0059808258584c559193998be2601324fdeafb1 \ + --hash=sha256:1872df69a4de6aead3491198eaf13810b565bdbeec3ae2dc8780f14458ec73ce \ + --hash=sha256:1b4b79e8ebf6b55351f0d91fe80f893b4743f104bff22e90697db1590e47a218 \ + --hash=sha256:1b52b4fb9df4eb9ae465f8d0c228a00624de2334f216f178a995ccdcf82c4634 \ + --hash=sha256:1ba88449deb3de88bd40044603fafffb7bc2b055d626a330323a9ed736661695 \ + --hash=sha256:1cc7ea17a6824959616c525620e387f6dd30fec8cb44f649e31712db02123dad \ + --hash=sha256:218551f6df4868a8d527e3062d0fb968682fe92054e89978594c28e642c43a73 \ + --hash=sha256:26a5784ded40c9e318cfc2bdb30fe164bdb8665ded9cd64d500a34fb42067b1c \ + --hash=sha256:2713baf880df847f2bece4230d4d094280f4e67b1e813eec43b4c0e144a34ffe \ + --hash=sha256:2a15a08b17dd94c53a1da0438822d70ebcd13f8c3a95abe3a9ef9f11a94830aa \ + --hash=sha256:2f981d352f04553a7171b8e44369f2af4055f888dfb147d55e42d29e29e74559 \ + --hash=sha256:32001d6a8fc98c8cb5c947787c5d08b0a50663d139f1305bac5885d98d9b40fa \ + --hash=sha256:3524b778fe5cfb3452a09d31e7b5adefeea8c5be1d43c4f810ba09f2ceb29d37 \ + --hash=sha256:3537e01efc9d4dccdf77221fb1cb3b8e1a38d5428920e0657ce299b20324d758 \ + --hash=sha256:35add3b638a5d900e807944a078b51922212fb3dedb01633a8defc4b01a3c85f \ + --hash=sha256:38664109c14ffc9e7437e86b4dceb442b0096dfe3541d7864d9cbe1da4cf36c8 \ + --hash=sha256:3a7e8ae81ae39e62a41ec302f972ba6ae23a5c5396c8e60113e9066ef893da0d \ + --hash=sha256:3b562dd9e9ea93f13d53989d23a7e775fdfd1066c33494ff43f5418bc8c58a5c \ + --hash=sha256:457a69a9577064c05a97c41f4e65148652db078a3a509039e64d3467b9e7ef97 \ + --hash=sha256:4bd4cd07944443f5a265608cc6aab442e4f74dff8088b0dfc8238647b8f6ae9a \ + --hash=sha256:4e885a3d1efa2eadc93c894a21770e4bc67899e3543680313b09f139e149ab19 \ + --hash=sha256:4faffd047e07c38848ce017e8725090413cd80cbc23d86e55c587bf979e579c9 \ + --hash=sha256:509fa21c6deb7a7a273d629cf5ec029bc209d1a51178615ddf718f5918992ab9 \ + --hash=sha256:5678211cb9333a6468fb8d8be0305520aa073f50d17f089b5b4b477ea6e67fdc \ + --hash=sha256:591ae9f2a647529ca990bc681daebdd52c8791ff06c2bfa05b65163e28102ef2 \ + --hash=sha256:5a7d5dc5140555cf21a6fefbdbf8723f06fcd2f63ef108f2854de715e4422cb4 \ + --hash=sha256:69c0b73548bc525c8cb9a251cddf1931d1db4d2258e9599c28c07ef3580ef354 \ + --hash=sha256:6b5420a1d9450023228968e7e6a9ce57f65d148ab56d2313fcd589eee96a7a50 \ + --hash=sha256:722695808f4b6457b320fdc131280796bdceb04ab50fe1795cd540799ebe1698 \ + --hash=sha256:729586769a26dbceff69f7a7dbbf59ab6572b99d94576a5592625d5b411576b9 \ + --hash=sha256:77f0643abe7495da77fb436f50f8dab76dbc6e5fd25d39589a0f1fe6548bfa2b \ + --hash=sha256:795e7751525cae078558e679d646ae45574b47ed6e7771863fcc079a6171a0fc \ + --hash=sha256:7be7b61bb172e1ed687f1754f8e7484f1c8019780f6f6b0786e76bb01c2ae115 \ + --hash=sha256:7c3fb7d25180895632e5d3148dbdc29ea38ccb7fd210aa27acbd1201a1902c6e \ + --hash=sha256:7e68f88e5b8799aa49c85cd116c932a1ac15caaa3f5db09087854d218359e485 \ + --hash=sha256:83891d0e9fb81a825d9a6d61e3f07550ca70a076484292a70fde82c4b807286f \ + --hash=sha256:8485f406a96febb5140bfeca44a73e3ce5116b2501ac54fe953e488fb1d03b12 \ + --hash=sha256:8709b08f4a89aa7586de0aadc8da56180242ee0ada3999749b183aa23df95025 \ + --hash=sha256:8f71bc33915be5186016f675cd83a1e08523649b0e33efdb898db577ef5bb009 \ + --hash=sha256:915c04ba3851909ce68ccc2b8e2cd691618c4dc4c4232fb7982bca3f41fd8c3d \ + --hash=sha256:949b8d66bc381ee8b007cd945914c721d9aba8e27f71959d750a46f7c282b20b \ + --hash=sha256:94c6f0bb423f739146aec64595853541634bde58b2135f27f61c1ffd1cd4d16a \ + --hash=sha256:9a1abfdc021a164803f4d485104931fb8f8c1efd55bc6b748d2f5774e78b62c5 \ + --hash=sha256:9b79b7a16f7fedff2495d684f2b59b0457c3b493778c9eed31111be64d58279f \ + --hash=sha256:a320721ab5a1aba0a233739394eb907f8c8da5c98c9181d1161e77a0c8e36f2d \ + --hash=sha256:a4afe79fb3de0b7097d81da19090f4df4f8d3a2b3adaa8764138aac2e44f3af1 \ + --hash=sha256:ad2cf8aa28b8c020ab2fc8287b0f823d0a7d8630784c31e9ee5edea20f406287 \ + --hash=sha256:b8512a91625c9b3da6f127803b166b629725e68af71f8184ae7e7d54686a56d6 \ + --hash=sha256:bc51efed119bc9cfdf792cdeaa4d67e8f6fcccab66ed4bfdd6bde3e59bfcbb2f \ + --hash=sha256:bdc919ead48f234740ad807933cdf545180bfbe9342c2bb451556db2ed958581 \ + --hash=sha256:bdd37121970bfd8be76c5fb069c7751683bdf373db1ed6c010162b2a130248ed \ + --hash=sha256:be8813b57049a7dc738189df53d69395eba14fb99345e0a5994914a3864c8a4b \ + --hash=sha256:c0c0b3ade1c0b13b936d7970b1d37a57acde9199dc2aecc4c336773e1d86049c \ + --hash=sha256:c47a551199eb8eb2121d4f0f15ae0f923d31350ab9280078d1e5f12b249e0026 \ + --hash=sha256:c4ffb7ebf07cfe8931028e3e4c85f0357459a3f9f9490886198848f4fa002ec8 \ + --hash=sha256:ccfcd093f13f0f0b7fdd0f198b90053bf7b2f02a3927a30e63f3ccc9df56b676 \ + --hash=sha256:d2ee202e79d8ed691ceebae8e0486bd9a2cd4794cec4824e1c99b6f5009502f6 \ + --hash=sha256:d53197da72cc091b024dd97249dfc7794d6a56530370992a5e1a08983ad9230e \ + --hash=sha256:d6dd0be5b5b189d31db7cda48b91d7e0a9795f31430b7f271219ab30f1d3ac9d \ + --hash=sha256:d88b440e37a16e651bda4c7c2b930eb586fd15ca7406cb39e211fcff3bf3017d \ + --hash=sha256:de8a88e63464af587c950061a5e6a67d3632e36df62b986892331d4620a35c01 \ + --hash=sha256:df2449253ef108a379b8b5d6b43f4b1a8e81a061d6537becd5582fba5f9196d7 \ + --hash=sha256:e1c1493fb6e50ab01d20a22826e57520f1284df32f2d8601fdd90b6304601419 \ + --hash=sha256:e1cf1972137e83c5d4c136c43ced9ac51d0e124706ee1c8aa8532c1287fa8795 \ + --hash=sha256:e2103a929dfa2fcaf9bb4e7c091983a49c9ac3b19c9061b6d5427dd7d14d81a1 \ + --hash=sha256:e56b7d45a839a697b5eb268c82a71bd8c7f6c94d6fd50c3d577fa39a9f1409f5 \ + --hash=sha256:e8afc3f2ccfa24215f8cb28dcf43f0113ac3c37c2f0f0806d8c70e4228c5cf4d \ + --hash=sha256:e8fc20152abba6b83724d7ff268c249fa196d8259ff481f3b1476383f8f24e42 \ + --hash=sha256:eaa9599de571d72e2daf60164784109f19978b327a3910d3e9de8c97b5b70cfe \ + --hash=sha256:ec15a59cf5af7be74194f7ab02d0f59a62bdcf1a537677ce67a2537c9b87fcda \ + --hash=sha256:f190daf01f13c72eac4efd5c430a8de82489d9cff23c364c3ea822545032993e \ + --hash=sha256:f34c41761022dd093b4b6896d4810782ffbabe30f2d443ff5f083e0cbbb8c737 \ + --hash=sha256:f3e98bb3798ead92273dc0e5fd0f31ade220f59a266ffd8a4f6065e0a3ce0523 \ + --hash=sha256:f42d0984e947b8adf7dd6dde396e720934d12c506ce84eea8476409563607591 \ + --hash=sha256:f71a396b3bf33ecaa1626c255855702aca4d3d9fea5e051b41ac59a9c1c41edc \ + --hash=sha256:f9e130248f4462aaa8e2552d547f36ddadbeaa573879158d721bbd33dfe4743a \ + --hash=sha256:fed51ac40f757d41b7c48425901843666a6677e3e8eb0abcff09e4ba6e664f50 + # via + # rules-python-dev (dev/pyproject.toml) + # jinja2 +mdit-py-plugins==0.6.1 \ + --hash=sha256:214c82fb2ac524472ab6a5bcab1de80f73b50443e187f401bfd77efbc7c6481d \ + --hash=sha256:a2bca0f039f39dbd35fb74ae1b5f998608c437463371f0ff7f49a19a17a114d0 + # via myst-parser +mdurl==0.1.2 \ + --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ + --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba + # via markdown-it-py +myst-parser==4.0.1 ; python_full_version < '3.11' \ + --hash=sha256:5cfea715e4f3574138aecbf7d54132296bfd72bb614d31168f48c477a830a7c4 \ + --hash=sha256:9134e88959ec3b5780aedf8a99680ea242869d012e8821db3126d427edc9c95d + # via rules-python-dev (dev/pyproject.toml) +myst-parser==5.1.0 ; python_full_version >= '3.11' \ + --hash=sha256:9c91c52b3cdb4d94a6506e4fab4e2f296c7623a0da0dcbe6de1565c3dad67a8a \ + --hash=sha256:ab69322dc6719dcc7f296479dbb70181b66df6ed315064f92dbc85c0e1bf2f02 + # via rules-python-dev (dev/pyproject.toml) +packaging==26.3 \ + --hash=sha256:94edc256424af38762eb31306eed28beb9f0efc50a8837492c9d6fd6004aed79 \ + --hash=sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c + # via + # pytest + # readthedocs-sphinx-ext + # sphinx +pefile==2024.8.26 \ + --hash=sha256:3ff6c5d8b43e8c37bb6e6dd5085658d658a7a0bdcd20b6a07b1fcfc1c4e9d632 \ + --hash=sha256:76f8b485dcd3b1bb8166f1128d395fa3d87af26360c2358fb75b80019b957c6f + # via rules-python-dev (dev/pyproject.toml) +pluggy==1.6.0 \ + --hash=sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3 \ + --hash=sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746 + # via pytest +pyelftools==0.33 \ + --hash=sha256:660d82dcbeb8e83d1702bd97f223f761625da06111c0cc988eac6b8ab0c1b61f \ + --hash=sha256:f215ad5f47d3f1373a21496a6c9e0707c622840d0622f23ff7ce08678b020036 + # via rules-python-dev (dev/pyproject.toml) +pygments==2.20.0 \ + --hash=sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f \ + --hash=sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176 + # via + # pytest + # sphinx +pytest==9.1.1 \ + --hash=sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313 \ + --hash=sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c + # via + # rules-python-dev (dev/pyproject.toml) + # pytest-bazel + # pytest-mock +pytest-bazel==0.1.6 \ + --hash=sha256:a29e80e1d67c3db801bdd4d0b6b742f2bfb48cd6841caa33401458e5c4e29c21 + # via rules-python-dev (dev/pyproject.toml) +pytest-mock==3.15.1 \ + --hash=sha256:0a25e2eb88fe5168d535041d09a4529a188176ae608a6d249ee65abc0949630d \ + --hash=sha256:1849a238f6f396da19762269de72cb1814ab44416fa73a8686deac10b0d87a0f + # via rules-python-dev (dev/pyproject.toml) +pyyaml==6.0.3 \ + --hash=sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c \ + --hash=sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a \ + --hash=sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3 \ + --hash=sha256:02ea2dfa234451bbb8772601d7b8e426c2bfa197136796224e50e35a78777956 \ + --hash=sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6 \ + --hash=sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c \ + --hash=sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65 \ + --hash=sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a \ + --hash=sha256:1ebe39cb5fc479422b83de611d14e2c0d3bb2a18bbcb01f229ab3cfbd8fee7a0 \ + --hash=sha256:214ed4befebe12df36bcc8bc2b64b396ca31be9304b8f59e25c11cf94a4c033b \ + --hash=sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1 \ + --hash=sha256:22ba7cfcad58ef3ecddc7ed1db3409af68d023b7f940da23c6c2a1890976eda6 \ + --hash=sha256:27c0abcb4a5dac13684a37f76e701e054692a9b2d3064b70f5e4eb54810553d7 \ + --hash=sha256:28c8d926f98f432f88adc23edf2e6d4921ac26fb084b028c733d01868d19007e \ + --hash=sha256:2e71d11abed7344e42a8849600193d15b6def118602c4c176f748e4583246007 \ + --hash=sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310 \ + --hash=sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4 \ + --hash=sha256:3c5677e12444c15717b902a5798264fa7909e41153cdf9ef7ad571b704a63dd9 \ + --hash=sha256:3ff07ec89bae51176c0549bc4c63aa6202991da2d9a6129d7aef7f1407d3f295 \ + --hash=sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea \ + --hash=sha256:418cf3f2111bc80e0933b2cd8cd04f286338bb88bdc7bc8e6dd775ebde60b5e0 \ + --hash=sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e \ + --hash=sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac \ + --hash=sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9 \ + --hash=sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7 \ + --hash=sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35 \ + --hash=sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb \ + --hash=sha256:5cf4e27da7e3fbed4d6c3d8e797387aaad68102272f8f9752883bc32d61cb87b \ + --hash=sha256:5e0b74767e5f8c593e8c9b5912019159ed0533c70051e9cce3e8b6aa699fcd69 \ + --hash=sha256:5ed875a24292240029e4483f9d4a4b8a1ae08843b9c54f43fcc11e404532a8a5 \ + --hash=sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b \ + --hash=sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c \ + --hash=sha256:6344df0d5755a2c9a276d4473ae6b90647e216ab4757f8426893b5dd2ac3f369 \ + --hash=sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd \ + --hash=sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824 \ + --hash=sha256:66291b10affd76d76f54fad28e22e51719ef9ba22b29e1d7d03d6777a9174198 \ + --hash=sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065 \ + --hash=sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c \ + --hash=sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c \ + --hash=sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764 \ + --hash=sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196 \ + --hash=sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b \ + --hash=sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00 \ + --hash=sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac \ + --hash=sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8 \ + --hash=sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e \ + --hash=sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28 \ + --hash=sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3 \ + --hash=sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5 \ + --hash=sha256:9c57bb8c96f6d1808c030b1687b9b5fb476abaa47f0db9c0101f5e9f394e97f4 \ + --hash=sha256:9c7708761fccb9397fe64bbc0395abcae8c4bf7b0eac081e12b809bf47700d0b \ + --hash=sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf \ + --hash=sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5 \ + --hash=sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702 \ + --hash=sha256:b30236e45cf30d2b8e7b3e85881719e98507abed1011bf463a8fa23e9c3e98a8 \ + --hash=sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788 \ + --hash=sha256:b865addae83924361678b652338317d1bd7e79b1f4596f96b96c77a5a34b34da \ + --hash=sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d \ + --hash=sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc \ + --hash=sha256:bdb2c67c6c1390b63c6ff89f210c8fd09d9a1217a465701eac7316313c915e4c \ + --hash=sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba \ + --hash=sha256:c2514fceb77bc5e7a2f7adfaa1feb2fb311607c9cb518dbc378688ec73d8292f \ + --hash=sha256:c3355370a2c156cffb25e876646f149d5d68f5e0a3ce86a5084dd0b64a994917 \ + --hash=sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5 \ + --hash=sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26 \ + --hash=sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f \ + --hash=sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b \ + --hash=sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be \ + --hash=sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c \ + --hash=sha256:efd7b85f94a6f21e4932043973a7ba2613b059c4a000551892ac9f1d11f5baf3 \ + --hash=sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6 \ + --hash=sha256:fa160448684b4e94d80416c0fa4aac48967a969efe22931448d853ada8baf926 \ + --hash=sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0 + # via myst-parser +readthedocs-sphinx-ext==2.2.5 \ + --hash=sha256:ee5fd5b99db9f0c180b2396cbce528aa36671951b9526bb0272dbfce5517bd27 \ + --hash=sha256:f8c56184ea011c972dd45a90122568587cc85b0127bc9cf064d17c68bc809daa + # via rules-python-dev (dev/pyproject.toml) +requests==2.34.2 \ + --hash=sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0 \ + --hash=sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed + # via + # readthedocs-sphinx-ext + # sphinx +roman-numerals==4.1.0 ; python_full_version >= '3.11' \ + --hash=sha256:1af8b147eb1405d5839e78aeb93131690495fe9da5c91856cb33ad55a7f1e5b2 \ + --hash=sha256:647ba99caddc2cc1e55a51e4360689115551bf4476d90e8162cf8c345fe233c7 + # via sphinx +snowballstemmer==3.1.1 \ + --hash=sha256:7e207fa178741da09cdee59d3ecec3827ad5f92b1fc5c9ff3755b639f71f5752 \ + --hash=sha256:e07bbc54a0d798fe6010a12398422e62a8bfbba95c394fd0956ef58cb4d3e260 + # via sphinx +sphinx==8.1.3 ; python_full_version < '3.11' \ + --hash=sha256:09719015511837b76bf6e03e42eb7595ac8c2e41eeb9c29c5b755c6b677992a2 \ + --hash=sha256:43c1911eecb0d3e161ad78611bc905d1ad0e523e4ddc202a58a821773dc4c927 + # via + # rules-python-dev (dev/pyproject.toml) + # myst-parser + # sphinx-reredirects + # sphinx-rtd-theme + # sphinxcontrib-jquery +sphinx==9.0.4 ; python_full_version == '3.11.*' \ + --hash=sha256:594ef59d042972abbc581d8baa577404abe4e6c3b04ef61bd7fc2acbd51f3fa3 \ + --hash=sha256:5bebc595a5e943ea248b99c13814c1c5e10b3ece718976824ffa7959ff95fffb + # via + # rules-python-dev (dev/pyproject.toml) + # myst-parser + # sphinx-reredirects + # sphinx-rtd-theme + # sphinxcontrib-jquery +sphinx==9.1.0 ; python_full_version >= '3.12' \ + --hash=sha256:7741722357dd75f8190766926071fed3bdc211c74dd2d7d4df5404da95930ddb \ + --hash=sha256:c84fdd4e782504495fe4f2c0b3413d6c2bf388589bb352d439b2a3bb99991978 + # via + # rules-python-dev (dev/pyproject.toml) + # myst-parser + # sphinx-reredirects + # sphinx-rtd-theme + # sphinxcontrib-jquery +sphinx-autodoc2==0.5.0 \ + --hash=sha256:7d76044aa81d6af74447080182b6868c7eb066874edc835e8ddf810735b6565a \ + --hash=sha256:e867013b1512f9d6d7e6f6799f8b537d6884462acd118ef361f3f619a60b5c9e + # via rules-python-dev (dev/pyproject.toml) +sphinx-reredirects==0.1.6 ; python_full_version < '3.11' \ + --hash=sha256:c491cba545f67be9697508727818d8626626366245ae64456fe29f37e9bbea64 \ + --hash=sha256:efd50c766fbc5bf40cd5148e10c00f2c00d143027de5c5e48beece93cc40eeea + # via rules-python-dev (dev/pyproject.toml) +sphinx-reredirects==1.1.0 ; python_full_version >= '3.11' \ + --hash=sha256:4b5692273c72cd2d4d917f4c6f87d5919e4d6114a752d4be033f7f5f6310efd9 \ + --hash=sha256:fb9b195335ab14b43f8273287d0c7eeb637ba6c56c66581c11b47202f6718b29 + # via rules-python-dev (dev/pyproject.toml) +sphinx-rtd-theme==3.1.0 \ + --hash=sha256:1785824ae8e6632060490f67cf3a72d404a85d2d9fc26bce3619944de5682b89 \ + --hash=sha256:b44276f2c276e909239a4f6c955aa667aaafeb78597923b1c60babc76db78e4c + # via rules-python-dev (dev/pyproject.toml) +sphinxcontrib-applehelp==2.0.0 \ + --hash=sha256:2f29ef331735ce958efa4734873f084941970894c6090408b079c61b2e1c06d1 \ + --hash=sha256:4cd3f0ec4ac5dd9c17ec65e9ab272c9b867ea77425228e68ecf08d6b28ddbdb5 + # via sphinx +sphinxcontrib-devhelp==2.0.0 \ + --hash=sha256:411f5d96d445d1d73bb5d52133377b4248ec79db5c793ce7dbe59e074b4dd1ad \ + --hash=sha256:aefb8b83854e4b0998877524d1029fd3e6879210422ee3780459e28a1f03a8a2 + # via sphinx +sphinxcontrib-htmlhelp==2.1.0 \ + --hash=sha256:166759820b47002d22914d64a075ce08f4c46818e17cfc9470a9786b759b19f8 \ + --hash=sha256:c9e2916ace8aad64cc13a0d233ee22317f2b9025b9cf3295249fa985cc7082e9 + # via sphinx +sphinxcontrib-jquery==4.1 \ + --hash=sha256:1620739f04e36a2c779f1a131a2dfd49b2fd07351bf1968ced074365933abc7a \ + --hash=sha256:f936030d7d0147dd026a4f2b5a57343d233f1fc7b363f68b3d4f1cb0993878ae + # via sphinx-rtd-theme +sphinxcontrib-jsmath==1.0.1 \ + --hash=sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178 \ + --hash=sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8 + # via sphinx +sphinxcontrib-qthelp==2.0.0 \ + --hash=sha256:4fe7d0ac8fc171045be623aba3e2a8f613f8682731f9153bb2e40ece16b9bbab \ + --hash=sha256:b18a828cdba941ccd6ee8445dbe72ffa3ef8cbe7505d8cd1fa0d42d3f2d5f3eb + # via sphinx +sphinxcontrib-serializinghtml==2.0.0 \ + --hash=sha256:6e2cb0eef194e10c27ec0023bfeb25badbbb5868244cf5bc5bdc04e4464bf331 \ + --hash=sha256:e9d912827f872c029017a53f0ef2180b327c3f7fd23c87229f7a8e8b70031d4d + # via sphinx +tomli==2.4.1 ; python_full_version < '3.11' \ + --hash=sha256:01f520d4f53ef97964a240a035ec2a869fe1a37dde002b57ebc4417a27ccd853 \ + --hash=sha256:0d85819802132122da43cb86656f8d1f8c6587d54ae7dcaf30e90533028b49fe \ + --hash=sha256:136443dbd7e1dee43c68ac2694fde36b2849865fa258d39bf822c10e8068eac5 \ + --hash=sha256:1d8591993e228b0c930c4bb0db464bdad97b3289fb981255d6c9a41aedc84b2d \ + --hash=sha256:2190f2e9dd7508d2a90ded5ed369255980a1bcdd58e52f7fe24b8162bf9fedbd \ + --hash=sha256:2c1c351919aca02858f740c6d33adea0c5deea37f9ecca1cc1ef9e884a619d26 \ + --hash=sha256:36d2bd2ad5fb9eaddba5226aa02c8ec3fa4f192631e347b3ed28186d43be6b54 \ + --hash=sha256:3d48a93ee1c9b79c04bb38772ee1b64dcf18ff43085896ea460ca8dec96f35f6 \ + --hash=sha256:47149d5bd38761ac8be13a84864bf0b7b70bc051806bc3669ab1cbc56216b23c \ + --hash=sha256:4ab97e64ccda8756376892c53a72bd1f964e519c77236368527f758fbc36a53a \ + --hash=sha256:4b605484e43cdc43f0954ddae319fb75f04cc10dd80d830540060ee7cd0243cd \ + --hash=sha256:504aa796fe0569bb43171066009ead363de03675276d2d121ac1a4572397870f \ + --hash=sha256:51529d40e3ca50046d7606fa99ce3956a617f9b36380da3b7f0dd3dd28e68cb5 \ + --hash=sha256:52c8ef851d9a240f11a88c003eacb03c31fc1c9c4ec64a99a0f922b93874fda9 \ + --hash=sha256:559db847dc486944896521f68d8190be1c9e719fced785720d2216fe7022b662 \ + --hash=sha256:5a881ab208c0baf688221f8cecc5401bd291d67e38a1ac884d6736cbcd8247e9 \ + --hash=sha256:5cb41aa38891e073ee49d55fbc7839cfdb2bc0e600add13874d048c94aadddd1 \ + --hash=sha256:5e262d41726bc187e69af7825504c933b6794dc3fbd5945e41a79bb14c31f585 \ + --hash=sha256:5ee18d9ebdb417e384b58fe414e8d6af9f4e7a0ae761519fb50f721de398dd4e \ + --hash=sha256:7008df2e7655c495dd12d2a4ad038ff878d4ca4b81fccaf82b714e07eae4402c \ + --hash=sha256:734e20b57ba95624ecf1841e72b53f6e186355e216e5412de414e3c51e5e3c41 \ + --hash=sha256:7c7e1a961a0b2f2472c1ac5b69affa0ae1132c39adcb67aba98568702b9cc23f \ + --hash=sha256:7f86fd587c4ed9dd76f318225e7d9b29cfc5a9d43de44e5754db8d1128487085 \ + --hash=sha256:7f94b27a62cfad8496c8d2513e1a222dd446f095fca8987fceef261225538a15 \ + --hash=sha256:88dceee75c2c63af144e456745e10101eb67361050196b0b6af5d717254dddf7 \ + --hash=sha256:8a650c2dbafa08d42e51ba0b62740dae4ecb9338eefa093aa5c78ceb546fcd5c \ + --hash=sha256:8d65a2fbf9d2f8352685bc1364177ee3923d6baf5e7f43ea4959d7d8bc326a36 \ + --hash=sha256:96481a5786729fd470164b47cdb3e0e58062a496f455ee41b4403be77cb5a076 \ + --hash=sha256:a120733b01c45e9a0c34aeef92bf0cf1d56cfe81ed9d47d562f9ed591a9828ac \ + --hash=sha256:b1d22e6e9387bf4739fbe23bfa80e93f6b0373a7f1b96c6227c32bef95a4d7a8 \ + --hash=sha256:b8c198f8c1805dc42708689ed6864951fd2494f924149d3e4bce7710f8eb5232 \ + --hash=sha256:c2541745709bad0264b7d4705ad453b76ccd191e64aa6f0fc66b69a293a45ece \ + --hash=sha256:c742f741d58a28940ce01d58f0ab2ea3ced8b12402f162f4d534dfe18ba1cd6a \ + --hash=sha256:c7f2c7f2b9ca6bdeef8f0fa897f8e05085923eb091721675170254cbc5b02897 \ + --hash=sha256:d312ef37c91508b0ab2cee7da26ec0b3ed2f03ce12bd87a588d771ae15dcf82d \ + --hash=sha256:d4d8fe59808a54658fcc0160ecfb1b30f9089906c50b23bcb4c69eddc19ec2b4 \ + --hash=sha256:da25dc3563bff5965356133435b757a795a17b17d01dbc0f42fb32447ddfd917 \ + --hash=sha256:eab21f45c7f66c13f2a9e0e1535309cee140182a9cdae1e041d02e47291e8396 \ + --hash=sha256:eb0dc4e38e6a1fd579e5d50369aa2e10acfc9cace504579b2faabb478e76941a \ + --hash=sha256:ec9bfaf3ad2df51ace80688143a6a4ebc09a248f6ff781a9945e51937008fcbc \ + --hash=sha256:ede3e6487c5ef5d28634ba3f31f989030ad6af71edfb0055cbbd14189ff240ba \ + --hash=sha256:f3c6818a1a86dd6dca7ddcaaf76947d5ba31aecc28cb1b67009a5877c9a64f3f \ + --hash=sha256:f758f1b9299d059cc3f6546ae2af89670cb1c4d48ea29c3cacc4fe7de3058257 \ + --hash=sha256:f8f0fc26ec2cc2b965b7a3b87cd19c5c6b8c5e5f436b984e85f486d652285c30 \ + --hash=sha256:fd0409a3653af6c147209d267a0e4243f0ae46b011aa978b1080359fddc9b6cf \ + --hash=sha256:ff18e6a727ee0ab0388507b89d1bc6a22b138d1e2fa56d1ad494586d61d2eae9 \ + --hash=sha256:ff2983983d34813c1aeb0fa89091e76c3a22889ee83ab27c5eeb45100560c049 + # via + # pytest + # sphinx + # sphinx-autodoc2 +typing-extensions==4.16.0 \ + --hash=sha256:481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8 \ + --hash=sha256:dc983d19a509c94dba722ee6abd33940f7c05a89e243c47e907eb4db6f1a43e5 + # via + # rules-python-dev (dev/pyproject.toml) + # astroid + # exceptiongroup + # sphinx-autodoc2 +urllib3==2.7.0 \ + --hash=sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c \ + --hash=sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897 + # via requests diff --git a/dev/uv.lock b/dev/uv.lock new file mode 100644 index 0000000000..b524eb1f61 --- /dev/null +++ b/dev/uv.lock @@ -0,0 +1,985 @@ +version = 1 +revision = 3 +requires-python = ">=3.10" +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", + "python_full_version < '3.11'", +] + +[[package]] +name = "absl-py" +version = "2.4.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/64/c7/8de93764ad66968d19329a7e0c147a2bb3c7054c554d4a119111b8f9440f/absl_py-2.4.0.tar.gz", hash = "sha256:8c6af82722b35cf71e0f4d1d47dcaebfff286e27110a99fc359349b247dfb5d4", size = 116543, upload-time = "2026-01-28T10:17:05.322Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/18/a6/907a406bb7d359e6a63f99c313846d9eec4f7e6f7437809e03aa00fa3074/absl_py-2.4.0-py3-none-any.whl", hash = "sha256:88476fd881ca8aab94ffa78b7b6c632a782ab3ba1cd19c9bd423abc4fb4cd28d", size = 135750, upload-time = "2026-01-28T10:17:04.19Z" }, +] + +[[package]] +name = "alabaster" +version = "1.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/a6/f8/d9c74d0daf3f742840fd818d69cfae176fa332022fd44e3469487d5a9420/alabaster-1.0.0.tar.gz", hash = "sha256:c00dca57bca26fa62a6d7d0a9fcce65f3e026e9bfe33e9c538fd3fbb2144fd9e", size = 24210, upload-time = "2024-07-26T18:15:03.762Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7e/b3/6b4067be973ae96ba0d615946e314c5ae35f9f993eca561b356540bb0c2b/alabaster-1.0.0-py3-none-any.whl", hash = "sha256:fc6786402dc3fcb2de3cabd5fe455a2db534b371124f1f21de8731783dec828b", size = 13929, upload-time = "2024-07-26T18:15:02.05Z" }, +] + +[[package]] +name = "altgraph" +version = "0.17.5" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/7e/f8/97fdf103f38fed6792a1601dbc16cc8aac56e7459a9fff08c812d8ae177a/altgraph-0.17.5.tar.gz", hash = "sha256:c87b395dd12fabde9c99573a9749d67da8d29ef9de0125c7f536699b4a9bc9e7", size = 48428, upload-time = "2025-11-21T20:35:50.583Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a9/ba/000a1996d4308bc65120167c21241a3b205464a2e0b58deda26ae8ac21d1/altgraph-0.17.5-py2.py3-none-any.whl", hash = "sha256:f3a22400bce1b0c701683820ac4f3b159cd301acab067c51c653e06961600597", size = 21228, upload-time = "2025-11-21T20:35:49.444Z" }, +] + +[[package]] +name = "astroid" +version = "3.3.11" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions", marker = "python_full_version < '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/18/74/dfb75f9ccd592bbedb175d4a32fc643cf569d7c218508bfbd6ea7ef9c091/astroid-3.3.11.tar.gz", hash = "sha256:1e5a5011af2920c7c67a53f65d536d65bfa7116feeaf2354d8b94f29573bb0ce", size = 400439, upload-time = "2025-07-13T18:04:23.177Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/af/0f/3b8fdc946b4d9cc8cc1e8af42c4e409468c84441b933d037e101b3d72d86/astroid-3.3.11-py3-none-any.whl", hash = "sha256:54c760ae8322ece1abd213057c4b5bba7c49818853fc901ef09719a60dbf9dec", size = 275612, upload-time = "2025-07-13T18:04:21.07Z" }, +] + +[[package]] +name = "babel" +version = "2.18.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/7d/b2/51899539b6ceeeb420d40ed3cd4b7a40519404f9baf3d4ac99dc413a834b/babel-2.18.0.tar.gz", hash = "sha256:b80b99a14bd085fcacfa15c9165f651fbb3406e66cc603abf11c5750937c992d", size = 9959554, upload-time = "2026-02-01T12:30:56.078Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/77/f5/21d2de20e8b8b0408f0681956ca2c69f1320a3848ac50e6e7f39c6159675/babel-2.18.0-py3-none-any.whl", hash = "sha256:e2b422b277c2b9a9630c1d7903c2a00d0830c409c59ac8cae9081c92f1aeba35", size = 10196845, upload-time = "2026-02-01T12:30:53.445Z" }, +] + +[[package]] +name = "certifi" +version = "2026.4.22" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/25/ee/6caf7a40c36a1220410afe15a1cc64993a1f864871f698c0f93acb72842a/certifi-2026.4.22.tar.gz", hash = "sha256:8d455352a37b71bf76a79caa83a3d6c25afee4a385d632127b6afb3963f1c580", size = 137077, upload-time = "2026-04-22T11:26:11.191Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/22/30/7cd8fdcdfbc5b869528b079bfb76dcdf6056b1a2097a662e5e8c04f42965/certifi-2026.4.22-py3-none-any.whl", hash = "sha256:3cb2210c8f88ba2318d29b0388d1023c8492ff72ecdde4ebdaddbb13a31b1c4a", size = 135707, upload-time = "2026-04-22T11:26:09.372Z" }, +] + +[[package]] +name = "charset-normalizer" +version = "3.4.7" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/e7/a1/67fe25fac3c7642725500a3f6cfe5821ad557c3abb11c9d20d12c7008d3e/charset_normalizer-3.4.7.tar.gz", hash = "sha256:ae89db9e5f98a11a4bf50407d4363e7b09b31e55bc117b4f7d80aab97ba009e5", size = 144271, upload-time = "2026-04-02T09:28:39.342Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/26/08/0f303cb0b529e456bb116f2d50565a482694fbb94340bf56d44677e7ed03/charset_normalizer-3.4.7-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:cdd68a1fb318e290a2077696b7eb7a21a49163c455979c639bf5a5dcdc46617d", size = 315182, upload-time = "2026-04-02T09:25:40.673Z" }, + { url = "https://files.pythonhosted.org/packages/24/47/b192933e94b546f1b1fe4df9cc1f84fcdbf2359f8d1081d46dd029b50207/charset_normalizer-3.4.7-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e17b8d5d6a8c47c85e68ca8379def1303fd360c3e22093a807cd34a71cd082b8", size = 209329, upload-time = "2026-04-02T09:25:42.354Z" }, + { url = "https://files.pythonhosted.org/packages/c2/b4/01fa81c5ca6141024d89a8fc15968002b71da7f825dd14113207113fabbd/charset_normalizer-3.4.7-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:511ef87c8aec0783e08ac18565a16d435372bc1ac25a91e6ac7f5ef2b0bff790", size = 231230, upload-time = "2026-04-02T09:25:44.281Z" }, + { url = "https://files.pythonhosted.org/packages/20/f7/7b991776844dfa058017e600e6e55ff01984a063290ca5622c0b63162f68/charset_normalizer-3.4.7-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:007d05ec7321d12a40227aae9e2bc6dca73f3cb21058999a1df9e193555a9dcc", size = 225890, upload-time = "2026-04-02T09:25:45.475Z" }, + { url = "https://files.pythonhosted.org/packages/20/e7/bed0024a0f4ab0c8a9c64d4445f39b30c99bd1acd228291959e3de664247/charset_normalizer-3.4.7-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:cf29836da5119f3c8a8a70667b0ef5fdca3bb12f80fd06487cfa575b3909b393", size = 216930, upload-time = "2026-04-02T09:25:46.58Z" }, + { url = "https://files.pythonhosted.org/packages/e2/ab/b18f0ab31cdd7b3ddb8bb76c4a414aeb8160c9810fdf1bc62f269a539d87/charset_normalizer-3.4.7-cp310-cp310-manylinux_2_31_armv7l.whl", hash = "sha256:12d8baf840cc7889b37c7c770f478adea7adce3dcb3944d02ec87508e2dcf153", size = 202109, upload-time = "2026-04-02T09:25:48.031Z" }, + { url = "https://files.pythonhosted.org/packages/82/e5/7e9440768a06dfb3075936490cb82dbf0ee20a133bf0dd8551fa096914ec/charset_normalizer-3.4.7-cp310-cp310-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:d560742f3c0d62afaccf9f41fe485ed69bd7661a241f86a3ef0f0fb8b1a397af", size = 214684, upload-time = "2026-04-02T09:25:49.245Z" }, + { url = "https://files.pythonhosted.org/packages/71/94/8c61d8da9f062fdf457c80acfa25060ec22bf1d34bbeaca4350f13bcfd07/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:b14b2d9dac08e28bb8046a1a0434b1750eb221c8f5b87a68f4fa11a6f97b5e34", size = 212785, upload-time = "2026-04-02T09:25:50.671Z" }, + { url = "https://files.pythonhosted.org/packages/66/cd/6e9889c648e72c0ab2e5967528bb83508f354d706637bc7097190c874e13/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_armv7l.whl", hash = "sha256:bc17a677b21b3502a21f66a8cc64f5bfad4df8a0b8434d661666f8ce90ac3af1", size = 203055, upload-time = "2026-04-02T09:25:51.802Z" }, + { url = "https://files.pythonhosted.org/packages/92/2e/7a951d6a08aefb7eb8e1b54cdfb580b1365afdd9dd484dc4bee9e5d8f258/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_ppc64le.whl", hash = "sha256:750e02e074872a3fad7f233b47734166440af3cdea0add3e95163110816d6752", size = 232502, upload-time = "2026-04-02T09:25:53.388Z" }, + { url = "https://files.pythonhosted.org/packages/58/d5/abcf2d83bf8e0a1286df55cd0dc1d49af0da4282aa77e986df343e7de124/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_riscv64.whl", hash = "sha256:4e5163c14bffd570ef2affbfdd77bba66383890797df43dc8b4cc7d6f500bf53", size = 214295, upload-time = "2026-04-02T09:25:54.765Z" }, + { url = "https://files.pythonhosted.org/packages/47/3a/7d4cd7ed54be99973a0dc176032cba5cb1f258082c31fa6df35cff46acfc/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_s390x.whl", hash = "sha256:6ed74185b2db44f41ef35fd1617c5888e59792da9bbc9190d6c7300617182616", size = 227145, upload-time = "2026-04-02T09:25:55.904Z" }, + { url = "https://files.pythonhosted.org/packages/1d/98/3a45bf8247889cf28262ebd3d0872edff11565b2a1e3064ccb132db3fbb0/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:94e1885b270625a9a828c9793b4d52a64445299baa1fea5a173bf1d3dd9a1a5a", size = 218884, upload-time = "2026-04-02T09:25:57.074Z" }, + { url = "https://files.pythonhosted.org/packages/ad/80/2e8b7f8915ed5c9ef13aa828d82738e33888c485b65ebf744d615040c7ea/charset_normalizer-3.4.7-cp310-cp310-win32.whl", hash = "sha256:6785f414ae0f3c733c437e0f3929197934f526d19dfaa75e18fdb4f94c6fb374", size = 148343, upload-time = "2026-04-02T09:25:58.199Z" }, + { url = "https://files.pythonhosted.org/packages/35/1b/3b8c8c77184af465ee9ad88b5aea46ea6b2e1f7b9dc9502891e37af21e30/charset_normalizer-3.4.7-cp310-cp310-win_amd64.whl", hash = "sha256:6696b7688f54f5af4462118f0bfa7c1621eeb87154f77fa04b9295ce7a8f2943", size = 159174, upload-time = "2026-04-02T09:25:59.322Z" }, + { url = "https://files.pythonhosted.org/packages/be/c1/feb40dca40dbb21e0a908801782d9288c64fc8d8e562c2098e9994c8c21b/charset_normalizer-3.4.7-cp310-cp310-win_arm64.whl", hash = "sha256:66671f93accb62ed07da56613636f3641f1a12c13046ce91ffc923721f23c008", size = 147805, upload-time = "2026-04-02T09:26:00.756Z" }, + { url = "https://files.pythonhosted.org/packages/c2/d7/b5b7020a0565c2e9fa8c09f4b5fa6232feb326b8c20081ccded47ea368fd/charset_normalizer-3.4.7-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:7641bb8895e77f921102f72833904dcd9901df5d6d72a2ab8f31d04b7e51e4e7", size = 309705, upload-time = "2026-04-02T09:26:02.191Z" }, + { url = "https://files.pythonhosted.org/packages/5a/53/58c29116c340e5456724ecd2fff4196d236b98f3da97b404bc5e51ac3493/charset_normalizer-3.4.7-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:202389074300232baeb53ae2569a60901f7efadd4245cf3a3bf0617d60b439d7", size = 206419, upload-time = "2026-04-02T09:26:03.583Z" }, + { url = "https://files.pythonhosted.org/packages/b2/02/e8146dc6591a37a00e5144c63f29fb7c97a734ea8a111190783c0e60ab63/charset_normalizer-3.4.7-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:30b8d1d8c52a48c2c5690e152c169b673487a2a58de1ec7393196753063fcd5e", size = 227901, upload-time = "2026-04-02T09:26:04.738Z" }, + { url = "https://files.pythonhosted.org/packages/fb/73/77486c4cd58f1267bf17db420e930c9afa1b3be3fe8c8b8ebbebc9624359/charset_normalizer-3.4.7-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:532bc9bf33a68613fd7d65e4b1c71a6a38d7d42604ecf239c77392e9b4e8998c", size = 222742, upload-time = "2026-04-02T09:26:06.36Z" }, + { url = "https://files.pythonhosted.org/packages/a1/fa/f74eb381a7d94ded44739e9d94de18dc5edc9c17fb8c11f0a6890696c0a9/charset_normalizer-3.4.7-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2fe249cb4651fd12605b7288b24751d8bfd46d35f12a20b1ba33dea122e690df", size = 214061, upload-time = "2026-04-02T09:26:08.347Z" }, + { url = "https://files.pythonhosted.org/packages/dc/92/42bd3cefcf7687253fb86694b45f37b733c97f59af3724f356fa92b8c344/charset_normalizer-3.4.7-cp311-cp311-manylinux_2_31_armv7l.whl", hash = "sha256:65bcd23054beab4d166035cabbc868a09c1a49d1efe458fe8e4361215df40265", size = 199239, upload-time = "2026-04-02T09:26:09.823Z" }, + { url = "https://files.pythonhosted.org/packages/4c/3d/069e7184e2aa3b3cddc700e3dd267413dc259854adc3380421c805c6a17d/charset_normalizer-3.4.7-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:08e721811161356f97b4059a9ba7bafb23ea5ee2255402c42881c214e173c6b4", size = 210173, upload-time = "2026-04-02T09:26:10.953Z" }, + { url = "https://files.pythonhosted.org/packages/62/51/9d56feb5f2e7074c46f93e0ebdbe61f0848ee246e2f0d89f8e20b89ebb8f/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:e060d01aec0a910bdccb8be71faf34e7799ce36950f8294c8bf612cba65a2c9e", size = 209841, upload-time = "2026-04-02T09:26:12.142Z" }, + { url = "https://files.pythonhosted.org/packages/d2/59/893d8f99cc4c837dda1fe2f1139079703deb9f321aabcb032355de13b6c7/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:38c0109396c4cfc574d502df99742a45c72c08eff0a36158b6f04000043dbf38", size = 200304, upload-time = "2026-04-02T09:26:13.711Z" }, + { url = "https://files.pythonhosted.org/packages/7d/1d/ee6f3be3464247578d1ed5c46de545ccc3d3ff933695395c402c21fa6b77/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:1c2a768fdd44ee4a9339a9b0b130049139b8ce3c01d2ce09f67f5a68048d477c", size = 229455, upload-time = "2026-04-02T09:26:14.941Z" }, + { url = "https://files.pythonhosted.org/packages/54/bb/8fb0a946296ea96a488928bdce8ef99023998c48e4713af533e9bb98ef07/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:1a87ca9d5df6fe460483d9a5bbf2b18f620cbed41b432e2bddb686228282d10b", size = 210036, upload-time = "2026-04-02T09:26:16.478Z" }, + { url = "https://files.pythonhosted.org/packages/9a/bc/015b2387f913749f82afd4fcba07846d05b6d784dd16123cb66860e0237d/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_s390x.whl", hash = "sha256:d635aab80466bc95771bb78d5370e74d36d1fe31467b6b29b8b57b2a3cd7d22c", size = 224739, upload-time = "2026-04-02T09:26:17.751Z" }, + { url = "https://files.pythonhosted.org/packages/17/ab/63133691f56baae417493cba6b7c641571a2130eb7bceba6773367ab9ec5/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:ae196f021b5e7c78e918242d217db021ed2a6ace2bc6ae94c0fc596221c7f58d", size = 216277, upload-time = "2026-04-02T09:26:18.981Z" }, + { url = "https://files.pythonhosted.org/packages/06/6d/3be70e827977f20db77c12a97e6a9f973631a45b8d186c084527e53e77a4/charset_normalizer-3.4.7-cp311-cp311-win32.whl", hash = "sha256:adb2597b428735679446b46c8badf467b4ca5f5056aae4d51a19f9570301b1ad", size = 147819, upload-time = "2026-04-02T09:26:20.295Z" }, + { url = "https://files.pythonhosted.org/packages/20/d9/5f67790f06b735d7c7637171bbfd89882ad67201891b7275e51116ed8207/charset_normalizer-3.4.7-cp311-cp311-win_amd64.whl", hash = "sha256:8e385e4267ab76874ae30db04c627faaaf0b509e1ccc11a95b3fc3e83f855c00", size = 159281, upload-time = "2026-04-02T09:26:21.74Z" }, + { url = "https://files.pythonhosted.org/packages/ca/83/6413f36c5a34afead88ce6f66684d943d91f233d76dd083798f9602b75ae/charset_normalizer-3.4.7-cp311-cp311-win_arm64.whl", hash = "sha256:d4a48e5b3c2a489fae013b7589308a40146ee081f6f509e047e0e096084ceca1", size = 147843, upload-time = "2026-04-02T09:26:22.901Z" }, + { url = "https://files.pythonhosted.org/packages/0c/eb/4fc8d0a7110eb5fc9cc161723a34a8a6c200ce3b4fbf681bc86feee22308/charset_normalizer-3.4.7-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:eca9705049ad3c7345d574e3510665cb2cf844c2f2dcfe675332677f081cbd46", size = 311328, upload-time = "2026-04-02T09:26:24.331Z" }, + { url = "https://files.pythonhosted.org/packages/f8/e3/0fadc706008ac9d7b9b5be6dc767c05f9d3e5df51744ce4cc9605de7b9f4/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6178f72c5508bfc5fd446a5905e698c6212932f25bcdd4b47a757a50605a90e2", size = 208061, upload-time = "2026-04-02T09:26:25.568Z" }, + { url = "https://files.pythonhosted.org/packages/42/f0/3dd1045c47f4a4604df85ec18ad093912ae1344ac706993aff91d38773a2/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e1421b502d83040e6d7fb2fb18dff63957f720da3d77b2fbd3187ceb63755d7b", size = 229031, upload-time = "2026-04-02T09:26:26.865Z" }, + { url = "https://files.pythonhosted.org/packages/dc/67/675a46eb016118a2fbde5a277a5d15f4f69d5f3f5f338e5ee2f8948fcf43/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:edac0f1ab77644605be2cbba52e6b7f630731fc42b34cb0f634be1a6eface56a", size = 225239, upload-time = "2026-04-02T09:26:28.044Z" }, + { url = "https://files.pythonhosted.org/packages/4b/f8/d0118a2f5f23b02cd166fa385c60f9b0d4f9194f574e2b31cef350ad7223/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5649fd1c7bade02f320a462fdefd0b4bd3ce036065836d4f42e0de958038e116", size = 216589, upload-time = "2026-04-02T09:26:29.239Z" }, + { url = "https://files.pythonhosted.org/packages/b1/f1/6d2b0b261b6c4ceef0fcb0d17a01cc5bc53586c2d4796fa04b5c540bc13d/charset_normalizer-3.4.7-cp312-cp312-manylinux_2_31_armv7l.whl", hash = "sha256:203104ed3e428044fd943bc4bf45fa73c0730391f9621e37fe39ecf477b128cb", size = 202733, upload-time = "2026-04-02T09:26:30.5Z" }, + { url = "https://files.pythonhosted.org/packages/6f/c0/7b1f943f7e87cc3db9626ba17807d042c38645f0a1d4415c7a14afb5591f/charset_normalizer-3.4.7-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:298930cec56029e05497a76988377cbd7457ba864beeea92ad7e844fe74cd1f1", size = 212652, upload-time = "2026-04-02T09:26:31.709Z" }, + { url = "https://files.pythonhosted.org/packages/38/dd/5a9ab159fe45c6e72079398f277b7d2b523e7f716acc489726115a910097/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:708838739abf24b2ceb208d0e22403dd018faeef86ddac04319a62ae884c4f15", size = 211229, upload-time = "2026-04-02T09:26:33.282Z" }, + { url = "https://files.pythonhosted.org/packages/d5/ff/531a1cad5ca855d1c1a8b69cb71abfd6d85c0291580146fda7c82857caa1/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:0f7eb884681e3938906ed0434f20c63046eacd0111c4ba96f27b76084cd679f5", size = 203552, upload-time = "2026-04-02T09:26:34.845Z" }, + { url = "https://files.pythonhosted.org/packages/c1/4c/a5fb52d528a8ca41f7598cb619409ece30a169fbdf9cdce592e53b46c3a6/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:4dc1e73c36828f982bfe79fadf5919923f8a6f4df2860804db9a98c48824ce8d", size = 230806, upload-time = "2026-04-02T09:26:36.152Z" }, + { url = "https://files.pythonhosted.org/packages/59/7a/071feed8124111a32b316b33ae4de83d36923039ef8cf48120266844285b/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:aed52fea0513bac0ccde438c188c8a471c4e0f457c2dd20cdbf6ea7a450046c7", size = 212316, upload-time = "2026-04-02T09:26:37.672Z" }, + { url = "https://files.pythonhosted.org/packages/fd/35/f7dba3994312d7ba508e041eaac39a36b120f32d4c8662b8814dab876431/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:fea24543955a6a729c45a73fe90e08c743f0b3334bbf3201e6c4bc1b0c7fa464", size = 227274, upload-time = "2026-04-02T09:26:38.93Z" }, + { url = "https://files.pythonhosted.org/packages/8a/2d/a572df5c9204ab7688ec1edc895a73ebded3b023bb07364710b05dd1c9be/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:bb6d88045545b26da47aa879dd4a89a71d1dce0f0e549b1abcb31dfe4a8eac49", size = 218468, upload-time = "2026-04-02T09:26:40.17Z" }, + { url = "https://files.pythonhosted.org/packages/86/eb/890922a8b03a568ca2f336c36585a4713c55d4d67bf0f0c78924be6315ca/charset_normalizer-3.4.7-cp312-cp312-win32.whl", hash = "sha256:2257141f39fe65a3fdf38aeccae4b953e5f3b3324f4ff0daf9f15b8518666a2c", size = 148460, upload-time = "2026-04-02T09:26:41.416Z" }, + { url = "https://files.pythonhosted.org/packages/35/d9/0e7dffa06c5ab081f75b1b786f0aefc88365825dfcd0ac544bdb7b2b6853/charset_normalizer-3.4.7-cp312-cp312-win_amd64.whl", hash = "sha256:5ed6ab538499c8644b8a3e18debabcd7ce684f3fa91cf867521a7a0279cab2d6", size = 159330, upload-time = "2026-04-02T09:26:42.554Z" }, + { url = "https://files.pythonhosted.org/packages/9e/5d/481bcc2a7c88ea6b0878c299547843b2521ccbc40980cb406267088bc701/charset_normalizer-3.4.7-cp312-cp312-win_arm64.whl", hash = "sha256:56be790f86bfb2c98fb742ce566dfb4816e5a83384616ab59c49e0604d49c51d", size = 147828, upload-time = "2026-04-02T09:26:44.075Z" }, + { url = "https://files.pythonhosted.org/packages/c1/3b/66777e39d3ae1ddc77ee606be4ec6d8cbd4c801f65e5a1b6f2b11b8346dd/charset_normalizer-3.4.7-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:f496c9c3cc02230093d8330875c4c3cdfc3b73612a5fd921c65d39cbcef08063", size = 309627, upload-time = "2026-04-02T09:26:45.198Z" }, + { url = "https://files.pythonhosted.org/packages/2e/4e/b7f84e617b4854ade48a1b7915c8ccfadeba444d2a18c291f696e37f0d3b/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0ea948db76d31190bf08bd371623927ee1339d5f2a0b4b1b4a4439a65298703c", size = 207008, upload-time = "2026-04-02T09:26:46.824Z" }, + { url = "https://files.pythonhosted.org/packages/c4/bb/ec73c0257c9e11b268f018f068f5d00aa0ef8c8b09f7753ebd5f2880e248/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a277ab8928b9f299723bc1a2dabb1265911b1a76341f90a510368ca44ad9ab66", size = 228303, upload-time = "2026-04-02T09:26:48.397Z" }, + { url = "https://files.pythonhosted.org/packages/85/fb/32d1f5033484494619f701e719429c69b766bfc4dbc61aa9e9c8c166528b/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:3bec022aec2c514d9cf199522a802bd007cd588ab17ab2525f20f9c34d067c18", size = 224282, upload-time = "2026-04-02T09:26:49.684Z" }, + { url = "https://files.pythonhosted.org/packages/fa/07/330e3a0dda4c404d6da83b327270906e9654a24f6c546dc886a0eb0ffb23/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e044c39e41b92c845bc815e5ae4230804e8e7bc29e399b0437d64222d92809dd", size = 215595, upload-time = "2026-04-02T09:26:50.915Z" }, + { url = "https://files.pythonhosted.org/packages/e3/7c/fc890655786e423f02556e0216d4b8c6bcb6bdfa890160dc66bf52dee468/charset_normalizer-3.4.7-cp313-cp313-manylinux_2_31_armv7l.whl", hash = "sha256:f495a1652cf3fbab2eb0639776dad966c2fb874d79d87ca07f9d5f059b8bd215", size = 201986, upload-time = "2026-04-02T09:26:52.197Z" }, + { url = "https://files.pythonhosted.org/packages/d8/97/bfb18b3db2aed3b90cf54dc292ad79fdd5ad65c4eae454099475cbeadd0d/charset_normalizer-3.4.7-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e712b419df8ba5e42b226c510472b37bd57b38e897d3eca5e8cfd410a29fa859", size = 211711, upload-time = "2026-04-02T09:26:53.49Z" }, + { url = "https://files.pythonhosted.org/packages/6f/a5/a581c13798546a7fd557c82614a5c65a13df2157e9ad6373166d2a3e645d/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:7804338df6fcc08105c7745f1502ba68d900f45fd770d5bdd5288ddccb8a42d8", size = 210036, upload-time = "2026-04-02T09:26:54.975Z" }, + { url = "https://files.pythonhosted.org/packages/8c/bf/b3ab5bcb478e4193d517644b0fb2bf5497fbceeaa7a1bc0f4d5b50953861/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:481551899c856c704d58119b5025793fa6730adda3571971af568f66d2424bb5", size = 202998, upload-time = "2026-04-02T09:26:56.303Z" }, + { url = "https://files.pythonhosted.org/packages/e7/4e/23efd79b65d314fa320ec6017b4b5834d5c12a58ba4610aa353af2e2f577/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:f59099f9b66f0d7145115e6f80dd8b1d847176df89b234a5a6b3f00437aa0832", size = 230056, upload-time = "2026-04-02T09:26:57.554Z" }, + { url = "https://files.pythonhosted.org/packages/b9/9f/1e1941bc3f0e01df116e68dc37a55c4d249df5e6fa77f008841aef68264f/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:f59ad4c0e8f6bba240a9bb85504faa1ab438237199d4cce5f622761507b8f6a6", size = 211537, upload-time = "2026-04-02T09:26:58.843Z" }, + { url = "https://files.pythonhosted.org/packages/80/0f/088cbb3020d44428964a6c97fe1edfb1b9550396bf6d278330281e8b709c/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:3dedcc22d73ec993f42055eff4fcfed9318d1eeb9a6606c55892a26964964e48", size = 226176, upload-time = "2026-04-02T09:27:00.437Z" }, + { url = "https://files.pythonhosted.org/packages/6a/9f/130394f9bbe06f4f63e22641d32fc9b202b7e251c9aef4db044324dac493/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:64f02c6841d7d83f832cd97ccf8eb8a906d06eb95d5276069175c696b024b60a", size = 217723, upload-time = "2026-04-02T09:27:02.021Z" }, + { url = "https://files.pythonhosted.org/packages/73/55/c469897448a06e49f8fa03f6caae97074fde823f432a98f979cc42b90e69/charset_normalizer-3.4.7-cp313-cp313-win32.whl", hash = "sha256:4042d5c8f957e15221d423ba781e85d553722fc4113f523f2feb7b188cc34c5e", size = 148085, upload-time = "2026-04-02T09:27:03.192Z" }, + { url = "https://files.pythonhosted.org/packages/5d/78/1b74c5bbb3f99b77a1715c91b3e0b5bdb6fe302d95ace4f5b1bec37b0167/charset_normalizer-3.4.7-cp313-cp313-win_amd64.whl", hash = "sha256:3946fa46a0cf3e4c8cb1cc52f56bb536310d34f25f01ca9b6c16afa767dab110", size = 158819, upload-time = "2026-04-02T09:27:04.454Z" }, + { url = "https://files.pythonhosted.org/packages/68/86/46bd42279d323deb8687c4a5a811fd548cb7d1de10cf6535d099877a9a9f/charset_normalizer-3.4.7-cp313-cp313-win_arm64.whl", hash = "sha256:80d04837f55fc81da168b98de4f4b797ef007fc8a79ab71c6ec9bc4dd662b15b", size = 147915, upload-time = "2026-04-02T09:27:05.971Z" }, + { url = "https://files.pythonhosted.org/packages/97/c8/c67cb8c70e19ef1960b97b22ed2a1567711de46c4ddf19799923adc836c2/charset_normalizer-3.4.7-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:c36c333c39be2dbca264d7803333c896ab8fa7d4d6f0ab7edb7dfd7aea6e98c0", size = 309234, upload-time = "2026-04-02T09:27:07.194Z" }, + { url = "https://files.pythonhosted.org/packages/99/85/c091fdee33f20de70d6c8b522743b6f831a2f1cd3ff86de4c6a827c48a76/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1c2aed2e5e41f24ea8ef1590b8e848a79b56f3a5564a65ceec43c9d692dc7d8a", size = 208042, upload-time = "2026-04-02T09:27:08.749Z" }, + { url = "https://files.pythonhosted.org/packages/87/1c/ab2ce611b984d2fd5d86a5a8a19c1ae26acac6bad967da4967562c75114d/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:54523e136b8948060c0fa0bc7b1b50c32c186f2fceee897a495406bb6e311d2b", size = 228706, upload-time = "2026-04-02T09:27:09.951Z" }, + { url = "https://files.pythonhosted.org/packages/a8/29/2b1d2cb00bf085f59d29eb773ce58ec2d325430f8c216804a0a5cd83cbca/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:715479b9a2802ecac752a3b0efa2b0b60285cf962ee38414211abdfccc233b41", size = 224727, upload-time = "2026-04-02T09:27:11.175Z" }, + { url = "https://files.pythonhosted.org/packages/47/5c/032c2d5a07fe4d4855fea851209cca2b6f03ebeb6d4e3afdb3358386a684/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:bd6c2a1c7573c64738d716488d2cdd3c00e340e4835707d8fdb8dc1a66ef164e", size = 215882, upload-time = "2026-04-02T09:27:12.446Z" }, + { url = "https://files.pythonhosted.org/packages/2c/c2/356065d5a8b78ed04499cae5f339f091946a6a74f91e03476c33f0ab7100/charset_normalizer-3.4.7-cp314-cp314-manylinux_2_31_armv7l.whl", hash = "sha256:c45e9440fb78f8ddabcf714b68f936737a121355bf59f3907f4e17721b9d1aae", size = 200860, upload-time = "2026-04-02T09:27:13.721Z" }, + { url = "https://files.pythonhosted.org/packages/0c/cd/a32a84217ced5039f53b29f460962abb2d4420def55afabe45b1c3c7483d/charset_normalizer-3.4.7-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:3534e7dcbdcf757da6b85a0bbf5b6868786d5982dd959b065e65481644817a18", size = 211564, upload-time = "2026-04-02T09:27:15.272Z" }, + { url = "https://files.pythonhosted.org/packages/44/86/58e6f13ce26cc3b8f4a36b94a0f22ae2f00a72534520f4ae6857c4b81f89/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:e8ac484bf18ce6975760921bb6148041faa8fef0547200386ea0b52b5d27bf7b", size = 211276, upload-time = "2026-04-02T09:27:16.834Z" }, + { url = "https://files.pythonhosted.org/packages/8f/fe/d17c32dc72e17e155e06883efa84514ca375f8a528ba2546bee73fc4df81/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:a5fe03b42827c13cdccd08e6c0247b6a6d4b5e3cdc53fd1749f5896adcdc2356", size = 201238, upload-time = "2026-04-02T09:27:18.229Z" }, + { url = "https://files.pythonhosted.org/packages/6a/29/f33daa50b06525a237451cdb6c69da366c381a3dadcd833fa5676bc468b3/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:2d6eb928e13016cea4f1f21d1e10c1cebd5a421bc57ddf5b1142ae3f86824fab", size = 230189, upload-time = "2026-04-02T09:27:19.445Z" }, + { url = "https://files.pythonhosted.org/packages/b6/6e/52c84015394a6a0bdcd435210a7e944c5f94ea1055f5cc5d56c5fe368e7b/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:e74327fb75de8986940def6e8dee4f127cc9752bee7355bb323cc5b2659b6d46", size = 211352, upload-time = "2026-04-02T09:27:20.79Z" }, + { url = "https://files.pythonhosted.org/packages/8c/d7/4353be581b373033fb9198bf1da3cf8f09c1082561e8e922aa7b39bf9fe8/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:d6038d37043bced98a66e68d3aa2b6a35505dc01328cd65217cefe82f25def44", size = 227024, upload-time = "2026-04-02T09:27:22.063Z" }, + { url = "https://files.pythonhosted.org/packages/30/45/99d18aa925bd1740098ccd3060e238e21115fffbfdcb8f3ece837d0ace6c/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:7579e913a5339fb8fa133f6bbcfd8e6749696206cf05acdbdca71a1b436d8e72", size = 217869, upload-time = "2026-04-02T09:27:23.486Z" }, + { url = "https://files.pythonhosted.org/packages/5c/05/5ee478aa53f4bb7996482153d4bfe1b89e0f087f0ab6b294fcf92d595873/charset_normalizer-3.4.7-cp314-cp314-win32.whl", hash = "sha256:5b77459df20e08151cd6f8b9ef8ef1f961ef73d85c21a555c7eed5b79410ec10", size = 148541, upload-time = "2026-04-02T09:27:25.146Z" }, + { url = "https://files.pythonhosted.org/packages/48/77/72dcb0921b2ce86420b2d79d454c7022bf5be40202a2a07906b9f2a35c97/charset_normalizer-3.4.7-cp314-cp314-win_amd64.whl", hash = "sha256:92a0a01ead5e668468e952e4238cccd7c537364eb7d851ab144ab6627dbbe12f", size = 159634, upload-time = "2026-04-02T09:27:26.642Z" }, + { url = "https://files.pythonhosted.org/packages/c6/a3/c2369911cd72f02386e4e340770f6e158c7980267da16af8f668217abaa0/charset_normalizer-3.4.7-cp314-cp314-win_arm64.whl", hash = "sha256:67f6279d125ca0046a7fd386d01b311c6363844deac3e5b069b514ba3e63c246", size = 148384, upload-time = "2026-04-02T09:27:28.271Z" }, + { url = "https://files.pythonhosted.org/packages/94/09/7e8a7f73d24dba1f0035fbbf014d2c36828fc1bf9c88f84093e57d315935/charset_normalizer-3.4.7-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:effc3f449787117233702311a1b7d8f59cba9ced946ba727bdc329ec69028e24", size = 330133, upload-time = "2026-04-02T09:27:29.474Z" }, + { url = "https://files.pythonhosted.org/packages/8d/da/96975ddb11f8e977f706f45cddd8540fd8242f71ecdb5d18a80723dcf62c/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:fbccdc05410c9ee21bbf16a35f4c1d16123dcdeb8a1d38f33654fa21d0234f79", size = 216257, upload-time = "2026-04-02T09:27:30.793Z" }, + { url = "https://files.pythonhosted.org/packages/e5/e8/1d63bf8ef2d388e95c64b2098f45f84758f6d102a087552da1485912637b/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:733784b6d6def852c814bce5f318d25da2ee65dd4839a0718641c696e09a2960", size = 234851, upload-time = "2026-04-02T09:27:32.44Z" }, + { url = "https://files.pythonhosted.org/packages/9b/40/e5ff04233e70da2681fa43969ad6f66ca5611d7e669be0246c4c7aaf6dc8/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a89c23ef8d2c6b27fd200a42aa4ac72786e7c60d40efdc76e6011260b6e949c4", size = 233393, upload-time = "2026-04-02T09:27:34.03Z" }, + { url = "https://files.pythonhosted.org/packages/be/c1/06c6c49d5a5450f76899992f1ee40b41d076aee9279b49cf9974d2f313d5/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6c114670c45346afedc0d947faf3c7f701051d2518b943679c8ff88befe14f8e", size = 223251, upload-time = "2026-04-02T09:27:35.369Z" }, + { url = "https://files.pythonhosted.org/packages/2b/9f/f2ff16fb050946169e3e1f82134d107e5d4ae72647ec8a1b1446c148480f/charset_normalizer-3.4.7-cp314-cp314t-manylinux_2_31_armv7l.whl", hash = "sha256:a180c5e59792af262bf263b21a3c49353f25945d8d9f70628e73de370d55e1e1", size = 206609, upload-time = "2026-04-02T09:27:36.661Z" }, + { url = "https://files.pythonhosted.org/packages/69/d5/a527c0cd8d64d2eab7459784fb4169a0ac76e5a6fc5237337982fd61347e/charset_normalizer-3.4.7-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:3c9a494bc5ec77d43cea229c4f6db1e4d8fe7e1bbffa8b6f0f0032430ff8ab44", size = 220014, upload-time = "2026-04-02T09:27:38.019Z" }, + { url = "https://files.pythonhosted.org/packages/7e/80/8a7b8104a3e203074dc9aa2c613d4b726c0e136bad1cc734594b02867972/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:8d828b6667a32a728a1ad1d93957cdf37489c57b97ae6c4de2860fa749b8fc1e", size = 218979, upload-time = "2026-04-02T09:27:39.37Z" }, + { url = "https://files.pythonhosted.org/packages/02/9a/b759b503d507f375b2b5c153e4d2ee0a75aa215b7f2489cf314f4541f2c0/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:cf1493cd8607bec4d8a7b9b004e699fcf8f9103a9284cc94962cb73d20f9d4a3", size = 209238, upload-time = "2026-04-02T09:27:40.722Z" }, + { url = "https://files.pythonhosted.org/packages/c2/4e/0f3f5d47b86bdb79256e7290b26ac847a2832d9a4033f7eb2cd4bcf4bb5b/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:0c96c3b819b5c3e9e165495db84d41914d6894d55181d2d108cc1a69bfc9cce0", size = 236110, upload-time = "2026-04-02T09:27:42.33Z" }, + { url = "https://files.pythonhosted.org/packages/96/23/bce28734eb3ed2c91dcf93abeb8a5cf393a7b2749725030bb630e554fdd8/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:752a45dc4a6934060b3b0dab47e04edc3326575f82be64bc4fc293914566503e", size = 219824, upload-time = "2026-04-02T09:27:43.924Z" }, + { url = "https://files.pythonhosted.org/packages/2c/6f/6e897c6984cc4d41af319b077f2f600fc8214eb2fe2d6bcb79141b882400/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:8778f0c7a52e56f75d12dae53ae320fae900a8b9b4164b981b9c5ce059cd1fcb", size = 233103, upload-time = "2026-04-02T09:27:45.348Z" }, + { url = "https://files.pythonhosted.org/packages/76/22/ef7bd0fe480a0ae9b656189ec00744b60933f68b4f42a7bb06589f6f576a/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:ce3412fbe1e31eb81ea42f4169ed94861c56e643189e1e75f0041f3fe7020abe", size = 225194, upload-time = "2026-04-02T09:27:46.706Z" }, + { url = "https://files.pythonhosted.org/packages/c5/a7/0e0ab3e0b5bc1219bd80a6a0d4d72ca74d9250cb2382b7c699c147e06017/charset_normalizer-3.4.7-cp314-cp314t-win32.whl", hash = "sha256:c03a41a8784091e67a39648f70c5f97b5b6a37f216896d44d2cdcb82615339a0", size = 159827, upload-time = "2026-04-02T09:27:48.053Z" }, + { url = "https://files.pythonhosted.org/packages/7a/1d/29d32e0fb40864b1f878c7f5a0b343ae676c6e2b271a2d55cc3a152391da/charset_normalizer-3.4.7-cp314-cp314t-win_amd64.whl", hash = "sha256:03853ed82eeebbce3c2abfdbc98c96dc205f32a79627688ac9a27370ea61a49c", size = 174168, upload-time = "2026-04-02T09:27:49.795Z" }, + { url = "https://files.pythonhosted.org/packages/de/32/d92444ad05c7a6e41fb2036749777c163baf7a0301a040cb672d6b2b1ae9/charset_normalizer-3.4.7-cp314-cp314t-win_arm64.whl", hash = "sha256:c35abb8bfff0185efac5878da64c45dafd2b37fb0383add1be155a763c1f083d", size = 153018, upload-time = "2026-04-02T09:27:51.116Z" }, + { url = "https://files.pythonhosted.org/packages/db/8f/61959034484a4a7c527811f4721e75d02d653a35afb0b6054474d8185d4c/charset_normalizer-3.4.7-py3-none-any.whl", hash = "sha256:3dce51d0f5e7951f8bb4900c257dad282f49190fdbebecd4ba99bcc41fef404d", size = 61958, upload-time = "2026-04-02T09:28:37.794Z" }, +] + +[[package]] +name = "colorama" +version = "0.4.6" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" }, +] + +[[package]] +name = "docutils" +version = "0.21.2" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.11'", +] +sdist = { url = "https://files.pythonhosted.org/packages/ae/ed/aefcc8cd0ba62a0560c3c18c33925362d46c6075480bfa4df87b28e169a9/docutils-0.21.2.tar.gz", hash = "sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f", size = 2204444, upload-time = "2024-04-23T18:57:18.24Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/8f/d7/9322c609343d929e75e7e5e6255e614fcc67572cfd083959cdef3b7aad79/docutils-0.21.2-py3-none-any.whl", hash = "sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2", size = 587408, upload-time = "2024-04-23T18:57:14.835Z" }, +] + +[[package]] +name = "docutils" +version = "0.22.4" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", +] +sdist = { url = "https://files.pythonhosted.org/packages/ae/b6/03bb70946330e88ffec97aefd3ea75ba575cb2e762061e0e62a213befee8/docutils-0.22.4.tar.gz", hash = "sha256:4db53b1fde9abecbb74d91230d32ab626d94f6badfc575d6db9194a49df29968", size = 2291750, upload-time = "2025-12-18T19:00:26.443Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/02/10/5da547df7a391dcde17f59520a231527b8571e6f46fc8efb02ccb370ab12/docutils-0.22.4-py3-none-any.whl", hash = "sha256:d0013f540772d1420576855455d050a2180186c91c15779301ac2ccb3eeb68de", size = 633196, upload-time = "2025-12-18T19:00:18.077Z" }, +] + +[[package]] +name = "exceptiongroup" +version = "1.3.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions", marker = "python_full_version < '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/50/79/66800aadf48771f6b62f7eb014e352e5d06856655206165d775e675a02c9/exceptiongroup-1.3.1.tar.gz", hash = "sha256:8b412432c6055b0b7d14c310000ae93352ed6754f70fa8f7c34141f91c4e3219", size = 30371, upload-time = "2025-11-21T23:01:54.787Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/8a/0e/97c33bf5009bdbac74fd2beace167cab3f978feb69cc36f1ef79360d6c4e/exceptiongroup-1.3.1-py3-none-any.whl", hash = "sha256:a7a39a3bd276781e98394987d3a5701d0c4edffb633bb7a5144577f82c773598", size = 16740, upload-time = "2025-11-21T23:01:53.443Z" }, +] + +[[package]] +name = "idna" +version = "3.15" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/82/77/7b3966d0b9d1d31a36ddf1746926a11dface89a83409bf1483f0237aa758/idna-3.15.tar.gz", hash = "sha256:ca962446ea538f7092a95e057da437618e886f4d349216d2b1e294abfdb65fdc", size = 199245, upload-time = "2026-05-12T22:45:57.011Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d2/23/408243171aa9aaba178d3e2559159c24c1171a641aa83b67bdd3394ead8e/idna-3.15-py3-none-any.whl", hash = "sha256:048adeaf8c2d788c40fee287673ccaa74c24ffd8dcf09ffa555a2fbb59f10ac8", size = 72340, upload-time = "2026-05-12T22:45:55.733Z" }, +] + +[[package]] +name = "imagesize" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/6c/e6/7bf14eeb8f8b7251141944835abd42eb20a658d89084b7e1f3e5fe394090/imagesize-2.0.0.tar.gz", hash = "sha256:8e8358c4a05c304f1fccf7ff96f036e7243a189e9e42e90851993c558cfe9ee3", size = 1773045, upload-time = "2026-03-03T14:18:29.941Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5f/53/fb7122b71361a0d121b669dcf3d31244ef75badbbb724af388948de543e2/imagesize-2.0.0-py2.py3-none-any.whl", hash = "sha256:5667c5bbb57ab3f1fa4bc366f4fbc971db3d5ed011fd2715fd8001f782718d96", size = 9441, upload-time = "2026-03-03T14:18:27.892Z" }, +] + +[[package]] +name = "iniconfig" +version = "2.3.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/72/34/14ca021ce8e5dfedc35312d08ba8bf51fdd999c576889fc2c24cb97f4f10/iniconfig-2.3.0.tar.gz", hash = "sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730", size = 20503, upload-time = "2025-10-18T21:55:43.219Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" }, +] + +[[package]] +name = "jinja2" +version = "3.1.6" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "markupsafe" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/df/bf/f7da0350254c0ed7c72f3e33cef02e048281fec7ecec5f032d4aac52226b/jinja2-3.1.6.tar.gz", hash = "sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d", size = 245115, upload-time = "2025-03-05T20:05:02.478Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/62/a1/3d680cbfd5f4b8f15abc1d571870c5fc3e594bb582bc3b64ea099db13e56/jinja2-3.1.6-py3-none-any.whl", hash = "sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67", size = 134899, upload-time = "2025-03-05T20:05:00.369Z" }, +] + +[[package]] +name = "macholib" +version = "1.16.4" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "altgraph" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/10/2f/97589876ea967487978071c9042518d28b958d87b17dceb7cdc1d881f963/macholib-1.16.4.tar.gz", hash = "sha256:f408c93ab2e995cd2c46e34fe328b130404be143469e41bc366c807448979362", size = 59427, upload-time = "2025-11-22T08:28:38.373Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c7/d1/a9f36f8ecdf0fb7c9b1e78c8d7af12b8c8754e74851ac7b94a8305540fc7/macholib-1.16.4-py2.py3-none-any.whl", hash = "sha256:da1a3fa8266e30f0ce7e97c6a54eefaae8edd1e5f86f3eb8b95457cae90265ea", size = 38117, upload-time = "2025-11-22T08:28:36.939Z" }, +] + +[[package]] +name = "markdown-it-py" +version = "3.0.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.11'", +] +dependencies = [ + { name = "mdurl", marker = "python_full_version < '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/38/71/3b932df36c1a044d397a1f92d1cf91ee0a503d91e470cbd670aa66b07ed0/markdown-it-py-3.0.0.tar.gz", hash = "sha256:e3f60a94fa066dc52ec76661e37c851cb232d92f9886b15cb560aaada2df8feb", size = 74596, upload-time = "2023-06-03T06:41:14.443Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/42/d7/1ec15b46af6af88f19b8e5ffea08fa375d433c998b8a7639e76935c14f1f/markdown_it_py-3.0.0-py3-none-any.whl", hash = "sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1", size = 87528, upload-time = "2023-06-03T06:41:11.019Z" }, +] + +[[package]] +name = "markdown-it-py" +version = "4.2.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", +] +dependencies = [ + { name = "mdurl", marker = "python_full_version >= '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/06/ff/7841249c247aa650a76b9ee4bbaeae59370dc8bfd2f6c01f3630c35eb134/markdown_it_py-4.2.0.tar.gz", hash = "sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49", size = 82454, upload-time = "2026-05-07T12:08:28.36Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b3/81/4da04ced5a082363ecfa159c010d200ecbd959ae410c10c0264a38cac0f5/markdown_it_py-4.2.0-py3-none-any.whl", hash = "sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a", size = 91687, upload-time = "2026-05-07T12:08:27.182Z" }, +] + +[[package]] +name = "markupsafe" +version = "3.0.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/7e/99/7690b6d4034fffd95959cbe0c02de8deb3098cc577c67bb6a24fe5d7caa7/markupsafe-3.0.3.tar.gz", hash = "sha256:722695808f4b6457b320fdc131280796bdceb04ab50fe1795cd540799ebe1698", size = 80313, upload-time = "2025-09-27T18:37:40.426Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e8/4b/3541d44f3937ba468b75da9eebcae497dcf67adb65caa16760b0a6807ebb/markupsafe-3.0.3-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:2f981d352f04553a7171b8e44369f2af4055f888dfb147d55e42d29e29e74559", size = 11631, upload-time = "2025-09-27T18:36:05.558Z" }, + { url = "https://files.pythonhosted.org/packages/98/1b/fbd8eed11021cabd9226c37342fa6ca4e8a98d8188a8d9b66740494960e4/markupsafe-3.0.3-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:e1c1493fb6e50ab01d20a22826e57520f1284df32f2d8601fdd90b6304601419", size = 12057, upload-time = "2025-09-27T18:36:07.165Z" }, + { url = "https://files.pythonhosted.org/packages/40/01/e560d658dc0bb8ab762670ece35281dec7b6c1b33f5fbc09ebb57a185519/markupsafe-3.0.3-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1ba88449deb3de88bd40044603fafffb7bc2b055d626a330323a9ed736661695", size = 22050, upload-time = "2025-09-27T18:36:08.005Z" }, + { url = "https://files.pythonhosted.org/packages/af/cd/ce6e848bbf2c32314c9b237839119c5a564a59725b53157c856e90937b7a/markupsafe-3.0.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f42d0984e947b8adf7dd6dde396e720934d12c506ce84eea8476409563607591", size = 20681, upload-time = "2025-09-27T18:36:08.881Z" }, + { url = "https://files.pythonhosted.org/packages/c9/2a/b5c12c809f1c3045c4d580b035a743d12fcde53cf685dbc44660826308da/markupsafe-3.0.3-cp310-cp310-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:c0c0b3ade1c0b13b936d7970b1d37a57acde9199dc2aecc4c336773e1d86049c", size = 20705, upload-time = "2025-09-27T18:36:10.131Z" }, + { url = "https://files.pythonhosted.org/packages/cf/e3/9427a68c82728d0a88c50f890d0fc072a1484de2f3ac1ad0bfc1a7214fd5/markupsafe-3.0.3-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:0303439a41979d9e74d18ff5e2dd8c43ed6c6001fd40e5bf2e43f7bd9bbc523f", size = 21524, upload-time = "2025-09-27T18:36:11.324Z" }, + { url = "https://files.pythonhosted.org/packages/bc/36/23578f29e9e582a4d0278e009b38081dbe363c5e7165113fad546918a232/markupsafe-3.0.3-cp310-cp310-musllinux_1_2_riscv64.whl", hash = "sha256:d2ee202e79d8ed691ceebae8e0486bd9a2cd4794cec4824e1c99b6f5009502f6", size = 20282, upload-time = "2025-09-27T18:36:12.573Z" }, + { url = "https://files.pythonhosted.org/packages/56/21/dca11354e756ebd03e036bd8ad58d6d7168c80ce1fe5e75218e4945cbab7/markupsafe-3.0.3-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:177b5253b2834fe3678cb4a5f0059808258584c559193998be2601324fdeafb1", size = 20745, upload-time = "2025-09-27T18:36:13.504Z" }, + { url = "https://files.pythonhosted.org/packages/87/99/faba9369a7ad6e4d10b6a5fbf71fa2a188fe4a593b15f0963b73859a1bbd/markupsafe-3.0.3-cp310-cp310-win32.whl", hash = "sha256:2a15a08b17dd94c53a1da0438822d70ebcd13f8c3a95abe3a9ef9f11a94830aa", size = 14571, upload-time = "2025-09-27T18:36:14.779Z" }, + { url = "https://files.pythonhosted.org/packages/d6/25/55dc3ab959917602c96985cb1253efaa4ff42f71194bddeb61eb7278b8be/markupsafe-3.0.3-cp310-cp310-win_amd64.whl", hash = "sha256:c4ffb7ebf07cfe8931028e3e4c85f0357459a3f9f9490886198848f4fa002ec8", size = 15056, upload-time = "2025-09-27T18:36:16.125Z" }, + { url = "https://files.pythonhosted.org/packages/d0/9e/0a02226640c255d1da0b8d12e24ac2aa6734da68bff14c05dd53b94a0fc3/markupsafe-3.0.3-cp310-cp310-win_arm64.whl", hash = "sha256:e2103a929dfa2fcaf9bb4e7c091983a49c9ac3b19c9061b6d5427dd7d14d81a1", size = 13932, upload-time = "2025-09-27T18:36:17.311Z" }, + { url = "https://files.pythonhosted.org/packages/08/db/fefacb2136439fc8dd20e797950e749aa1f4997ed584c62cfb8ef7c2be0e/markupsafe-3.0.3-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:1cc7ea17a6824959616c525620e387f6dd30fec8cb44f649e31712db02123dad", size = 11631, upload-time = "2025-09-27T18:36:18.185Z" }, + { url = "https://files.pythonhosted.org/packages/e1/2e/5898933336b61975ce9dc04decbc0a7f2fee78c30353c5efba7f2d6ff27a/markupsafe-3.0.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:4bd4cd07944443f5a265608cc6aab442e4f74dff8088b0dfc8238647b8f6ae9a", size = 12058, upload-time = "2025-09-27T18:36:19.444Z" }, + { url = "https://files.pythonhosted.org/packages/1d/09/adf2df3699d87d1d8184038df46a9c80d78c0148492323f4693df54e17bb/markupsafe-3.0.3-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6b5420a1d9450023228968e7e6a9ce57f65d148ab56d2313fcd589eee96a7a50", size = 24287, upload-time = "2025-09-27T18:36:20.768Z" }, + { url = "https://files.pythonhosted.org/packages/30/ac/0273f6fcb5f42e314c6d8cd99effae6a5354604d461b8d392b5ec9530a54/markupsafe-3.0.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0bf2a864d67e76e5c9a34dc26ec616a66b9888e25e7b9460e1c76d3293bd9dbf", size = 22940, upload-time = "2025-09-27T18:36:22.249Z" }, + { url = "https://files.pythonhosted.org/packages/19/ae/31c1be199ef767124c042c6c3e904da327a2f7f0cd63a0337e1eca2967a8/markupsafe-3.0.3-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:bc51efed119bc9cfdf792cdeaa4d67e8f6fcccab66ed4bfdd6bde3e59bfcbb2f", size = 21887, upload-time = "2025-09-27T18:36:23.535Z" }, + { url = "https://files.pythonhosted.org/packages/b2/76/7edcab99d5349a4532a459e1fe64f0b0467a3365056ae550d3bcf3f79e1e/markupsafe-3.0.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:068f375c472b3e7acbe2d5318dea141359e6900156b5b2ba06a30b169086b91a", size = 23692, upload-time = "2025-09-27T18:36:24.823Z" }, + { url = "https://files.pythonhosted.org/packages/a4/28/6e74cdd26d7514849143d69f0bf2399f929c37dc2b31e6829fd2045b2765/markupsafe-3.0.3-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:7be7b61bb172e1ed687f1754f8e7484f1c8019780f6f6b0786e76bb01c2ae115", size = 21471, upload-time = "2025-09-27T18:36:25.95Z" }, + { url = "https://files.pythonhosted.org/packages/62/7e/a145f36a5c2945673e590850a6f8014318d5577ed7e5920a4b3448e0865d/markupsafe-3.0.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:f9e130248f4462aaa8e2552d547f36ddadbeaa573879158d721bbd33dfe4743a", size = 22923, upload-time = "2025-09-27T18:36:27.109Z" }, + { url = "https://files.pythonhosted.org/packages/0f/62/d9c46a7f5c9adbeeeda52f5b8d802e1094e9717705a645efc71b0913a0a8/markupsafe-3.0.3-cp311-cp311-win32.whl", hash = "sha256:0db14f5dafddbb6d9208827849fad01f1a2609380add406671a26386cdf15a19", size = 14572, upload-time = "2025-09-27T18:36:28.045Z" }, + { url = "https://files.pythonhosted.org/packages/83/8a/4414c03d3f891739326e1783338e48fb49781cc915b2e0ee052aa490d586/markupsafe-3.0.3-cp311-cp311-win_amd64.whl", hash = "sha256:de8a88e63464af587c950061a5e6a67d3632e36df62b986892331d4620a35c01", size = 15077, upload-time = "2025-09-27T18:36:29.025Z" }, + { url = "https://files.pythonhosted.org/packages/35/73/893072b42e6862f319b5207adc9ae06070f095b358655f077f69a35601f0/markupsafe-3.0.3-cp311-cp311-win_arm64.whl", hash = "sha256:3b562dd9e9ea93f13d53989d23a7e775fdfd1066c33494ff43f5418bc8c58a5c", size = 13876, upload-time = "2025-09-27T18:36:29.954Z" }, + { url = "https://files.pythonhosted.org/packages/5a/72/147da192e38635ada20e0a2e1a51cf8823d2119ce8883f7053879c2199b5/markupsafe-3.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:d53197da72cc091b024dd97249dfc7794d6a56530370992a5e1a08983ad9230e", size = 11615, upload-time = "2025-09-27T18:36:30.854Z" }, + { url = "https://files.pythonhosted.org/packages/9a/81/7e4e08678a1f98521201c3079f77db69fb552acd56067661f8c2f534a718/markupsafe-3.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:1872df69a4de6aead3491198eaf13810b565bdbeec3ae2dc8780f14458ec73ce", size = 12020, upload-time = "2025-09-27T18:36:31.971Z" }, + { url = "https://files.pythonhosted.org/packages/1e/2c/799f4742efc39633a1b54a92eec4082e4f815314869865d876824c257c1e/markupsafe-3.0.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3a7e8ae81ae39e62a41ec302f972ba6ae23a5c5396c8e60113e9066ef893da0d", size = 24332, upload-time = "2025-09-27T18:36:32.813Z" }, + { url = "https://files.pythonhosted.org/packages/3c/2e/8d0c2ab90a8c1d9a24f0399058ab8519a3279d1bd4289511d74e909f060e/markupsafe-3.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d6dd0be5b5b189d31db7cda48b91d7e0a9795f31430b7f271219ab30f1d3ac9d", size = 22947, upload-time = "2025-09-27T18:36:33.86Z" }, + { url = "https://files.pythonhosted.org/packages/2c/54/887f3092a85238093a0b2154bd629c89444f395618842e8b0c41783898ea/markupsafe-3.0.3-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:94c6f0bb423f739146aec64595853541634bde58b2135f27f61c1ffd1cd4d16a", size = 21962, upload-time = "2025-09-27T18:36:35.099Z" }, + { url = "https://files.pythonhosted.org/packages/c9/2f/336b8c7b6f4a4d95e91119dc8521402461b74a485558d8f238a68312f11c/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:be8813b57049a7dc738189df53d69395eba14fb99345e0a5994914a3864c8a4b", size = 23760, upload-time = "2025-09-27T18:36:36.001Z" }, + { url = "https://files.pythonhosted.org/packages/32/43/67935f2b7e4982ffb50a4d169b724d74b62a3964bc1a9a527f5ac4f1ee2b/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:83891d0e9fb81a825d9a6d61e3f07550ca70a076484292a70fde82c4b807286f", size = 21529, upload-time = "2025-09-27T18:36:36.906Z" }, + { url = "https://files.pythonhosted.org/packages/89/e0/4486f11e51bbba8b0c041098859e869e304d1c261e59244baa3d295d47b7/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:77f0643abe7495da77fb436f50f8dab76dbc6e5fd25d39589a0f1fe6548bfa2b", size = 23015, upload-time = "2025-09-27T18:36:37.868Z" }, + { url = "https://files.pythonhosted.org/packages/2f/e1/78ee7a023dac597a5825441ebd17170785a9dab23de95d2c7508ade94e0e/markupsafe-3.0.3-cp312-cp312-win32.whl", hash = "sha256:d88b440e37a16e651bda4c7c2b930eb586fd15ca7406cb39e211fcff3bf3017d", size = 14540, upload-time = "2025-09-27T18:36:38.761Z" }, + { url = "https://files.pythonhosted.org/packages/aa/5b/bec5aa9bbbb2c946ca2733ef9c4ca91c91b6a24580193e891b5f7dbe8e1e/markupsafe-3.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:26a5784ded40c9e318cfc2bdb30fe164bdb8665ded9cd64d500a34fb42067b1c", size = 15105, upload-time = "2025-09-27T18:36:39.701Z" }, + { url = "https://files.pythonhosted.org/packages/e5/f1/216fc1bbfd74011693a4fd837e7026152e89c4bcf3e77b6692fba9923123/markupsafe-3.0.3-cp312-cp312-win_arm64.whl", hash = "sha256:35add3b638a5d900e807944a078b51922212fb3dedb01633a8defc4b01a3c85f", size = 13906, upload-time = "2025-09-27T18:36:40.689Z" }, + { url = "https://files.pythonhosted.org/packages/38/2f/907b9c7bbba283e68f20259574b13d005c121a0fa4c175f9bed27c4597ff/markupsafe-3.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:e1cf1972137e83c5d4c136c43ced9ac51d0e124706ee1c8aa8532c1287fa8795", size = 11622, upload-time = "2025-09-27T18:36:41.777Z" }, + { url = "https://files.pythonhosted.org/packages/9c/d9/5f7756922cdd676869eca1c4e3c0cd0df60ed30199ffd775e319089cb3ed/markupsafe-3.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:116bb52f642a37c115f517494ea5feb03889e04df47eeff5b130b1808ce7c219", size = 12029, upload-time = "2025-09-27T18:36:43.257Z" }, + { url = "https://files.pythonhosted.org/packages/00/07/575a68c754943058c78f30db02ee03a64b3c638586fba6a6dd56830b30a3/markupsafe-3.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:133a43e73a802c5562be9bbcd03d090aa5a1fe899db609c29e8c8d815c5f6de6", size = 24374, upload-time = "2025-09-27T18:36:44.508Z" }, + { url = "https://files.pythonhosted.org/packages/a9/21/9b05698b46f218fc0e118e1f8168395c65c8a2c750ae2bab54fc4bd4e0e8/markupsafe-3.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ccfcd093f13f0f0b7fdd0f198b90053bf7b2f02a3927a30e63f3ccc9df56b676", size = 22980, upload-time = "2025-09-27T18:36:45.385Z" }, + { url = "https://files.pythonhosted.org/packages/7f/71/544260864f893f18b6827315b988c146b559391e6e7e8f7252839b1b846a/markupsafe-3.0.3-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:509fa21c6deb7a7a273d629cf5ec029bc209d1a51178615ddf718f5918992ab9", size = 21990, upload-time = "2025-09-27T18:36:46.916Z" }, + { url = "https://files.pythonhosted.org/packages/c2/28/b50fc2f74d1ad761af2f5dcce7492648b983d00a65b8c0e0cb457c82ebbe/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a4afe79fb3de0b7097d81da19090f4df4f8d3a2b3adaa8764138aac2e44f3af1", size = 23784, upload-time = "2025-09-27T18:36:47.884Z" }, + { url = "https://files.pythonhosted.org/packages/ed/76/104b2aa106a208da8b17a2fb72e033a5a9d7073c68f7e508b94916ed47a9/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:795e7751525cae078558e679d646ae45574b47ed6e7771863fcc079a6171a0fc", size = 21588, upload-time = "2025-09-27T18:36:48.82Z" }, + { url = "https://files.pythonhosted.org/packages/b5/99/16a5eb2d140087ebd97180d95249b00a03aa87e29cc224056274f2e45fd6/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:8485f406a96febb5140bfeca44a73e3ce5116b2501ac54fe953e488fb1d03b12", size = 23041, upload-time = "2025-09-27T18:36:49.797Z" }, + { url = "https://files.pythonhosted.org/packages/19/bc/e7140ed90c5d61d77cea142eed9f9c303f4c4806f60a1044c13e3f1471d0/markupsafe-3.0.3-cp313-cp313-win32.whl", hash = "sha256:bdd37121970bfd8be76c5fb069c7751683bdf373db1ed6c010162b2a130248ed", size = 14543, upload-time = "2025-09-27T18:36:51.584Z" }, + { url = "https://files.pythonhosted.org/packages/05/73/c4abe620b841b6b791f2edc248f556900667a5a1cf023a6646967ae98335/markupsafe-3.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:9a1abfdc021a164803f4d485104931fb8f8c1efd55bc6b748d2f5774e78b62c5", size = 15113, upload-time = "2025-09-27T18:36:52.537Z" }, + { url = "https://files.pythonhosted.org/packages/f0/3a/fa34a0f7cfef23cf9500d68cb7c32dd64ffd58a12b09225fb03dd37d5b80/markupsafe-3.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:7e68f88e5b8799aa49c85cd116c932a1ac15caaa3f5db09087854d218359e485", size = 13911, upload-time = "2025-09-27T18:36:53.513Z" }, + { url = "https://files.pythonhosted.org/packages/e4/d7/e05cd7efe43a88a17a37b3ae96e79a19e846f3f456fe79c57ca61356ef01/markupsafe-3.0.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:218551f6df4868a8d527e3062d0fb968682fe92054e89978594c28e642c43a73", size = 11658, upload-time = "2025-09-27T18:36:54.819Z" }, + { url = "https://files.pythonhosted.org/packages/99/9e/e412117548182ce2148bdeacdda3bb494260c0b0184360fe0d56389b523b/markupsafe-3.0.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:3524b778fe5cfb3452a09d31e7b5adefeea8c5be1d43c4f810ba09f2ceb29d37", size = 12066, upload-time = "2025-09-27T18:36:55.714Z" }, + { url = "https://files.pythonhosted.org/packages/bc/e6/fa0ffcda717ef64a5108eaa7b4f5ed28d56122c9a6d70ab8b72f9f715c80/markupsafe-3.0.3-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4e885a3d1efa2eadc93c894a21770e4bc67899e3543680313b09f139e149ab19", size = 25639, upload-time = "2025-09-27T18:36:56.908Z" }, + { url = "https://files.pythonhosted.org/packages/96/ec/2102e881fe9d25fc16cb4b25d5f5cde50970967ffa5dddafdb771237062d/markupsafe-3.0.3-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8709b08f4a89aa7586de0aadc8da56180242ee0ada3999749b183aa23df95025", size = 23569, upload-time = "2025-09-27T18:36:57.913Z" }, + { url = "https://files.pythonhosted.org/packages/4b/30/6f2fce1f1f205fc9323255b216ca8a235b15860c34b6798f810f05828e32/markupsafe-3.0.3-cp313-cp313t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b8512a91625c9b3da6f127803b166b629725e68af71f8184ae7e7d54686a56d6", size = 23284, upload-time = "2025-09-27T18:36:58.833Z" }, + { url = "https://files.pythonhosted.org/packages/58/47/4a0ccea4ab9f5dcb6f79c0236d954acb382202721e704223a8aafa38b5c8/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:9b79b7a16f7fedff2495d684f2b59b0457c3b493778c9eed31111be64d58279f", size = 24801, upload-time = "2025-09-27T18:36:59.739Z" }, + { url = "https://files.pythonhosted.org/packages/6a/70/3780e9b72180b6fecb83a4814d84c3bf4b4ae4bf0b19c27196104149734c/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_riscv64.whl", hash = "sha256:12c63dfb4a98206f045aa9563db46507995f7ef6d83b2f68eda65c307c6829eb", size = 22769, upload-time = "2025-09-27T18:37:00.719Z" }, + { url = "https://files.pythonhosted.org/packages/98/c5/c03c7f4125180fc215220c035beac6b9cb684bc7a067c84fc69414d315f5/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:8f71bc33915be5186016f675cd83a1e08523649b0e33efdb898db577ef5bb009", size = 23642, upload-time = "2025-09-27T18:37:01.673Z" }, + { url = "https://files.pythonhosted.org/packages/80/d6/2d1b89f6ca4bff1036499b1e29a1d02d282259f3681540e16563f27ebc23/markupsafe-3.0.3-cp313-cp313t-win32.whl", hash = "sha256:69c0b73548bc525c8cb9a251cddf1931d1db4d2258e9599c28c07ef3580ef354", size = 14612, upload-time = "2025-09-27T18:37:02.639Z" }, + { url = "https://files.pythonhosted.org/packages/2b/98/e48a4bfba0a0ffcf9925fe2d69240bfaa19c6f7507b8cd09c70684a53c1e/markupsafe-3.0.3-cp313-cp313t-win_amd64.whl", hash = "sha256:1b4b79e8ebf6b55351f0d91fe80f893b4743f104bff22e90697db1590e47a218", size = 15200, upload-time = "2025-09-27T18:37:03.582Z" }, + { url = "https://files.pythonhosted.org/packages/0e/72/e3cc540f351f316e9ed0f092757459afbc595824ca724cbc5a5d4263713f/markupsafe-3.0.3-cp313-cp313t-win_arm64.whl", hash = "sha256:ad2cf8aa28b8c020ab2fc8287b0f823d0a7d8630784c31e9ee5edea20f406287", size = 13973, upload-time = "2025-09-27T18:37:04.929Z" }, + { url = "https://files.pythonhosted.org/packages/33/8a/8e42d4838cd89b7dde187011e97fe6c3af66d8c044997d2183fbd6d31352/markupsafe-3.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:eaa9599de571d72e2daf60164784109f19978b327a3910d3e9de8c97b5b70cfe", size = 11619, upload-time = "2025-09-27T18:37:06.342Z" }, + { url = "https://files.pythonhosted.org/packages/b5/64/7660f8a4a8e53c924d0fa05dc3a55c9cee10bbd82b11c5afb27d44b096ce/markupsafe-3.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:c47a551199eb8eb2121d4f0f15ae0f923d31350ab9280078d1e5f12b249e0026", size = 12029, upload-time = "2025-09-27T18:37:07.213Z" }, + { url = "https://files.pythonhosted.org/packages/da/ef/e648bfd021127bef5fa12e1720ffed0c6cbb8310c8d9bea7266337ff06de/markupsafe-3.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f34c41761022dd093b4b6896d4810782ffbabe30f2d443ff5f083e0cbbb8c737", size = 24408, upload-time = "2025-09-27T18:37:09.572Z" }, + { url = "https://files.pythonhosted.org/packages/41/3c/a36c2450754618e62008bf7435ccb0f88053e07592e6028a34776213d877/markupsafe-3.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:457a69a9577064c05a97c41f4e65148652db078a3a509039e64d3467b9e7ef97", size = 23005, upload-time = "2025-09-27T18:37:10.58Z" }, + { url = "https://files.pythonhosted.org/packages/bc/20/b7fdf89a8456b099837cd1dc21974632a02a999ec9bf7ca3e490aacd98e7/markupsafe-3.0.3-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e8afc3f2ccfa24215f8cb28dcf43f0113ac3c37c2f0f0806d8c70e4228c5cf4d", size = 22048, upload-time = "2025-09-27T18:37:11.547Z" }, + { url = "https://files.pythonhosted.org/packages/9a/a7/591f592afdc734f47db08a75793a55d7fbcc6902a723ae4cfbab61010cc5/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:ec15a59cf5af7be74194f7ab02d0f59a62bdcf1a537677ce67a2537c9b87fcda", size = 23821, upload-time = "2025-09-27T18:37:12.48Z" }, + { url = "https://files.pythonhosted.org/packages/7d/33/45b24e4f44195b26521bc6f1a82197118f74df348556594bd2262bda1038/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:0eb9ff8191e8498cca014656ae6b8d61f39da5f95b488805da4bb029cccbfbaf", size = 21606, upload-time = "2025-09-27T18:37:13.485Z" }, + { url = "https://files.pythonhosted.org/packages/ff/0e/53dfaca23a69fbfbbf17a4b64072090e70717344c52eaaaa9c5ddff1e5f0/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:2713baf880df847f2bece4230d4d094280f4e67b1e813eec43b4c0e144a34ffe", size = 23043, upload-time = "2025-09-27T18:37:14.408Z" }, + { url = "https://files.pythonhosted.org/packages/46/11/f333a06fc16236d5238bfe74daccbca41459dcd8d1fa952e8fbd5dccfb70/markupsafe-3.0.3-cp314-cp314-win32.whl", hash = "sha256:729586769a26dbceff69f7a7dbbf59ab6572b99d94576a5592625d5b411576b9", size = 14747, upload-time = "2025-09-27T18:37:15.36Z" }, + { url = "https://files.pythonhosted.org/packages/28/52/182836104b33b444e400b14f797212f720cbc9ed6ba34c800639d154e821/markupsafe-3.0.3-cp314-cp314-win_amd64.whl", hash = "sha256:bdc919ead48f234740ad807933cdf545180bfbe9342c2bb451556db2ed958581", size = 15341, upload-time = "2025-09-27T18:37:16.496Z" }, + { url = "https://files.pythonhosted.org/packages/6f/18/acf23e91bd94fd7b3031558b1f013adfa21a8e407a3fdb32745538730382/markupsafe-3.0.3-cp314-cp314-win_arm64.whl", hash = "sha256:5a7d5dc5140555cf21a6fefbdbf8723f06fcd2f63ef108f2854de715e4422cb4", size = 14073, upload-time = "2025-09-27T18:37:17.476Z" }, + { url = "https://files.pythonhosted.org/packages/3c/f0/57689aa4076e1b43b15fdfa646b04653969d50cf30c32a102762be2485da/markupsafe-3.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:1353ef0c1b138e1907ae78e2f6c63ff67501122006b0f9abad68fda5f4ffc6ab", size = 11661, upload-time = "2025-09-27T18:37:18.453Z" }, + { url = "https://files.pythonhosted.org/packages/89/c3/2e67a7ca217c6912985ec766c6393b636fb0c2344443ff9d91404dc4c79f/markupsafe-3.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1085e7fbddd3be5f89cc898938f42c0b3c711fdcb37d75221de2666af647c175", size = 12069, upload-time = "2025-09-27T18:37:19.332Z" }, + { url = "https://files.pythonhosted.org/packages/f0/00/be561dce4e6ca66b15276e184ce4b8aec61fe83662cce2f7d72bd3249d28/markupsafe-3.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1b52b4fb9df4eb9ae465f8d0c228a00624de2334f216f178a995ccdcf82c4634", size = 25670, upload-time = "2025-09-27T18:37:20.245Z" }, + { url = "https://files.pythonhosted.org/packages/50/09/c419f6f5a92e5fadde27efd190eca90f05e1261b10dbd8cbcb39cd8ea1dc/markupsafe-3.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fed51ac40f757d41b7c48425901843666a6677e3e8eb0abcff09e4ba6e664f50", size = 23598, upload-time = "2025-09-27T18:37:21.177Z" }, + { url = "https://files.pythonhosted.org/packages/22/44/a0681611106e0b2921b3033fc19bc53323e0b50bc70cffdd19f7d679bb66/markupsafe-3.0.3-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:f190daf01f13c72eac4efd5c430a8de82489d9cff23c364c3ea822545032993e", size = 23261, upload-time = "2025-09-27T18:37:22.167Z" }, + { url = "https://files.pythonhosted.org/packages/5f/57/1b0b3f100259dc9fffe780cfb60d4be71375510e435efec3d116b6436d43/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e56b7d45a839a697b5eb268c82a71bd8c7f6c94d6fd50c3d577fa39a9f1409f5", size = 24835, upload-time = "2025-09-27T18:37:23.296Z" }, + { url = "https://files.pythonhosted.org/packages/26/6a/4bf6d0c97c4920f1597cc14dd720705eca0bf7c787aebc6bb4d1bead5388/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:f3e98bb3798ead92273dc0e5fd0f31ade220f59a266ffd8a4f6065e0a3ce0523", size = 22733, upload-time = "2025-09-27T18:37:24.237Z" }, + { url = "https://files.pythonhosted.org/packages/14/c7/ca723101509b518797fedc2fdf79ba57f886b4aca8a7d31857ba3ee8281f/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:5678211cb9333a6468fb8d8be0305520aa073f50d17f089b5b4b477ea6e67fdc", size = 23672, upload-time = "2025-09-27T18:37:25.271Z" }, + { url = "https://files.pythonhosted.org/packages/fb/df/5bd7a48c256faecd1d36edc13133e51397e41b73bb77e1a69deab746ebac/markupsafe-3.0.3-cp314-cp314t-win32.whl", hash = "sha256:915c04ba3851909ce68ccc2b8e2cd691618c4dc4c4232fb7982bca3f41fd8c3d", size = 14819, upload-time = "2025-09-27T18:37:26.285Z" }, + { url = "https://files.pythonhosted.org/packages/1a/8a/0402ba61a2f16038b48b39bccca271134be00c5c9f0f623208399333c448/markupsafe-3.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4faffd047e07c38848ce017e8725090413cd80cbc23d86e55c587bf979e579c9", size = 15426, upload-time = "2025-09-27T18:37:27.316Z" }, + { url = "https://files.pythonhosted.org/packages/70/bc/6f1c2f612465f5fa89b95bead1f44dcb607670fd42891d8fdcd5d039f4f4/markupsafe-3.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:32001d6a8fc98c8cb5c947787c5d08b0a50663d139f1305bac5885d98d9b40fa", size = 14146, upload-time = "2025-09-27T18:37:28.327Z" }, +] + +[[package]] +name = "mdit-py-plugins" +version = "0.6.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "markdown-it-py", version = "3.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "markdown-it-py", version = "4.2.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/59/fc/f8d0863f8862f25602c0404d75568e89fb6b4109804645e5cdfb1be5cf56/mdit_py_plugins-0.6.1.tar.gz", hash = "sha256:a2bca0f039f39dbd35fb74ae1b5f998608c437463371f0ff7f49a19a17a114d0", size = 56114, upload-time = "2026-05-13T09:03:38.91Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a5/69/6da5581c6a7fede7dc261bf4e67d6adca4196f176b43288b55b3db395b6e/mdit_py_plugins-0.6.1-py3-none-any.whl", hash = "sha256:214c82fb2ac524472ab6a5bcab1de80f73b50443e187f401bfd77efbc7c6481d", size = 66663, upload-time = "2026-05-13T09:03:37.76Z" }, +] + +[[package]] +name = "mdurl" +version = "0.1.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d6/54/cfe61301667036ec958cb99bd3efefba235e65cdeb9c84d24a8293ba1d90/mdurl-0.1.2.tar.gz", hash = "sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba", size = 8729, upload-time = "2022-08-14T12:40:10.846Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl", hash = "sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", size = 9979, upload-time = "2022-08-14T12:40:09.779Z" }, +] + +[[package]] +name = "myst-parser" +version = "4.0.1" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.11'", +] +dependencies = [ + { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "jinja2", marker = "python_full_version < '3.11'" }, + { name = "markdown-it-py", version = "3.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "mdit-py-plugins", marker = "python_full_version < '3.11'" }, + { name = "pyyaml", marker = "python_full_version < '3.11'" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/66/a5/9626ba4f73555b3735ad86247a8077d4603aa8628537687c839ab08bfe44/myst_parser-4.0.1.tar.gz", hash = "sha256:5cfea715e4f3574138aecbf7d54132296bfd72bb614d31168f48c477a830a7c4", size = 93985, upload-time = "2025-02-12T10:53:03.833Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5f/df/76d0321c3797b54b60fef9ec3bd6f4cfd124b9e422182156a1dd418722cf/myst_parser-4.0.1-py3-none-any.whl", hash = "sha256:9134e88959ec3b5780aedf8a99680ea242869d012e8821db3126d427edc9c95d", size = 84579, upload-time = "2025-02-12T10:53:02.078Z" }, +] + +[[package]] +name = "myst-parser" +version = "5.1.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", +] +dependencies = [ + { name = "docutils", version = "0.22.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "jinja2", marker = "python_full_version >= '3.11'" }, + { name = "markdown-it-py", version = "4.2.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "mdit-py-plugins", marker = "python_full_version >= '3.11'" }, + { name = "pyyaml", marker = "python_full_version >= '3.11'" }, + { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/21/dc/603751677fff302f34396e206b610f556a59d7fe58b9a2145f54e96b48e8/myst_parser-5.1.0.tar.gz", hash = "sha256:ab69322dc6719dcc7f296479dbb70181b66df6ed315064f92dbc85c0e1bf2f02", size = 101182, upload-time = "2026-05-13T09:38:19.361Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/09/dc/f3dfb7488b770f3f67e6545085bf2abea5172e88f57b8ad25ef860ca704c/myst_parser-5.1.0-py3-none-any.whl", hash = "sha256:9c91c52b3cdb4d94a6506e4fab4e2f296c7623a0da0dcbe6de1565c3dad67a8a", size = 85817, upload-time = "2026-05-13T09:38:17.904Z" }, +] + +[[package]] +name = "packaging" +version = "26.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d7/f1/e7a6dd94a8d4a5626c03e4e99c87f241ba9e350cd9e6d75123f992427270/packaging-26.2.tar.gz", hash = "sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661", size = 228134, upload-time = "2026-04-24T20:15:23.917Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/df/b2/87e62e8c3e2f4b32e5fe99e0b86d576da1312593b39f47d8ceef365e95ed/packaging-26.2-py3-none-any.whl", hash = "sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e", size = 100195, upload-time = "2026-04-24T20:15:22.081Z" }, +] + +[[package]] +name = "pefile" +version = "2024.8.26" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/03/4f/2750f7f6f025a1507cd3b7218691671eecfd0bbebebe8b39aa0fe1d360b8/pefile-2024.8.26.tar.gz", hash = "sha256:3ff6c5d8b43e8c37bb6e6dd5085658d658a7a0bdcd20b6a07b1fcfc1c4e9d632", size = 76008, upload-time = "2024-08-26T20:58:38.155Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/54/16/12b82f791c7f50ddec566873d5bdd245baa1491bac11d15ffb98aecc8f8b/pefile-2024.8.26-py3-none-any.whl", hash = "sha256:76f8b485dcd3b1bb8166f1128d395fa3d87af26360c2358fb75b80019b957c6f", size = 74766, upload-time = "2024-08-26T21:01:02.632Z" }, +] + +[[package]] +name = "pluggy" +version = "1.6.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/f9/e2/3e91f31a7d2b083fe6ef3fa267035b518369d9511ffab804f839851d2779/pluggy-1.6.0.tar.gz", hash = "sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3", size = 69412, upload-time = "2025-05-15T12:30:07.975Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" }, +] + +[[package]] +name = "pyelftools" +version = "0.32" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/b9/ab/33968940b2deb3d92f5b146bc6d4009a5f95d1d06c148ea2f9ee965071af/pyelftools-0.32.tar.gz", hash = "sha256:6de90ee7b8263e740c8715a925382d4099b354f29ac48ea40d840cf7aa14ace5", size = 15047199, upload-time = "2025-02-19T14:20:05.549Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/af/43/700932c4f0638c3421177144a2e86448c0d75dbaee2c7936bda3f9fd0878/pyelftools-0.32-py3-none-any.whl", hash = "sha256:013df952a006db5e138b1edf6d8a68ecc50630adbd0d83a2d41e7f846163d738", size = 188525, upload-time = "2025-02-19T14:19:59.919Z" }, +] + +[[package]] +name = "pygments" +version = "2.20.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz", hash = "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", size = 4955991, upload-time = "2026-03-29T13:29:33.898Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" }, +] + +[[package]] +name = "pytest" +version = "9.1.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "colorama", marker = "sys_platform == 'win32'" }, + { name = "exceptiongroup", marker = "python_full_version < '3.11'" }, + { name = "iniconfig" }, + { name = "packaging" }, + { name = "pluggy" }, + { name = "pygments" }, + { name = "tomli", marker = "python_full_version < '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/e4/47/b9efed96c114afcfa3c9d3fe98a76a1d14c74a9e266d397cf6eb64be5e01/pytest-9.1.1.tar.gz", hash = "sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313", size = 1636369, upload-time = "2026-06-19T10:58:32.857Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/24/25/1de2678b631f5a49215c6c96fff41ba892b0a34df68d6d80292b1b48aa7f/pytest-9.1.1-py3-none-any.whl", hash = "sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c", size = 386536, upload-time = "2026-06-19T10:58:31.347Z" }, +] + +[[package]] +name = "pytest-bazel" +version = "0.1.6" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pytest" }, +] +wheels = [ + { url = "https://files.pythonhosted.org/packages/c2/90/4726a39728fb5a5f77a773c5e9d7bfd3a0120aa2029963a88fd06e2c5fb2/pytest_bazel-0.1.6-py3-none-any.whl", hash = "sha256:a29e80e1d67c3db801bdd4d0b6b742f2bfb48cd6841caa33401458e5c4e29c21", size = 10126, upload-time = "2025-10-31T08:41:17.165Z" }, +] + +[[package]] +name = "pytest-mock" +version = "3.15.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pytest" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/68/14/eb014d26be205d38ad5ad20d9a80f7d201472e08167f0bb4361e251084a9/pytest_mock-3.15.1.tar.gz", hash = "sha256:1849a238f6f396da19762269de72cb1814ab44416fa73a8686deac10b0d87a0f", size = 34036, upload-time = "2025-09-16T16:37:27.081Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5a/cc/06253936f4a7fa2e0f48dfe6d851d9c56df896a9ab09ac019d70b760619c/pytest_mock-3.15.1-py3-none-any.whl", hash = "sha256:0a25e2eb88fe5168d535041d09a4529a188176ae608a6d249ee65abc0949630d", size = 10095, upload-time = "2025-09-16T16:37:25.734Z" }, +] + +[[package]] +name = "pyyaml" +version = "6.0.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/05/8e/961c0007c59b8dd7729d542c61a4d537767a59645b82a0b521206e1e25c2/pyyaml-6.0.3.tar.gz", hash = "sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f", size = 130960, upload-time = "2025-09-25T21:33:16.546Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f4/a0/39350dd17dd6d6c6507025c0e53aef67a9293a6d37d3511f23ea510d5800/pyyaml-6.0.3-cp310-cp310-macosx_10_13_x86_64.whl", hash = "sha256:214ed4befebe12df36bcc8bc2b64b396ca31be9304b8f59e25c11cf94a4c033b", size = 184227, upload-time = "2025-09-25T21:31:46.04Z" }, + { url = "https://files.pythonhosted.org/packages/05/14/52d505b5c59ce73244f59c7a50ecf47093ce4765f116cdb98286a71eeca2/pyyaml-6.0.3-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:02ea2dfa234451bbb8772601d7b8e426c2bfa197136796224e50e35a78777956", size = 174019, upload-time = "2025-09-25T21:31:47.706Z" }, + { url = "https://files.pythonhosted.org/packages/43/f7/0e6a5ae5599c838c696adb4e6330a59f463265bfa1e116cfd1fbb0abaaae/pyyaml-6.0.3-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b30236e45cf30d2b8e7b3e85881719e98507abed1011bf463a8fa23e9c3e98a8", size = 740646, upload-time = "2025-09-25T21:31:49.21Z" }, + { url = "https://files.pythonhosted.org/packages/2f/3a/61b9db1d28f00f8fd0ae760459a5c4bf1b941baf714e207b6eb0657d2578/pyyaml-6.0.3-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:66291b10affd76d76f54fad28e22e51719ef9ba22b29e1d7d03d6777a9174198", size = 840793, upload-time = "2025-09-25T21:31:50.735Z" }, + { url = "https://files.pythonhosted.org/packages/7a/1e/7acc4f0e74c4b3d9531e24739e0ab832a5edf40e64fbae1a9c01941cabd7/pyyaml-6.0.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9c7708761fccb9397fe64bbc0395abcae8c4bf7b0eac081e12b809bf47700d0b", size = 770293, upload-time = "2025-09-25T21:31:51.828Z" }, + { url = "https://files.pythonhosted.org/packages/8b/ef/abd085f06853af0cd59fa5f913d61a8eab65d7639ff2a658d18a25d6a89d/pyyaml-6.0.3-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:418cf3f2111bc80e0933b2cd8cd04f286338bb88bdc7bc8e6dd775ebde60b5e0", size = 732872, upload-time = "2025-09-25T21:31:53.282Z" }, + { url = "https://files.pythonhosted.org/packages/1f/15/2bc9c8faf6450a8b3c9fc5448ed869c599c0a74ba2669772b1f3a0040180/pyyaml-6.0.3-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:5e0b74767e5f8c593e8c9b5912019159ed0533c70051e9cce3e8b6aa699fcd69", size = 758828, upload-time = "2025-09-25T21:31:54.807Z" }, + { url = "https://files.pythonhosted.org/packages/a3/00/531e92e88c00f4333ce359e50c19b8d1de9fe8d581b1534e35ccfbc5f393/pyyaml-6.0.3-cp310-cp310-win32.whl", hash = "sha256:28c8d926f98f432f88adc23edf2e6d4921ac26fb084b028c733d01868d19007e", size = 142415, upload-time = "2025-09-25T21:31:55.885Z" }, + { url = "https://files.pythonhosted.org/packages/2a/fa/926c003379b19fca39dd4634818b00dec6c62d87faf628d1394e137354d4/pyyaml-6.0.3-cp310-cp310-win_amd64.whl", hash = "sha256:bdb2c67c6c1390b63c6ff89f210c8fd09d9a1217a465701eac7316313c915e4c", size = 158561, upload-time = "2025-09-25T21:31:57.406Z" }, + { url = "https://files.pythonhosted.org/packages/6d/16/a95b6757765b7b031c9374925bb718d55e0a9ba8a1b6a12d25962ea44347/pyyaml-6.0.3-cp311-cp311-macosx_10_13_x86_64.whl", hash = "sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e", size = 185826, upload-time = "2025-09-25T21:31:58.655Z" }, + { url = "https://files.pythonhosted.org/packages/16/19/13de8e4377ed53079ee996e1ab0a9c33ec2faf808a4647b7b4c0d46dd239/pyyaml-6.0.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824", size = 175577, upload-time = "2025-09-25T21:32:00.088Z" }, + { url = "https://files.pythonhosted.org/packages/0c/62/d2eb46264d4b157dae1275b573017abec435397aa59cbcdab6fc978a8af4/pyyaml-6.0.3-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c", size = 775556, upload-time = "2025-09-25T21:32:01.31Z" }, + { url = "https://files.pythonhosted.org/packages/10/cb/16c3f2cf3266edd25aaa00d6c4350381c8b012ed6f5276675b9eba8d9ff4/pyyaml-6.0.3-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00", size = 882114, upload-time = "2025-09-25T21:32:03.376Z" }, + { url = "https://files.pythonhosted.org/packages/71/60/917329f640924b18ff085ab889a11c763e0b573da888e8404ff486657602/pyyaml-6.0.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d", size = 806638, upload-time = "2025-09-25T21:32:04.553Z" }, + { url = "https://files.pythonhosted.org/packages/dd/6f/529b0f316a9fd167281a6c3826b5583e6192dba792dd55e3203d3f8e655a/pyyaml-6.0.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a", size = 767463, upload-time = "2025-09-25T21:32:06.152Z" }, + { url = "https://files.pythonhosted.org/packages/f2/6a/b627b4e0c1dd03718543519ffb2f1deea4a1e6d42fbab8021936a4d22589/pyyaml-6.0.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4", size = 794986, upload-time = "2025-09-25T21:32:07.367Z" }, + { url = "https://files.pythonhosted.org/packages/45/91/47a6e1c42d9ee337c4839208f30d9f09caa9f720ec7582917b264defc875/pyyaml-6.0.3-cp311-cp311-win32.whl", hash = "sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b", size = 142543, upload-time = "2025-09-25T21:32:08.95Z" }, + { url = "https://files.pythonhosted.org/packages/da/e3/ea007450a105ae919a72393cb06f122f288ef60bba2dc64b26e2646fa315/pyyaml-6.0.3-cp311-cp311-win_amd64.whl", hash = "sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf", size = 158763, upload-time = "2025-09-25T21:32:09.96Z" }, + { url = "https://files.pythonhosted.org/packages/d1/33/422b98d2195232ca1826284a76852ad5a86fe23e31b009c9886b2d0fb8b2/pyyaml-6.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196", size = 182063, upload-time = "2025-09-25T21:32:11.445Z" }, + { url = "https://files.pythonhosted.org/packages/89/a0/6cf41a19a1f2f3feab0e9c0b74134aa2ce6849093d5517a0c550fe37a648/pyyaml-6.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0", size = 173973, upload-time = "2025-09-25T21:32:12.492Z" }, + { url = "https://files.pythonhosted.org/packages/ed/23/7a778b6bd0b9a8039df8b1b1d80e2e2ad78aa04171592c8a5c43a56a6af4/pyyaml-6.0.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28", size = 775116, upload-time = "2025-09-25T21:32:13.652Z" }, + { url = "https://files.pythonhosted.org/packages/65/30/d7353c338e12baef4ecc1b09e877c1970bd3382789c159b4f89d6a70dc09/pyyaml-6.0.3-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c", size = 844011, upload-time = "2025-09-25T21:32:15.21Z" }, + { url = "https://files.pythonhosted.org/packages/8b/9d/b3589d3877982d4f2329302ef98a8026e7f4443c765c46cfecc8858c6b4b/pyyaml-6.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc", size = 807870, upload-time = "2025-09-25T21:32:16.431Z" }, + { url = "https://files.pythonhosted.org/packages/05/c0/b3be26a015601b822b97d9149ff8cb5ead58c66f981e04fedf4e762f4bd4/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e", size = 761089, upload-time = "2025-09-25T21:32:17.56Z" }, + { url = "https://files.pythonhosted.org/packages/be/8e/98435a21d1d4b46590d5459a22d88128103f8da4c2d4cb8f14f2a96504e1/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea", size = 790181, upload-time = "2025-09-25T21:32:18.834Z" }, + { url = "https://files.pythonhosted.org/packages/74/93/7baea19427dcfbe1e5a372d81473250b379f04b1bd3c4c5ff825e2327202/pyyaml-6.0.3-cp312-cp312-win32.whl", hash = "sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5", size = 137658, upload-time = "2025-09-25T21:32:20.209Z" }, + { url = "https://files.pythonhosted.org/packages/86/bf/899e81e4cce32febab4fb42bb97dcdf66bc135272882d1987881a4b519e9/pyyaml-6.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b", size = 154003, upload-time = "2025-09-25T21:32:21.167Z" }, + { url = "https://files.pythonhosted.org/packages/1a/08/67bd04656199bbb51dbed1439b7f27601dfb576fb864099c7ef0c3e55531/pyyaml-6.0.3-cp312-cp312-win_arm64.whl", hash = "sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd", size = 140344, upload-time = "2025-09-25T21:32:22.617Z" }, + { url = "https://files.pythonhosted.org/packages/d1/11/0fd08f8192109f7169db964b5707a2f1e8b745d4e239b784a5a1dd80d1db/pyyaml-6.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8", size = 181669, upload-time = "2025-09-25T21:32:23.673Z" }, + { url = "https://files.pythonhosted.org/packages/b1/16/95309993f1d3748cd644e02e38b75d50cbc0d9561d21f390a76242ce073f/pyyaml-6.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1", size = 173252, upload-time = "2025-09-25T21:32:25.149Z" }, + { url = "https://files.pythonhosted.org/packages/50/31/b20f376d3f810b9b2371e72ef5adb33879b25edb7a6d072cb7ca0c486398/pyyaml-6.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c", size = 767081, upload-time = "2025-09-25T21:32:26.575Z" }, + { url = "https://files.pythonhosted.org/packages/49/1e/a55ca81e949270d5d4432fbbd19dfea5321eda7c41a849d443dc92fd1ff7/pyyaml-6.0.3-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5", size = 841159, upload-time = "2025-09-25T21:32:27.727Z" }, + { url = "https://files.pythonhosted.org/packages/74/27/e5b8f34d02d9995b80abcef563ea1f8b56d20134d8f4e5e81733b1feceb2/pyyaml-6.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6", size = 801626, upload-time = "2025-09-25T21:32:28.878Z" }, + { url = "https://files.pythonhosted.org/packages/f9/11/ba845c23988798f40e52ba45f34849aa8a1f2d4af4b798588010792ebad6/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6", size = 753613, upload-time = "2025-09-25T21:32:30.178Z" }, + { url = "https://files.pythonhosted.org/packages/3d/e0/7966e1a7bfc0a45bf0a7fb6b98ea03fc9b8d84fa7f2229e9659680b69ee3/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be", size = 794115, upload-time = "2025-09-25T21:32:31.353Z" }, + { url = "https://files.pythonhosted.org/packages/de/94/980b50a6531b3019e45ddeada0626d45fa85cbe22300844a7983285bed3b/pyyaml-6.0.3-cp313-cp313-win32.whl", hash = "sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26", size = 137427, upload-time = "2025-09-25T21:32:32.58Z" }, + { url = "https://files.pythonhosted.org/packages/97/c9/39d5b874e8b28845e4ec2202b5da735d0199dbe5b8fb85f91398814a9a46/pyyaml-6.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c", size = 154090, upload-time = "2025-09-25T21:32:33.659Z" }, + { url = "https://files.pythonhosted.org/packages/73/e8/2bdf3ca2090f68bb3d75b44da7bbc71843b19c9f2b9cb9b0f4ab7a5a4329/pyyaml-6.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb", size = 140246, upload-time = "2025-09-25T21:32:34.663Z" }, + { url = "https://files.pythonhosted.org/packages/9d/8c/f4bd7f6465179953d3ac9bc44ac1a8a3e6122cf8ada906b4f96c60172d43/pyyaml-6.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac", size = 181814, upload-time = "2025-09-25T21:32:35.712Z" }, + { url = "https://files.pythonhosted.org/packages/bd/9c/4d95bb87eb2063d20db7b60faa3840c1b18025517ae857371c4dd55a6b3a/pyyaml-6.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310", size = 173809, upload-time = "2025-09-25T21:32:36.789Z" }, + { url = "https://files.pythonhosted.org/packages/92/b5/47e807c2623074914e29dabd16cbbdd4bf5e9b2db9f8090fa64411fc5382/pyyaml-6.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7", size = 766454, upload-time = "2025-09-25T21:32:37.966Z" }, + { url = "https://files.pythonhosted.org/packages/02/9e/e5e9b168be58564121efb3de6859c452fccde0ab093d8438905899a3a483/pyyaml-6.0.3-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788", size = 836355, upload-time = "2025-09-25T21:32:39.178Z" }, + { url = "https://files.pythonhosted.org/packages/88/f9/16491d7ed2a919954993e48aa941b200f38040928474c9e85ea9e64222c3/pyyaml-6.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5", size = 794175, upload-time = "2025-09-25T21:32:40.865Z" }, + { url = "https://files.pythonhosted.org/packages/dd/3f/5989debef34dc6397317802b527dbbafb2b4760878a53d4166579111411e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764", size = 755228, upload-time = "2025-09-25T21:32:42.084Z" }, + { url = "https://files.pythonhosted.org/packages/d7/ce/af88a49043cd2e265be63d083fc75b27b6ed062f5f9fd6cdc223ad62f03e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35", size = 789194, upload-time = "2025-09-25T21:32:43.362Z" }, + { url = "https://files.pythonhosted.org/packages/23/20/bb6982b26a40bb43951265ba29d4c246ef0ff59c9fdcdf0ed04e0687de4d/pyyaml-6.0.3-cp314-cp314-win_amd64.whl", hash = "sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac", size = 156429, upload-time = "2025-09-25T21:32:57.844Z" }, + { url = "https://files.pythonhosted.org/packages/f4/f4/a4541072bb9422c8a883ab55255f918fa378ecf083f5b85e87fc2b4eda1b/pyyaml-6.0.3-cp314-cp314-win_arm64.whl", hash = "sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3", size = 143912, upload-time = "2025-09-25T21:32:59.247Z" }, + { url = "https://files.pythonhosted.org/packages/7c/f9/07dd09ae774e4616edf6cda684ee78f97777bdd15847253637a6f052a62f/pyyaml-6.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3", size = 189108, upload-time = "2025-09-25T21:32:44.377Z" }, + { url = "https://files.pythonhosted.org/packages/4e/78/8d08c9fb7ce09ad8c38ad533c1191cf27f7ae1effe5bb9400a46d9437fcf/pyyaml-6.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba", size = 183641, upload-time = "2025-09-25T21:32:45.407Z" }, + { url = "https://files.pythonhosted.org/packages/7b/5b/3babb19104a46945cf816d047db2788bcaf8c94527a805610b0289a01c6b/pyyaml-6.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c", size = 831901, upload-time = "2025-09-25T21:32:48.83Z" }, + { url = "https://files.pythonhosted.org/packages/8b/cc/dff0684d8dc44da4d22a13f35f073d558c268780ce3c6ba1b87055bb0b87/pyyaml-6.0.3-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702", size = 861132, upload-time = "2025-09-25T21:32:50.149Z" }, + { url = "https://files.pythonhosted.org/packages/b1/5e/f77dc6b9036943e285ba76b49e118d9ea929885becb0a29ba8a7c75e29fe/pyyaml-6.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c", size = 839261, upload-time = "2025-09-25T21:32:51.808Z" }, + { url = "https://files.pythonhosted.org/packages/ce/88/a9db1376aa2a228197c58b37302f284b5617f56a5d959fd1763fb1675ce6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065", size = 805272, upload-time = "2025-09-25T21:32:52.941Z" }, + { url = "https://files.pythonhosted.org/packages/da/92/1446574745d74df0c92e6aa4a7b0b3130706a4142b2d1a5869f2eaa423c6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65", size = 829923, upload-time = "2025-09-25T21:32:54.537Z" }, + { url = "https://files.pythonhosted.org/packages/f0/7a/1c7270340330e575b92f397352af856a8c06f230aa3e76f86b39d01b416a/pyyaml-6.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9", size = 174062, upload-time = "2025-09-25T21:32:55.767Z" }, + { url = "https://files.pythonhosted.org/packages/f1/12/de94a39c2ef588c7e6455cfbe7343d3b2dc9d6b6b2f40c4c6565744c873d/pyyaml-6.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b", size = 149341, upload-time = "2025-09-25T21:32:56.828Z" }, +] + +[[package]] +name = "readthedocs-sphinx-ext" +version = "2.2.5" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "jinja2" }, + { name = "packaging" }, + { name = "requests" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/e8/ce/38130d8dec600bf5413eb89a3413dd38f204c7c728c4947e12ff8cb793b7/readthedocs-sphinx-ext-2.2.5.tar.gz", hash = "sha256:ee5fd5b99db9f0c180b2396cbce528aa36671951b9526bb0272dbfce5517bd27", size = 12303, upload-time = "2023-12-19T10:00:49.573Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/64/71/c89e7709a0d4f93af1848e9855112299a820b470d84f917b4dd5998bdd07/readthedocs_sphinx_ext-2.2.5-py2.py3-none-any.whl", hash = "sha256:f8c56184ea011c972dd45a90122568587cc85b0127bc9cf064d17c68bc809daa", size = 11332, upload-time = "2023-12-19T10:00:43.972Z" }, +] + +[[package]] +name = "requests" +version = "2.34.2" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "certifi" }, + { name = "charset-normalizer" }, + { name = "idna" }, + { name = "urllib3" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/ac/c3/e2a2b89f2d3e2179abd6d00ebd70bff6273f37fb3e0cc209f48b39d00cbf/requests-2.34.2.tar.gz", hash = "sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed", size = 142856, upload-time = "2026-05-14T19:25:27.735Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a0/f4/c67b0b3f1b9245e8d266f0f112c500d50e5b4e83cb6f3b71b6528104182a/requests-2.34.2-py3-none-any.whl", hash = "sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0", size = 73075, upload-time = "2026-05-14T19:25:26.443Z" }, +] + +[[package]] +name = "roman-numerals" +version = "4.1.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/ae/f9/41dc953bbeb056c17d5f7a519f50fdf010bd0553be2d630bc69d1e022703/roman_numerals-4.1.0.tar.gz", hash = "sha256:1af8b147eb1405d5839e78aeb93131690495fe9da5c91856cb33ad55a7f1e5b2", size = 9077, upload-time = "2025-12-17T18:25:34.381Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/04/54/6f679c435d28e0a568d8e8a7c0a93a09010818634c3c3907fc98d8983770/roman_numerals-4.1.0-py3-none-any.whl", hash = "sha256:647ba99caddc2cc1e55a51e4360689115551bf4476d90e8162cf8c345fe233c7", size = 7676, upload-time = "2025-12-17T18:25:33.098Z" }, +] + +[[package]] +name = "rules-python-dev" +version = "0.0.0" +source = { virtual = "." } +dependencies = [ + { name = "absl-py" }, + { name = "macholib" }, + { name = "markupsafe" }, + { name = "myst-parser", version = "4.0.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "myst-parser", version = "5.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "pefile" }, + { name = "pyelftools" }, + { name = "pytest" }, + { name = "pytest-bazel" }, + { name = "pytest-mock" }, + { name = "readthedocs-sphinx-ext" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "sphinx-autodoc2" }, + { name = "sphinx-reredirects", version = "0.1.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "sphinx-reredirects", version = "1.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "sphinx-rtd-theme" }, + { name = "typing-extensions" }, +] + +[package.metadata] +requires-dist = [ + { name = "absl-py" }, + { name = "macholib" }, + { name = "markupsafe" }, + { name = "myst-parser" }, + { name = "pefile" }, + { name = "pyelftools" }, + { name = "pytest" }, + { name = "pytest-bazel" }, + { name = "pytest-mock" }, + { name = "readthedocs-sphinx-ext" }, + { name = "sphinx" }, + { name = "sphinx-autodoc2" }, + { name = "sphinx-reredirects" }, + { name = "sphinx-rtd-theme", specifier = ">=2.0" }, + { name = "typing-extensions" }, +] + +[[package]] +name = "snowballstemmer" +version = "3.0.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/75/a7/9810d872919697c9d01295633f5d574fb416d47e535f258272ca1f01f447/snowballstemmer-3.0.1.tar.gz", hash = "sha256:6d5eeeec8e9f84d4d56b847692bacf79bc2c8e90c7f80ca4444ff8b6f2e52895", size = 105575, upload-time = "2025-05-09T16:34:51.843Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c8/78/3565d011c61f5a43488987ee32b6f3f656e7f107ac2782dd57bdd7d91d9a/snowballstemmer-3.0.1-py3-none-any.whl", hash = "sha256:6cd7b3897da8d6c9ffb968a6781fa6532dce9c3618a4b127d920dab764a19064", size = 103274, upload-time = "2025-05-09T16:34:50.371Z" }, +] + +[[package]] +name = "sphinx" +version = "8.1.3" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.11'", +] +dependencies = [ + { name = "alabaster", marker = "python_full_version < '3.11'" }, + { name = "babel", marker = "python_full_version < '3.11'" }, + { name = "colorama", marker = "python_full_version < '3.11' and sys_platform == 'win32'" }, + { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "imagesize", marker = "python_full_version < '3.11'" }, + { name = "jinja2", marker = "python_full_version < '3.11'" }, + { name = "packaging", marker = "python_full_version < '3.11'" }, + { name = "pygments", marker = "python_full_version < '3.11'" }, + { name = "requests", marker = "python_full_version < '3.11'" }, + { name = "snowballstemmer", marker = "python_full_version < '3.11'" }, + { name = "sphinxcontrib-applehelp", marker = "python_full_version < '3.11'" }, + { name = "sphinxcontrib-devhelp", marker = "python_full_version < '3.11'" }, + { name = "sphinxcontrib-htmlhelp", marker = "python_full_version < '3.11'" }, + { name = "sphinxcontrib-jsmath", marker = "python_full_version < '3.11'" }, + { name = "sphinxcontrib-qthelp", marker = "python_full_version < '3.11'" }, + { name = "sphinxcontrib-serializinghtml", marker = "python_full_version < '3.11'" }, + { name = "tomli", marker = "python_full_version < '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/6f/6d/be0b61178fe2cdcb67e2a92fc9ebb488e3c51c4f74a36a7824c0adf23425/sphinx-8.1.3.tar.gz", hash = "sha256:43c1911eecb0d3e161ad78611bc905d1ad0e523e4ddc202a58a821773dc4c927", size = 8184611, upload-time = "2024-10-13T20:27:13.93Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/26/60/1ddff83a56d33aaf6f10ec8ce84b4c007d9368b21008876fceda7e7381ef/sphinx-8.1.3-py3-none-any.whl", hash = "sha256:09719015511837b76bf6e03e42eb7595ac8c2e41eeb9c29c5b755c6b677992a2", size = 3487125, upload-time = "2024-10-13T20:27:10.448Z" }, +] + +[[package]] +name = "sphinx" +version = "9.0.4" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version == '3.11.*'", +] +dependencies = [ + { name = "alabaster", marker = "python_full_version == '3.11.*'" }, + { name = "babel", marker = "python_full_version == '3.11.*'" }, + { name = "colorama", marker = "python_full_version == '3.11.*' and sys_platform == 'win32'" }, + { name = "docutils", version = "0.22.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "imagesize", marker = "python_full_version == '3.11.*'" }, + { name = "jinja2", marker = "python_full_version == '3.11.*'" }, + { name = "packaging", marker = "python_full_version == '3.11.*'" }, + { name = "pygments", marker = "python_full_version == '3.11.*'" }, + { name = "requests", marker = "python_full_version == '3.11.*'" }, + { name = "roman-numerals", marker = "python_full_version == '3.11.*'" }, + { name = "snowballstemmer", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-applehelp", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-devhelp", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-htmlhelp", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-jsmath", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-qthelp", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-serializinghtml", marker = "python_full_version == '3.11.*'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/42/50/a8c6ccc36d5eacdfd7913ddccd15a9cee03ecafc5ee2bc40e1f168d85022/sphinx-9.0.4.tar.gz", hash = "sha256:594ef59d042972abbc581d8baa577404abe4e6c3b04ef61bd7fc2acbd51f3fa3", size = 8710502, upload-time = "2025-12-04T07:45:27.343Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c6/3f/4bbd76424c393caead2e1eb89777f575dee5c8653e2d4b6afd7a564f5974/sphinx-9.0.4-py3-none-any.whl", hash = "sha256:5bebc595a5e943ea248b99c13814c1c5e10b3ece718976824ffa7959ff95fffb", size = 3917713, upload-time = "2025-12-04T07:45:24.944Z" }, +] + +[[package]] +name = "sphinx" +version = "9.1.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", +] +dependencies = [ + { name = "alabaster", marker = "python_full_version >= '3.12'" }, + { name = "babel", marker = "python_full_version >= '3.12'" }, + { name = "colorama", marker = "python_full_version >= '3.12' and sys_platform == 'win32'" }, + { name = "docutils", version = "0.22.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "imagesize", marker = "python_full_version >= '3.12'" }, + { name = "jinja2", marker = "python_full_version >= '3.12'" }, + { name = "packaging", marker = "python_full_version >= '3.12'" }, + { name = "pygments", marker = "python_full_version >= '3.12'" }, + { name = "requests", marker = "python_full_version >= '3.12'" }, + { name = "roman-numerals", marker = "python_full_version >= '3.12'" }, + { name = "snowballstemmer", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-applehelp", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-devhelp", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-htmlhelp", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-jsmath", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-qthelp", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-serializinghtml", marker = "python_full_version >= '3.12'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/cd/bd/f08eb0f4eed5c83f1ba2a3bd18f7745a2b1525fad70660a1c00224ec468a/sphinx-9.1.0.tar.gz", hash = "sha256:7741722357dd75f8190766926071fed3bdc211c74dd2d7d4df5404da95930ddb", size = 8718324, upload-time = "2025-12-31T15:09:27.646Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/73/f7/b1884cb3188ab181fc81fa00c266699dab600f927a964df02ec3d5d1916a/sphinx-9.1.0-py3-none-any.whl", hash = "sha256:c84fdd4e782504495fe4f2c0b3413d6c2bf388589bb352d439b2a3bb99991978", size = 3921742, upload-time = "2025-12-31T15:09:25.561Z" }, +] + +[[package]] +name = "sphinx-autodoc2" +version = "0.5.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "astroid" }, + { name = "tomli", marker = "python_full_version < '3.11'" }, + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/17/5f/5350046d1aa1a56b063ae08b9ad871025335c9d55fe2372896ea48711da9/sphinx_autodoc2-0.5.0.tar.gz", hash = "sha256:7d76044aa81d6af74447080182b6868c7eb066874edc835e8ddf810735b6565a", size = 115077, upload-time = "2023-11-27T07:27:51.407Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/19/e6/48d47961bbdae755ba9c17dfc65d89356312c67668dcb36c87cfadfa1964/sphinx_autodoc2-0.5.0-py3-none-any.whl", hash = "sha256:e867013b1512f9d6d7e6f6799f8b537d6884462acd118ef361f3f619a60b5c9e", size = 43385, upload-time = "2023-11-27T07:27:49.929Z" }, +] + +[[package]] +name = "sphinx-reredirects" +version = "0.1.6" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.11'", +] +dependencies = [ + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/16/6b/bcca2785de4071f604a722444d4d7ba8a9d40de3c14ad52fce93e6d92694/sphinx_reredirects-0.1.6.tar.gz", hash = "sha256:c491cba545f67be9697508727818d8626626366245ae64456fe29f37e9bbea64", size = 7080, upload-time = "2025-03-22T10:52:30.271Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ac/6f/0b3625be30a1a50f9e4c2cb2ec147b08f15ed0e9f8444efcf274b751300b/sphinx_reredirects-0.1.6-py3-none-any.whl", hash = "sha256:efd50c766fbc5bf40cd5148e10c00f2c00d143027de5c5e48beece93cc40eeea", size = 5675, upload-time = "2025-03-22T10:52:29.113Z" }, +] + +[[package]] +name = "sphinx-reredirects" +version = "1.1.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", +] +dependencies = [ + { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/1b/8d/0e39fe2740d7d71417edf9a6424aa80ca2c27c17fc21282cdc39f90d5a40/sphinx_reredirects-1.1.0.tar.gz", hash = "sha256:fb9b195335ab14b43f8273287d0c7eeb637ba6c56c66581c11b47202f6718b29", size = 614624, upload-time = "2025-12-22T08:28:02.792Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/51/81/b5dd07067f3daac6d23687ec737b2d593740671ebcd145830c8f92d381c5/sphinx_reredirects-1.1.0-py3-none-any.whl", hash = "sha256:4b5692273c72cd2d4d917f4c6f87d5919e4d6114a752d4be033f7f5f6310efd9", size = 6351, upload-time = "2025-12-22T08:27:59.724Z" }, +] + +[[package]] +name = "sphinx-rtd-theme" +version = "3.1.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "docutils", version = "0.22.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-jquery" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/84/68/a1bfbf38c0f7bccc9b10bbf76b94606f64acb1552ae394f0b8285bfaea25/sphinx_rtd_theme-3.1.0.tar.gz", hash = "sha256:b44276f2c276e909239a4f6c955aa667aaafeb78597923b1c60babc76db78e4c", size = 7620915, upload-time = "2026-01-12T16:03:31.17Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/87/c7/b5c8015d823bfda1a346adb2c634a2101d50bb75d421eb6dcb31acd25ebc/sphinx_rtd_theme-3.1.0-py2.py3-none-any.whl", hash = "sha256:1785824ae8e6632060490f67cf3a72d404a85d2d9fc26bce3619944de5682b89", size = 7655617, upload-time = "2026-01-12T16:03:28.101Z" }, +] + +[[package]] +name = "sphinxcontrib-applehelp" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/ba/6e/b837e84a1a704953c62ef8776d45c3e8d759876b4a84fe14eba2859106fe/sphinxcontrib_applehelp-2.0.0.tar.gz", hash = "sha256:2f29ef331735ce958efa4734873f084941970894c6090408b079c61b2e1c06d1", size = 20053, upload-time = "2024-07-29T01:09:00.465Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5d/85/9ebeae2f76e9e77b952f4b274c27238156eae7979c5421fba91a28f4970d/sphinxcontrib_applehelp-2.0.0-py3-none-any.whl", hash = "sha256:4cd3f0ec4ac5dd9c17ec65e9ab272c9b867ea77425228e68ecf08d6b28ddbdb5", size = 119300, upload-time = "2024-07-29T01:08:58.99Z" }, +] + +[[package]] +name = "sphinxcontrib-devhelp" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/f6/d2/5beee64d3e4e747f316bae86b55943f51e82bb86ecd325883ef65741e7da/sphinxcontrib_devhelp-2.0.0.tar.gz", hash = "sha256:411f5d96d445d1d73bb5d52133377b4248ec79db5c793ce7dbe59e074b4dd1ad", size = 12967, upload-time = "2024-07-29T01:09:23.417Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/35/7a/987e583882f985fe4d7323774889ec58049171828b58c2217e7f79cdf44e/sphinxcontrib_devhelp-2.0.0-py3-none-any.whl", hash = "sha256:aefb8b83854e4b0998877524d1029fd3e6879210422ee3780459e28a1f03a8a2", size = 82530, upload-time = "2024-07-29T01:09:21.945Z" }, +] + +[[package]] +name = "sphinxcontrib-htmlhelp" +version = "2.1.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/43/93/983afd9aa001e5201eab16b5a444ed5b9b0a7a010541e0ddfbbfd0b2470c/sphinxcontrib_htmlhelp-2.1.0.tar.gz", hash = "sha256:c9e2916ace8aad64cc13a0d233ee22317f2b9025b9cf3295249fa985cc7082e9", size = 22617, upload-time = "2024-07-29T01:09:37.889Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0a/7b/18a8c0bcec9182c05a0b3ec2a776bba4ead82750a55ff798e8d406dae604/sphinxcontrib_htmlhelp-2.1.0-py3-none-any.whl", hash = "sha256:166759820b47002d22914d64a075ce08f4c46818e17cfc9470a9786b759b19f8", size = 98705, upload-time = "2024-07-29T01:09:36.407Z" }, +] + +[[package]] +name = "sphinxcontrib-jquery" +version = "4.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/de/f3/aa67467e051df70a6330fe7770894b3e4f09436dea6881ae0b4f3d87cad8/sphinxcontrib-jquery-4.1.tar.gz", hash = "sha256:1620739f04e36a2c779f1a131a2dfd49b2fd07351bf1968ced074365933abc7a", size = 122331, upload-time = "2023-03-14T15:01:01.944Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/76/85/749bd22d1a68db7291c89e2ebca53f4306c3f205853cf31e9de279034c3c/sphinxcontrib_jquery-4.1-py2.py3-none-any.whl", hash = "sha256:f936030d7d0147dd026a4f2b5a57343d233f1fc7b363f68b3d4f1cb0993878ae", size = 121104, upload-time = "2023-03-14T15:01:00.356Z" }, +] + +[[package]] +name = "sphinxcontrib-jsmath" +version = "1.0.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/b2/e8/9ed3830aeed71f17c026a07a5097edcf44b692850ef215b161b8ad875729/sphinxcontrib-jsmath-1.0.1.tar.gz", hash = "sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8", size = 5787, upload-time = "2019-01-21T16:10:16.347Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c2/42/4c8646762ee83602e3fb3fbe774c2fac12f317deb0b5dbeeedd2d3ba4b77/sphinxcontrib_jsmath-1.0.1-py2.py3-none-any.whl", hash = "sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178", size = 5071, upload-time = "2019-01-21T16:10:14.333Z" }, +] + +[[package]] +name = "sphinxcontrib-qthelp" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/68/bc/9104308fc285eb3e0b31b67688235db556cd5b0ef31d96f30e45f2e51cae/sphinxcontrib_qthelp-2.0.0.tar.gz", hash = "sha256:4fe7d0ac8fc171045be623aba3e2a8f613f8682731f9153bb2e40ece16b9bbab", size = 17165, upload-time = "2024-07-29T01:09:56.435Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/27/83/859ecdd180cacc13b1f7e857abf8582a64552ea7a061057a6c716e790fce/sphinxcontrib_qthelp-2.0.0-py3-none-any.whl", hash = "sha256:b18a828cdba941ccd6ee8445dbe72ffa3ef8cbe7505d8cd1fa0d42d3f2d5f3eb", size = 88743, upload-time = "2024-07-29T01:09:54.885Z" }, +] + +[[package]] +name = "sphinxcontrib-serializinghtml" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/3b/44/6716b257b0aa6bfd51a1b31665d1c205fb12cb5ad56de752dfa15657de2f/sphinxcontrib_serializinghtml-2.0.0.tar.gz", hash = "sha256:e9d912827f872c029017a53f0ef2180b327c3f7fd23c87229f7a8e8b70031d4d", size = 16080, upload-time = "2024-07-29T01:10:09.332Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/52/a7/d2782e4e3f77c8450f727ba74a8f12756d5ba823d81b941f1b04da9d033a/sphinxcontrib_serializinghtml-2.0.0-py3-none-any.whl", hash = "sha256:6e2cb0eef194e10c27ec0023bfeb25badbbb5868244cf5bc5bdc04e4464bf331", size = 92072, upload-time = "2024-07-29T01:10:08.203Z" }, +] + +[[package]] +name = "tomli" +version = "2.4.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/22/de/48c59722572767841493b26183a0d1cc411d54fd759c5607c4590b6563a6/tomli-2.4.1.tar.gz", hash = "sha256:7c7e1a961a0b2f2472c1ac5b69affa0ae1132c39adcb67aba98568702b9cc23f", size = 17543, upload-time = "2026-03-25T20:22:03.828Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f4/11/db3d5885d8528263d8adc260bb2d28ebf1270b96e98f0e0268d32b8d9900/tomli-2.4.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:f8f0fc26ec2cc2b965b7a3b87cd19c5c6b8c5e5f436b984e85f486d652285c30", size = 154704, upload-time = "2026-03-25T20:21:10.473Z" }, + { url = "https://files.pythonhosted.org/packages/6d/f7/675db52c7e46064a9aa928885a9b20f4124ecb9bc2e1ce74c9106648d202/tomli-2.4.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:4ab97e64ccda8756376892c53a72bd1f964e519c77236368527f758fbc36a53a", size = 149454, upload-time = "2026-03-25T20:21:12.036Z" }, + { url = "https://files.pythonhosted.org/packages/61/71/81c50943cf953efa35bce7646caab3cf457a7d8c030b27cfb40d7235f9ee/tomli-2.4.1-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:96481a5786729fd470164b47cdb3e0e58062a496f455ee41b4403be77cb5a076", size = 237561, upload-time = "2026-03-25T20:21:13.098Z" }, + { url = "https://files.pythonhosted.org/packages/48/c1/f41d9cb618acccca7df82aaf682f9b49013c9397212cb9f53219e3abac37/tomli-2.4.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5a881ab208c0baf688221f8cecc5401bd291d67e38a1ac884d6736cbcd8247e9", size = 243824, upload-time = "2026-03-25T20:21:14.569Z" }, + { url = "https://files.pythonhosted.org/packages/22/e4/5a816ecdd1f8ca51fb756ef684b90f2780afc52fc67f987e3c61d800a46d/tomli-2.4.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:47149d5bd38761ac8be13a84864bf0b7b70bc051806bc3669ab1cbc56216b23c", size = 242227, upload-time = "2026-03-25T20:21:15.712Z" }, + { url = "https://files.pythonhosted.org/packages/6b/49/2b2a0ef529aa6eec245d25f0c703e020a73955ad7edf73e7f54ddc608aa5/tomli-2.4.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:ec9bfaf3ad2df51ace80688143a6a4ebc09a248f6ff781a9945e51937008fcbc", size = 247859, upload-time = "2026-03-25T20:21:17.001Z" }, + { url = "https://files.pythonhosted.org/packages/83/bd/6c1a630eaca337e1e78c5903104f831bda934c426f9231429396ce3c3467/tomli-2.4.1-cp311-cp311-win32.whl", hash = "sha256:ff2983983d34813c1aeb0fa89091e76c3a22889ee83ab27c5eeb45100560c049", size = 97204, upload-time = "2026-03-25T20:21:18.079Z" }, + { url = "https://files.pythonhosted.org/packages/42/59/71461df1a885647e10b6bb7802d0b8e66480c61f3f43079e0dcd315b3954/tomli-2.4.1-cp311-cp311-win_amd64.whl", hash = "sha256:5ee18d9ebdb417e384b58fe414e8d6af9f4e7a0ae761519fb50f721de398dd4e", size = 108084, upload-time = "2026-03-25T20:21:18.978Z" }, + { url = "https://files.pythonhosted.org/packages/b8/83/dceca96142499c069475b790e7913b1044c1a4337e700751f48ed723f883/tomli-2.4.1-cp311-cp311-win_arm64.whl", hash = "sha256:c2541745709bad0264b7d4705ad453b76ccd191e64aa6f0fc66b69a293a45ece", size = 95285, upload-time = "2026-03-25T20:21:20.309Z" }, + { url = "https://files.pythonhosted.org/packages/c1/ba/42f134a3fe2b370f555f44b1d72feebb94debcab01676bf918d0cb70e9aa/tomli-2.4.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:c742f741d58a28940ce01d58f0ab2ea3ced8b12402f162f4d534dfe18ba1cd6a", size = 155924, upload-time = "2026-03-25T20:21:21.626Z" }, + { url = "https://files.pythonhosted.org/packages/dc/c7/62d7a17c26487ade21c5422b646110f2162f1fcc95980ef7f63e73c68f14/tomli-2.4.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:7f86fd587c4ed9dd76f318225e7d9b29cfc5a9d43de44e5754db8d1128487085", size = 150018, upload-time = "2026-03-25T20:21:23.002Z" }, + { url = "https://files.pythonhosted.org/packages/5c/05/79d13d7c15f13bdef410bdd49a6485b1c37d28968314eabee452c22a7fda/tomli-2.4.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ff18e6a727ee0ab0388507b89d1bc6a22b138d1e2fa56d1ad494586d61d2eae9", size = 244948, upload-time = "2026-03-25T20:21:24.04Z" }, + { url = "https://files.pythonhosted.org/packages/10/90/d62ce007a1c80d0b2c93e02cab211224756240884751b94ca72df8a875ca/tomli-2.4.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:136443dbd7e1dee43c68ac2694fde36b2849865fa258d39bf822c10e8068eac5", size = 253341, upload-time = "2026-03-25T20:21:25.177Z" }, + { url = "https://files.pythonhosted.org/packages/1a/7e/caf6496d60152ad4ed09282c1885cca4eea150bfd007da84aea07bcc0a3e/tomli-2.4.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:5e262d41726bc187e69af7825504c933b6794dc3fbd5945e41a79bb14c31f585", size = 248159, upload-time = "2026-03-25T20:21:26.364Z" }, + { url = "https://files.pythonhosted.org/packages/99/e7/c6f69c3120de34bbd882c6fba7975f3d7a746e9218e56ab46a1bc4b42552/tomli-2.4.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:5cb41aa38891e073ee49d55fbc7839cfdb2bc0e600add13874d048c94aadddd1", size = 253290, upload-time = "2026-03-25T20:21:27.46Z" }, + { url = "https://files.pythonhosted.org/packages/d6/2f/4a3c322f22c5c66c4b836ec58211641a4067364f5dcdd7b974b4c5da300c/tomli-2.4.1-cp312-cp312-win32.whl", hash = "sha256:da25dc3563bff5965356133435b757a795a17b17d01dbc0f42fb32447ddfd917", size = 98141, upload-time = "2026-03-25T20:21:28.492Z" }, + { url = "https://files.pythonhosted.org/packages/24/22/4daacd05391b92c55759d55eaee21e1dfaea86ce5c571f10083360adf534/tomli-2.4.1-cp312-cp312-win_amd64.whl", hash = "sha256:52c8ef851d9a240f11a88c003eacb03c31fc1c9c4ec64a99a0f922b93874fda9", size = 108847, upload-time = "2026-03-25T20:21:29.386Z" }, + { url = "https://files.pythonhosted.org/packages/68/fd/70e768887666ddd9e9f5d85129e84910f2db2796f9096aa02b721a53098d/tomli-2.4.1-cp312-cp312-win_arm64.whl", hash = "sha256:f758f1b9299d059cc3f6546ae2af89670cb1c4d48ea29c3cacc4fe7de3058257", size = 95088, upload-time = "2026-03-25T20:21:30.677Z" }, + { url = "https://files.pythonhosted.org/packages/07/06/b823a7e818c756d9a7123ba2cda7d07bc2dd32835648d1a7b7b7a05d848d/tomli-2.4.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:36d2bd2ad5fb9eaddba5226aa02c8ec3fa4f192631e347b3ed28186d43be6b54", size = 155866, upload-time = "2026-03-25T20:21:31.65Z" }, + { url = "https://files.pythonhosted.org/packages/14/6f/12645cf7f08e1a20c7eb8c297c6f11d31c1b50f316a7e7e1e1de6e2e7b7e/tomli-2.4.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:eb0dc4e38e6a1fd579e5d50369aa2e10acfc9cace504579b2faabb478e76941a", size = 149887, upload-time = "2026-03-25T20:21:33.028Z" }, + { url = "https://files.pythonhosted.org/packages/5c/e0/90637574e5e7212c09099c67ad349b04ec4d6020324539297b634a0192b0/tomli-2.4.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c7f2c7f2b9ca6bdeef8f0fa897f8e05085923eb091721675170254cbc5b02897", size = 243704, upload-time = "2026-03-25T20:21:34.51Z" }, + { url = "https://files.pythonhosted.org/packages/10/8f/d3ddb16c5a4befdf31a23307f72828686ab2096f068eaf56631e136c1fdd/tomli-2.4.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f3c6818a1a86dd6dca7ddcaaf76947d5ba31aecc28cb1b67009a5877c9a64f3f", size = 251628, upload-time = "2026-03-25T20:21:36.012Z" }, + { url = "https://files.pythonhosted.org/packages/e3/f1/dbeeb9116715abee2485bf0a12d07a8f31af94d71608c171c45f64c0469d/tomli-2.4.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:d312ef37c91508b0ab2cee7da26ec0b3ed2f03ce12bd87a588d771ae15dcf82d", size = 247180, upload-time = "2026-03-25T20:21:37.136Z" }, + { url = "https://files.pythonhosted.org/packages/d3/74/16336ffd19ed4da28a70959f92f506233bd7cfc2332b20bdb01591e8b1d1/tomli-2.4.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:51529d40e3ca50046d7606fa99ce3956a617f9b36380da3b7f0dd3dd28e68cb5", size = 251674, upload-time = "2026-03-25T20:21:38.298Z" }, + { url = "https://files.pythonhosted.org/packages/16/f9/229fa3434c590ddf6c0aa9af64d3af4b752540686cace29e6281e3458469/tomli-2.4.1-cp313-cp313-win32.whl", hash = "sha256:2190f2e9dd7508d2a90ded5ed369255980a1bcdd58e52f7fe24b8162bf9fedbd", size = 97976, upload-time = "2026-03-25T20:21:39.316Z" }, + { url = "https://files.pythonhosted.org/packages/6a/1e/71dfd96bcc1c775420cb8befe7a9d35f2e5b1309798f009dca17b7708c1e/tomli-2.4.1-cp313-cp313-win_amd64.whl", hash = "sha256:8d65a2fbf9d2f8352685bc1364177ee3923d6baf5e7f43ea4959d7d8bc326a36", size = 108755, upload-time = "2026-03-25T20:21:40.248Z" }, + { url = "https://files.pythonhosted.org/packages/83/7a/d34f422a021d62420b78f5c538e5b102f62bea616d1d75a13f0a88acb04a/tomli-2.4.1-cp313-cp313-win_arm64.whl", hash = "sha256:4b605484e43cdc43f0954ddae319fb75f04cc10dd80d830540060ee7cd0243cd", size = 95265, upload-time = "2026-03-25T20:21:41.219Z" }, + { url = "https://files.pythonhosted.org/packages/3c/fb/9a5c8d27dbab540869f7c1f8eb0abb3244189ce780ba9cd73f3770662072/tomli-2.4.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:fd0409a3653af6c147209d267a0e4243f0ae46b011aa978b1080359fddc9b6cf", size = 155726, upload-time = "2026-03-25T20:21:42.23Z" }, + { url = "https://files.pythonhosted.org/packages/62/05/d2f816630cc771ad836af54f5001f47a6f611d2d39535364f148b6a92d6b/tomli-2.4.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:a120733b01c45e9a0c34aeef92bf0cf1d56cfe81ed9d47d562f9ed591a9828ac", size = 149859, upload-time = "2026-03-25T20:21:43.386Z" }, + { url = "https://files.pythonhosted.org/packages/ce/48/66341bdb858ad9bd0ceab5a86f90eddab127cf8b046418009f2125630ecb/tomli-2.4.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:559db847dc486944896521f68d8190be1c9e719fced785720d2216fe7022b662", size = 244713, upload-time = "2026-03-25T20:21:44.474Z" }, + { url = "https://files.pythonhosted.org/packages/df/6d/c5fad00d82b3c7a3ab6189bd4b10e60466f22cfe8a08a9394185c8a8111c/tomli-2.4.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:01f520d4f53ef97964a240a035ec2a869fe1a37dde002b57ebc4417a27ccd853", size = 252084, upload-time = "2026-03-25T20:21:45.62Z" }, + { url = "https://files.pythonhosted.org/packages/00/71/3a69e86f3eafe8c7a59d008d245888051005bd657760e96d5fbfb0b740c2/tomli-2.4.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7f94b27a62cfad8496c8d2513e1a222dd446f095fca8987fceef261225538a15", size = 247973, upload-time = "2026-03-25T20:21:46.937Z" }, + { url = "https://files.pythonhosted.org/packages/67/50/361e986652847fec4bd5e4a0208752fbe64689c603c7ae5ea7cb16b1c0ca/tomli-2.4.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:ede3e6487c5ef5d28634ba3f31f989030ad6af71edfb0055cbbd14189ff240ba", size = 256223, upload-time = "2026-03-25T20:21:48.467Z" }, + { url = "https://files.pythonhosted.org/packages/8c/9a/b4173689a9203472e5467217e0154b00e260621caa227b6fa01feab16998/tomli-2.4.1-cp314-cp314-win32.whl", hash = "sha256:3d48a93ee1c9b79c04bb38772ee1b64dcf18ff43085896ea460ca8dec96f35f6", size = 98973, upload-time = "2026-03-25T20:21:49.526Z" }, + { url = "https://files.pythonhosted.org/packages/14/58/640ac93bf230cd27d002462c9af0d837779f8773bc03dee06b5835208214/tomli-2.4.1-cp314-cp314-win_amd64.whl", hash = "sha256:88dceee75c2c63af144e456745e10101eb67361050196b0b6af5d717254dddf7", size = 109082, upload-time = "2026-03-25T20:21:50.506Z" }, + { url = "https://files.pythonhosted.org/packages/d5/2f/702d5e05b227401c1068f0d386d79a589bb12bf64c3d2c72ce0631e3bc49/tomli-2.4.1-cp314-cp314-win_arm64.whl", hash = "sha256:b8c198f8c1805dc42708689ed6864951fd2494f924149d3e4bce7710f8eb5232", size = 96490, upload-time = "2026-03-25T20:21:51.474Z" }, + { url = "https://files.pythonhosted.org/packages/45/4b/b877b05c8ba62927d9865dd980e34a755de541eb65fffba52b4cc495d4d2/tomli-2.4.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:d4d8fe59808a54658fcc0160ecfb1b30f9089906c50b23bcb4c69eddc19ec2b4", size = 164263, upload-time = "2026-03-25T20:21:52.543Z" }, + { url = "https://files.pythonhosted.org/packages/24/79/6ab420d37a270b89f7195dec5448f79400d9e9c1826df982f3f8e97b24fd/tomli-2.4.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:7008df2e7655c495dd12d2a4ad038ff878d4ca4b81fccaf82b714e07eae4402c", size = 160736, upload-time = "2026-03-25T20:21:53.674Z" }, + { url = "https://files.pythonhosted.org/packages/02/e0/3630057d8eb170310785723ed5adcdfb7d50cb7e6455f85ba8a3deed642b/tomli-2.4.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1d8591993e228b0c930c4bb0db464bdad97b3289fb981255d6c9a41aedc84b2d", size = 270717, upload-time = "2026-03-25T20:21:55.129Z" }, + { url = "https://files.pythonhosted.org/packages/7a/b4/1613716072e544d1a7891f548d8f9ec6ce2faf42ca65acae01d76ea06bb0/tomli-2.4.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:734e20b57ba95624ecf1841e72b53f6e186355e216e5412de414e3c51e5e3c41", size = 278461, upload-time = "2026-03-25T20:21:56.228Z" }, + { url = "https://files.pythonhosted.org/packages/05/38/30f541baf6a3f6df77b3df16b01ba319221389e2da59427e221ef417ac0c/tomli-2.4.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:8a650c2dbafa08d42e51ba0b62740dae4ecb9338eefa093aa5c78ceb546fcd5c", size = 274855, upload-time = "2026-03-25T20:21:57.653Z" }, + { url = "https://files.pythonhosted.org/packages/77/a3/ec9dd4fd2c38e98de34223b995a3b34813e6bdadf86c75314c928350ed14/tomli-2.4.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:504aa796fe0569bb43171066009ead363de03675276d2d121ac1a4572397870f", size = 283144, upload-time = "2026-03-25T20:21:59.089Z" }, + { url = "https://files.pythonhosted.org/packages/ef/be/605a6261cac79fba2ec0c9827e986e00323a1945700969b8ee0b30d85453/tomli-2.4.1-cp314-cp314t-win32.whl", hash = "sha256:b1d22e6e9387bf4739fbe23bfa80e93f6b0373a7f1b96c6227c32bef95a4d7a8", size = 108683, upload-time = "2026-03-25T20:22:00.214Z" }, + { url = "https://files.pythonhosted.org/packages/12/64/da524626d3b9cc40c168a13da8335fe1c51be12c0a63685cc6db7308daae/tomli-2.4.1-cp314-cp314t-win_amd64.whl", hash = "sha256:2c1c351919aca02858f740c6d33adea0c5deea37f9ecca1cc1ef9e884a619d26", size = 121196, upload-time = "2026-03-25T20:22:01.169Z" }, + { url = "https://files.pythonhosted.org/packages/5a/cd/e80b62269fc78fc36c9af5a6b89c835baa8af28ff5ad28c7028d60860320/tomli-2.4.1-cp314-cp314t-win_arm64.whl", hash = "sha256:eab21f45c7f66c13f2a9e0e1535309cee140182a9cdae1e041d02e47291e8396", size = 100393, upload-time = "2026-03-25T20:22:02.137Z" }, + { url = "https://files.pythonhosted.org/packages/7b/61/cceae43728b7de99d9b847560c262873a1f6c98202171fd5ed62640b494b/tomli-2.4.1-py3-none-any.whl", hash = "sha256:0d85819802132122da43cb86656f8d1f8c6587d54ae7dcaf30e90533028b49fe", size = 14583, upload-time = "2026-03-25T20:22:03.012Z" }, +] + +[[package]] +name = "typing-extensions" +version = "4.15.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/72/94/1a15dd82efb362ac84269196e94cf00f187f7ed21c242792a923cdb1c61f/typing_extensions-4.15.0.tar.gz", hash = "sha256:0cea48d173cc12fa28ecabc3b837ea3cf6f38c6d1136f85cbaaf598984861466", size = 109391, upload-time = "2025-08-25T13:49:26.313Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl", hash = "sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548", size = 44614, upload-time = "2025-08-25T13:49:24.86Z" }, +] + +[[package]] +name = "urllib3" +version = "2.7.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/53/0c/06f8b233b8fd13b9e5ee11424ef85419ba0d8ba0b3138bf360be2ff56953/urllib3-2.7.0.tar.gz", hash = "sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c", size = 433602, upload-time = "2026-05-07T16:13:18.596Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7f/3e/5db95bcf282c52709639744ca2a8b149baccf648e39c8cc87553df9eae0c/urllib3-2.7.0-py3-none-any.whl", hash = "sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897", size = 131087, upload-time = "2026-05-07T16:13:17.151Z" }, +] diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index a9a1db02a8..baa1a1e3db 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -12,15 +12,14 @@ # See the License for the specific language governing permissions and # limitations under the License. -load("@bazel_skylib//:bzl_library.bzl", "bzl_library") -load("@dev_pip//:requirements.bzl", "requirement") +load("@bazel_skylib//rules:build_test.bzl", "build_test") +load("@sphinxdocs//sphinxdocs:readthedocs.bzl", "readthedocs_install") +load("@sphinxdocs//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs") +load("@sphinxdocs//sphinxdocs:sphinx_docs_library.bzl", "sphinx_docs_library") +load("@sphinxdocs//sphinxdocs:sphinx_stardoc.bzl", "sphinx_stardoc", "sphinx_stardocs") +load("//python:defs.bzl", "py_binary") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility -load("//python/uv/private:lock.bzl", "lock") # buildifier: disable=bzl-visibility -load("//sphinxdocs:readthedocs.bzl", "readthedocs_install") -load("//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs") -load("//sphinxdocs:sphinx_docs_library.bzl", "sphinx_docs_library") -load("//sphinxdocs:sphinx_stardoc.bzl", "sphinx_stardoc", "sphinx_stardocs") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility package(default_visibility = ["//:__subpackages__"]) @@ -38,6 +37,27 @@ _TARGET_COMPATIBLE_WITH = select({ "//conditions:default": ["@platforms//:incompatible"], }) if BZLMOD_ENABLED else ["@platforms//:incompatible"] +py_binary( + name = "merge_changelog", + srcs = ["merge_changelog.py"], + target_compatible_with = _TARGET_COMPATIBLE_WITH, + deps = [ + "//tools/private/release:changelog_news", + ], +) + +genrule( + name = "merged_changelog", + srcs = [ + "//:CHANGELOG.md", + "//news:news_files", + ], + outs = ["changelog.md"], + cmd = "$(location :merge_changelog) --changelog $(location //:CHANGELOG.md) --news-dir news --output $@", + target_compatible_with = _TARGET_COMPATIBLE_WITH, + tools = [":merge_changelog"], +) + # See README.md for instructions. Short version: # * `bazel run //docs:docs.serve` in a separate terminal # * `ibazel build //docs:docs` to automatically rebuild docs @@ -55,15 +75,15 @@ sphinx_docs( "_*", "*.inv*", ], - ), + ) + ["//gazelle/docs"], config = "conf.py", formats = [ "html", ], renamed_srcs = { - "//:CHANGELOG.md": "changelog.md", "//:CONTRIBUTING.md": "contributing.md", - "//sphinxdocs/inventories:bazel_inventory": "bazel_inventory.inv", + ":merged_changelog": "changelog.md", + "@sphinxdocs//sphinxdocs/inventories:bazel_inventory": "bazel_inventory.inv", }, sphinx = ":sphinx-build", strip_prefix = package_name() + "/", @@ -73,43 +93,73 @@ sphinx_docs( ":bzl_api_docs", ":py_api_srcs", ":py_runtime_pair", - "//sphinxdocs/docs:docs_lib", + "@sphinxdocs//docs:docs_lib", ], ) +build_test( + name = "docs_build_test", + targets = [":docs"], +) + sphinx_stardocs( name = "bzl_api_docs", srcs = [ - "//python:defs_bzl", - "//python:packaging_bzl", - "//python:pip_bzl", - "//python:py_binary_bzl", - "//python:py_cc_link_params_info_bzl", - "//python:py_exec_tools_info_bzl", - "//python:py_exec_tools_toolchain_bzl", - "//python:py_executable_info_bzl", - "//python:py_library_bzl", - "//python:py_runtime_bzl", - "//python:py_runtime_info_bzl", - "//python:py_test_bzl", - "//python:repositories_bzl", - "//python/api:api_bzl", - "//python/cc:py_cc_toolchain_bzl", - "//python/cc:py_cc_toolchain_info_bzl", - "//python/entry_points:py_console_script_binary_bzl", - "//python/private:py_binary_rule_bzl", - "//python/private:py_cc_toolchain_rule_bzl", - "//python/private:py_library_rule_bzl", - "//python/private:py_runtime_rule_bzl", - "//python/private:py_test_rule_bzl", - "//python/private/api:py_common_api_bzl", + "//python:defs", + "//python:features", + "//python:packaging", + "//python:pip", + "//python:proto", + "//python:py_binary", + "//python:py_cc_link_params_info", + "//python:py_exec_tools_info", + "//python:py_exec_tools_toolchain", + "//python:py_executable_info", + "//python:py_info", + "//python:py_library", + "//python:py_runtime", + "//python:py_runtime_info", + "//python:py_test", + "//python:repositories", + "//python/api", + "//python/api:attr_builders", + "//python/api:executables", + "//python/api:libraries", + "//python/api:rule_builders", + "//python/cc:py_cc_toolchain", + "//python/cc:py_cc_toolchain_info", + "//python/cc:py_extension", + "//python/entry_points:py_console_script_binary", + "//python/extensions:config", + "//python/extensions:python", + "//python/local_toolchains:repos", + "//python/private:attr_builders", + "//python/private:builders", + "//python/private:builders_util", + "//python/private:py_binary_rule", + "//python/private:py_cc_toolchain_rule", + "//python/private:py_info", + "//python/private:py_library_rule", + "//python/private:py_runtime_rule", + "//python/private:py_test_rule", + "//python/private:rule_builders", + "//python/private/api:py_common_api", + "//python/private/pypi:config_settings", + "//python/private/pypi:env_marker_info", + "//python/private/pypi:pkg_aliases", + "//python/private/pypi:whl_config_setting", + "//python/private/pypi:whl_library", + "//python/private/zipapp:py_zipapp_rule", + "//python/uv", + "//python/uv:lock", + "//python/uv:uv_toolchain", + "//python/uv:uv_toolchain_info", + "//python/zipapp:py_zipapp_binary", + "//python/zipapp:py_zipapp_test", ] + ([ - # Bazel 6 + Stardoc isn't able to parse something about the python bzlmod extension - "//python/extensions:python_bzl", - ] if IS_BAZEL_7_OR_HIGHER else []) + ([ # This depends on @pythons_hub, which is only created under bzlmod, - "//python/extensions:pip_bzl", - ] if IS_BAZEL_7_OR_HIGHER and BZLMOD_ENABLED else []), + "//python/extensions:pip", + ] if BZLMOD_ENABLED else []), prefix = "api/rules_python/", tags = ["docs"], target_compatible_with = _TARGET_COMPATIBLE_WITH, @@ -117,7 +167,7 @@ sphinx_stardocs( sphinx_stardoc( name = "py_runtime_pair", - src = "//python/private:py_runtime_pair_rule_bzl", + src = "//python/private:py_runtime_pair_rule", prefix = "api/rules_python/", tags = ["docs"], target_compatible_with = _TARGET_COMPATIBLE_WITH, @@ -139,59 +189,22 @@ readthedocs_install( sphinx_build_binary( name = "sphinx-build", + config_settings = { + labels.BOOTSTRAP_IMPL: "script", + labels.VENVS_SITE_PACKAGES: "yes", + labels.PY_FREETHREADED: "yes", + labels.PYTHON_VERSION: "3.14", + }, target_compatible_with = _TARGET_COMPATIBLE_WITH, deps = [ - requirement("sphinx"), - requirement("sphinx_rtd_theme"), - requirement("myst_parser"), - requirement("readthedocs_sphinx_ext"), - requirement("typing_extensions"), - requirement("sphinx_autodoc2"), - requirement("sphinx_reredirects"), - "//sphinxdocs/src/sphinx_bzl", + "@dev_pip//myst_parser", + "@dev_pip//pytest", + "@dev_pip//readthedocs_sphinx_ext", + "@dev_pip//sphinx", + "@dev_pip//sphinx_autodoc2", + "@dev_pip//sphinx_reredirects", + "@dev_pip//sphinx_rtd_theme", + "@dev_pip//typing_extensions", + "@sphinxdocs//sphinxdocs/src/sphinx_bzl", ], ) - -# Run bazel run //docs:requirements.update -lock( - name = "requirements", - srcs = ["pyproject.toml"], - out = "requirements.txt", - upgrade = True, - visibility = ["//private:__pkg__"], -) - -# Temporary compatibility aliases for some other projects depending on the old -# bzl_library targets. -alias( - name = "defs", - actual = "//python:defs_bzl", - deprecation = "Use //python:defs_bzl instead; targets under //docs are internal.", - visibility = ["//visibility:public"], -) - -alias( - name = "bazel_repo_tools", - actual = "//python/private:bazel_tools_bzl", - deprecation = "Use @bazel_tools//tools:bzl_srcs instead; targets under //docs are internal.", - visibility = ["//visibility:public"], -) - -bzl_library( - name = "pip_install_bzl", - deprecation = "Use //python:pip_bzl or //python/pip_install:pip_repository_bzl instead; " + - "targets under //docs are internal.", - visibility = ["//visibility:public"], - deps = [ - "//python:pip_bzl", - "//python/pip_install:pip_repository_bzl", - ], -) - -alias( - name = "requirements_parser_bzl", - actual = "//python/pip_install:pip_repository_bzl", - deprecation = "Use //python/pip_install:pip_repository_bzl instead; Both the requirements " + - "parser and targets under //docs are internal", - visibility = ["//visibility:public"], -) diff --git a/docs/README.md b/docs/README.md index d98be41232..1316d733bb 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,14 +1,14 @@ # rules_python Sphinx docs generation The docs for rules_python are generated using a combination of Sphinx, Bazel, -and Readthedocs.org. The Markdown files in source control are unlikely to render +and Read the Docs. The Markdown files in source control are unlikely to render properly without the Sphinx processing step because they rely on Sphinx and MyST-specific Markdown functionality. The actual sources that Sphinx consumes are in this directory, with Stardoc -generating additional sources or Sphinx. +generating additional sources for Sphinx. -Manually building the docs isn't necessary -- readthedocs.org will +Manually building the docs isn't necessary -- Read the Docs will automatically build and deploy them when commits are pushed to the repo. ## Generating docs for development @@ -28,11 +28,25 @@ changes and re-run the build process, and you can simply refresh your browser to see the changes. Using ibazel is not required; you can manually run the equivalent bazel command if desired. +An alternative to `ibazel` is using `inotify` on Linux systems: + +``` +inotifywait --event modify --monitor . --recursive --includei '^.*\.md$' | +while read -r dir events filename; do bazel build //docs:docs; done; +``` + +And lastly, a poor-man's `ibazel` and `inotify` is simply `watch` with +a reasonable interval like 10s: + +``` +watch --interval 10 bazel build //docs:docs +``` + ### Installing ibazel The `ibazel` tool can be used to automatically rebuild the docs as you -development them. See the [ibazel docs](https://github.com/bazelbuild/bazel-watcher) for -how to install it. The quick start for linux is: +develop them. See the [ibazel docs](https://github.com/bazelbuild/bazel-watcher) for +how to install it. The quick start for Linux is: ``` sudo apt install npm @@ -57,9 +71,9 @@ docs/. The Sphinx configuration is `docs/conf.py`. See https://www.sphinx-doc.org/ for details about the configuration file. -## Readthedocs configuration +## Read the Docs configuration -There's two basic parts to the readthedocs configuration: +There's two basic parts to the Read the Docs configuration: * `.readthedocs.yaml`: This configuration file controls most settings, such as the OS version used to build, Python version, dependencies, what Bazel @@ -69,4 +83,4 @@ There's two basic parts to the readthedocs configuration: controls additional settings such as permissions, what versions are published, when to publish changes, etc. -For more readthedocs configuration details, see docs.readthedocs.io. +For more Read the Docs configuration details, see docs.readthedocs.io. diff --git a/docs/_includes/experimental_api.md b/docs/_includes/experimental_api.md new file mode 100644 index 0000000000..45473a7cbf --- /dev/null +++ b/docs/_includes/experimental_api.md @@ -0,0 +1,5 @@ +:::{warning} + +**Experimental API.** This API is still under development and may change or be +removed without notice. +::: diff --git a/docs/_includes/field_kwargs_doc.md b/docs/_includes/field_kwargs_doc.md new file mode 100644 index 0000000000..0241947b43 --- /dev/null +++ b/docs/_includes/field_kwargs_doc.md @@ -0,0 +1,11 @@ +:::{field} kwargs +:type: dict[str, Any] + +Additional kwargs to use when building. This is to allow manipulations that +aren't directly supported by the builder's API. The state of this dict +may or may not reflect prior API calls, and subsequent API calls may +modify this dict. The general contract is that modifications to this will +be respected when `build()` is called, assuming there were no API calls +in between. +::: + diff --git a/docs/_includes/py_console_script_binary.md b/docs/_includes/py_console_script_binary.md index 7373c8a7b2..c9686f2eee 100644 --- a/docs/_includes/py_console_script_binary.md +++ b/docs/_includes/py_console_script_binary.md @@ -1,8 +1,8 @@ This rule is to make it easier to generate `console_script` entry points as per Python [specification]. -Generate a `py_binary` target for a particular console_script `entry_point` -from a PyPI package, e.g. for creating an executable `pylint` target use: +Generate a `py_binary` target for a particular `console_script` entry_point +from a PyPI package, e.g. for creating an executable `pylint` target, use: ```starlark load("@rules_python//python/entry_points:py_console_script_binary.bzl", "py_console_script_binary") @@ -12,10 +12,13 @@ py_console_script_binary( ) ``` -Or for more advanced setups you can also specify extra dependencies and the -exact script name you want to call. It is useful for tools like `flake8`, `pylint`, -`pytest`, which have plugin discovery methods and discover dependencies from the -PyPI packages available in the `PYTHONPATH`. +:::{rubric} Specifying extra dependencies +::: +You can also specify extra dependencies and the +exact script name you want to call. This is useful for tools like `flake8`, +`pylint`, and `pytest`, which have plugin discovery methods and discover +dependencies from the PyPI packages available in the `PYTHONPATH`. + ```starlark load("@rules_python//python/entry_points:py_console_script_binary.bzl", "py_console_script_binary") @@ -34,18 +37,50 @@ py_console_script_binary( ) ``` -A specific Python version can be forced by using the generated version-aware -wrappers, e.g. to force Python 3.9: +:::{rubric} Using a specific Python version +::: + +A specific Python version can be forced by passing the desired Python version, e.g. to force Python 3.9: ```starlark -load("@python_versions//3.9:defs.bzl", "py_console_script_binary") +load("@rules_python//python/entry_points:py_console_script_binary.bzl", "py_console_script_binary") py_console_script_binary( name = "yamllint", pkg = "@pip//yamllint", + python_version = "3.9", +) +``` + +:::{rubric} Adding a Shebang Line +::: + +You can specify a shebang line for the generated binary. This is useful for Unix-like +systems where the shebang line determines which interpreter is used to execute +the script, per [PEP441]: + +```starlark +load("@rules_python//python/entry_points:py_console_script_binary.bzl", "py_console_script_binary") + +py_console_script_binary( + name = "black", + pkg = "@pip//black", + shebang = "#!/usr/bin/env python3", ) ``` -Alternatively, the [`py_console_script_binary.binary_rule`] arg can be passed +Note that to execute via the shebang line, you need to ensure the specified +Python interpreter is available in the environment. + + +:::{rubric} Using a specific Python Version directly from a Toolchain +::: +:::{deprecated} 1.1.0 +The toolchain-specific `py_binary` and `py_test` symbols are aliases to the regular rules. +For example, `load("@python_versions//3.11:defs.bzl", "py_binary")` and `load("@python_versions//3.11:defs.bzl", "py_test")` are deprecated. + +You should instead specify the desired Python version with `python_version`; see the example above. +::: +Alternatively, the {obj}`py_console_script_binary.binary_rule` arg can be passed the version-bound `py_binary` symbol, or any other `py_binary`-compatible rule of your choosing: ```starlark @@ -61,4 +96,4 @@ py_console_script_binary( [specification]: https://packaging.python.org/en/latest/specifications/entry-points/ [`py_console_script_binary.binary_rule`]: #py_console_script_binary_binary_rule - +[PEP441]: https://peps.python.org/pep-0441/#minimal-tooling-the-zipapp-module diff --git a/docs/_includes/volatile_api.md b/docs/_includes/volatile_api.md new file mode 100644 index 0000000000..b79f5f7061 --- /dev/null +++ b/docs/_includes/volatile_api.md @@ -0,0 +1,5 @@ +:::{important} + +**Public, but volatile, API.** Some parts are stable, while others are +implementation details and may change more frequently. +::: diff --git a/docs/api/rules_python/command_line_option/index.md b/docs/api/rules_python/command_line_option/index.md new file mode 100644 index 0000000000..02d29206f5 --- /dev/null +++ b/docs/api/rules_python/command_line_option/index.md @@ -0,0 +1,76 @@ +:::{default-domain} bzl +::: +:::{bzl:currentfile} //command_line_option:BUILD.bazel +::: + +# //command_line_option + +This package provides special targets that correspond to the Bazel-builtin +`//command_line_option` psuedo-targets. These can be used with the {obj}`config_settings` +attribute on Python rules to transition specific command line flags for a target. + +:::{note} +These targets are not actual `alias()` targets, Starlark flags, nor are they the +actual builtin command line flags. They are regular targets that the +`config_settings` transition logic specially recognizes and handles as if they +were the builtin `//command_line_option` psuedo-targets. +::: + +While this package only provides a subset of builtin Bazel flags, additional +ones can be introduced by: + +* Define your own `@foo//command_line_flag:` target. It **must** be + in a top-level `command_line_flag` directory. +* Use {obj}`config.add_transition_setting` to make the rules transition on + the corresponding `//command_line_option:` builtin Bazel psuedo-target. + +:::{seealso} +The `config_settings` attribute documentation on: +* {obj}`py_binary.config_settings` +* {obj}`py_test.config_settings` +::: + +## build_runfile_links + +:::{bzl:target} build_runfile_links + +Special target for the Bazel-builtin `//command_line_option:build_runfile_links` flag. + +See the [Bazel documentation for --build_runfile_links](https://bazel.build/reference/command-line-reference#flag--build_runfile_links). + +The special value `INHERIT` can be specified to use the existing flag value. +::: + +## enable_runfiles + +:::{bzl:target} enable_runfiles + +Special target for the Bazel-builtin `//command_line_option:enable_runfiles` flag. + +See the [Bazel documentation for --enable_runfiles](https://bazel.build/reference/command-line-reference#flag--enable_runfiles). + +The special value `INHERIT` can be specified to use the existing flag value. +::: + +## extra_toolchains + +:::{bzl:target} extra_toolchains + +Special target for the Bazel-builtin `//command_line_option:extra_toolchains` +flag. + +See the [Bazel documentation for --extra_toolchains](https://bazel.build/reference/command-line-reference#flag--extra_toolchains). + +The special value `INHERIT` can be specified to use the existing flag value. + +:::{note} +Unlike the normal Bazel built-in flag, which accepts a list of labels, this +pseudo-flag must be specified as a single, comma-separated string when set +using the `config_settings` attribute. For example: + +```python +"//command_line_option:extra_toolchains": "//my/tc1,//my/tc2" +``` +::: +::: + diff --git a/docs/api/rules_python/python/bin/index.md b/docs/api/rules_python/python/bin/index.md new file mode 100644 index 0000000000..873b644341 --- /dev/null +++ b/docs/api/rules_python/python/bin/index.md @@ -0,0 +1,42 @@ +:::{default-domain} bzl +::: +:::{bzl:currentfile} //python/bin:BUILD.bazel +::: + +# //python/bin + +:::{bzl:target} python + +A target to directly run a Python interpreter. + +By default, it uses the Python version that toolchain resolution matches +(typically the one set with `python.defaults(python_version = ...)` in +`MODULE.bazel`). + +This runs a Python interpreter in a similar manner as when running `python3` +on the command line. It can be invoked using `bazel run`. Remember that in +order to pass flags onto the program `--` must be specified to separate +Bazel flags from the program flags. + +An example that will run Python 3.12 and have it print the version + +``` +bazel run @rules_python//python/bin:python \ + `--@rule_python//python/config_settings:python_verion=3.12 \ + -- \ + --version +``` + +::::{seealso} +The {flag}`--python_src` flag for using the intepreter a binary/test uses. +:::: + +::::{versionadded} 1.3.0 +:::: +::: + +:::{bzl:flag} python_src + +The target (one providing `PyRuntimeInfo`) whose python interpreter to use for +{obj}`:python`. +::: diff --git a/docs/api/rules_python/python/cc/index.md b/docs/api/rules_python/python/cc/index.md index 82c59343be..98d68dacd5 100644 --- a/docs/api/rules_python/python/cc/index.md +++ b/docs/api/rules_python/python/cc/index.md @@ -4,7 +4,7 @@ ::: # //python/cc -:::{bzl:target} current_py_cc_headers +::::{bzl:target} current_py_cc_headers A convenience target that provides the Python headers. It uses toolchain resolution to find the headers for the Python runtime matching the interpreter @@ -14,7 +14,32 @@ that will be used. This basically forwards the underlying This target provides: * `CcInfo`: The C++ information about the Python headers. + +:::{seealso} + +The {obj}`:current_py_cc_headers_abi3` target for explicitly using the +stable ABI. +::: + +:::: + +::::{bzl:target} current_py_cc_headers_abi3 + +A convenience target that provides the Python ABI3 headers (stable ABI headers). +It uses toolchain resolution to find the headers for the Python runtime matching +the interpreter that will be used. This basically forwards the underlying +`cc_library(name="python_headers_abi3")` target defined in the `@python_X_Y` +repo. + +This target provides: + +* `CcInfo`: The C++ information about the Python ABI3 headers. + +:::{versionadded} 1.7.0 +The {obj}`features.headers_abi3` attribute can be used to detect if this target +is available or not. ::: +:::: :::{bzl:target} current_py_cc_libs diff --git a/docs/api/rules_python/python/config_settings/index.md b/docs/api/rules_python/python/config_settings/index.md index ef829bab76..9ae0665da2 100644 --- a/docs/api/rules_python/python/config_settings/index.md +++ b/docs/api/rules_python/python/config_settings/index.md @@ -24,6 +24,86 @@ This is a transition flag and will be removed in a subsequent release. :::: ::: +::::{bzl:flag} build_python_zip +Controls if a `py_binary/py_test` output is a self-executable zipapp. + +When enabled, the output of `py_binary` or `py_test` targets will be a +self-executable zipapp. + +:::{note} +This affects _all_ `py_binary` and `py_test` targets in the build, not +only the target(s) specified on the command line. +::: + +Values: +* `true` +* `false` + +This flag replaces the Bazel builtin `--build_python_zip` flag. + +:::{versionadded} 1.7.0 +::: +:::: + +::::{bzl:flag} debugger +A target for providing a custom debugger dependency. + +This flag is roughly equivalent to putting a target in `deps`. It allows +injecting a dependency into executables (`py_binary`, `py_test`) without having +to modify their deps. The expectation is it points to a target that provides an +alternative debugger (pudb, winpdb, debugpy, etc). + +* Must provide {obj}`PyInfo`. +* This dependency is only used for the target config, i.e. build tools don't + have it added. + +:::{note} +Setting this flag adds the debugger dependency, but doesn't automatically set +`PYTHONBREAKPOINT` to change `breakpoint()` behavior. +::: + +:::{versionadded} 1.9.0 +::: +:::: + +::::{bzl:flag} experimental_python_import_all_repositories +Controls whether repository directories are added to the import path. + +When enabled, the top-level directories in the runfiles root directory (which +are presumbed to be repository directories) are added to the Python import +search path. + +It's recommended to set this to **`false`** to avoid external dependencies +unexpectedly interferring with import searching. + +Values; +* `true` (default) +* `false` + +This flag replaces the Bazel builtin +`--experimental_python_import_all_repositories` flag. + +:::{versionadded} 1.7.0 +::: +:::: + +::::{bzl:flag} python_path +A fallback path to use for Python for particular legacy Windows-specific code paths. + +Deprecated, do not use. This flag is largely a no-op and was replaced by +toolchains. It only remains for some legacy Windows code-paths that will +be removed. + +This flag replaces the Bazel builtin `--python_path` flag. + +:::{deprecated} 1.7.0 +Use toolchains instead. +::: + +:::{versionadded} 1.7.0 +::: +:::: + :::{bzl:flag} python_version Determines the default hermetic Python toolchain version. This can be set to one of the values that `rules_python` maintains. @@ -33,6 +113,29 @@ one of the values that `rules_python` maintains. Parses the value of the `python_version` and transforms it into a `X.Y` value. ::: +::::{bzl:flag} incompatible_default_to_explicit_init_py +Controls if missing `__init__.py` files are generated or not. + +If false, `py_binary` and `py_test` will, for every `*.py` and `*.so` file, +create `__init__.py` files for the containing directory, and all parent +directories, that do not already have an `__init__.py` file. If true, this +behavior is disabled. + +It's recommended to disable this behavior to avoid surprising import effects +from directories being importable when they otherwise wouldn't be, and for +how it can interfere with implicit namespace packages. + +Values: +* `true`: do not generate missing `__init__.py` files +* `false` (default): generate missing `__init__.py` files + +This flag replaces the Bazel builtin +`--incompatible_default_to_explicit_init_py` flag. + +:::{versionadded} 1.7.0 +::: +:::: + :::{bzl:target} is_python_* config_settings to match Python versions @@ -46,7 +149,7 @@ of builtin, known versions. If you need to match a version that isn't present, then you have two options: 1. Manually define a `config_setting` and have it match {obj}`--python_version` - or {ob}`python_version_major_minor`. This works best when you don't control the + or {obj}`python_version_major_minor`. This works best when you don't control the root module, or don't want to rely on the MODULE.bazel configuration. Such a config settings would look like: ``` @@ -66,7 +169,7 @@ If you need to match a version that isn't present, then you have two options: ) ``` -2. Use {obj}`python.single_override` to re-introduce the desired version so +2. Use {obj}`python.single_version_override` to re-introduce the desired version so that the corresponding `//python/config_setting:is_python_XXX` target is generated. ::: @@ -139,6 +242,44 @@ The `auto` value The `omit_if_generated_source` value was removed :::: +::::{bzl:flag} validate_test_main +Determines if `py_test` runs a build-time validation that its main module +actually runs tests. + +A common `py_test` pitfall is to define test classes or functions but forget +to add code that runs them (for example, assuming `py_test` automatically +invokes `unittest` or `pytest`). When that happens, the test does nothing and +silently passes. + +When enabled, a validation action statically analyzes the main module and +fails the build if it defines test classes or functions but its top-level body +is "inert" -- i.e. it only contains definitions, imports, assignments, and +docstrings, with nothing that actually runs tests (such as an +`if __name__ == "__main__":` guard that invokes a test runner). + +A module that defines no classes or functions at all (for example, one that +only imports other modules) is always allowed, since it isn't the +"defined some tests but forgot to run them" case this check targets. + +This is only applicable to `py_test` targets that have a `main` source file; +targets using `main_module` are not checked. + +Values: + +* `auto`: (default) Automatically decide the effective value; the current + behavior is `disabled`. +* `enabled`: Run the validation action. +* `disabled`: Don't run the validation action. + +:::{note} +Enabling this requires the exec tools toolchain (with an exec interpreter) to +be registered, which is the case for the default hermetic toolchains. +::: + +:::{versionadded} 2.3.0 +::: +:::: + ::::{bzl:flag} py_linux_libc Set what libc is used for the target platform. This will affect which whl binaries will be pulled and what toolchain will be auto-detected. Currently `rules_python` only supplies toolchains compatible with `glibc`. @@ -159,64 +300,52 @@ Values: ::: :::: -::::{bzl:flag} pip_whl -Set what distributions are used in the `pip` integration. +::::{bzl:flag} pip_env_marker_config +The target that provides the values for pip env marker evaluation. -Values: -* `auto`: Prefer `whl` distributions if they are compatible with a target - platform, but fallback to `sdist`. This is the default. -* `only`: Only use `whl` distributions and error out if it is not available. -* `no`: Only use `sdist` distributions. The wheels will be built non-hermetically in the `whl_library` repository rule. -:::{versionadded} 0.33.0 -::: -:::: +Default: `//python/config_settings:_pip_env_marker_default_config` -::::{bzl:flag} pip_whl_osx_arch -Set what wheel types we should prefer when building on the OSX platform. +This flag points to a target providing {obj}`EnvMarkerInfo`, which determines +the values used when environment markers are resolved at build time. -Values: -* `arch`: Prefer architecture specific wheels. -* `universal`: Prefer universal wheels that usually are bigger and contain binaries for both, Intel and ARM architectures in the same wheel. -:::{versionadded} 0.33.0 +:::{versionadded} 1.5.0 ::: :::: -::::{bzl:flag} pip_whl_glibc_version -Set the minimum `glibc` version that the `py_binary` using `whl` distributions from a PyPI index should support. +::::{bzl:flag} pip_whl_osx_version +Set the minimum `osx` version that the `py_binary` using `whl` distributions from a PyPI index should support. Values: * `""`: Select the lowest available version of each wheel giving you the maximum compatibility. This is the default. -* `X.Y`: The string representation of a `glibc` version. The allowed values depend on the `requirements.txt` lock file contents. +* `X.Y`: The string representation of the MacOS version. The allowed values depend on the `requirements.txt` lock file contents. + :::{versionadded} 0.33.0 ::: :::: -::::{bzl:flag} pip_whl_muslc_version -Set the minimum `muslc` version that the `py_binary` using `whl` distributions from a PyPI index should support. -Values: -* `""`: Select the lowest available version of each wheel giving you the maximum compatibility. This is the default. -* `X.Y`: The string representation of a `muslc` version. The allowed values depend on the `requirements.txt` lock file contents. -:::{versionadded} 0.33.0 -::: :::: -::::{bzl:flag} pip_whl_osx_version -Set the minimum `osx` version that the `py_binary` using `whl` distributions from a PyPI index should support. +:::{flag} venvs_site_packages -Values: -* `""`: Select the lowest available version of each wheel giving you the maximum compatibility. This is the default. -* `X.Y`: The string representation of the MacOS version. The allowed values depend on the `requirements.txt` lock file contents. +Determines if libraries use a site-packages layout for their files. -:::{versionadded} 0.33.0 +Note this flag only affects PyPI dependencies of `--bootstrap_impl=script` binaries + +:::{include} /_includes/experimental_api.md ::: + + +Values: +* `no` (default): Make libraries importable by adding to `sys.path` +* `yes`: Make libraries importable by creating paths in a binary's site-packages directory. :::: ::::{bzl:flag} bootstrap_impl Determine how programs implement their startup process. Values: -* `system_python`: Use a bootstrap that requires a system Python available +* `system_python`: (default) Use a bootstrap that requires a system Python available in order to start programs. This requires {obj}`PyRuntimeInfo.bootstrap_template` to be a Python program. * `script`: Use a bootstrap that uses an arbitrary executable script (usually a @@ -240,3 +369,73 @@ instead. ::: :::: + +::::{bzl:flag} current_config +Fail the build if the current build configuration does not match the +{obj}`pip.parse` defined wheels. + +Values: +* `fail`: Will fail in the build action ensuring that we get the error + message no matter the action cache. +* ``: (empty string) The default value, that will just print a warning. + +:::{seealso} +{obj}`pip.parse` +::: + +:::{versionadded} 1.1.0 +::: + +:::: + +::::{bzl:flag} venv +Determines which PyPI repository hub is used when resolving package dependencies. + +This flag is transitioned on automatically by executable targets (`py_binary`, `py_test`) +to select the appropriate concrete PyPI hub (e.g., when fallback or disjoint packages exist across multiple hubs). + +Values: +* `auto`: (default) Resolves dependencies using the fallback or first available hub. +* ``: Explicitly forces resolution of packages from the + specified concrete PyPI hub (corresponding to a + {obj}`pip.parse.hub_name` value). + +:::{versionadded} 2.2.0 +::: +:::: + +::::{bzl:flag} venvs_use_declare_symlink + +Determines if relative symlinks are created using `declare_symlink()` at build +time. + +This is only intended to work around +[#2489](https://github.com/bazel-contrib/rules_python/issues/2489), where some +packaging rules don't support `declare_symlink()` artifacts. + +Values: +* `yes`: Use `declare_symlink()` and create relative symlinks at build time. +* `no`: Do not use `declare_symlink()`. Instead, the venv will be created at + runtime. + +:::{seealso} +{envvar}`RULES_PYTHON_EXTRACT_ROOT` for customizing where the runtime venv +is created. +::: + +:::{versionadded} 1.2.0 +::: +:::: + + +## Removed Flags + +:::{versionremoved} 2.1.0 +The following flags were removed: + +* `pip_whl` +* `pip_whl_osx_arch` +* `pip_whl_glibc_version` +* `pip_whl_muslc_version` +::: + diff --git a/docs/conf.py b/docs/conf.py index 4c8e4a2a6b..750e344285 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -1,13 +1,16 @@ # Configuration file for the Sphinx documentation builder. import os +import re + +from docutils import nodes # -- Project information project = "rules_python" copyright = "2023, The Bazel Authors" author = "Bazel" -# NOTE: These are overriden by -D flags via --//sphinxdocs:extra_defines +# NOTE: These are overriden by -D flags via --@sphinxdocs//sphinxdocs:extra_defines version = "0.0.0" release = version @@ -87,53 +90,57 @@ "api/sphinxdocs/sphinx": "/api/sphinxdocs/sphinxdocs/sphinx.html", "api/sphinxdocs/sphinx_stardoc": "/api/sphinxdocs/sphinxdocs/sphinx_stardoc.html", "api/sphinxdocs/readthedocs": "/api/sphinxdocs/sphinxdocs/readthedocs.html", - "api/sphinxdocs/index": "/api/sphinxdocs/sphinxdocs/index.html", + "api/sphinxdocs/index": "sphinxdocs/index.html", "api/sphinxdocs/private/sphinx_docs_library": "/api/sphinxdocs/sphinxdocs/private/sphinx_docs_library.html", "api/sphinxdocs/sphinx_docs_library": "/api/sphinxdocs/sphinxdocs/sphinx_docs_library.html", "api/sphinxdocs/inventories/index": "/api/sphinxdocs/sphinxdocs/inventories/index.html", + "pip.html": "pypi/index.html", + "pypi-dependencies.html": "pypi/index.html", } -# Adapted from the template code: -# https://github.com/readthedocs/readthedocs.org/blob/main/readthedocs/doc_builder/templates/doc_builder/conf.py.tmpl -if os.environ.get("READTHEDOCS") == "True": - # Must come first because it can interfere with other extensions, according - # to the original conf.py template comments - extensions.insert(0, "readthedocs_ext.readthedocs") - - if os.environ.get("READTHEDOCS_VERSION_TYPE") == "external": - # Insert after the main extension - extensions.insert(1, "readthedocs_ext.external_version_warning") - readthedocs_vcs_url = ( - "http://github.com/bazelbuild/rules_python/pull/{}".format( - os.environ.get("READTHEDOCS_VERSION", "") - ) - ) - # The build id isn't directly available, but it appears to be encoded - # into the host name, so we can parse it from that. The format appears - # to be `build-X-project-Y-Z`, where: - # * X is an integer build id - # * Y is an integer project id - # * Z is the project name - _build_id = os.environ.get("HOSTNAME", "build-0-project-0-rules-python") - _build_id = _build_id.split("-")[1] - readthedocs_build_url = ( - f"https://readthedocs.org/projects/rules-python/builds/{_build_id}" - ) - exclude_patterns = ["_includes/*"] templates_path = ["_templates"] primary_domain = None # The default is 'py', which we don't make much use of nitpicky = True +# Ignore nitpicks for missing cross-references to external objects. +# These are typically objects that aren't documented or aren't easily linked +# via intersphinx mapping, so we suppress warnings for them to keep the build clean. +nitpick_ignore_regex = [ + ("py:class", r"docutils\..*"), + ("py:obj", r"sphinx\.util\.docutils\..*"), + ("py:obj", r"sphinx\.util\.docfields\..*"), + ("py:class", r"sphinx\.util\.typing\..*"), + ("py:class", r"sphinx_bzl\.bzl\..*"), + ("py:class", r"typing_extensions\.TypeAlias"), + ("bzl:obj", r":current_py_cc_headers_abi3"), + ("bzl:obj", r":python"), + ("bzl:type", r"T"), + ("bzl:type", r"input_value"), + ("bzl:type", r"DepsetBuilder"), + ("bzl:type", r"RunfilesBuilder"), + ("bzl:type", r"BuiltinPyInfo"), + ("bzl:type", r".*SentinelInfo"), + ("bzl:type", r".*SphinxDocsLibraryInfo"), + ("bzl:type", r".*_SphinxRunInfo"), +] + # --- Intersphinx configuration intersphinx_mapping = { + "python": ("https://docs.python.org/3", None), + "sphinx": ("https://www.sphinx-doc.org/en/master", None), "bazel": ("https://bazel.build/", "bazel_inventory.inv"), } # --- Extlinks configuration extlinks = { - "gh-path": (f"https://github.com/bazelbuild/rules_python/tree/main/%s", "%s"), + "gh-issue": ( + "https://github.com/bazel-contrib/rules_python/issues/%s", + "#%s issue", + ), + "gh-path": ("https://github.com/bazel-contrib/rules_python/tree/main/%s", "%s"), + "gh-pr": ("https://github.com/bazel-contrib/rules_python/pull/%s", "#%s PR"), } # --- MyST configuration @@ -215,9 +222,19 @@ ] +def _pep_role(name, rawtext, text, lineno, inliner, options={}, content=[]): + match = re.search(r"\d+", text) + pep_num = match.group(0) if match else text + pep_url = f"https://peps.python.org/pep-{pep_num.zfill(4)}/" + display_text = text if text.startswith("PEP") else f"PEP {pep_num}" + node = nodes.reference(rawtext, display_text, refuri=pep_url, **options) + return [node], [] + + def setup(app): # Pygments says it supports starlark, but it doesn't seem to actually # recognize `starlark` as a name. So just manually map it to python. from sphinx.highlighting import lexer_classes app.add_lexer("starlark", lexer_classes["python"]) + app.add_role("pep", _pep_role) diff --git a/docs/coverage.md b/docs/coverage.md index 3e0e67368c..8c162d86fd 100644 --- a/docs/coverage.md +++ b/docs/coverage.md @@ -9,7 +9,7 @@ when configuring toolchains. ## Enabling `rules_python` coverage support Enabling the coverage support bundled with `rules_python` just requires setting an -argument when registerting toolchains. +argument when registering toolchains. For Bzlmod: @@ -32,11 +32,25 @@ python_register_toolchains( This will implicitly add the version of `coverage` bundled with `rules_python` to the dependencies of `py_test` rules when `bazel coverage` is run. If a target already transitively depends on a different version of -`coverage`, then behavior is undefined -- it is undefined which version comes +`coverage`, then the behavior is undefined -- it is undefined which version comes first in the import path. If you find yourself in this situation, then you'll need to manually configure coverage (see below). ::: +:::{note} +The bundled `coverage` wheel set covers CPython 3.9 through 3.14 (with +freethreaded variants for 3.13+), and not every platform within that range. +When the interpreter that `bazel coverage` actually selects has no bundled +wheel, `configure_coverage_tool = True` produces no coverage tool and +`bazel coverage` emits empty lcov data; manually configure coverage (see +below) instead. + +`py_runtime` warns when this happens. The warning is emitted during analysis, +for the runtime toolchain resolution selected and only when coverage is being +collected, so it does not fire for the many platforms toolchains are +registered for but a given build never uses. +::: + ## Manually configuring coverage To manually configure coverage support, you'll need to set the diff --git a/docs/devguide.md b/docs/devguide.md new file mode 100644 index 0000000000..86a9d5b542 --- /dev/null +++ b/docs/devguide.md @@ -0,0 +1,151 @@ +# Dev Guide + +This document covers tips and guidance for working on the `rules_python` code +base. Its primary audience is first-time contributors. + +## Running tests + +Running tests is particularly easy thanks to Bazel, simply run: + +``` +bazel test //... +``` + +And it will run all the tests it can find. The first time you do this, it will +probably take a long time because various dependencies will need to be downloaded +and set up. Subsequent runs will be faster, but there are many tests, and some of +them are slow. If you're working on a particular area of code, you can run just +the tests in those directories instead, which can speed up your edit-run cycle. + +## Writing Tests + +Most code should have tests of some sort. This helps us have confidence that +refactors didn't break anything and that releases won't have regressions. + +We don't require 100% test coverage; testing certain Bazel functionality is +difficult, and some edge cases are simply too hard to test or not worth the +extra complexity. We try to judiciously decide when not having tests is a good +idea. + +Tests go under `tests/`. They are loosely organized into directories for the +particular subsystem or functionality they are testing. If an existing directory +doesn't seem like a good match for the functionality being tested, then it's +fine to create a new directory. + +Re-usable test helpers and support code go in `tests/support`. Tests don't need +to be perfectly factored and not every common thing a test does needs to be +factored into a more generally reusable piece. Copying and pasting is fine. It's +more important for tests to balance understandability and maintainability. + +### Test utilities + +General code to support testing is in {gh-path}`tests/support`. It has a variety +of functions, constants, rules etc, to make testing easier. Below are some +common utilities that are frequently used. + +### sh_py_run_test + +The {gh-path}`sh_py_run_test + # for example: + # bazel run //tools/private/update_deps:update_coverage_deps 7.10.7 + ``` + +## Updating tool dependencies + +It's suggested to routinely update the tool versions within our repo. Some of the +tools are using requirement files compiled by `uv`, and others use other means. In order +to have everything self-documented, we have a special target, +`//private:requirements.update`, which uses `rules_multirun` to run all +of the requirement-updating scripts in sequence in one go. This can be done once per release as +we prepare for releases. + +(creating-backport-prs)= +## Creating Backport PRs + +The steps to create a backport PR are: + +1. Create an issue for the patch release; use the [patch release + template][patch-release-issue]. +2. Create a fork of `rules_python`. +3. Checkout the `release/X.Y` branch. +4. Use `git cherry-pick -x` to cherry pick the desired fixes. +5. Update the release's `CHANGELOG.md` file: + * Add a Major.Minor.Patch section if one doesn't exist + * Copy the changelog text from `main` to the release's changelog. +6. Send a PR with the backport's changes. + * The title should be `backport: PR#N to Major.Minor` + * The body must preserve the original PR's number, commit hash, description, + and authorship. + Use the following format (`git cherry-pick` will use this format): + ``` + + + + (cherry picked from commit ) + ----- + Co-authored-by: + ``` + * If the PR contains multiple backport commits, separate each's description + with `-----`. +7. Send a PR to update the `main` branch's `CHANGELOG.md` to reflect the + changes done in the patched release. + +[patch-release-issue]: https://github.com/bazelbuild/rules_python/issues/new?template=patch_release.md diff --git a/docs/environment-variables.md b/docs/environment-variables.md index 12c1bcf0c2..3eb62221c0 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -1,29 +1,85 @@ # Environment Variables -:::{envvar} RULES_PYTHON_REPO_DEBUG +::::{envvar} RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS -When `1`, repository rules will print debug information about what they're -doing. This is mostly useful for development to debug errors. +This variable allows for additional arguments to be provided to the Python interpreter +at bootstrap time when the `bash` bootstrap is used. If +`RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` were provided as `-Xaaa`, then the command +would be: + +``` +python -Xaaa /path/to/file.py +``` + +This feature is useful for the integration of debuggers. For example, +it would be possible to configure `RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` to +be set to `/path/to/debugger.py --port 12344 --file`, resulting +in the command executed being: + +``` +python /path/to/debugger.py --port 12345 --file /path/to/file.py +``` + +:::{seealso} +The {bzl:obj}`interpreter_args` attribute. + +The guide on {any}`How to integrate a debugger` ::: -:::{envvar} RULES_PYTHON_REPO_DEBUG_VERBOSITY +:::{versionadded} 1.3.0 +::: +:::{versionchanged} 1.7.0 +Support added for {bzl:flag}`--bootstrap_impl=system_python`. +::: -Determines the verbosity of logging output for repo rules. Valid values: +:::: -* `DEBUG` -* `INFO` -* `TRACE` +:::{envvar} RULES_PYTHON_BOOTSTRAP_VERBOSE + +When `1`, debug information about bootstrapping of a program is printed to +stderr. ::: -:::{envvar} RULES_PYTHON_REPO_TOOLCHAIN_VERSION_OS_ARCH +:::{envvar} RULES_PYTHON_BZLMOD_DEBUG -Determines the python interpreter platform to be used for a particular -interpreter `(version, os, arch)` triple to be used in repository rules. -Replace the `VERSION_OS_ARCH` part with actual values when using, e.g. -`3_13_0_linux_x86_64`. The version values must have `_` instead of `.` and the -os, arch values are the same as the ones mentioned in the -`//python:versions.bzl` file. +When `1`, bzlmod extensions will print debug information about what they're +doing. This is mostly useful for development to debug errors. +::: + +:::{envvar} RULES_PYTHON_DEPRECATION_WARNINGS + +When `1`, `rules_python` will warn users about deprecated functionality that will +be removed in a subsequent major `rules_python` version. Defaults to `0` if unset. +::: + +::::{envvar} RULES_PYTHON_EXTRACT_ROOT + +Directory to use as the root for creating files necessary for bootstrapping so +that a binary can run. + +Only applicable when {bzl:flag}`--venvs_use_declare_symlink=no` is used. + +When set, a binary will attempt to find a unique, reusable, location within this +directory for the files it needs to create to aid startup. The files may not be +deleted upon program exit; it is the responsibility of the caller to ensure +cleanup. + +Manually specifying the directory is useful to lower the overhead of +extracting/creating files on every program execution. By using a location +outside /tmp, longer lived programs don't have to worry about files in /tmp +being cleaned up by the OS. + +If not set, then a temporary directory will be created and deleted upon program +exit. + +:::{versionadded} 1.2.0 ::: +:::: + +:::{envvar} RULES_PYTHON_GAZELLE_VERBOSE + +When `1`, debug information from Gazelle is printed to stderr. +:::: :::{envvar} RULES_PYTHON_PIP_ISOLATED @@ -34,22 +90,71 @@ Valid values: * Other non-empty values mean to use isolated mode. ::: -:::{envvar} RULES_PYTHON_BZLMOD_DEBUG +:::{envvar} RULES_PYTHON_PYCACHE_DIR -When `1`, bzlmod extensions will print debug information about what they're +Determines the directory that runtime-generated pyc cache files will +be stored in. + +This directory may be reused between invocations, depending on the sandboxing +configuration. Setting it to `/dev/null` will, in effect, disable runtime +pyc caching. By setting e.g. +`--sandbox_add_mount_pair=/tmp/rules_python_pycache`, it's possible for pyc +caching to persist across invocations. + +**Behavior specific to downloaded runtimes:** +First `RULES_PYTHON_PYCACHE_DIR` is checked. If set, it is used as-is for +the root pycache directory. + +Otherwise, the following environment variables are checked in the following +order. Their values will have `rules_python_pycache` appended to them to form +the root pycache directory: +1. `XDG_CACHE_HOME`. +2. `TMP` (non-Windows) or `TEMP` (Windows). +3. The common platform-specific temporary directory (`/tmp` (non-Windows) or + `C:\Temp` (Windows)). + +If such a diretory cannot be found, or created, then `/dev/null` will be used, +which will effectively disable pyc caching. + +::: + +:::{envvar} RULES_PYTHON_PYPI_HUB_RESERVED + +When `1`, any PyPI hub named `"pypi"` will be renamed to `_pypi` +to prevent name collisions with the unified `@pypi` proxy repository, and +a warning is printed indicating that the renaming occurred. If not set (defaulting +to `0`), a warning is printed advising to rename the hub, and the collision +is not resolved. + +:::{versionadded} 2.2.0 +::: + +::: + +:::{envvar} RULES_PYTHON_REPO_DEBUG + +When `1`, repository rules will print debug information about what they're doing. This is mostly useful for development to debug errors. ::: -:::{envvar} RULES_PYTHON_ENABLE_PYSTAR +:::{envvar} RULES_PYTHON_REPO_DEBUG_VERBOSITY -When `1`, the rules_python Starlark implementation of the core rules is used -instead of the Bazel-builtin rules. Note this requires Bazel 7+. +Determines the verbosity of logging output for repo rules. Valid values: + +* `DEBUG` +* `FAIL` +* `INFO` +* `TRACE` ::: -:::{envvar} RULES_PYTHON_BOOTSTRAP_VERBOSE +:::{envvar} RULES_PYTHON_REPO_TOOLCHAIN_VERSION_OS_ARCH -When `1`, debug information about bootstrapping of a program is printed to -stderr. +Determines the Python interpreter platform to be used for a particular +interpreter `(version, os, arch)` triple to be used in repository rules. +Replace the `VERSION_OS_ARCH` part with actual values when using, e.g., +`3_13_0_linux_x86_64`. The version values must have `_` instead of `.` and the +os, arch values are the same as the ones mentioned in the +`//python:versions.bzl` file. ::: :::{envvar} VERBOSE_COVERAGE @@ -57,8 +162,13 @@ stderr. When `1`, debug information about coverage behavior is printed to stderr. ::: +## Removed Environment Variables -:::{envvar} RULES_PYTHON_GAZELLE_VERBOSE +:::{versionremoved} 2.1.0 +The following environment variables were removed: -When `1`, debug information from gazelle is printed to stderr. +* `RULES_PYTHON_ENABLE_PYSTAR`: Used to enable the Starlark implementation of + core rules. +* `RULES_PYTHON_ENABLE_PIPSTAR`: Used to enable the Starlark implementation of + PyPI integration. ::: diff --git a/docs/extending.md b/docs/extending.md new file mode 100644 index 0000000000..1c91b01795 --- /dev/null +++ b/docs/extending.md @@ -0,0 +1,143 @@ +# Extending the rules + +:::{important} +**This is public, but volatile, functionality.** + +Extending and customizing the rules is supported functionality, but with weaker +backwards compatibility guarantees, and is not fully subject to the normal +backwards compatibility procedures and policies. It's simply not feasible to +support every possible customization with strong backwards compatibility +guarantees. +::: + +Because of the rich ecosystem of tools and variety of use cases, APIs are +provided to make it easy to create custom rules using the existing rules as a +basis. This allows implementing behaviors that aren't possible using +wrapper macros around the core rules, and can make certain types of changes +much easier and transparent to implement. + +:::{note} +It is not required to extend a core rule. The minimum requirement for a custom +rule is to return the appropriate provider (e.g. {bzl:obj}`PyInfo` etc). +Extending the core rules is most useful when you want all or most of the +behavior of a core rule. +::: + +Follow or comment on {gh-issue}`1647` +for the development of APIs to support custom derived rules. + +## Creating custom rules + +Custom rules can be created using the core rules as a basis by using their rule +builder APIs. + +* [`//python/apis:executables.bzl`](#python-apis-executables-bzl): builders for + executables. +* [`//python/apis:libraries.bzl`](#python-apis-libraries-bzl): builders for + libraries. + +These builders create {bzl:obj}`ruleb.Rule` objects, which are thin +wrappers around the keyword arguments eventually passed to the `rule()` +function. These builder APIs give access to the _entire_ rule definition and +allow arbitrary modifications. + +This level of control is powerful but also volatile. A rule definition +contains many details that _must_ change as the implementation changes. What +is more or less likely to change isn't known in advance, but some general +rules of thumb are: + +* Additive behavior to public attributes will be less prone to breaking. +* Internal attributes that directly support a public attribute are likely + reliable. +* Internal attributes that support an action are more likely to change. +* Rule toolchains are moderately stable (toolchains are mostly internal to + how a rule works, but custom toolchains are supported). + +## Example: validating a source file + +In this example, we derive a custom rule from `py_library` that verifies source +code contains the word "snakes". It does this by: + +* Adding an implicit dependency on a checker program +* Calling the base implementation function +* Running the checker on the srcs files +* Adding the result to the `_validation` output group (a special output + group for validation behaviors). + +To users, they can use `has_snakes_library` the same as `py_library`. The same +is true for other targets that might consume the rule. + +``` +load("@rules_python//python/api:libraries.bzl", "libraries") +load("@rules_python//python/api:attr_builders.bzl", "attrb") + +def _has_snakes_impl(ctx, base): + providers = base(ctx) + + out = ctx.actions.declare_file(ctx.label.name + "_snakes.check") + ctx.actions.run( + inputs = ctx.files.srcs, + outputs = [out], + executable = ctx.attr._checker[DefaultInfo].files_to_run, + args = [out.path] + [f.path for f in ctx.files.srcs], + ) + prior_ogi = None + for i, p in enumerate(providers): + if type(p) == "OutputGroupInfo": + prior_ogi = (i, p) + break + if prior_ogi: + groups = {k: getattr(prior_ogi[1], k) for k in dir(prior_ogi)} + if "_validation" in groups: + groups["_validation"] = depset([out], transitive=groups["_validation"]) + else: + groups["_validation"] = depset([out]) + providers[prior_ogi[0]] = OutputGroupInfo(**groups) + else: + providers.append(OutputGroupInfo(_validation=depset([out]))) + return providers + +def create_has_snakes_rule(): + r = libraries.py_library_builder() + base_impl = r.implementation() + r.set_implementation(lambda ctx: _has_snakes_impl(ctx, base_impl)) + r.attrs["_checker"] = attrb.Label( + default="//:checker", + executable = True, + ) + return r.build() +has_snakes_library = create_has_snakes_rule() +``` + +## Example: adding transitions + +In this example, we derive a custom rule from `py_binary` to force building for a particular +platform. We do this by: + +* Adding an additional output to the rule's cfg +* Calling the base transition function +* Returning the new transition outputs + +```starlark + +load("@rules_python//python/api:executables.bzl", "executables") + +def _force_linux_impl(settings, attr, base_impl): + settings = base_impl(settings, attr) + settings["//command_line_option:platforms"] = ["//my/platforms:linux"] + return settings + +def create_rule(): + r = executables.py_binary_rule_builder() + base_impl = r.cfg.implementation() + r.cfg.set_implementation( + lambda settings, attr: _force_linux_impl(settings, attr, base_impl) + ) + r.cfg.add_output("//command_line_option:platforms") + return r.build() + +py_linux_binary = create_rule() +``` + +Users can then use `py_linux_binary` the same as a regular `py_binary`. It will +act as if `--platforms=//my/platforms:linux` was specified when building it. diff --git a/docs/gazelle.md b/docs/gazelle.md deleted file mode 100644 index 89f26d67bb..0000000000 --- a/docs/gazelle.md +++ /dev/null @@ -1,9 +0,0 @@ -# Gazelle plugin - -[Gazelle](https://github.com/bazelbuild/bazel-gazelle) -is a build file generator for Bazel projects. It can create new `BUILD.bazel` files for a project that follows language conventions and update existing build files to include new sources, dependencies, and options. - -Bazel may run Gazelle using the Gazelle rule, or it may be installed and run as a command line tool. - -See the documentation for Gazelle with rules_python in the {gh-path}`gazelle` -directory. diff --git a/docs/getting-started.md b/docs/getting-started.md index 9f52243fd1..6494b37c51 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -1,24 +1,26 @@ # Getting started -This doc is a simplified guide to help get started quickly. It provides +This document is a simplified guide to help you get started quickly. It provides a simplified introduction to having a working Python program for both `bzlmod` and the older way of using `WORKSPACE`. It assumes you have a `requirements.txt` file with your PyPI dependencies. -For more details information about configuring `rules_python`, see: -* [Configuring the runtime](toolchains) -* [Configuring third party dependencies (pip/pypi)](pypi-dependencies) +For more detailed information about configuring `rules_python`, see: +* [Configuring the runtime](configuring-toolchains) +* [Configuring third-party dependencies (pip/PyPI)](./pypi/index) * [API docs](api/index) -## Using bzlmod +A [screen recording](https://www.youtube.com/watch?v=Xtuh-WipOnk) on configuring rules_python for beginners is also available. -The first step to using rules_python with bzlmod is to add the dependency to -your MODULE.bazel file: +## Including dependencies + +The first step to using `rules_python` is to add the dependency to +your `MODULE.bazel` file: ```starlark # Update the version "0.0.0" to the release found here: -# https://github.com/bazelbuild/rules_python/releases. +# https://github.com/bazel-contrib/rules_python/releases. bazel_dep(name = "rules_python", version = "0.0.0") pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") @@ -30,22 +32,22 @@ pip.parse( use_repo(pip, "pypi") ``` -## Using a WORKSPACE file +### Using a WORKSPACE file -Using WORKSPACE is deprecated, but still supported, and a bit more involved than +Using `WORKSPACE` is deprecated but still supported, and it's a bit more involved than using Bzlmod. Here is a simplified setup to download the prebuilt runtimes. ```starlark load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") # Update the snippet based on the latest release below -# https://github.com/bazelbuild/rules_python/releases +# https://github.com/bazel-contrib/rules_python/releases http_archive( name = "rules_python", sha256 = "ca77768989a7f311186a29747e3e95c936a41dffac779aff6b443db22290d913", strip_prefix = "rules_python-0.36.0", - url = "https://github.com/bazelbuild/rules_python/releases/download/0.36.0/rules_python-0.36.0.tar.gz", + url = "https://github.com/bazel-contrib/rules_python/releases/download/0.36.0/rules_python-0.36.0.tar.gz", ) load("@rules_python//python:repositories.bzl", "py_repositories") @@ -72,11 +74,11 @@ pip_parse( ## "Hello World" -Once you've imported the rule set using either Bzlmod or WORKSPACE, you can then +Once you've imported the rule set using either Bzlmod or `WORKSPACE`, you can then load the core rules in your `BUILD` files with the following: ```starlark -load("@rules_python//python:defs.bzl", "py_binary") +load("@rules_python//python:py_binary.bzl", "py_binary") py_binary( name = "main", diff --git a/docs/glossary.md b/docs/glossary.md index 9afbcffb92..c9bd03fd0e 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -5,7 +5,7 @@ common attributes : Every rule has a set of common attributes. See Bazel's [Common attributes](https://bazel.build/reference/be/common-definitions#common-attributes) - for a complete listing + for a complete listing. in-build runtime : An in-build runtime is one where the Python runtime, and all its files, are @@ -21,9 +21,9 @@ which can be a significant number of files. platform runtime : A platform runtime is a Python runtime that is assumed to be installed on the -system where a Python binary runs, whereever that may be. For example, using `/usr/bin/python3` +system where a Python binary runs, wherever that may be. For example, using `/usr/bin/python3` as the interpreter is a platform runtime -- it assumes that, wherever the binary -runs (your local machine, a remote worker, within a container, etc), that path +runs (your local machine, a remote worker, within a container, etc.), that path is available. Such runtimes are _not_ part of a binary's runfiles. The main advantage of platform runtimes is they are lightweight insofar as @@ -42,8 +42,8 @@ rule callable accepted; refer to the respective API accepting this type. simple label -: A `str` or `Label` object but not a _direct_ `select` object. These usually - mean a string manipulation is occuring, which can't be done on `select` + A `str` or `Label` object but not a _direct_ `select` object. This usually + means a string manipulation is occurring, which can't be done on `select` objects. Such attributes are usually still configurable if an alias is used, and a reference to the alias is passed instead. diff --git a/docs/howto/build-a-wheel.md b/docs/howto/build-a-wheel.md new file mode 100644 index 0000000000..0248ff802b --- /dev/null +++ b/docs/howto/build-a-wheel.md @@ -0,0 +1,87 @@ +:::{default-domain} bzl +::: + +# How to build a wheel + +This guide explains how to use the `py_wheel` rule to build a wheel +file from a `py_library`. + +## Basic usage + +The `py_wheel` rule takes any file-providing target as input and put its files +into a wheel. Because `py_library` provides its source files, simple cases can +pass `py_library` directly to `py_wheel`: + +```starlark +# BUILD.bazel + +py_library( + name = "my_project_lib", + srcs = glob(["my_project/**/*.py"]), + # ... +) + +py_wheel( + name = "my_project_wheel", + distribution = "my-project", + version = "0.1.0", + deps = [":my_project_lib"], +) +``` + +The above will include the *default outputs* of the `py_library`, which are the +direct `.py` files listed in the py library. It does **not** include transitive +dependencies. + +## Including and filtering transitive dependencies + + +Use the `py_package` rule to include and filter the transitive parts of +a `py_library` target. + +The `py_package` rule has a `packages` attribute that takes a list of dotted +Python package names to include. All files and dependencies of those packages +are included. + +Here is an example: + +```starlark +# BUILD.bazel + +py_library( + name = "my_project_lib", + srcs = glob(["my_project/**/*.py"]), + deps = ["@pypi//some_dep"], +) + +py_package( + name = "my_project_package", + # This will only include files for the "my_package" package; other files + # will be excluded. + packages = ["my_project"], +) + +py_wheel( + name = "my_project_wheel", + distribution = "my-project", + version = "0.1.0", + # The `py_wheel` rule takes the `py_package` target in the `deps` + # attribute. + deps = [":my_project_package"], +) +``` + +## Disabling `__init__.py` generation + +By default, Bazel automatically creates `__init__.py` files in directories to +make them importable. This can sometimes be undesirable when building wheels +because it interfers with namespace packages or makes directories importable +that shouldn't be importable. + +It's highly recommended to disable this behavior by setting a flag in your +`.bazelrc` file: + +``` +# .bazelrc +build --incompatible_default_to_explicit_init_py=true +``` \ No newline at end of file diff --git a/docs/howto/common-deps-with-multiple-pypi-versions.md b/docs/howto/common-deps-with-multiple-pypi-versions.md new file mode 100644 index 0000000000..f6887e3667 --- /dev/null +++ b/docs/howto/common-deps-with-multiple-pypi-versions.md @@ -0,0 +1,144 @@ +(common-deps-with-multiple-pypi-versions)= +# How to use a common set of dependencies with multiple PyPI versions + +In this guide, we show how to handle a situation common to monorepos that +extensively share code: How do multiple binaries utilize distinct requirements +files while pulling from shared internal libraries, without requiring +manually-maintained `select()` logic inside dependency targets? + +Stated as code, consider this example: + +```bzl +# When building bin_alpha, requests and more_itertools should resolve +# from requirements_alpha.txt +py_binary( + name = "bin_alpha", + deps = [ + "@pypi//requests", + ":common", + ], +) + +# When building bin_beta, requests and more_itertools should resolve +# from requirements_beta.txt +py_binary( + name = "bin_beta", + deps = [ + "@pypi//requests", + ":common", + ], +) + +# Transitive dependencies like more_itertools are requested here, but +# must automatically match whichever dependency track is active for the binary. +py_library( + name = "common", + deps = ["@pypi//more_itertools"], +) +``` + +## Defining dependency tracks via custom platforms + +The solution involves defining custom "platforms" mapped to separate +dependency tracks inside `MODULE.bazel`. Using custom platforms via +{obj}`pip.default` and associating requirements files to them through the +`requirements_by_platform` attribute on {obj}`pip.parse` instructs +`rules_python` to generate `select()` logic behind a unified hub. + +Binaries configure their execution requirements by forcing flag transition +attributes using custom build setting flags. + +In this example, we define custom string flag named `//:pypi_hub`, setup +distinct custom platforms for `"alpha"` and `"beta"` profiles, and register +associated requirements lock files grouped inside the `@pypi` hub. + +```starlark +# File: MODULE.bazel + +rules_python_config = use_extension( + "@rules_python//python/extensions:config.bzl", + "config", +) +rules_python_config.add_transition_setting( + setting = "//:pypi_hub", +) + +pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") + +pip.default( + platform = "alpha", + config_settings = ["@//:is_pypi_alpha"], +) + +pip.default( + platform = "beta", + config_settings = ["@//:is_pypi_beta"], +) + +pip.parse( + hub_name = "pypi", + python_version = "3.14", + requirements_by_platform = { + "//:requirements_alpha.txt": "alpha", + "//:requirements_beta.txt": "beta", + }, +) + +use_repo(pip, "pypi") +``` + +```starlark +# File: BUILD.bazel + +load("@bazel_skylib//rules:common_settings.bzl", "string_flag") + +string_flag( + name = "pypi_hub", + build_setting_default = "none", +) + +config_setting( + name = "is_pypi_alpha", + flag_values = {"//:pypi_hub": "alpha"}, +) + +config_setting( + name = "is_pypi_beta", + flag_values = {"//:pypi_hub": "beta"}, +) + +py_binary( + name = "bin_alpha", + srcs = ["bin_alpha.py"], + config_settings = { + "//:pypi_hub": "alpha", + }, + deps = [ + "@pypi//requests", + ":common", + ], +) + +py_binary( + name = "bin_beta", + srcs = ["bin_beta.py"], + config_settings = { + "//:pypi_hub": "beta", + }, + deps = [ + "@pypi//requests", + ":common", + ], +) + +py_library( + name = "common", + deps = ["@pypi//more_itertools"], +) +``` + +When building `bin_alpha` or `bin_beta`, they set `//:pypi_hub` via target +transitions. The generated aliased dependencies inside the `@pypi` hub will +evaluate that Bazel configuration, automatically delivering corresponding +Python wheels from targeted lock files. + diff --git a/docs/howto/debuggers.md b/docs/howto/debuggers.md new file mode 100644 index 0000000000..199a366675 --- /dev/null +++ b/docs/howto/debuggers.md @@ -0,0 +1,206 @@ +:::{default-domain} bzl +::: + +# How to integrate a debugger + +This guide explains how to integrate a debugger +with your Python applications built with `rules_python`. + +There are two ways available: the {obj}`--debugger` flag, and the {any}`RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` environment variable. + +## {obj}`--debugger` flag + +### Basic Usage + +The {obj}`--debugger` flag allows you to inject an extra dependency into `py_test` +and `py_binary` targets so that they have a custom debugger available at +runtime. The flag is roughly equivalent to manually adding it to `deps` of +the target under test. + +To use the debugger, you typically provide the `--debugger` flag to your `bazel run` command. + +Example command line: + +```bash +bazel run --@rules_python//python/config_settings:debugger=@pypi//pudb \ + //path/to:my_python_binary +``` + +This will launch the Python program with the `@pypi//pudb` dependency added. + +The exact behavior (e.g., waiting for attachment, breaking at the first line) +depends on the specific debugger and its configuration. + +:::{note} +The specified target must be in the requirements.txt file used with +`pip.parse()` to make it available to Bazel. +::: + +### Python `PYTHONBREAKPOINT` Environment Variable + +For more fine-grained control over debugging, especially for programmatic breakpoints, +you can leverage the Python built-in `breakpoint()` function and the +`PYTHONBREAKPOINT` environment variable. + +The `breakpoint()` built-in function, available since Python 3.7, +can be called anywhere in your code to invoke a debugger. The `PYTHONBREAKPOINT` +environment variable can be set to specify which debugger to use. + +For example, to use `pdb` (the Python Debugger) when `breakpoint()` is called: + +```bash +PYTHONBREAKPOINT=pudb.set_trace bazel run \ + --@rules_python//python/config_settings:debugger=@pypi//pudb \ + //path/to:my_python_binary +``` + +For more details on `PYTHONBREAKPOINT`, refer to the [Python documentation](https://docs.python.org/3/library/functions.html#breakpoint). + +### Setting a default debugger + +By adding settings to your user or project `.bazelrc` files, you can have +these settings automatically added to your bazel invocations. e.g. + +``` +common --@rules_python//python/config_settings:debugger=@pypi//pudb +common --test_env=PYTHONBREAKPOINT=pudb.set_trace +``` + +Note that `--test_env` isn't strictly necessary. The `py_test` and `py_binary` +rules will respect the `PYTHONBREAKPOINT` environment variable in your shell. + +## debugpy (e.g. vscode) + +You can integrate `debugpy` (i.e. the debugger used in vscode or PyCharm) by using a launcher script. This method leverages {any}`RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` to inject the debugger into the Bazel-managed Python process. + +For the remainder of this document, we assume you are using vscode. + +![VS Code debugpy demo](https://raw.githubusercontent.com/shayanhoshyari/issue-reports/refs/heads/main/rules_python/vscode_debugger/docs/demo.gif) + + +1. **Create a launcher script**: Save the following Python script as `.vscode/debugpy/launch.py` (or another location, adjusting `launch.json` accordingly). This script bridges VS Code's debugger with Bazel. + +
+ launch.py + + ```python + """ + Launcher script for VS Code (debugpy). + + This script is not managed by Bazel; it is invoked by VS Code's launch.json to + wrap the Bazel command, injecting the debugger into the runtime environment. + """ + + import argparse + import os + import shlex + import subprocess + import sys + from typing import cast + + def main() -> None: + parser = argparse.ArgumentParser(description="Launch bazel debugpy with test or run.") + parser.add_argument("mode", choices=["test", "run"], help="Choose whether to run a bazel test or run.") + parser.add_argument("args", help="The bazel target to test or run (e.g., //foo:bar) and any additional args") + args = parser.parse_args() + + # Import debugpy, provided by VS Code + try: + # debugpy._vendored is needed for force_pydevd to perform path manipulation. + import debugpy._vendored # type: ignore[import-not-found] + + # pydev_monkey patches os and subprocess functions to handle new launched processes. + from _pydev_bundle import pydev_monkey # type: ignore[import-not-found] + except ImportError as exc: + print(f"Error: This script must be run via VS Code's debug adapter. Details: {exc}") + sys.exit(-1) + + # Prepare arguments for the monkey-patched process. + # is_exec=False ensures we don't replace the current process immediately. + patched_args = cast(list[str], pydev_monkey.patch_args(["python", "dummy.py"], is_exec=False)) + pydev_monkey.send_process_created_message() + + # Extract the injected arguments (skipping the dummy python executable and script). + # These args invoke the pydevd entrypoint which connects back to the debugger. + rules_python_interpreter_args = " ".join(patched_args[1:-1]) + + bzl_args = shlex.split(args.args) + if not bzl_args: + print("Error: At least one argument (the target) is required.") + sys.exit(-1) + + cmd = [ + "bazel", + args.mode, + # Propagate environment variables to the test/run environment. + "--test_env=PYDEVD_RESOLVE_SYMLINKS", + "--test_env=RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS", + "--test_env=IDE_PROJECT_ROOTS", + bzl_args[0], + ] + + if bzl_args[1:]: + if args.mode == "run": + # Append extra arguments for 'run' mode. + cmd.append("--") + cmd.extend(bzl_args[1:]) + elif args.mode == "test": + # Append extra arguments for 'test' mode. + cmd.extend([f"--test_arg={arg}" for arg in bzl_args[1:]]) + + env = { + **os.environ.copy(), + # Inject the debugger arguments into the rules_python toolchain. + "RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS": rules_python_interpreter_args, + # Ensure breakpoints hit the original source files, not Bazel's symlinks. + "PYDEVD_RESOLVE_SYMLINKS": "1", + } + + # Execute Bazel. + result = subprocess.run(cmd, env=env, check=False) + sys.exit(result.returncode) + + if __name__ == "__main__": + main() + ``` +
+ +2. **Configure `launch.json`**: Add the following configurations to your `.vscode/launch.json`. This tells VS Code to use the launcher script. + +
+ launch.json + + ```json + { + "version": "0.2.0", + "configurations": [ + { + "name": "Python: Bazel py run", + "type": "debugpy", + "request": "launch", + "program": "${workspaceFolder}/.vscode/debugpy/launch.py", + "args": ["run", "${input:BazelArgs}"], + "console": "integratedTerminal" + }, + { + "name": "Python: Bazel py test", + "type": "debugpy", + "request": "launch", + "program": "${workspaceFolder}/.vscode/debugpy/launch.py", + "args": ["test", "${input:BazelArgs}"], + "console": "integratedTerminal" + } + ], + "inputs": [ + { + "id": "BazelArgs", + "type": "promptString", + "description": "Bazel target and arguments (e.g., //foo:bar --my-arg)" + } + ] + } + ``` +
+ + Note: If you find `justMyCode` behavior is incompatible with Bazel's symlinks (causing breakpoints to be missed), you can set `"justMyCode": false` in `launch.json` and use the `IDE_PROJECT_ROOTS` environment variable (set to `"${workspaceFolder}"`) to explicitly map your workspace. + diff --git a/docs/howto/get-python-version.md b/docs/howto/get-python-version.md new file mode 100644 index 0000000000..8ff9cffd71 --- /dev/null +++ b/docs/howto/get-python-version.md @@ -0,0 +1,61 @@ +:::{default-domain} bzl +::: + +# How to get the current Python version + +This guide explains how to use a [toolchain](toolchains) to get the current Python +version and, as an example, write it to a file. + +You can create a simple rule that accesses the Python toolchain and retrieves +the version string. + +## The rule implementation + +Create a file named `my_rule.bzl`: + +```starlark +# my_rule.bzl +def _my_rule_impl(ctx): + toolchain = ctx.toolchains["@rules_python//python:toolchain_type"] + info = toolchain.py3_runtime.interpreter_version_info + python_version = str(info.major) + "." + str(info.minor) + "." + str(info.micro) + + output_file = ctx.actions.declare_file(ctx.attr.name + ".txt") + ctx.actions.write( + output = output_file, + content = python_version, + ) + + return [DefaultInfo(files = depset([output_file]))] + +my_rule = rule( + implementation = _my_rule_impl, + attrs = {}, + toolchains = ["@rules_python//python:toolchain_type"], +) +``` + +The `info` variable above is a {obj}`PyRuntimeInfo` object, which contains +information about the Python runtime. It contains more than just the version; +see the {obj}`PyRuntimeInfo` docs for its API documentation. + +## Using the rule + +In your `BUILD.bazel` file, you can use the rule like this: + +```starlark +# BUILD.bazel +load(":my_rule.bzl", "my_rule") + +my_rule( + name = "show_python_version", +) +``` + +When you build this target, it will generate a file named +`show_python_version.txt` containing the Python version (e.g., `3.9`). + +```starlark +bazel build :show_python_version +cat bazel-bin/show_python_version.txt +``` diff --git a/docs/howto/index.md b/docs/howto/index.md new file mode 100644 index 0000000000..e6ddf70325 --- /dev/null +++ b/docs/howto/index.md @@ -0,0 +1,13 @@ +:::{default-domain} bzl +::: + +# How-to Guides + +This section contains a collection of how-to guides for accomplishing specific tasks with `rules_python`. + +```{toctree} +:maxdepth: 1 +:glob: + +* +``` \ No newline at end of file diff --git a/docs/howto/linking-libpython.md b/docs/howto/linking-libpython.md new file mode 100644 index 0000000000..4d70b322c9 --- /dev/null +++ b/docs/howto/linking-libpython.md @@ -0,0 +1,66 @@ +:::{default-domain} bzl +::: + +# How to link to libpython + +This guide explains how to use the Python [toolchain](toolchains) to get the linker +flags required for linking against `libpython`. This is often necessary when +embedding Python in a C/C++ application. + +Currently, the `:current_py_cc_libs` target does *not* include `-lpython` et al +linker flags. This is intentional because it forces dynamic linking (via the +dynamic linker processing `DT_NEEDED` entries), which prevents users who want +to load it in some more custom way. + +## Exposing linker flags in a rule + +You can create a rule that gets the Python version from the toolchain and +constructs the correct linker flag. This rule can then provide the flag to +other C/C++ rules via the `CcInfo` provider. + +Here's an example of a rule that creates the `-lpython` flag: + +```starlark +# python_libs.bzl +load("@bazel_tools//tools/cpp:toolchain_utils.bzl", "cc_common") + +def _python_libs_impl(ctx): + toolchain = ctx.toolchains["@rules_python//python:toolchain_type"] + info = toolchain.py3_runtime.interpreter_version_info + link_flag = "-lpython{}.{}".format(info.major, info.minor) + + cc_info = CcInfo( + linking_context = cc_common.create_linking_context( + user_link_flags = [link_flag], + ), + ) + return [cc_info] + +python_libs = rule( + implementation = _python_libs_impl, + toolchains = ["@rules_python//python:toolchain_type"], +) +``` + +## Using the rule + +In your `BUILD.bazel` file, define a target using this rule and add it to the +`deps` of your `cc_binary` or `cc_library`. + +```starlark +# BUILD.bazel +load(":python_libs.bzl", "python_libs") + +python_libs( + name = "py_libs", +) + +cc_binary( + name = "my_app", + srcs = ["my_app.c"], + deps = [ + ":py_libs", + # Other dependencies + ], +) +``` diff --git a/docs/howto/multi-platform-pypi-deps.md b/docs/howto/multi-platform-pypi-deps.md new file mode 100644 index 0000000000..f329ccdf9e --- /dev/null +++ b/docs/howto/multi-platform-pypi-deps.md @@ -0,0 +1,194 @@ +:::{default-domain} bzl +::: + +# How-to: Multi-Platform PyPI Dependencies + +When developing applications that need to run on a wide variety of platforms, +managing PyPI dependencies can become complex. You might need different sets of +dependencies for different combinations of Python version, threading model, +operating system, CPU architecture, libc, and even hardware accelerators like +GPUs. + +This guide demonstrates how to manage this complexity using `rules_python` with +bzlmod. If you prefer to learn by example, complete example code is provided at +the end. + +In this how to guide, we configure for using 4 requirements files, each +for a different variation using Python 3.14 on Linux: + +* Regular (non-freethreaded) Python +* Freethreaded Python +* Regular Python for CUDA 12.9 +* Freethreaded Python for ARM and Musl + +## Mapping requirements files to Bazel configuration settings + +Unfortunately, a requirements file doesn't tell what it's compatible with, +so we have to manually specify the Bazel configuration settings for it. To do +that using rules_python, there are two steps: defining a platform, then +associating a requirements file with the platform. + +### Defining a platform + +First, we define a "platform" using {obj}`pip.default`. This associates an +arbitrary name with a list of Bazel {obj}`config_setting` targets. While any +name can be used for a platform (its name has no inherent semantic meaning), it +should encode all the relevant dimensions that distinguish a requirements file. +For example, if a requirements file is specifically for the combination of CUDA +12.9 and NumPy 2.0, then the platform name should represent that. + +The convention is to follow the format of `{os}_{cpu}{threading}`, where: + +* `{os}` is the operating system (`linux`, `osx`, `windows`). +* `{cpu}` is the architecture (`x86_64`, `aarch64`). +* `{threading}` is `_freethreaded` for a freethreaded Python runtime, or an + empty string for the regular runtime. + +Additional dimensions should be appended and separated with an underscore (e.g. +`linux_x86_64_musl_cuda12.9_numpy2`). + +The platform name should not include the Python version. That is handled by +{attr}`pip.parse.python_version` separately. + +:::{note} +The term _platform_ here has nothing to do with Bazel's `platform()` rule. +::: + +#### Defining custom settings + +Because {attr}`pip.default.config_settings` is a list of arbitrary `config_setting` +targets, you can define your own flags or implement custom config matching +logic. This allows you to model settings that aren't inherently part of +rules_python. + +This is typically done using [bazel_skylib flags](https://bazel.build/extending/config), but any [Starlark +defined build setting](https://bazel.build/extending/config) can be used. Just +remember to use `config_setting()` to match a particular value of the flag. + +In our example below, we define a custom flag for CUDA version. + +#### Predefined and common build settings + +rules_python has some predefined build settings you can use. Commonly used ones +are: + +* {obj}`@rules_python//python/config_settings:py_linux_libc` +* {obj}`@rules_python//python/config_settings:py_freethreaded` + +Additionally, [Bazel @platforms](https://github.com/bazelbuild/platforms) +contains commonly used settings for OS and CPU: + +* `@platforms//os:windows` +* `@platforms//os:linux` +* `@platforms//os:osx` +* `@platforms//cpu:x86_64` +* `@platforms//cpu:aarch64` + +Note that these are the raw flag names. In order to use them with `pip.default`, +you must use {obj}`config_setting` to match a particular value for them. + +### Associating Requirements to Platforms + +Next, we associate a requirements file with a platform using +{obj}`pip.parse.requirements_by_platform`. This is a dictionary attribute where +the keys are requirements files and the value is a platform name. The platform +value can use a trailing or leading `*` to match multiple platforms. It can also +specify multiple platform names using commas to separate them. + +Note that the Python version is _not_ part of the platform name. + +Under the hood, `pip.parse` merges all the requirements (for a `hub_name`) and +constructs `select()` expressions to route to the appropriate dependencies. + +### Using it in practice + +Finally, to make use of what we've configured, perform a build and set +command line flags to the appropriate values. + +```shell +# Build for CUDA +bazel build --//:cuda_version=12.9 //:binary + +# Build for ARM with musl +bazel build --@rules_python//python/config_settings:py_linux_libc=musl \ + --cpu=aarch64 //:binary + +# Build for freethreaded +bazel build --@rules_python//python/config_settings:py_freethreaded=yes //:binary +``` + +Note that certain combinations of flags may result in an error or undefined +behavior. For example, trying to set both freethreaded and CUDA at the same +time would result in an error because no requirements file was registered +to match that combination. + +## Multiple Python Versions + +Having multiple Python versions is fully supported. Simply add a `pip.parse()` +call and set `python_version` appropriately. + +## Multiple hubs + +Having multiple `pip.parse` calls with different `hub_name` values is fully +supported. Each hub only contains the requirements registered to it. + +## Complete Example + +Here is a complete example that puts all the pieces together. + +```starlark +# File: BUILD.bazel +load("@bazel_skylib//rules:common_settings.bzl", "string_flag") + +# A custom flag for controlling the CUDA version +string_flag( + name = "cuda_version", + build_setting_default = "none", +) + +config_setting( + name = "is_cuda_12_9", + flag_values = {":cuda_version": "12.9"}, +) + +# A config_setting that uses the built-in libc flag from rules_python +config_setting( + name = "is_musl", + flag_values = {"@rules_python//python/config_settings:py_linux_libc": "muslc"}, +) + +# File: MODULE.bazel +pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") + +# A custom platform for CUDA on glibc linux +pip.default( + platform = "linux_x86_64_cuda12.9", + arch_name = "x86_64", + os_name = "linux", + config_settings = ["@//:is_cuda_12_9"], +) + +# A custom platform for musl on linux +pip.default( + platform = "linux_aarch64_musl", + os_name = "linux", + arch_name = "aarch64", + config_settings = ["@//:is_musl"], +) + +pip.parse( + hub_name = "my_deps", + python_version = "3.14", + requirements_by_platform = { + # Map to default platform names + "//:py3.14-regular-linux-x86-glibc-cpu.txt": "linux_x86_64", + "//:py3.14-freethreaded-linux-x86-glibc-cpu.txt": "linux_x86_64_freethreaded", + + # Map to our custom platform names + "//:py3.14-regular-linux-x86-glibc-cuda12.9.txt": "linux_x86_64_cuda12.9", + "//:py3.14-freethreaded-linux-arm-musl-cpu.txt": "linux_aarch64_musl", + }, +) + +use_repo(pip, "my_deps") +``` diff --git a/docs/howto/optional-feature-loading.md b/docs/howto/optional-feature-loading.md new file mode 100644 index 0000000000..144c0f65cb --- /dev/null +++ b/docs/howto/optional-feature-loading.md @@ -0,0 +1,180 @@ +:::{default-domain} bzl +::: + +# How to optionally load symbols using `features.loadable_symbols` + +When writing Bazel rules, macros, or repository extensions that support +multiple versions of `rules_python`, you may want to detect whether a public +symbol (such as {obj}`py_extension` in `//python/cc:py_extension.bzl`) is +available before attempting to load or use it. + +Because Starlark `load()` statements are evaluated at parse time and must be at +the top level of a `.bzl` file, unconditionally loading a symbol that does not +exist in older versions of `rules_python` will cause a build error. + +The {bzl:obj}`features.loadable_symbols` dictionary in `//python:features.bzl` +allows you to programmatically inspect which symbols are exported by `.bzl` +files in the current `rules_python` version. + +## The `features.loadable_symbols` structure + +{bzl:obj}`features.loadable_symbols` is a `dict[str, list[str]]` mapping label +strings of `.bzl` files to the list of public symbols they export: + +```starlark +load("@rules_python//python:features.bzl", "features") + +# Example structure of features.loadable_symbols: +# { +# "//python/cc:py_extension.bzl": [ +# "py_extension", +# ], +# "//python:py_info.bzl": [ +# "PyInfo", +# ], +# } +``` + +## Using load() with optional symbols + +In repository rules or Bazel module extensions (`repository_ctx` or +`module_ctx`), you generate `.bzl` files dynamically. You can inspect +`features.loadable_symbols` to determine which `load()` statements to write into +a generated compatibility repository. + +Re-export the symbol under its standard name if available, or set it to `None` +if it is absent. By generating compatibility files and empty `BUILD.bazel` +files at the exact same relative package paths as `rules_python`, the only +difference in downstream `load()` statements is the repository name: + +```starlark +load("@rules_python//python:features.bzl", "features") + +def _rules_python_compat_impl(rctx): + for bzl, symbol_list in rctx.attr.symbols.items(): + loadable = features.loadable_symbols.get(bzl, []) + lines = [] + for symbol in symbol_list: + if symbol in loadable: + lines.append( + 'load("{}", _{} = "{}")'.format(bzl, symbol, symbol), + ) + lines.append("{} = _{}".format(symbol, symbol)) + else: + lines.append("{} = None".format(symbol)) + + package, _, filename = bzl.lstrip("/").partition(":") + path = package + "/" + filename if package else filename + build_path = package + "/BUILD.bazel" if package else "BUILD.bazel" + + rctx.file(path, content = "\n".join(lines) + "\n") + rctx.file(build_path, content = "") + +rules_python_compat = repository_rule( + implementation = _rules_python_compat_impl, + attrs = { + "symbols": attr.string_list_dict( + mandatory = True, + doc = "Map of bzl paths to lists of symbols to optionally load", + ), + }, +) +``` + +Instantiate the repository rule by providing a mapping of `.bzl` paths to their +symbols of interest: + +```starlark +rules_python_compat( + name = "rules_python_compat", + symbols = { + "//python/cc:py_extension.bzl": ["py_extension"], + }, +) +``` + +### Using the generated compatibility files + +Your macros and rules can load from `@rules_python_compat` using the same +file path as `@rules_python`, testing whether the symbol is `None` before +using it: + +```starlark +load("@rules_python_compat//python/cc:py_extension.bzl", "py_extension") + +def my_macro(name, **kwargs): + if py_extension != None: + py_extension( + name = name + "_ext", + **kwargs + ) + else: + # Fall back to default behavior for older rules_python versions + pass +``` + +## Handling optional targets + +In addition to symbol loading, you may need to check whether a specific Bazel +target exists in `rules_python` before referencing its label in dependencies, +toolchains, or attribute defaults. + +The {bzl:obj}`features.targets` dictionary in `//python:features.bzl` is a +`dict[str, bool]` mapping public API target labels to `True` when available. + +In a macro: + +```starlark +load("@rules_python//python:features.bzl", "features") + +def my_cc_extension_macro(name, deps = [], **kwargs): + if features.targets.get("//python/cc:current_py_cc_headers_abi3"): + deps = deps + ["@rules_python//python/cc:current_py_cc_headers_abi3"] + + # ... define target with deps +``` + +In a `BUILD` file: + +```starlark +load("@rules_python//python:features.bzl", "features") +load("@rules_python//python:py_library.bzl", "py_library") + +py_library( + name = "my_lib", + srcs = ["my_lib.py"], + deps = [ + "//my/app:base_lib", + ] + ( + ["@rules_python//python/cc:current_py_cc_headers_abi3"] + if features.targets.get("//python/cc:current_py_cc_headers_abi3") + else [] + ), +) +``` + +## Checking versions with `features.version` + +When a behavioral change or capability is not directly reflected by a public +target or loadable symbol, you can inspect {bzl:obj}`features.version` in +`//python:features.bzl`. + +{bzl:obj}`features.version` returns a semver-formatted version string (such as +`"1.0.0"`, `"2.0.0-rc2"`, or `""` for unreleased development builds): + +```starlark +load("@rules_python//python:features.bzl", "features") + +def _to_tuple(v): + return tuple([ + int(x) if x.isdigit() else x + for x in v.replace("-", ".").split(".") + ]) + +def has_foo(): + # If version is empty, it is an unreleased build from main which includes + # all features. + if not features.version: + return True + return _to_tuple(features.version) >= _to_tuple("0.38.0") +``` diff --git a/docs/howto/pypi-headers.md b/docs/howto/pypi-headers.md new file mode 100644 index 0000000000..3675031096 --- /dev/null +++ b/docs/howto/pypi-headers.md @@ -0,0 +1,109 @@ +:::{default-domain} bzl +::: + +# How to expose headers from a PyPI package + +When you depend on a PyPI package that includes C headers (like `numpy`), you +need to make those headers available to your `cc_library` or +`cc_binary` targets. + +The recommended way to do this is to inject a `BUILD.bazel` file into the +external repository for the package. This `BUILD` file will create +a `cc_library` target that exposes the header files. + +First, create a `.bzl` file that has the extra logic we'll inject. Putting it +in a separate bzl file avoids having to redownload and extract the whl file +when our logic changes. + +```bzl + +# pypi_extra_targets.bzl +load("@rules_cc//cc:cc_library.bzl", "cc_library") + +def extra_numpy_targets(): + cc_library( + name = "headers", + hdrs = glob(["**/*.h"]), + visibility = ["//visibility:public"], + ) +``` + +## Bzlmod setup + +In your `MODULE.bazel` file, use the `build_file_content` attribute of +`pip.parse` to inject the `BUILD` file content for the `numpy` package. + +```bazel +# MODULE.bazel +load("@rules_python//python/extensions:pip.bzl", "parse", "whl_mods") +pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") +whl_mods = use_extension("@rules_python//python/extensions:pip.bzl", "whl_mods") + + +# Define a specific modification for a wheel +whl_mods( + hub_name = "pypi_mods", + whl_name = "numpy-1.0.0-py3-none-any.whl", # The exact wheel filename + additive_build_content = """ +load("@//:pypi_extra_targets.bzl", "numpy_hdrs") + +extra_numpy_targets() +""", +) +pip.parse( + hub_name = "pypi", + wheel_name = "numpy", + requirements_lock = "//:requirements.txt", + whl_modifications = { + "@pypi_mods//:numpy.json": "numpy", + }, + extra_hub_aliases = { + "numpy": ["headers"], + } +) +``` + +## WORKSPACE setup + +In your `WORKSPACE` file, use the `annotations` attribute of `pip_parse` to +inject additional `BUILD` file content, then use `extra_hub_targets` to expose +that target in the `@pypi` hub repo. + +The {obj}`package_annotation` helper can be used to construct the value for the +`annotations` attribute. + +```starlark +# WORKSPACE +load("@rules_python//python:pip.bzl", "package_annotation", "pip_parse") + +pip_parse( + name = "pypi", + requirements_lock = "//:requirements.txt", + annotations = { + "numpy": package_annotation( + additive_build_content = """\ +load("@//:pypi_extra_targets.bzl", "numpy_hdrs") + +extra_numpy_targets() +""" + ), + }, + extra_hub_targets = { + "numpy": ["headers"], + }, +) +``` + +## Using the headers + +In your `BUILD.bazel` file, you can now depend on the generated `headers` +target. + +```bazel +# BUILD.bazel +cc_library( + name = "my_c_extension", + srcs = ["my_c_extension.c"], + deps = ["@pypi//numpy:headers"], +) +``` diff --git a/docs/howto/python-headers.md b/docs/howto/python-headers.md new file mode 100644 index 0000000000..fa8c2cece5 --- /dev/null +++ b/docs/howto/python-headers.md @@ -0,0 +1,54 @@ +:::{default-domain} bzl +::: + +# How to get Python headers for C extensions + +When building a Python C extension, you need access to the Python header +files. This guide shows how to get the necessary include paths from the Python +[toolchain](toolchains). + +The recommended way to get the headers is to depend on the +{obj}`@rules_python//python/cc:current_py_cc_headers` or +{obj}`@rules_python//python/cc:current_py_cc_headers_abi3` +targets. These are convenience targets that use toolchain resolution to find +the correct headers for the target platform. + +## Using the headers + +In your `BUILD.bazel` file, you can add `@rules_python//python/cc:current_py_cc_headers` +to the `deps` of a `cc_library` or `cc_binary` target. + +```bazel +# BUILD.bazel +cc_library( + name = "my_c_extension", + srcs = ["my_c_extension.c"], + deps = ["@rules_python//python/cc:current_py_cc_headers"], +) +``` + +This setup ensures that your C extension code can find and use the Python +headers during compilation. + +:::{note} +The `:current_py_cc_headers` target provides all the Python headers. This _may_ +include ABI-specific information. +::: + +## Using the stable ABI headers + +If you're building for the [Python stable ABI](https://docs.python.org/3/c-api/stable.html), +then depend on {obj}`@rules_python//python/cc:current_py_cc_headers_abi3`. This +target contains only objects relevant to the Python stable ABI. Remember to +define +[`Py_LIMITED_API`](https://docs.python.org/3/c-api/stable.html#c.Py_LIMITED_API) +when building such extensions. + +```bazel +# BUILD.bazel +cc_library( + name = "my_stable_abi_extension", + srcs = ["my_stable_abi_extension.c"], + deps = ["@rules_python//python/cc:current_py_cc_headers_abi3"], +) +``` diff --git a/docs/index.md b/docs/index.md index 378aac2faa..0204f38ec7 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,19 +1,19 @@ # Python Rules for Bazel -`rules_python` is the home for 4 major components with varying maturity levels. +`rules_python` is the home for four major components with varying maturity levels. :::{topic} Core rules The core Python rules -- `py_library`, `py_binary`, `py_test`, -`py_proto_library`, and related symbols that provide the basis for Python +and related symbols that provide the basis for Python support in Bazel. When using Bazel 6 (or earlier), the core rules are bundled into the Bazel binary, and the symbols -in this repository are simple aliases. On Bazel 7 and above `rules_python` uses -a separate Starlark implementation, +in this repository are simple aliases. On Bazel 7 and above, `rules_python` uses +a separate Starlark implementation; see {ref}`Migrating from the Bundled Rules` below. -Once rules_python 1.0 is released, they will follow +This repository follows [semantic versioning](https://semver.org) and the breaking change policy outlined in the [support](support) page. @@ -21,12 +21,12 @@ outlined in the [support](support) page. :::{topic} PyPI integration -Package installation rules for integrating with PyPI and other SimpleAPI +Package installation rules for integrating with PyPI and other Simple API- compatible indexes. These rules work and can be used in production, but the cross-platform building that supports pulling PyPI dependencies for a target platform that is different -from the host platform is still in beta and the APIs that are subject to potential +from the host platform is still in beta, and the APIs that are subject to potential change are marked as `experimental`. ::: @@ -36,9 +36,9 @@ change are marked as `experimental`. `sphinxdocs` rules allow users to generate documentation using Sphinx powered by Bazel, with additional functionality for documenting Starlark and Bazel code. -The functionality is exposed because other projects find it useful, but -it is available as is and **the semantic versioning and -compatibility policy used by `rules_python` does not apply**. +The functionality is exposed because other projects find it useful, but +it is available "as is", and **the semantic versioning and +compatibility policy used by `rules_python` does not apply**. ::: @@ -47,7 +47,7 @@ compatibility policy used by `rules_python` does not apply**. `gazelle` plugin for generating `BUILD.bazel` files based on Python source code. -This is available as is and the semantic versioning used by `rules_python` does +This is available "as is", and the semantic versioning used by `rules_python` does not apply. ::: @@ -67,7 +67,7 @@ components have examples in the {gh-path}`examples` directory. The core rules are currently available in Bazel as built-in symbols, but this form is deprecated. Instead, you should depend on rules_python in your `WORKSPACE` or `MODULE.bazel` file and load the Python rules from -`@rules_python//python:defs.bzl` or load paths described in the API documentation. +`@rules_python//python:.bzl` or load paths described in the API documentation. A [buildifier](https://github.com/bazelbuild/buildtools/blob/master/buildifier/README.md) fix is available to automatically migrate `BUILD` and `.bzl` files to add the @@ -78,7 +78,7 @@ appropriate `load()` statements and rewrite uses of `native.py_*`. buildifier --lint=fix --warnings=native-py ``` -Currently, the `WORKSPACE` file needs to be updated manually as per +Currently, the `WORKSPACE` file needs to be updated manually as per [Getting started](getting-started). Note that Starlark-defined bundled symbols underneath @@ -87,7 +87,7 @@ by buildifier. ## Migrating to bzlmod -See {gh-path}`Bzlmod support ` for any behaviour differences between +See {gh-path}`Bzlmod support ` for any behavioral differences between `bzlmod` and `WORKSPACE`. @@ -95,13 +95,16 @@ See {gh-path}`Bzlmod support ` for any behaviour differences :hidden: self getting-started -pypi-dependencies +pypi/index Toolchains -pip coverage precompiling -gazelle +gazelle/docs/index +REPL +Extending +How-to Guides Contributing +devguide support Changelog api/index diff --git a/docs/merge_changelog.py b/docs/merge_changelog.py new file mode 100644 index 0000000000..47af98b2a2 --- /dev/null +++ b/docs/merge_changelog.py @@ -0,0 +1,38 @@ +"""A tool to merge news entries into CHANGELOG.md for documentation preview.""" + +import argparse +import pathlib + +from tools.private.release import changelog_news + + +def main(): + parser = argparse.ArgumentParser(description="Merge news entries into CHANGELOG.md") + parser.add_argument( + "--changelog", type=pathlib.Path, required=True, help="Path to CHANGELOG.md" + ) + parser.add_argument( + "--news-dir", type=pathlib.Path, required=True, help="Path to news directory" + ) + parser.add_argument( + "--output", + type=pathlib.Path, + required=True, + help="Path to output merged changelog", + ) + args = parser.parse_args() + + # Call the public merge_new_into_changelog function + # Using deterministic date "0000-00-00" and version "0.0.0" + changelog_news.merge_new_into_changelog( + changelog_path=args.changelog, + output_path=args.output, + news_dir=args.news_dir, + version="unreleased", + release_date="0000-00-00", + delete_news=False, + ) + + +if __name__ == "__main__": + main() diff --git a/docs/pip.md b/docs/pip.md deleted file mode 100644 index 43d8fc4978..0000000000 --- a/docs/pip.md +++ /dev/null @@ -1,4 +0,0 @@ -(pip-integration)= -# Pip Integration - -See [PyPI dependencies](./pypi-dependencies). diff --git a/docs/precompiling.md b/docs/precompiling.md index a46608f77e..ea978cddce 100644 --- a/docs/precompiling.md +++ b/docs/precompiling.md @@ -1,6 +1,6 @@ # Precompiling -Precompiling is compiling Python source files (`.py` files) into byte code +Precompiling is compiling Python source files (`.py` files) into bytecode (`.pyc` files) at build time instead of runtime. Doing it at build time can improve performance by skipping that work at runtime. @@ -15,12 +15,12 @@ While precompiling helps runtime performance, it has two main costs: a `.pyc` file. Compiled files are generally around the same size as the source files, so it approximately doubles the disk usage. 2. Precompiling requires running an extra action at build time. While - compiling itself isn't that expensive, the overhead can become noticable + compiling itself isn't that expensive, the overhead can become noticeable as more files need to be compiled. ## Binary-level opt-in -Binary-level opt-in allows enabling precompiling on a per-target basic. This is +Binary-level opt-in allows enabling precompiling on a per-target basis. This is useful for situations such as: * Globally enabling precompiling in your `.bazelrc` isn't feasible. This may @@ -41,7 +41,7 @@ can use an opt-in or opt-out approach by setting its value: ## Pyc-only builds -A pyc-only build (aka "source less" builds) is when only `.pyc` files are +A pyc-only build (aka "sourceless" builds) is when only `.pyc` files are included; the source `.py` files are not included. To enable this, set @@ -55,8 +55,8 @@ The advantage of pyc-only builds are: The disadvantages are: * Error messages will be less precise because the precise line and offset - information isn't in an pyc file. -* pyc files are Python major-version specific. + information isn't in a pyc file. +* pyc files are Python major-version-specific. :::{note} pyc files are not a form of hiding source code. They are trivial to uncompile, @@ -75,11 +75,11 @@ mechanisms are available: the {bzl:attr}`precompiler` attribute. Arbitrary binaries are supported. * The execution requirements can be customized using `--@rules_python//tools/precompiler:execution_requirements`. This is a list - flag that can be repeated. Each entry is a key=value that is added to the + flag that can be repeated. Each entry is a `key=value` pair that is added to the execution requirements of the `PyCompile` action. Note that this flag - is specific to the rules_python precompiler. If a custom binary is used, + is specific to the `rules_python` precompiler. If a custom binary is used, this flag will have to be propagated from the custom binary using the - `testing.ExecutionInfo` provider; refer to the `py_interpreter_program` an + `testing.ExecutionInfo` provider; refer to the `py_interpreter_program` example. The default precompiler implementation is an asynchronous/concurrent implementation. If you find it has bugs or hangs, please report them. In the @@ -90,18 +90,18 @@ as well, but is less likely to have issues. The `execution_requirements` keys of most relevance are: * `supports-workers`: 1 or 0, to indicate if a regular persistent worker is desired. -* `supports-multiplex-workers`: 1 o 0, to indicate if a multiplexed persistent +* `supports-multiplex-workers`: `1` or `0`, to indicate if a multiplexed persistent worker is desired. -* `requires-worker-protocol`: json or proto; the rules_python precompiler - currently only supports json. -* `supports-multiplex-sandboxing`: 1 or 0, to indicate if sanboxing is of the +* `requires-worker-protocol`: `json` or `proto`; the `rules_python` precompiler + currently only supports `json`. +* `supports-multiplex-sandboxing`: `1` or `0`, to indicate if sandboxing of the worker is supported. -* `supports-worker-cancellation`: 1 or 1, to indicate if requests to the worker +* `supports-worker-cancellation`: `1` or `0`, to indicate if requests to the worker can be cancelled. Note that any execution requirements values can be specified in the flag. -## Known issues, caveats, and idiosyncracies +## Known issues, caveats, and idiosyncrasies * Precompiling requires Bazel 7+ with the Pystar rule implementation enabled. * Mixing rules_python PyInfo with Bazel builtin PyInfo will result in pyc files @@ -111,14 +111,14 @@ Note that any execution requirements values can be specified in the flag. causes the module to be found in the workspace source directory instead of within the binary's runfiles directory (where the pyc files are). This can usually be worked around by removing `sys.path[0]` (or otherwise ensuring the - runfiles directory comes before the repos source directory in `sys.path`). -* The pyc filename does not include the optimization level (e.g. - `foo.cpython-39.opt-2.pyc`). This works fine (it's all byte code), but also + runfiles directory comes before the repo's source directory in `sys.path`). +* The pyc filename does not include the optimization level (e.g., + `foo.cpython-39.opt-2.pyc`). This works fine (it's all bytecode), but also means the interpreter `-O` argument can't be used -- doing so will cause the interpreter to look for the non-existent `opt-N` named files. -* Targets with the same source files and different exec properites will result +* Targets with the same source files and different exec properties will result in action conflicts. This most commonly occurs when a `py_binary` and - `py_library` have the same source files. To fix, modify both targets so + a `py_library` have the same source files. To fix this, modify both targets so they have the same exec properties. If this is difficult because unsupported exec groups end up being passed to the Python rules, please file an issue to have those exec groups added to the Python rules. diff --git a/docs/pypi-dependencies.md b/docs/pypi-dependencies.md deleted file mode 100644 index 28e630c61d..0000000000 --- a/docs/pypi-dependencies.md +++ /dev/null @@ -1,456 +0,0 @@ -:::{default-domain} bzl -::: - -# Using dependencies from PyPI - -Using PyPI packages (aka "pip install") involves two main steps. - -1. [Installing third party packages](#installing-third-party-packages) -2. [Using third party packages as dependencies](#using-third-party-packages) - -{#installing-third-party-packages} -## Installing third party packages - -### Using bzlmod - -To add pip dependencies to your `MODULE.bazel` file, use the `pip.parse` -extension, and call it to create the central external repo and individual wheel -external repos. Include in the `MODULE.bazel` the toolchain extension as shown -in the first bzlmod example above. - -```starlark -pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") -pip.parse( - hub_name = "my_deps", - python_version = "3.11", - requirements_lock = "//:requirements_lock_3_11.txt", -) -use_repo(pip, "my_deps") -``` -For more documentation, including how the rules can update/create a requirements -file, see the bzlmod examples under the {gh-path}`examples` folder or the documentation -for the {obj}`@rules_python//python/extensions:pip.bzl` extension. - -```{note} -We are using a host-platform compatible toolchain by default to setup pip dependencies. -During the setup phase, we create some symlinks, which may be inefficient on Windows -by default. In that case use the following `.bazelrc` options to improve performance if -you have admin privileges: - - startup --windows_enable_symlinks - -This will enable symlinks on Windows and help with bootstrap performance of setting up the -hermetic host python interpreter on this platform. Linux and OSX users should see no -difference. -``` - -### Using a WORKSPACE file - -To add pip dependencies to your `WORKSPACE`, load the `pip_parse` function and -call it to create the central external repo and individual wheel external repos. - -```starlark -load("@rules_python//python:pip.bzl", "pip_parse") - -# Create a central repo that knows about the dependencies needed from -# requirements_lock.txt. -pip_parse( - name = "my_deps", - requirements_lock = "//path/to:requirements_lock.txt", -) -# Load the starlark macro, which will define your dependencies. -load("@my_deps//:requirements.bzl", "install_deps") -# Call it to define repos for your requirements. -install_deps() -``` - -(vendoring-requirements)= -#### Vendoring the requirements.bzl file - -In some cases you may not want to generate the requirements.bzl file as a repository rule -while Bazel is fetching dependencies. For example, if you produce a reusable Bazel module -such as a ruleset, you may want to include the requirements.bzl file rather than make your users -install the WORKSPACE setup to generate it. -See https://github.com/bazelbuild/rules_python/issues/608 - -This is the same workflow as Gazelle, which creates `go_repository` rules with -[`update-repos`](https://github.com/bazelbuild/bazel-gazelle#update-repos) - -To do this, use the "write to source file" pattern documented in -https://blog.aspect.dev/bazel-can-write-to-the-source-folder -to put a copy of the generated requirements.bzl into your project. -Then load the requirements.bzl file directly rather than from the generated repository. -See the example in rules_python/examples/pip_parse_vendored. - -(per-os-arch-requirements)= -### Requirements for a specific OS/Architecture - -In some cases you may need to use different requirements files for different OS, Arch combinations. This is enabled via the `requirements_by_platform` attribute in `pip.parse` extension and the `pip_parse` repository rule. The keys of the dictionary are labels to the file and the values are a list of comma separated target (os, arch) tuples. - -For example: -```starlark - # ... - requirements_by_platform = { - "requirements_linux_x86_64.txt": "linux_x86_64", - "requirements_osx.txt": "osx_*", - "requirements_linux_exotic.txt": "linux_exotic", - "requirements_some_platforms.txt": "linux_aarch64,windows_*", - }, - # For the list of standard platforms that the rules_python has toolchains for, default to - # the following requirements file. - requirements_lock = "requirements_lock.txt", -``` - -In case of duplicate platforms, `rules_python` will raise an error as there has -to be unambiguous mapping of the requirement files to the (os, arch) tuples. - -An alternative way is to use per-OS requirement attributes. -```starlark - # ... - requirements_windows = "requirements_windows.txt", - requirements_darwin = "requirements_darwin.txt", - # For the remaining platforms (which is basically only linux OS), use this file. - requirements_lock = "requirements_lock.txt", -) -``` - -### pip rules - -Note that since `pip_parse` and `pip.parse` are executed at evaluation time, -Bazel has no information about the Python toolchain and cannot enforce that the -interpreter used to invoke `pip` matches the interpreter used to run -`py_binary` targets. By default, `pip_parse` uses the system command -`"python3"`. To override this, pass in the `python_interpreter` attribute or -`python_interpreter_target` attribute to `pip_parse`. The `pip.parse` `bzlmod` extension -by default uses the hermetic python toolchain for the host platform. - -You can have multiple `pip_parse`s in the same workspace, or use the pip -extension multiple times when using bzlmod. This configuration will create -multiple external repos that have no relation to one another and may result in -downloading the same wheels numerous times. - -As with any repository rule, if you would like to ensure that `pip_parse` is -re-executed to pick up a non-hermetic change to your environment (e.g., updating -your system `python` interpreter), you can force it to re-execute by running -`bazel sync --only [pip_parse name]`. - -{#using-third-party-packages} -## Using third party packages as dependencies - -Each extracted wheel repo contains a `py_library` target representing -the wheel's contents. There are two ways to access this library. The -first uses the `requirement()` function defined in the central -repo's `//:requirements.bzl` file. This function maps a pip package -name to a label: - -```starlark -load("@my_deps//:requirements.bzl", "requirement") - -py_library( - name = "mylib", - srcs = ["mylib.py"], - deps = [ - ":myotherlib", - requirement("some_pip_dep"), - requirement("another_pip_dep"), - ] -) -``` - -The reason `requirement()` exists is to insulate from -changes to the underlying repository and label strings. However, those -labels have become directly used, so aren't able to easily change regardless. - -On the other hand, using `requirement()` has several drawbacks; see -[this issue][requirements-drawbacks] for an enumeration. If you don't -want to use `requirement()`, you can use the library -labels directly instead. For `pip_parse`, the labels are of the following form: - -```starlark -@{name}//{package} -``` - -Here `name` is the `name` attribute that was passed to `pip_parse` and -`package` is the pip package name with characters that are illegal in -Bazel label names (e.g. `-`, `.`) replaced with `_`. If you need to -update `name` from "old" to "new", then you can run the following -buildozer command: - -```shell -buildozer 'substitute deps @old//([^/]+) @new//${1}' //...:* -``` - -[requirements-drawbacks]: https://github.com/bazelbuild/rules_python/issues/414 - -### Entry points - -If you would like to access [entry points][whl_ep], see the `py_console_script_binary` rule documentation, -which can help you create a `py_binary` target for a particular console script exposed by a package. - -[whl_ep]: https://packaging.python.org/specifications/entry-points/ - -### 'Extras' dependencies - -Any 'extras' specified in the requirements lock file will be automatically added -as transitive dependencies of the package. In the example above, you'd just put -`requirement("useful_dep")` or `@pypi//useful_dep`. - -### Consuming Wheel Dists Directly - -If you need to depend on the wheel dists themselves, for instance, to pass them -to some other packaging tool, you can get a handle to them with the -`whl_requirement` macro. For example: - -```starlark -load("@pypi//:requirements.bzl", "whl_requirement") - -filegroup( - name = "whl_files", - data = [ - # This is equivalent to "@pypi//boto3:whl" - whl_requirement("boto3"), - ] -) -``` - -### Creating a filegroup of files within a whl - -The rule {obj}`whl_filegroup` exists as an easy way to extract the necessary files -from a whl file without the need to modify the `BUILD.bazel` contents of the -whl repositories generated via `pip_repository`. Use it similarly to the `filegroup` -above. See the API docs for more information. - -(advance-topics)= -## Advanced topics - -(circular-deps)= -### Circular dependencies - -Sometimes PyPi packages contain dependency cycles -- for instance a particular -version `sphinx` (this is no longer the case in the latest version as of -2024-06-02) depends on `sphinxcontrib-serializinghtml`. When using them as -`requirement()`s, ala - -``` -py_binary( - name = "doctool", - ... - deps = [ - requirement("sphinx"), - ], -) -``` - -Bazel will protest because it doesn't support cycles in the build graph -- - -``` -ERROR: .../external/pypi_sphinxcontrib_serializinghtml/BUILD.bazel:44:6: in alias rule @pypi_sphinxcontrib_serializinghtml//:pkg: cycle in dependency graph: - //:doctool (...) - @pypi//sphinxcontrib_serializinghtml:pkg (...) -.-> @pypi_sphinxcontrib_serializinghtml//:pkg (...) -| @pypi_sphinxcontrib_serializinghtml//:_pkg (...) -| @pypi_sphinx//:pkg (...) -| @pypi_sphinx//:_pkg (...) -`-- @pypi_sphinxcontrib_serializinghtml//:pkg (...) -``` - -The `experimental_requirement_cycles` argument allows you to work around these -issues by specifying groups of packages which form cycles. `pip_parse` will -transparently fix the cycles for you and provide the cyclic dependencies -simultaneously. - -```starlark -pip_parse( - ... - experimental_requirement_cycles = { - "sphinx": [ - "sphinx", - "sphinxcontrib-serializinghtml", - ] - }, -) -``` - -`pip_parse` supports fixing multiple cycles simultaneously, however cycles must -be distinct. `apache-airflow` for instance has dependency cycles with a number -of its optional dependencies, which means those optional dependencies must all -be a part of the `airflow` cycle. For instance -- - -```starlark -pip_parse( - ... - experimental_requirement_cycles = { - "airflow": [ - "apache-airflow", - "apache-airflow-providers-common-sql", - "apache-airflow-providers-postgres", - "apache-airflow-providers-sqlite", - ] - } -) -``` - -Alternatively, one could resolve the cycle by removing one leg of it. - -For example while `apache-airflow-providers-sqlite` is "baked into" the Airflow -package, `apache-airflow-providers-postgres` is not and is an optional feature. -Rather than listing `apache-airflow[postgres]` in your `requirements.txt` which -would expose a cycle via the extra, one could either _manually_ depend on -`apache-airflow` and `apache-airflow-providers-postgres` separately as -requirements. Bazel rules which need only `apache-airflow` can take it as a -dependency, and rules which explicitly want to mix in -`apache-airflow-providers-postgres` now can. - -Alternatively, one could use `rules_python`'s patching features to remove one -leg of the dependency manually. For instance by making -`apache-airflow-providers-postgres` not explicitly depend on `apache-airflow` or -perhaps `apache-airflow-providers-common-sql`. - - -(bazel-downloader)= -### Multi-platform support - -Multi-platform support of cross-building the wheels can be done in two ways - either -using {bzl:attr}`experimental_index_url` for the {bzl:obj}`pip.parse` bzlmod tag class -or by using the {bzl:attr}`pip.parse.download_only` setting. In this section we -are going to outline quickly how one can use the latter option. - -Let's say you have 2 requirements files: -``` -# requirements.linux_x86_64.txt ---platform=manylinux_2_17_x86_64 ---python-version=39 ---implementation=cp ---abi=cp39 - -foo==0.0.1 --hash=sha256:deadbeef -bar==0.0.1 --hash=sha256:deadb00f -``` - -``` -# requirements.osx_aarch64.txt contents ---platform=macosx_10_9_arm64 ---python-version=39 ---implementation=cp ---abi=cp39 - -foo==0.0.3 --hash=sha256:deadbaaf -``` - -With these 2 files your {bzl:obj}`pip.parse` could look like: -``` -pip.parse( - hub_name = "pip", - python_version = "3.9", - # Tell `pip` to ignore sdists - download_only = True, - requirements_by_platform = { - "requirements.linux_x86_64.txt": "linux_x86_64", - "requirements.osx_aarch64.txt": "osx_aarch64", - }, -) -``` - -With this, the `pip.parse` will create a hub repository that is going to -support only two platforms - `cp39_osx_aarch64` and `cp39_linux_x86_64` and it -will only use `wheels` and ignore any sdists that it may find on the PyPI -compatible indexes. - -```{note} -This is only supported on `bzlmd`. -``` - -(bazel-downloader)= -### Bazel downloader and multi-platform wheel hub repository. - -The `bzlmod` `pip.parse` call supports pulling information from `PyPI` (or a -compatible mirror) and it will ensure that the [bazel -downloader][bazel_downloader] is used for downloading the wheels. This allows -the users to use the [credential helper](#credential-helper) to authenticate -with the mirror and it also ensures that the distribution downloads are cached. -It also avoids using `pip` altogether and results in much faster dependency -fetching. - -This can be enabled by `experimental_index_url` and related flags as shown in -the {gh-path}`examples/bzlmod/MODULE.bazel` example. - -When using this feature during the `pip` extension evaluation you will see the accessed indexes similar to below: -```console -Loading: 0 packages loaded - currently loading: docs/ - Fetching module extension pip in @@//python/extensions:pip.bzl; starting - Fetching https://pypi.org/simple/twine/ -``` - -This does not mean that `rules_python` is fetching the wheels eagerly, but it -rather means that it is calling the PyPI server to get the Simple API response -to get the list of all available source and wheel distributions. Once it has -got all of the available distributions, it will select the right ones depending -on the `sha256` values in your `requirements_lock.txt` file. The compatible -distribution URLs will be then written to the `MODULE.bazel.lock` file. Currently -users wishing to use the lock file with `rules_python` with this feature have -to set an environment variable `RULES_PYTHON_OS_ARCH_LOCK_FILE=0` which will -become default in the next release. - -Fetching the distribution information from the PyPI allows `rules_python` to -know which `whl` should be used on which target platform and it will determine -that by parsing the `whl` filename based on [PEP600], [PEP656] standards. This -allows the user to configure the behaviour by using the following publicly -available flags: -* {obj}`--@rules_python//python/config_settings:py_linux_libc` for selecting the Linux libc variant. -* {obj}`--@rules_python//python/config_settings:pip_whl` for selecting `whl` distribution preference. -* {obj}`--@rules_python//python/config_settings:pip_whl_osx_arch` for selecting MacOS wheel preference. -* {obj}`--@rules_python//python/config_settings:pip_whl_glibc_version` for selecting the GLIBC version compatibility. -* {obj}`--@rules_python//python/config_settings:pip_whl_muslc_version` for selecting the musl version compatibility. -* {obj}`--@rules_python//python/config_settings:pip_whl_osx_version` for selecting MacOS version compatibility. - -[bazel_downloader]: https://bazel.build/rules/lib/builtins/repository_ctx#download -[pep600]: https://peps.python.org/pep-0600/ -[pep656]: https://peps.python.org/pep-0656/ - -(credential-helper)= -### Credential Helper - -The "use Bazel downloader for python wheels" experimental feature includes support for the Bazel -[Credential Helper][cred-helper-design]. - -Your python artifact registry may provide a credential helper for you. Refer to your index's docs -to see if one is provided. - -See the [Credential Helper Spec][cred-helper-spec] for details. - -[cred-helper-design]: https://github.com/bazelbuild/proposals/blob/main/designs/2022-06-07-bazel-credential-helpers.md -[cred-helper-spec]: https://github.com/EngFlow/credential-helper-spec/blob/main/spec.md - - -#### Basic Example: - -The simplest form of a credential helper is a bash script that accepts an arg and spits out JSON to -stdout. For a service like Google Artifact Registry that uses ['Basic' HTTP Auth][rfc7617] and does -not provide a credential helper that conforms to the [spec][cred-helper-spec], the script might -look like: - -```bash -#!/bin/bash -# cred_helper.sh -ARG=$1 # but we don't do anything with it as it's always "get" - -# formatting is optional -echo '{' -echo ' "headers": {' -echo ' "Authorization": ["Basic dGVzdDoxMjPCow=="]' -echo ' }' -echo '}' -``` - -Configure Bazel to use this credential helper for your python index `example.com`: - -``` -# .bazelrc -build --credential_helper=example.com=/full/path/to/cred_helper.sh -``` - -Bazel will call this file like `cred_helper.sh get` and use the returned JSON to inject headers -into whatever HTTP(S) request it performs against `example.com`. - -[rfc7617]: https://datatracker.ietf.org/doc/html/rfc7617 diff --git a/docs/pypi/circular-dependencies.md b/docs/pypi/circular-dependencies.md new file mode 100644 index 0000000000..62613f489e --- /dev/null +++ b/docs/pypi/circular-dependencies.md @@ -0,0 +1,82 @@ +:::{default-domain} bzl +::: + +# Circular dependencies + +Sometimes PyPI packages contain dependency cycles. For instance, a particular +version of `sphinx` (this is no longer the case in the latest version as of +2024-06-02) depends on `sphinxcontrib-serializinghtml`. When using them as +`requirement()`s, ala + +```starlark +py_binary( + name = "doctool", + ... + deps = [ + requirement("sphinx"), + ], +) +``` + +Bazel will protest because it doesn't support cycles in the build graph -- + +``` +ERROR: .../external/pypi_sphinxcontrib_serializinghtml/BUILD.bazel:44:6: in alias rule @pypi_sphinxcontrib_serializinghtml//:pkg: cycle in dependency graph: + //:doctool (...) + @pypi//sphinxcontrib_serializinghtml:pkg (...) +.-> @pypi_sphinxcontrib_serializinghtml//:pkg (...) +| @pypi_sphinxcontrib_serializinghtml//:_pkg (...) +| @pypi_sphinx//:pkg (...) +| @pypi_sphinx//:_pkg (...) +`-- @pypi_sphinxcontrib_serializinghtml//:pkg (...) +``` + +The `experimental_requirement_cycles` attribute allows you to work around these +issues by specifying groups of packages which form cycles. `pip_parse` will +transparently fix the cycles for you and provide the cyclic dependencies +simultaneously. + +```starlark + ... + experimental_requirement_cycles = { + "sphinx": [ + "sphinx", + "sphinxcontrib-serializinghtml", + ] + }, +) +``` + +`pip_parse` supports fixing multiple cycles simultaneously, however, cycles must +be distinct. `apache-airflow`, for instance, has dependency cycles with a number +of its optional dependencies, which means those optional dependencies must all +be a part of the `airflow` cycle. For instance: + +```starlark + ... + experimental_requirement_cycles = { + "airflow": [ + "apache-airflow", + "apache-airflow-providers-common-sql", + "apache-airflow-providers-postgres", + "apache-airflow-providers-sqlite", + ] + } +) +``` + +Alternatively, one could resolve the cycle by removing one leg of it. + +For example, while `apache-airflow-providers-sqlite` is "baked into" the Airflow +package, `apache-airflow-providers-postgres` is not and is an optional feature. +Rather than listing `apache-airflow[postgres]` in your `requirements.txt`, which +would expose a cycle via the extra, one could either _manually_ depend on +`apache-airflow` and `apache-airflow-providers-postgres` separately as +requirements. Bazel rules which need only `apache-airflow` can take it as a +dependency, and rules which explicitly want to mix in +`apache-airflow-providers-postgres` now can. + +Alternatively, one could use `rules_python`'s patching features to remove one +leg of the dependency manually, for instance, by making +`apache-airflow-providers-postgres` not explicitly depend on `apache-airflow` or +perhaps `apache-airflow-providers-common-sql`. diff --git a/docs/pypi/download-workspace.md b/docs/pypi/download-workspace.md new file mode 100644 index 0000000000..4912f71103 --- /dev/null +++ b/docs/pypi/download-workspace.md @@ -0,0 +1,106 @@ +:::{default-domain} bzl +::: + +# Download (WORKSPACE) + +This documentation page covers how to download PyPI dependencies in the legacy `WORKSPACE` setup. + +To add pip dependencies to your `WORKSPACE`, load the `pip_parse` function and +call it to create the central external repo and individual wheel external repos. + +```starlark +load("@rules_python//python:pip.bzl", "pip_parse") + +# Create a central repo that knows about the dependencies needed from +# requirements_lock.txt. +pip_parse( + name = "my_deps", + requirements_lock = "//path/to:requirements_lock.txt", +) + +# Load the starlark macro, which will define your dependencies. +load("@my_deps//:requirements.bzl", "install_deps") + +# Call it to define repos for your requirements. +install_deps() +``` + +## Interpreter selection + +Note that because `pip_parse` runs before Bazel decides which Python toolchain to use, it cannot +enforce that the interpreter used to invoke `pip` matches the interpreter used to run `py_binary` +targets. By default, `pip_parse` uses the system command `"python3"`. To override this, pass in the +{attr}`pip_parse.python_interpreter` attribute or {attr}`pip_parse.python_interpreter_target`. + +You can have multiple `pip_parse`s in the same workspace. This configuration will create multiple +external repos that have no relation to one another and may result in downloading the same wheels +numerous times. + +As with any repository rule, if you would like to ensure that `pip_parse` is +re-executed to pick up a non-hermetic change to your environment (e.g., updating +your system `python` interpreter), you can force it to re-execute by running +`bazel sync --only [pip_parse name]`. + +## Requirements for a specific OS/Architecture + +In some cases, you may need to use different requirements files for different OS and architecture combinations. +This is enabled via the {attr}`pip_parse.requirements_by_platform` attribute. The keys of the +dictionary are labels to the file, and the values are a list of comma-separated target (os, arch) +tuples. + +For example: +```starlark + # ... + requirements_by_platform = { + "requirements_linux_x86_64.txt": "linux_x86_64", + "requirements_osx.txt": "osx_*", + "requirements_linux_exotic.txt": "linux_exotic", + "requirements_some_platforms.txt": "linux_aarch64,windows_*", + }, + # For the list of standard platforms that the rules_python has toolchains for, default to + # the following requirements file. + requirements_lock = "requirements_lock.txt", +``` + +In case of duplicate platforms, `rules_python` will raise an error, as there has +to be an unambiguous mapping of the requirement files to the (os, arch) tuples. + +An alternative way is to use per-OS requirement attributes. +```starlark + # ... + requirements_windows = "requirements_windows.txt", + requirements_darwin = "requirements_darwin.txt", + # For the remaining platforms (which is basically only linux OS), use this file. + requirements_lock = "requirements_lock.txt", +) +``` + +:::{note} +If you are using a universal lock file but want to restrict the list of platforms that +the lock file will be evaluated against, consider using the aforementioned +`requirements_by_platform` attribute and listing the platforms explicitly. +::: + +(vendoring-requirements)= +## Vendoring the requirements.bzl file + +:::{note} +For `bzlmod`, refer to standard `bazel vendor` usage if you want to really vendor it, otherwise +just use the `pip` extension as you would normally. + +However, be aware that there are caveats when doing so. +::: + +In some cases you may not want to generate the requirements.bzl file as a repository rule +while Bazel is fetching dependencies. For example, if you produce a reusable Bazel module +such as a ruleset, you may want to include the `requirements.bzl` file rather than make your users +install the `WORKSPACE` setup to generate it, see {gh-issue}`608`. + +This is the same workflow as Gazelle, which creates `go_repository` rules with +[`update-repos`](https://github.com/bazelbuild/bazel-gazelle#update-repos) + +To do this, use the "write to source file" pattern documented in + +to put a copy of the generated `requirements.bzl` into your project. +Then load the requirements.bzl file directly rather than from the generated repository. +See the example in {gh-path}`examples/pip_parse_vendored`. diff --git a/docs/pypi/download.md b/docs/pypi/download.md new file mode 100644 index 0000000000..fb7f38dc24 --- /dev/null +++ b/docs/pypi/download.md @@ -0,0 +1,449 @@ +:::{default-domain} bzl +::: + +# Download (bzlmod) + +:::{seealso} +For WORKSPACE instructions see [here](./download-workspace). +::: + +To add PyPI dependencies to your `MODULE.bazel` file, use the `pip.parse` +extension and call it to create the central external repo and individual wheel +external repos. Include the toolchain extension in the `MODULE.bazel` file as shown +in the first bzlmod example above. + +```starlark +pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") + +pip.parse( + hub_name = "my_deps", + python_version = "3.13", + requirements_lock = "//:requirements_lock_3_11.txt", +) + +use_repo(pip, "my_deps") +``` + +For more documentation, see the Bzlmod examples under the {gh-path}`examples` folder or the documentation +for the {obj}`@rules_python//python/extensions:pip.bzl` extension. + +:::note} +We are using a host-platform compatible toolchain by default to setup pip dependencies. +During the setup phase, we create some symlinks, which may be inefficient on Windows +by default. In that case use the following `.bazelrc` options to improve performance if +you have admin privileges: + + startup --windows_enable_symlinks + +This will enable symlinks on Windows and help with bootstrap performance of setting up the +hermetic host python interpreter on this platform. Linux and OSX users should see no +difference. +::: + +## Interpreter selection + +The {obj}`pip.parse` `bzlmod` extension by default uses the hermetic Python toolchain for the host +platform, but you can customize the interpreter using {attr}`pip.parse.python_interpreter` and +{attr}`pip.parse.python_interpreter_target`. + +You can use the pip extension multiple times. This configuration will create +multiple external repos that have no relation to one another and may result in +downloading the same wheels numerous times. + +(unified-pypi-hub)= +## Unified `@pypi` Hub for Multi-Hub Configurations + +:::{versionadded} 2.2.0 +Unified `@pypi` hub repository for Bzlmod multi-hub configurations. +::: + +When you call the `pip` extension multiple times with different `hub_name` +attributes, `rules_python` automatically generates a unified `@pypi` hub +repository (unless one of your concrete hubs is explicitly named `"pypi"`). + +This unified `@pypi` repository acts as a dynamic proxy that routes package +dependencies to the active concrete hub at build time. This is especially +useful in monorepos where shared library targets need to depend on PyPI +packages without knowing which specific hub or requirements lock file the +consuming binary is using. + +#### Reserved `"pypi"` Hub Name + +The hub name `"pypi"` is **reserved** for the automatically generated unified +hub repository. Defining a concrete hub named `"pypi"` will cause a collision. + +For details on how this collision is handled and resolved via environment +variables, see the {envvar}`RULES_PYTHON_PYPI_HUB_RESERVED` documentation. + +#### Configuring the Unified Hub + +To configure the unified hub, define your concrete hubs as usual, and +optionally designate a default hub using the `pip.default` tag's +`default_hub` attribute: + +```starlark +pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") + +# Define concrete hub 'pypi_a' +pip.parse( + hub_name = "pypi_a", + python_version = "3.11", + requirements_lock = "//:requirements_a.txt", +) + +# Define concrete hub 'pypi_b' +pip.parse( + hub_name = "pypi_b", + python_version = "3.11", + requirements_lock = "//:requirements_b.txt", +) + +# Designate 'pypi_b' as the default hub for the unified '@pypi' repository +pip.default(default_hub = "pypi_b") + +# Import the unified hub repository +use_repo(pip, "pypi") +``` + +#### Dynamic Routing at Build Time + +By default, the unified `@pypi` repository will resolve packages from the +designated `default_hub`. You can dynamically switch the active hub for a build +using the `--@rules_python//python/config_settings:venv` command-line flag +or via target transitions: + +```bash +# Build using packages from 'pypi_a' +bazel build --@rules_python//python/config_settings:venv=pypi_a //my:binary +``` + +Shared library targets can simply depend on the unified hub (e.g., +`@pypi//numpy`), and the dependency will automatically resolve to the correct +wheel version from the active hub during the build. + +### Declaring Abstract Dependencies (pip.dep) + +:::{versionadded} 2.2.0 +Declaring abstract PyPI dependencies via `pip.dep` tags. +::: + +Sometimes a shared library target or a ruleset needs to depend on a PyPI +package (e.g., `@pypi//numpy`), but does not want to force a specific package +version or a concrete `requirements.txt` lock file on its consumers. + +Instead of calling `pip.parse()`, the module can declare its dependency using +the `pip.dep` tag: + +```starlark +pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") + +# Declare an abstract dependency on 'numpy' and specify extra targets that +# are expected to be available in the package. +pip.dep( + name = "numpy", + extra_targets = ["extra-alias"], +) +``` + +This ensures that the target structure `@pypi//numpy` (and +`@pypi//numpy:extra-alias`) exists in the unified `@pypi` hub repository, so the +declaring module can compile and analyze successfully without needing any local +requirements file. + +The actual concrete implementation and version of the package must be provided +by a downstream module calling `pip.parse`. + +If a downstream module attempts to build a target that depends on an abstract +dependency, but has not provided a concrete implementation for it via any +`pip.parse` call, the build will fail at execution time. + + +As with any repository rule or extension, if you would like to ensure that `pip_parse` is +re-executed to pick up a non-hermetic change to your environment (e.g., updating your system +`python` interpreter), you can force it to re-execute by running `bazel sync --only [pip_parse +name]`. + +(per-os-arch-requirements)= +## Requirements for a specific OS/Architecture + +In some cases, you may need to use different requirements files for different OS and architecture combinations. +This is enabled via the `requirements_by_platform` attribute in the `pip.parse` extension and the +{obj}`pip.parse` tag class. The keys of the dictionary are labels to the file, and the values are a +list of comma-separated target (os, arch) tuples. + +For example: +```starlark + # ... + requirements_by_platform = { + "requirements_linux_x86_64.txt": "linux_x86_64", + "requirements_osx.txt": "osx_*", + "requirements_linux_exotic.txt": "linux_exotic", + "requirements_some_platforms.txt": "linux_aarch64,windows_*", + }, + # For the list of standard platforms that the rules_python has toolchains for, default to + # the following requirements file. + requirements_lock = "requirements_lock.txt", +``` + +In case of duplicate platforms, `rules_python` will raise an error, as there has +to be an unambiguous mapping of the requirement files to the (os, arch) tuples. + +An alternative way is to use per-OS requirement attributes. +```starlark + # ... + requirements_windows = "requirements_windows.txt", + requirements_darwin = "requirements_darwin.txt", + # For the remaining platforms (which is basically only linux OS), use this file. + requirements_lock = "requirements_lock.txt", +) +``` + +:::{note} +If you are using a universal lock file but want to restrict the list of platforms that +the lock file will be evaluated against, consider using the aforementioned +`requirements_by_platform` attribute and listing the platforms explicitly. +::: + +## Multi-platform support + +Historically, the {obj}`pip_parse` and {obj}`pip.parse` have only been downloading/building +Python dependencies for the host platform that the `bazel` commands are executed on. Over +the years, people started needing support for building containers, and usually, that involves +fetching dependencies for a particular target platform that may be different from the host +platform. + +Multi-platform support for cross-building the wheels can be done by +using {attr}`target_platforms` for the {bzl:obj}`pip.parse` bzlmod tag class + +:::{warning} +This will not work for sdists with C extensions, but pure Python sdists may still work using the first +approach. +::: + +By default, `rules_python` selects the host `{os}_{arch}` platform from its `MODULE.bazel` +file. This means that `rules_python` by default does not provide cross-platform building support +because some packages have very large wheels and users should be able to use `bazel query` with +minimal overhead. As a result, users should configure their `pip.parse` +calls and select which platforms they want to target via the +{attr}`pip.parse.target_platforms` attribute: +```starlark + # Example of enabling free threaded and non-freethreaded switching on the host platform: + target_platforms = ["{os}_{arch}", "{os}_{arch}_freethreaded"], + + # As another example, to enable building for `linux_x86_64` containers and the host platform: + # target_platforms = ["{os}_{arch}", "linux_x86_64"], +) +``` + +### Using `download_only` attribute + +Let's say you have two requirements files: +``` +# requirements.linux_x86_64.txt +--platform=manylinux_2_17_x86_64 +--python-version=39 +--implementation=cp +--abi=cp39 + +foo==0.0.1 --hash=sha256:deadbeef +bar==0.0.1 --hash=sha256:deadb00f +``` + +``` +# requirements.osx_aarch64.txt contents +--platform=macosx_10_9_arm64 +--python-version=39 +--implementation=cp +--abi=cp39 + +foo==0.0.3 --hash=sha256:deadbaaf +``` + +With these 2 files your {bzl:obj}`pip.parse` could look like: +```starlark +pip.parse( + hub_name = "pip", + python_version = "3.9", + # Tell `pip` to ignore sdists + download_only = True, + requirements_by_platform = { + "requirements.linux_x86_64.txt": "linux_x86_64", + "requirements.osx_aarch64.txt": "osx_aarch64", + }, +) +``` + +With this, `pip.parse` will create a hub repository that is going to +support only two platforms - `cp39_osx_aarch64` and `cp39_linux_x86_64` - and it +will only use `wheels` and ignore any sdists that it may find on the PyPI- +compatible indexes. + +:::{warning} +Because bazel is not aware what exactly is downloaded, the same wheel may be downloaded +multiple times. +::: + +:::{note} +This will only work for wheel-only setups, i.e., all of your dependencies need to have wheels +available on the PyPI index that you use. +::: + +### Customizing `Requires-Dist` resolution + +In order to understand what dependencies to pull for a particular package, +`rules_python` parses the `whl` file [`METADATA`][metadata]. +Packages can express dependencies via `Requires-Dist`, and they can add conditions using +"environment markers", which represent the Python version, OS, etc. + +While the PyPI integration provides reasonable defaults to support most +platforms and environment markers, the values it uses can be customized in case +more esoteric configurations are needed. + +To customize the values used, you need to do two things: +1. Define a target that returns {obj}`EnvMarkerInfo` +2. Set the {obj}`//python/config_settings:pip_env_marker_config` flag to + the target defined in (1). + +The keys and values should be compatible with the [PyPA dependency specifiers +specification](https://packaging.python.org/en/latest/specifications/dependency-specifiers/). +This is not strictly enforced, however, so you can return a subset of keys or +additional keys, which become available during dependency evaluation. + +[metadata]: https://packaging.python.org/en/latest/specifications/core-metadata/ + +(bazel-downloader)= +### Bazel downloader and multi-platform wheel hub repository. + +The {obj}`pip` extension supports pulling information from `PyPI` (or a compatible mirror), and it +will ensure that the [bazel downloader][bazel_downloader] is used for downloading the wheels. + +This provides the following benefits: +* Integration with the [credential_helper](#credential-helper) to authenticate with private + mirrors. +* Cache the downloaded wheels speeding up the consecutive re-initialization of the repositories. +* Reuse the same instance of the wheel for multiple target platforms. +* Allow using transitions and targeting free-threaded and musl platforms more easily. +* Avoids `pip` for wheel fetching and results in much faster dependency fetching. + +Similar to [uv](https://docs.astral.sh/uv/configuration/indexes/), one can override the index that +is used for a single package. By default, we first search in the indexes specified by +`--extra-index-url`, then we fall back to the `--index-url` setting unless there are overrides +specified via {attr}`pip.parse.experimental_index_url_overrides`. + +When using this feature during the `pip` extension evaluation you will see the accessed indexes similar to below: +```console +Loading: 0 packages loaded + Fetching module extension @@//python/extensions:pip.bzl%pip; Fetch package lists from PyPI index + Fetching https://pypi.org/simple/jinja2/ + +``` + +This does not mean that `rules_python` is fetching the wheels eagerly; rather, +it means that it is calling the PyPI server to get the Simple API response +to get the list of all available source and wheel distributions. Once it has +gotten all of the available distributions, it will select the right ones depending +on the `--hash` values in your `requirements_lock.txt` file (any hash algorithm +advertised by the index can be matched, not only `sha256`). If hashes are not +present in the requirements file, we will fall back to matching by version +specified in the lock file. + +Fetching the distribution information from the PyPI allows `rules_python` to +know which `whl` should be used on which target platform and it will determine +that by parsing the `whl` filename based on [PEP600], [PEP656] standards. This +allows the user to configure the behaviour by using the following publicly +available flags: +* {obj}`--@rules_python//python/config_settings:py_linux_libc` for selecting the Linux libc variant. + +[bazel_downloader]: https://bazel.build/rules/lib/builtins/repository_ctx#download +[pep600]: https://peps.python.org/pep-0600/ +[pep656]: https://peps.python.org/pep-0656/ + +## Internal dependencies and private repositories + +The `rules_python` Bazel module downloads Python interpreters and +dependencies as part of its functionality. These artifacts are fetched +using Bazel's internal HTTP downloader, not using the `pip` tool. + +If you are in a network-restricted environment and must use internal +registries, you can configure the Bazel downloader to redirect all of +these downloads to a different registry. + +Example of a `bazel_downloader.cfg`: +```cfg +all_blocked_message See internal.mirror.lan/registry/ for more information +allow s3.amazon.com + +# Rewrite everything to files.pythonhosted to the internal mirror with two +# capture groups: the first group matches the host and is appended first, +# the second matches the entire path and is appended second +rewrite (files.pythonhosted.org)/(.*) internal.mirror.lan/python/$1/$2 +rewrite (pypi.python.org)/(.*) internal.mirror.lan/python/$1/$2 + +# Allow the internal mirror and block everything else +allow internal.mirror.lan +block * +``` + +Use the config file with `--experimental_downloader_config=bazel_downloader.cfg`. + +### How the config is parsed: + +* Uses Java regular expressions +* Matching is performed only on host and path components of the URL, not the scheme +* Directives are applied in the following order: `rewrite, allow, block` +* Back references are numbered starting from `$1` +* Expressions must match the entire string being tested, not just find a substring. + +If your patterns don't seem to match or rewrite: + +* Begin with simple patterns to ensure they match as expected. +* Be cautious when using `block` statements to avoid unintentionally blocking necessary downloads. Add `block` statements incrementally and test thoroughly after each change. + +### References: + +* [Configuring Bazel's Downloader](https://blog.aspect.build/configuring-bazels-downloader) +* [URLRewriterConfig.java Source Code](https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/bazel/repository/downloader/UrlRewriterConfig.java) +* [Issue 3519](https://github.com/bazel-contrib/rules_python/issues/3519) + +(credential-helper)= +## Credential Helper + +The [Bazel downloader](#bazel-downloader) usage allows for the Bazel +[Credential Helper][cred-helper-design]. +Your Python artifact registry may provide a credential helper for you. +Refer to your index's docs to see if one is provided. + +The simplest form of a credential helper is a bash script that accepts an argument and spits out JSON to +stdout. For a service like Google Artifact Registry that uses ['Basic' HTTP Auth][rfc7617] and does +not provide a credential helper that conforms to the [spec][cred-helper-spec], the script might +look like: + +```bash +#!/bin/bash +# cred_helper.sh +ARG=$1 # but we don't do anything with it as it's always "get" + +# formatting is optional +echo '{' +echo ' "headers": {' +echo ' "Authorization": ["Basic dGVzdDoxMjPCow=="]' +echo ' }' +echo '}' +``` + +Configure Bazel to use this credential helper for your Python index `example.com`: + +``` +# .bazelrc +build --credential_helper=example.com=/full/path/to/cred_helper.sh +``` + +Bazel will call this file like `cred_helper.sh get` and use the returned JSON to inject headers +into whatever HTTP(S) request it performs against `example.com`. + +See the [Credential Helper Spec][cred-helper-spec] for more details. + + +[rfc7617]: https://datatracker.ietf.org/doc/html/rfc7617 +[cred-helper-design]: https://github.com/bazelbuild/proposals/blob/main/designs/2022-06-07-bazel-credential-helpers.md +[cred-helper-spec]: https://github.com/EngFlow/credential-helper-spec/blob/main/spec.md diff --git a/docs/pypi/index.md b/docs/pypi/index.md new file mode 100644 index 0000000000..2d17dd5054 --- /dev/null +++ b/docs/pypi/index.md @@ -0,0 +1,32 @@ +:::{default-domain} bzl +::: + +(pypi-dependencies)= +# Using PyPI + +Using PyPI packages (aka "pip install") involves the following main steps: + +1. [Generating requirements file](./lock) +2. Installing third-party packages in [bzlmod](./download) or [WORKSPACE](./download-workspace). +3. [Using third-party packages as dependencies](./use) + +With the advanced topics covered separately: +* Dealing with [circular dependencies](./circular-dependencies). +* Handling [multi-platform dependencies](../howto/multi-platform-pypi-deps). + +```{toctree} +lock +download +download-workspace +use +``` + +## Advanced topics + +```{toctree} +:maxdepth: 1 + +circular-dependencies +patch +../howto/multi-platform-pypi-deps +``` diff --git a/docs/pypi/lock.md b/docs/pypi/lock.md new file mode 100644 index 0000000000..5c9f0646dc --- /dev/null +++ b/docs/pypi/lock.md @@ -0,0 +1,122 @@ +:::{default-domain} bzl +::: + +# Lock + +:::{note} +Currently `rules_python` only supports `requirements.txt` format. + +#{gh-issue}`2787` tracks `pylock.toml` support. +::: + +## requirements.txt + +### pip compile + +Generally, when working on a Python project, you'll have some dependencies that themselves have +other dependencies. You might also specify dependency bounds instead of specific versions. +So you'll need to generate a full list of all transitive dependencies and pinned versions +for every dependency. + +Typically, you'd have your project dependencies specified in `pyproject.toml` or `requirements.in` +and generate the full pinned list of dependencies in `requirements_lock.txt`, which you can +manage with {obj}`compile_pip_requirements`: + +```starlark +load("@rules_python//python:pip.bzl", "compile_pip_requirements") + +compile_pip_requirements( + name = "requirements", + src = "requirements.in", + requirements_txt = "requirements_lock.txt", +) +``` + +This rule generates two targets: +- `bazel run [name].update` will regenerate the `requirements_txt` file +- `bazel test [name]_test` will test that the `requirements_txt` file is up to date + +Once you generate this fully specified list of requirements, you can install the requirements ([bzlmod](./download)/[WORKSPACE](./download-workspace)). + +:::{warning} +If you're specifying dependencies in `pyproject.toml`, make sure to include the +`[build-system]` configuration, with pinned dependencies. +`compile_pip_requirements` will use the build system specified to read your +project's metadata, and you might see non-hermetic behavior if you don't pin the +build system. + +Not specifying `[build-system]` at all will result in using a default +`[build-system]` configuration, which uses unpinned versions +([ref](https://peps.python.org/pep-0518/#build-system-table)). +::: + + +#### pip compile Dependency groups + +pip-compile doesn't yet support pyproject.toml dependency groups. Follow +[pip-tools #2062](https://github.com/jazzband/pip-tools/issues/2062) +to see the status of their support. + +In the meantime, support can be emulated by passing multiple files to `srcs`: + +```starlark +compile_pip_requirements( + srcs = ["pyproject.toml", "requirements-dev.in"] + ... +) +``` + +### uv pip compile (bzlmod only) + +We also have an experimental setup for the `uv pip compile` way of generating lock files. +This is well tested with the public PyPI index, but you may hit some rough edges with private +mirrors. + +#### Example usage + +```starlark +load("@rules_python//python/uv:lock.bzl", "lock") + +lock( + name = "requirements", + srcs = ["pyproject.toml", "requirements.in"], + out = "requirements_lock.txt", +) +``` + +#### `[tool.uv]` settings from pyproject.toml + +When a `pyproject.toml` file is among the {attr}`lock.srcs`, the +{obj}`lock` rule auto-detects the project directory and passes +`--project ` to `uv pip compile`. This causes `uv` to read +`[tool.uv]` settings from that `pyproject.toml`, such as +`no-build-isolation`, `exclude-dependencies`, and workspace members. + +If multiple `pyproject.toml` files are in {attr}`lock.srcs`, the one +with the shortest directory path is selected (this heuristic works for +typical uv workspace layouts where the root configuration is at the +shortest path). + +If the auto-detection picks the wrong project directory, use the +`project` parameter to override: + +```starlark +lock( + name = "requirements", + srcs = ["pyproject.toml", "requirements.in"], + out = "requirements_lock.txt", + project = "subproject", +) +``` + +:::{warning} +**Known limitations of auto-detection** + +1. **Workspace heuristic** — the shortest-path selection may incorrectly assume the upper-most + workspace `pyproject.toml` is the correct one. For monorepos with multiple independent + sub-projects, you must set `project` explicitly for each {obj}`lock` target. +1. **No test target** — unlike {obj}`compile_pip_requirements`, no test target is auto-created; see + the {obj}`lock` docs for how to add one manually using `diff_test` from `bazel_skylib`. +::: + +For more documentation see {obj}`lock`. diff --git a/docs/pypi/patch.md b/docs/pypi/patch.md new file mode 100644 index 0000000000..7e3cb41981 --- /dev/null +++ b/docs/pypi/patch.md @@ -0,0 +1,10 @@ +:::{default-domain} bzl +::: + +# Patching wheels + +Sometimes the wheels have to be patched to: +* Workaround the lack of a standard `site-packages` layout ({gh-issue}`2156`). +* Include certain PRs of your choice on top of wheels and avoid building from sdist. + +You can patch the wheels by using the {attr}`pip.override.patches` attribute. diff --git a/docs/pypi/use.md b/docs/pypi/use.md new file mode 100644 index 0000000000..9d0c54c4ab --- /dev/null +++ b/docs/pypi/use.md @@ -0,0 +1,140 @@ +:::{default-domain} bzl +::: + +# Use in BUILD.bazel files + +Once you have set up the dependencies, you are ready to start using them in your `BUILD.bazel` +files. If you haven't done so yet, set it up by following these docs: +1. [WORKSPACE](./download-workspace) +2. [bzlmod](./download) + +To refer to targets in a hub repo `pypi`, you can do one of two things: +```starlark +py_library( + name = "my_lib", + deps = [ + "@pypi//numpy", + ], +) +``` + +Or use the `requirement` helper that needs to be loaded from the `hub` repo itself: +```starlark +load("@pypi//:requirements.bzl", "requirement") + +py_library( + deps = [ + requirement("numpy") + ], +) +``` + +Note that the usage of the `requirement` helper is not advised and can be problematic. See the +[notes below](#requirement-helper). + +Note that the hub repo contains the following targets for each package: +* `@pypi//numpy` - shorthand for `@pypi//numpy:numpy`. This is an {obj}`alias` to + `@pypi//numpy:pkg`. +* `@pypi//numpy:pkg` - the {obj}`py_library` target automatically generated by the repository + rules. +* `@pypi//numpy:data` - the {obj}`filegroup` for all of the extra files that are included + as data in the `pkg` target. +* `@pypi//numpy:dist_info` - the {obj}`filegroup` for all of the files in the `.distinfo` directory. +* `@pypi//numpy:extracted_whl_files` - a {obj}`filegroup` of all the files + extracted from the whl file. +* `@pypi//numpy:whl` - the {obj}`filegroup` that is the `.whl` file itself, which includes all + transitive dependencies via the {attr}`filegroup.data` attribute. + +:::{versionadded} 1.6.0 + +The `:extracted_whl_files` target was added +::: + +## Entry points + +If you would like to access [entry points][whl_ep], see the `py_console_script_binary` rule documentation, +which can help you create a `py_binary` target for a particular console script exposed by a package. + +[whl_ep]: https://packaging.python.org/specifications/entry-points/ + +## 'Extras' dependencies + +Any "extras" specified in the requirements lock file will be automatically added +as transitive dependencies of the package. In the example above, you'd just put +`requirement("useful_dep")` or `@pypi//useful_dep`. + +## Consuming Wheel Dists Directly + +If you need to depend on the wheel dists themselves (for instance, to pass them +to some other packaging tool), you can get a handle to them with the +`whl_requirement` macro. For example: + +```starlark +load("@pypi//:requirements.bzl", "whl_requirement") + +filegroup( + name = "whl_files", + data = [ + # This is equivalent to "@pypi//boto3:whl" + whl_requirement("boto3"), + ] +) +``` + +## Creating a filegroup of files within a whl + +The rule {obj}`whl_filegroup` exists as an easy way to extract the necessary files +from a whl file without needing to modify the `BUILD.bazel` contents of the +whl repositories generated via `pip_repository`. Use it similarly to the `filegroup` +above. See the API docs for more information. + +(requirement-helper)= +## A note about using the requirement helper + +Each extracted wheel repo contains a `py_library` target representing +the wheel's contents. There are two ways to access this library. The +first uses the `requirement()` function defined in the central +repo's `//:requirements.bzl` file. This function maps a pip package +name to a label: + +```starlark +load("@my_deps//:requirements.bzl", "requirement") + +py_library( + name = "mylib", + srcs = ["mylib.py"], + deps = [ + ":myotherlib", + requirement("some_pip_dep"), + requirement("another_pip_dep"), + ] +) +``` + +The reason `requirement()` exists is to insulate users from +changes to the underlying repository and label strings. However, those +labels have become directly used, so they aren't able to easily change regardless. + +On the other hand, using the `requirement()` helper has several drawbacks: + +- It doesn't work with `buildifier`. +- It doesn't work with `buildozer`. +- It adds an extra layer on top of normal mechanisms to refer to targets. +- It does not scale well, as each type of target needs a new macro to be loaded and imported. + +If you don't want to use `requirement()`, you can use the library labels directly instead. For +`pip_parse`, the labels are of the following form: + +```starlark +@{name}//{package} +``` + +Here `name` is the `name` attribute that was passed to `pip_parse` and +`package` is the pip package name with characters that are illegal in +Bazel label names (e.g. `-`, `.`) replaced with `_`. If you need to +update `name` from "old" to "new", then you can run the following +`buildozer` command: + +```shell +buildozer 'substitute deps @old//([^/]+) @new//${1}' //...:* +``` diff --git a/docs/readthedocs_build.sh b/docs/readthedocs_build.sh index 3f67310197..cd60792a3f 100755 --- a/docs/readthedocs_build.sh +++ b/docs/readthedocs_build.sh @@ -1,20 +1,20 @@ -#!/bin/bash +#!/usr/bin/env bash set -eou pipefail declare -a extra_env while IFS='=' read -r -d '' name value; do if [[ "$name" == READTHEDOCS* ]]; then - extra_env+=("--//sphinxdocs:extra_env=$name=$value") + extra_env+=("--@sphinxdocs//sphinxdocs:extra_env=$name=$value") fi done < <(env -0) # In order to get the build number, we extract it from the host name -extra_env+=("--//sphinxdocs:extra_env=HOSTNAME=$HOSTNAME") +extra_env+=("--@sphinxdocs//sphinxdocs:extra_env=HOSTNAME=$HOSTNAME") set -x bazel run \ --config=rtd \ - "--//sphinxdocs:extra_defines=version=$READTHEDOCS_VERSION" \ + "--@sphinxdocs//sphinxdocs:extra_defines=version=$READTHEDOCS_VERSION" \ "${extra_env[@]}" \ //docs:readthedocs_install diff --git a/docs/repl.md b/docs/repl.md new file mode 100644 index 0000000000..1434097fdf --- /dev/null +++ b/docs/repl.md @@ -0,0 +1,66 @@ +# Getting a REPL or Interactive Shell + +`rules_python` provides a REPL to help with debugging and developing. The goal of +the REPL is to present an environment identical to what a {bzl:obj}`py_binary` creates +for your code. + +## Usage + +Start the REPL with the following command: +```console +$ bazel run @rules_python//python/bin:repl +Python 3.11.11 (main, Mar 17 2025, 21:02:09) [Clang 20.1.0 ] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> +``` + +Settings like `//python/config_settings:python_version` will influence the exact +behaviour. +```console +$ bazel run @rules_python//python/bin:repl --@rules_python//python/config_settings:python_version=3.13 +Python 3.13.2 (main, Mar 17 2025, 21:02:54) [Clang 20.1.0 ] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> +``` + +See [//python/config_settings](api/rules_python/python/config_settings/index) +and [Environment Variables](environment-variables) for more settings. + +## Importing Python targets + +The `//python/bin:repl_dep` command line flag gives the REPL access to a target +that provides the {bzl:obj}`PyInfo` provider. + +```console +$ bazel run @rules_python//python/bin:repl --@rules_python//python/bin:repl_dep=@rules_python//tools:wheelmaker +Python 3.11.11 (main, Mar 17 2025, 21:02:09) [Clang 20.1.0 ] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> import tools.wheelmaker +>>> +``` + +## Customizing the shell + +By default, the `//python/bin:repl` target will invoke the shell from the `code` +module. It's possible to switch to another shell by writing a custom "stub" and +pointing the target at the necessary dependencies. + +### IPython Example + +For an IPython shell, create a file as follows. + +```python +import IPython +IPython.start_ipython() +``` + +Assuming the file is called `ipython_stub.py` and the `pip.parse` hub's name is +`my_deps`, set this up in the .bazelrc file: +``` +# Allow the REPL stub to import ipython. In this case, @my_deps is the hub name +# of the pip.parse() call. +build --@rules_python//python/bin:repl_stub_dep=@my_deps//ipython + +# Point the REPL at the stub created above. +build --@rules_python//python/bin:repl_stub=//path/to:ipython_stub.py +``` diff --git a/docs/requirements.txt b/docs/requirements.txt deleted file mode 100644 index 2e306cd5b5..0000000000 --- a/docs/requirements.txt +++ /dev/null @@ -1,376 +0,0 @@ -# This file was autogenerated by uv via the following command: -# bazel run //docs:requirements.update ---index-url https://pypi.org/simple - -absl-py==2.1.0 \ - --hash=sha256:526a04eadab8b4ee719ce68f204172ead1027549089702d99b9059f129ff1308 \ - --hash=sha256:7820790efbb316739cde8b4e19357243fc3608a152024288513dd968d7d959ff - # via rules-python-docs (docs/pyproject.toml) -alabaster==1.0.0 \ - --hash=sha256:c00dca57bca26fa62a6d7d0a9fcce65f3e026e9bfe33e9c538fd3fbb2144fd9e \ - --hash=sha256:fc6786402dc3fcb2de3cabd5fe455a2db534b371124f1f21de8731783dec828b - # via sphinx -astroid==3.3.5 \ - --hash=sha256:5cfc40ae9f68311075d27ef68a4841bdc5cc7f6cf86671b49f00607d30188e2d \ - --hash=sha256:a9d1c946ada25098d790e079ba2a1b112157278f3fb7e718ae6a9252f5835dc8 - # via sphinx-autodoc2 -babel==2.16.0 \ - --hash=sha256:368b5b98b37c06b7daf6696391c3240c938b37767d4584413e8438c5c435fa8b \ - --hash=sha256:d1f3554ca26605fe173f3de0c65f750f5a42f924499bf134de6423582298e316 - # via sphinx -certifi==2024.8.30 \ - --hash=sha256:922820b53db7a7257ffbda3f597266d435245903d80737e34f8a45ff3e3230d8 \ - --hash=sha256:bec941d2aa8195e248a60b31ff9f0558284cf01a52591ceda73ea9afffd69fd9 - # via requests -charset-normalizer==3.4.0 \ - --hash=sha256:0099d79bdfcf5c1f0c2c72f91516702ebf8b0b8ddd8905f97a8aecf49712c621 \ - --hash=sha256:0713f3adb9d03d49d365b70b84775d0a0d18e4ab08d12bc46baa6132ba78aaf6 \ - --hash=sha256:07afec21bbbbf8a5cc3651aa96b980afe2526e7f048fdfb7f1014d84acc8b6d8 \ - --hash=sha256:0b309d1747110feb25d7ed6b01afdec269c647d382c857ef4663bbe6ad95a912 \ - --hash=sha256:0d99dd8ff461990f12d6e42c7347fd9ab2532fb70e9621ba520f9e8637161d7c \ - --hash=sha256:0de7b687289d3c1b3e8660d0741874abe7888100efe14bd0f9fd7141bcbda92b \ - --hash=sha256:1110e22af8ca26b90bd6364fe4c763329b0ebf1ee213ba32b68c73de5752323d \ - --hash=sha256:130272c698667a982a5d0e626851ceff662565379baf0ff2cc58067b81d4f11d \ - --hash=sha256:136815f06a3ae311fae551c3df1f998a1ebd01ddd424aa5603a4336997629e95 \ - --hash=sha256:14215b71a762336254351b00ec720a8e85cada43b987da5a042e4ce3e82bd68e \ - --hash=sha256:1db4e7fefefd0f548d73e2e2e041f9df5c59e178b4c72fbac4cc6f535cfb1565 \ - --hash=sha256:1ffd9493de4c922f2a38c2bf62b831dcec90ac673ed1ca182fe11b4d8e9f2a64 \ - --hash=sha256:2006769bd1640bdf4d5641c69a3d63b71b81445473cac5ded39740a226fa88ab \ - --hash=sha256:20587d20f557fe189b7947d8e7ec5afa110ccf72a3128d61a2a387c3313f46be \ - --hash=sha256:223217c3d4f82c3ac5e29032b3f1c2eb0fb591b72161f86d93f5719079dae93e \ - --hash=sha256:27623ba66c183eca01bf9ff833875b459cad267aeeb044477fedac35e19ba907 \ - --hash=sha256:285e96d9d53422efc0d7a17c60e59f37fbf3dfa942073f666db4ac71e8d726d0 \ - --hash=sha256:2de62e8801ddfff069cd5c504ce3bc9672b23266597d4e4f50eda28846c322f2 \ - --hash=sha256:2f6c34da58ea9c1a9515621f4d9ac379871a8f21168ba1b5e09d74250de5ad62 \ - --hash=sha256:309a7de0a0ff3040acaebb35ec45d18db4b28232f21998851cfa709eeff49d62 \ - --hash=sha256:35c404d74c2926d0287fbd63ed5d27eb911eb9e4a3bb2c6d294f3cfd4a9e0c23 \ - --hash=sha256:3710a9751938947e6327ea9f3ea6332a09bf0ba0c09cae9cb1f250bd1f1549bc \ - --hash=sha256:3d59d125ffbd6d552765510e3f31ed75ebac2c7470c7274195b9161a32350284 \ - --hash=sha256:40d3ff7fc90b98c637bda91c89d51264a3dcf210cade3a2c6f838c7268d7a4ca \ - --hash=sha256:425c5f215d0eecee9a56cdb703203dda90423247421bf0d67125add85d0c4455 \ - --hash=sha256:43193c5cda5d612f247172016c4bb71251c784d7a4d9314677186a838ad34858 \ - --hash=sha256:44aeb140295a2f0659e113b31cfe92c9061622cadbc9e2a2f7b8ef6b1e29ef4b \ - --hash=sha256:47334db71978b23ebcf3c0f9f5ee98b8d65992b65c9c4f2d34c2eaf5bcaf0594 \ - --hash=sha256:4796efc4faf6b53a18e3d46343535caed491776a22af773f366534056c4e1fbc \ - --hash=sha256:4a51b48f42d9358460b78725283f04bddaf44a9358197b889657deba38f329db \ - --hash=sha256:4b67fdab07fdd3c10bb21edab3cbfe8cf5696f453afce75d815d9d7223fbe88b \ - --hash=sha256:4ec9dd88a5b71abfc74e9df5ebe7921c35cbb3b641181a531ca65cdb5e8e4dea \ - --hash=sha256:4f9fc98dad6c2eaa32fc3af1417d95b5e3d08aff968df0cd320066def971f9a6 \ - --hash=sha256:54b6a92d009cbe2fb11054ba694bc9e284dad30a26757b1e372a1fdddaf21920 \ - --hash=sha256:55f56e2ebd4e3bc50442fbc0888c9d8c94e4e06a933804e2af3e89e2f9c1c749 \ - --hash=sha256:5726cf76c982532c1863fb64d8c6dd0e4c90b6ece9feb06c9f202417a31f7dd7 \ - --hash=sha256:5d447056e2ca60382d460a604b6302d8db69476fd2015c81e7c35417cfabe4cd \ - --hash=sha256:5ed2e36c3e9b4f21dd9422f6893dec0abf2cca553af509b10cd630f878d3eb99 \ - --hash=sha256:5ff2ed8194587faf56555927b3aa10e6fb69d931e33953943bc4f837dfee2242 \ - --hash=sha256:62f60aebecfc7f4b82e3f639a7d1433a20ec32824db2199a11ad4f5e146ef5ee \ - --hash=sha256:63bc5c4ae26e4bc6be6469943b8253c0fd4e4186c43ad46e713ea61a0ba49129 \ - --hash=sha256:6b40e8d38afe634559e398cc32b1472f376a4099c75fe6299ae607e404c033b2 \ - --hash=sha256:6b493a043635eb376e50eedf7818f2f322eabbaa974e948bd8bdd29eb7ef2a51 \ - --hash=sha256:6dba5d19c4dfab08e58d5b36304b3f92f3bd5d42c1a3fa37b5ba5cdf6dfcbcee \ - --hash=sha256:6fd30dc99682dc2c603c2b315bded2799019cea829f8bf57dc6b61efde6611c8 \ - --hash=sha256:707b82d19e65c9bd28b81dde95249b07bf9f5b90ebe1ef17d9b57473f8a64b7b \ - --hash=sha256:7706f5850360ac01d80c89bcef1640683cc12ed87f42579dab6c5d3ed6888613 \ - --hash=sha256:7782afc9b6b42200f7362858f9e73b1f8316afb276d316336c0ec3bd73312742 \ - --hash=sha256:79983512b108e4a164b9c8d34de3992f76d48cadc9554c9e60b43f308988aabe \ - --hash=sha256:7f683ddc7eedd742e2889d2bfb96d69573fde1d92fcb811979cdb7165bb9c7d3 \ - --hash=sha256:82357d85de703176b5587dbe6ade8ff67f9f69a41c0733cf2425378b49954de5 \ - --hash=sha256:84450ba661fb96e9fd67629b93d2941c871ca86fc38d835d19d4225ff946a631 \ - --hash=sha256:86f4e8cca779080f66ff4f191a685ced73d2f72d50216f7112185dc02b90b9b7 \ - --hash=sha256:8cda06946eac330cbe6598f77bb54e690b4ca93f593dee1568ad22b04f347c15 \ - --hash=sha256:8ce7fd6767a1cc5a92a639b391891bf1c268b03ec7e021c7d6d902285259685c \ - --hash=sha256:8ff4e7cdfdb1ab5698e675ca622e72d58a6fa2a8aa58195de0c0061288e6e3ea \ - --hash=sha256:9289fd5dddcf57bab41d044f1756550f9e7cf0c8e373b8cdf0ce8773dc4bd417 \ - --hash=sha256:92a7e36b000bf022ef3dbb9c46bfe2d52c047d5e3f3343f43204263c5addc250 \ - --hash=sha256:92db3c28b5b2a273346bebb24857fda45601aef6ae1c011c0a997106581e8a88 \ - --hash=sha256:95c3c157765b031331dd4db3c775e58deaee050a3042fcad72cbc4189d7c8dca \ - --hash=sha256:980b4f289d1d90ca5efcf07958d3eb38ed9c0b7676bf2831a54d4f66f9c27dfa \ - --hash=sha256:9ae4ef0b3f6b41bad6366fb0ea4fc1d7ed051528e113a60fa2a65a9abb5b1d99 \ - --hash=sha256:9c98230f5042f4945f957d006edccc2af1e03ed5e37ce7c373f00a5a4daa6149 \ - --hash=sha256:9fa2566ca27d67c86569e8c85297aaf413ffab85a8960500f12ea34ff98e4c41 \ - --hash=sha256:a14969b8691f7998e74663b77b4c36c0337cb1df552da83d5c9004a93afdb574 \ - --hash=sha256:a8aacce6e2e1edcb6ac625fb0f8c3a9570ccc7bfba1f63419b3769ccf6a00ed0 \ - --hash=sha256:a8e538f46104c815be19c975572d74afb53f29650ea2025bbfaef359d2de2f7f \ - --hash=sha256:aa41e526a5d4a9dfcfbab0716c7e8a1b215abd3f3df5a45cf18a12721d31cb5d \ - --hash=sha256:aa693779a8b50cd97570e5a0f343538a8dbd3e496fa5dcb87e29406ad0299654 \ - --hash=sha256:ab22fbd9765e6954bc0bcff24c25ff71dcbfdb185fcdaca49e81bac68fe724d3 \ - --hash=sha256:ab2e5bef076f5a235c3774b4f4028a680432cded7cad37bba0fd90d64b187d19 \ - --hash=sha256:ab973df98fc99ab39080bfb0eb3a925181454d7c3ac8a1e695fddfae696d9e90 \ - --hash=sha256:af73657b7a68211996527dbfeffbb0864e043d270580c5aef06dc4b659a4b578 \ - --hash=sha256:b197e7094f232959f8f20541ead1d9862ac5ebea1d58e9849c1bf979255dfac9 \ - --hash=sha256:b295729485b06c1a0683af02a9e42d2caa9db04a373dc38a6a58cdd1e8abddf1 \ - --hash=sha256:b8831399554b92b72af5932cdbbd4ddc55c55f631bb13ff8fe4e6536a06c5c51 \ - --hash=sha256:b8dcd239c743aa2f9c22ce674a145e0a25cb1566c495928440a181ca1ccf6719 \ - --hash=sha256:bcb4f8ea87d03bc51ad04add8ceaf9b0f085ac045ab4d74e73bbc2dc033f0236 \ - --hash=sha256:bd7af3717683bea4c87acd8c0d3d5b44d56120b26fd3f8a692bdd2d5260c620a \ - --hash=sha256:bf4475b82be41b07cc5e5ff94810e6a01f276e37c2d55571e3fe175e467a1a1c \ - --hash=sha256:c3e446d253bd88f6377260d07c895816ebf33ffffd56c1c792b13bff9c3e1ade \ - --hash=sha256:c57516e58fd17d03ebe67e181a4e4e2ccab1168f8c2976c6a334d4f819fe5944 \ - --hash=sha256:c94057af19bc953643a33581844649a7fdab902624d2eb739738a30e2b3e60fc \ - --hash=sha256:cab5d0b79d987c67f3b9e9c53f54a61360422a5a0bc075f43cab5621d530c3b6 \ - --hash=sha256:ce031db0408e487fd2775d745ce30a7cd2923667cf3b69d48d219f1d8f5ddeb6 \ - --hash=sha256:cee4373f4d3ad28f1ab6290684d8e2ebdb9e7a1b74fdc39e4c211995f77bec27 \ - --hash=sha256:d5b054862739d276e09928de37c79ddeec42a6e1bfc55863be96a36ba22926f6 \ - --hash=sha256:dbe03226baf438ac4fda9e2d0715022fd579cb641c4cf639fa40d53b2fe6f3e2 \ - --hash=sha256:dc15e99b2d8a656f8e666854404f1ba54765871104e50c8e9813af8a7db07f12 \ - --hash=sha256:dcaf7c1524c0542ee2fc82cc8ec337f7a9f7edee2532421ab200d2b920fc97cf \ - --hash=sha256:dd4eda173a9fcccb5f2e2bd2a9f423d180194b1bf17cf59e3269899235b2a114 \ - --hash=sha256:dd9a8bd8900e65504a305bf8ae6fa9fbc66de94178c420791d0293702fce2df7 \ - --hash=sha256:de7376c29d95d6719048c194a9cf1a1b0393fbe8488a22008610b0361d834ecf \ - --hash=sha256:e7fdd52961feb4c96507aa649550ec2a0d527c086d284749b2f582f2d40a2e0d \ - --hash=sha256:e91f541a85298cf35433bf66f3fab2a4a2cff05c127eeca4af174f6d497f0d4b \ - --hash=sha256:e9e3c4c9e1ed40ea53acf11e2a386383c3304212c965773704e4603d589343ed \ - --hash=sha256:ee803480535c44e7f5ad00788526da7d85525cfefaf8acf8ab9a310000be4b03 \ - --hash=sha256:f09cb5a7bbe1ecae6e87901a2eb23e0256bb524a79ccc53eb0b7629fbe7677c4 \ - --hash=sha256:f19c1585933c82098c2a520f8ec1227f20e339e33aca8fa6f956f6691b784e67 \ - --hash=sha256:f1a2f519ae173b5b6a2c9d5fa3116ce16e48b3462c8b96dfdded11055e3d6365 \ - --hash=sha256:f28f891ccd15c514a0981f3b9db9aa23d62fe1a99997512b0491d2ed323d229a \ - --hash=sha256:f3e73a4255342d4eb26ef6df01e3962e73aa29baa3124a8e824c5d3364a65748 \ - --hash=sha256:f606a1881d2663630ea5b8ce2efe2111740df4b687bd78b34a8131baa007f79b \ - --hash=sha256:fe9f97feb71aa9896b81973a7bbada8c49501dc73e58a10fcef6663af95e5079 \ - --hash=sha256:ffc519621dce0c767e96b9c53f09c5d215578e10b02c285809f76509a3931482 - # via requests -colorama==0.4.6 ; sys_platform == 'win32' \ - --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \ - --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6 - # via sphinx -docutils==0.21.2 \ - --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ - --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2 - # via - # myst-parser - # sphinx - # sphinx-rtd-theme -idna==3.10 \ - --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ - --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3 - # via requests -imagesize==1.4.1 \ - --hash=sha256:0d8d18d08f840c19d0ee7ca1fd82490fdc3729b7ac93f49870406ddde8ef8d8b \ - --hash=sha256:69150444affb9cb0d5cc5a92b3676f0b2fb7cd9ae39e947a5e11a36b4497cd4a - # via sphinx -jinja2==3.1.4 \ - --hash=sha256:4a3aee7acbbe7303aede8e9648d13b8bf88a429282aa6122a993f0ac800cb369 \ - --hash=sha256:bc5dd2abb727a5319567b7a813e6a2e7318c39f4f487cfe6c89c6f9c7d25197d - # via - # myst-parser - # readthedocs-sphinx-ext - # sphinx -markdown-it-py==3.0.0 \ - --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ - --hash=sha256:e3f60a94fa066dc52ec76661e37c851cb232d92f9886b15cb560aaada2df8feb - # via - # mdit-py-plugins - # myst-parser -markupsafe==3.0.2 \ - --hash=sha256:0bff5e0ae4ef2e1ae4fdf2dfd5b76c75e5c2fa4132d05fc1b0dabcd20c7e28c4 \ - --hash=sha256:0f4ca02bea9a23221c0182836703cbf8930c5e9454bacce27e767509fa286a30 \ - --hash=sha256:1225beacc926f536dc82e45f8a4d68502949dc67eea90eab715dea3a21c1b5f0 \ - --hash=sha256:131a3c7689c85f5ad20f9f6fb1b866f402c445b220c19fe4308c0b147ccd2ad9 \ - --hash=sha256:15ab75ef81add55874e7ab7055e9c397312385bd9ced94920f2802310c930396 \ - --hash=sha256:1a9d3f5f0901fdec14d8d2f66ef7d035f2157240a433441719ac9a3fba440b13 \ - --hash=sha256:1c99d261bd2d5f6b59325c92c73df481e05e57f19837bdca8413b9eac4bd8028 \ - --hash=sha256:1e084f686b92e5b83186b07e8a17fc09e38fff551f3602b249881fec658d3eca \ - --hash=sha256:2181e67807fc2fa785d0592dc2d6206c019b9502410671cc905d132a92866557 \ - --hash=sha256:2cb8438c3cbb25e220c2ab33bb226559e7afb3baec11c4f218ffa7308603c832 \ - --hash=sha256:3169b1eefae027567d1ce6ee7cae382c57fe26e82775f460f0b2778beaad66c0 \ - --hash=sha256:3809ede931876f5b2ec92eef964286840ed3540dadf803dd570c3b7e13141a3b \ - --hash=sha256:38a9ef736c01fccdd6600705b09dc574584b89bea478200c5fbf112a6b0d5579 \ - --hash=sha256:3d79d162e7be8f996986c064d1c7c817f6df3a77fe3d6859f6f9e7be4b8c213a \ - --hash=sha256:444dcda765c8a838eaae23112db52f1efaf750daddb2d9ca300bcae1039adc5c \ - --hash=sha256:48032821bbdf20f5799ff537c7ac3d1fba0ba032cfc06194faffa8cda8b560ff \ - --hash=sha256:4aa4e5faecf353ed117801a068ebab7b7e09ffb6e1d5e412dc852e0da018126c \ - --hash=sha256:52305740fe773d09cffb16f8ed0427942901f00adedac82ec8b67752f58a1b22 \ - --hash=sha256:569511d3b58c8791ab4c2e1285575265991e6d8f8700c7be0e88f86cb0672094 \ - --hash=sha256:57cb5a3cf367aeb1d316576250f65edec5bb3be939e9247ae594b4bcbc317dfb \ - --hash=sha256:5b02fb34468b6aaa40dfc198d813a641e3a63b98c2b05a16b9f80b7ec314185e \ - --hash=sha256:6381026f158fdb7c72a168278597a5e3a5222e83ea18f543112b2662a9b699c5 \ - --hash=sha256:6af100e168aa82a50e186c82875a5893c5597a0c1ccdb0d8b40240b1f28b969a \ - --hash=sha256:6c89876f41da747c8d3677a2b540fb32ef5715f97b66eeb0c6b66f5e3ef6f59d \ - --hash=sha256:6e296a513ca3d94054c2c881cc913116e90fd030ad1c656b3869762b754f5f8a \ - --hash=sha256:70a87b411535ccad5ef2f1df5136506a10775d267e197e4cf531ced10537bd6b \ - --hash=sha256:7e94c425039cde14257288fd61dcfb01963e658efbc0ff54f5306b06054700f8 \ - --hash=sha256:846ade7b71e3536c4e56b386c2a47adf5741d2d8b94ec9dc3e92e5e1ee1e2225 \ - --hash=sha256:88416bd1e65dcea10bc7569faacb2c20ce071dd1f87539ca2ab364bf6231393c \ - --hash=sha256:88b49a3b9ff31e19998750c38e030fc7bb937398b1f78cfa599aaef92d693144 \ - --hash=sha256:8c4e8c3ce11e1f92f6536ff07154f9d49677ebaaafc32db9db4620bc11ed480f \ - --hash=sha256:8e06879fc22a25ca47312fbe7c8264eb0b662f6db27cb2d3bbbc74b1df4b9b87 \ - --hash=sha256:9025b4018f3a1314059769c7bf15441064b2207cb3f065e6ea1e7359cb46db9d \ - --hash=sha256:93335ca3812df2f366e80509ae119189886b0f3c2b81325d39efdb84a1e2ae93 \ - --hash=sha256:9778bd8ab0a994ebf6f84c2b949e65736d5575320a17ae8984a77fab08db94cf \ - --hash=sha256:9e2d922824181480953426608b81967de705c3cef4d1af983af849d7bd619158 \ - --hash=sha256:a123e330ef0853c6e822384873bef7507557d8e4a082961e1defa947aa59ba84 \ - --hash=sha256:a904af0a6162c73e3edcb969eeeb53a63ceeb5d8cf642fade7d39e7963a22ddb \ - --hash=sha256:ad10d3ded218f1039f11a75f8091880239651b52e9bb592ca27de44eed242a48 \ - --hash=sha256:b424c77b206d63d500bcb69fa55ed8d0e6a3774056bdc4839fc9298a7edca171 \ - --hash=sha256:b5a6b3ada725cea8a5e634536b1b01c30bcdcd7f9c6fff4151548d5bf6b3a36c \ - --hash=sha256:ba8062ed2cf21c07a9e295d5b8a2a5ce678b913b45fdf68c32d95d6c1291e0b6 \ - --hash=sha256:ba9527cdd4c926ed0760bc301f6728ef34d841f405abf9d4f959c478421e4efd \ - --hash=sha256:bbcb445fa71794da8f178f0f6d66789a28d7319071af7a496d4d507ed566270d \ - --hash=sha256:bcf3e58998965654fdaff38e58584d8937aa3096ab5354d493c77d1fdd66d7a1 \ - --hash=sha256:c0ef13eaeee5b615fb07c9a7dadb38eac06a0608b41570d8ade51c56539e509d \ - --hash=sha256:cabc348d87e913db6ab4aa100f01b08f481097838bdddf7c7a84b7575b7309ca \ - --hash=sha256:cdb82a876c47801bb54a690c5ae105a46b392ac6099881cdfb9f6e95e4014c6a \ - --hash=sha256:cfad01eed2c2e0c01fd0ecd2ef42c492f7f93902e39a42fc9ee1692961443a29 \ - --hash=sha256:d16a81a06776313e817c951135cf7340a3e91e8c1ff2fac444cfd75fffa04afe \ - --hash=sha256:d8213e09c917a951de9d09ecee036d5c7d36cb6cb7dbaece4c71a60d79fb9798 \ - --hash=sha256:e07c3764494e3776c602c1e78e298937c3315ccc9043ead7e685b7f2b8d47b3c \ - --hash=sha256:e17c96c14e19278594aa4841ec148115f9c7615a47382ecb6b82bd8fea3ab0c8 \ - --hash=sha256:e444a31f8db13eb18ada366ab3cf45fd4b31e4db1236a4448f68778c1d1a5a2f \ - --hash=sha256:e6a2a455bd412959b57a172ce6328d2dd1f01cb2135efda2e4576e8a23fa3b0f \ - --hash=sha256:eaa0a10b7f72326f1372a713e73c3f739b524b3af41feb43e4921cb529f5929a \ - --hash=sha256:eb7972a85c54febfb25b5c4b4f3af4dcc731994c7da0d8a0b4a6eb0640e1d178 \ - --hash=sha256:ee55d3edf80167e48ea11a923c7386f4669df67d7994554387f84e7d8b0a2bf0 \ - --hash=sha256:f3818cb119498c0678015754eba762e0d61e5b52d34c8b13d770f0719f7b1d79 \ - --hash=sha256:f8b3d067f2e40fe93e1ccdd6b2e1d16c43140e76f02fb1319a05cf2b79d99430 \ - --hash=sha256:fcabf5ff6eea076f859677f5f0b6b5c1a51e70a376b0579e0eadef8db48c6b50 - # via jinja2 -mdit-py-plugins==0.4.2 \ - --hash=sha256:0c673c3f889399a33b95e88d2f0d111b4447bdfea7f237dab2d488f459835636 \ - --hash=sha256:5f2cd1fdb606ddf152d37ec30e46101a60512bc0e5fa1a7002c36647b09e26b5 - # via myst-parser -mdurl==0.1.2 \ - --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ - --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba - # via markdown-it-py -myst-parser==4.0.0 \ - --hash=sha256:851c9dfb44e36e56d15d05e72f02b80da21a9e0d07cba96baf5e2d476bb91531 \ - --hash=sha256:b9317997552424448c6096c2558872fdb6f81d3ecb3a40ce84a7518798f3f28d - # via rules-python-docs (docs/pyproject.toml) -packaging==24.1 \ - --hash=sha256:026ed72c8ed3fcce5bf8950572258698927fd1dbda10a5e981cdf0ac37f4f002 \ - --hash=sha256:5b8f2217dbdbd2f7f384c41c628544e6d52f2d0f53c6d0c3ea61aa5d1d7ff124 - # via - # readthedocs-sphinx-ext - # sphinx -pygments==2.18.0 \ - --hash=sha256:786ff802f32e91311bff3889f6e9a86e81505fe99f2735bb6d60ae0c5004f199 \ - --hash=sha256:b8e6aca0523f3ab76fee51799c488e38782ac06eafcf95e7ba832985c8e7b13a - # via sphinx -pyyaml==6.0.2 \ - --hash=sha256:01179a4a8559ab5de078078f37e5c1a30d76bb88519906844fd7bdea1b7729ff \ - --hash=sha256:0833f8694549e586547b576dcfaba4a6b55b9e96098b36cdc7ebefe667dfed48 \ - --hash=sha256:0a9a2848a5b7feac301353437eb7d5957887edbf81d56e903999a75a3d743086 \ - --hash=sha256:0b69e4ce7a131fe56b7e4d770c67429700908fc0752af059838b1cfb41960e4e \ - --hash=sha256:0ffe8360bab4910ef1b9e87fb812d8bc0a308b0d0eef8c8f44e0254ab3b07133 \ - --hash=sha256:11d8f3dd2b9c1207dcaf2ee0bbbfd5991f571186ec9cc78427ba5bd32afae4b5 \ - --hash=sha256:17e311b6c678207928d649faa7cb0d7b4c26a0ba73d41e99c4fff6b6c3276484 \ - --hash=sha256:1e2120ef853f59c7419231f3bf4e7021f1b936f6ebd222406c3b60212205d2ee \ - --hash=sha256:1f71ea527786de97d1a0cc0eacd1defc0985dcf6b3f17bb77dcfc8c34bec4dc5 \ - --hash=sha256:23502f431948090f597378482b4812b0caae32c22213aecf3b55325e049a6c68 \ - --hash=sha256:24471b829b3bf607e04e88d79542a9d48bb037c2267d7927a874e6c205ca7e9a \ - --hash=sha256:29717114e51c84ddfba879543fb232a6ed60086602313ca38cce623c1d62cfbf \ - --hash=sha256:2e99c6826ffa974fe6e27cdb5ed0021786b03fc98e5ee3c5bfe1fd5015f42b99 \ - --hash=sha256:39693e1f8320ae4f43943590b49779ffb98acb81f788220ea932a6b6c51004d8 \ - --hash=sha256:3ad2a3decf9aaba3d29c8f537ac4b243e36bef957511b4766cb0057d32b0be85 \ - --hash=sha256:3b1fdb9dc17f5a7677423d508ab4f243a726dea51fa5e70992e59a7411c89d19 \ - --hash=sha256:41e4e3953a79407c794916fa277a82531dd93aad34e29c2a514c2c0c5fe971cc \ - --hash=sha256:43fa96a3ca0d6b1812e01ced1044a003533c47f6ee8aca31724f78e93ccc089a \ - --hash=sha256:50187695423ffe49e2deacb8cd10510bc361faac997de9efef88badc3bb9e2d1 \ - --hash=sha256:5ac9328ec4831237bec75defaf839f7d4564be1e6b25ac710bd1a96321cc8317 \ - --hash=sha256:5d225db5a45f21e78dd9358e58a98702a0302f2659a3c6cd320564b75b86f47c \ - --hash=sha256:6395c297d42274772abc367baaa79683958044e5d3835486c16da75d2a694631 \ - --hash=sha256:688ba32a1cffef67fd2e9398a2efebaea461578b0923624778664cc1c914db5d \ - --hash=sha256:68ccc6023a3400877818152ad9a1033e3db8625d899c72eacb5a668902e4d652 \ - --hash=sha256:70b189594dbe54f75ab3a1acec5f1e3faa7e8cf2f1e08d9b561cb41b845f69d5 \ - --hash=sha256:797b4f722ffa07cc8d62053e4cff1486fa6dc094105d13fea7b1de7d8bf71c9e \ - --hash=sha256:7c36280e6fb8385e520936c3cb3b8042851904eba0e58d277dca80a5cfed590b \ - --hash=sha256:7e7401d0de89a9a855c839bc697c079a4af81cf878373abd7dc625847d25cbd8 \ - --hash=sha256:80bab7bfc629882493af4aa31a4cfa43a4c57c83813253626916b8c7ada83476 \ - --hash=sha256:82d09873e40955485746739bcb8b4586983670466c23382c19cffecbf1fd8706 \ - --hash=sha256:8388ee1976c416731879ac16da0aff3f63b286ffdd57cdeb95f3f2e085687563 \ - --hash=sha256:8824b5a04a04a047e72eea5cec3bc266db09e35de6bdfe34c9436ac5ee27d237 \ - --hash=sha256:8b9c7197f7cb2738065c481a0461e50ad02f18c78cd75775628afb4d7137fb3b \ - --hash=sha256:9056c1ecd25795207ad294bcf39f2db3d845767be0ea6e6a34d856f006006083 \ - --hash=sha256:936d68689298c36b53b29f23c6dbb74de12b4ac12ca6cfe0e047bedceea56180 \ - --hash=sha256:9b22676e8097e9e22e36d6b7bda33190d0d400f345f23d4065d48f4ca7ae0425 \ - --hash=sha256:a4d3091415f010369ae4ed1fc6b79def9416358877534caf6a0fdd2146c87a3e \ - --hash=sha256:a8786accb172bd8afb8be14490a16625cbc387036876ab6ba70912730faf8e1f \ - --hash=sha256:a9f8c2e67970f13b16084e04f134610fd1d374bf477b17ec1599185cf611d725 \ - --hash=sha256:bc2fa7c6b47d6bc618dd7fb02ef6fdedb1090ec036abab80d4681424b84c1183 \ - --hash=sha256:c70c95198c015b85feafc136515252a261a84561b7b1d51e3384e0655ddf25ab \ - --hash=sha256:cc1c1159b3d456576af7a3e4d1ba7e6924cb39de8f67111c735f6fc832082774 \ - --hash=sha256:ce826d6ef20b1bc864f0a68340c8b3287705cae2f8b4b1d932177dcc76721725 \ - --hash=sha256:d584d9ec91ad65861cc08d42e834324ef890a082e591037abe114850ff7bbc3e \ - --hash=sha256:d7fded462629cfa4b685c5416b949ebad6cec74af5e2d42905d41e257e0869f5 \ - --hash=sha256:d84a1718ee396f54f3a086ea0a66d8e552b2ab2017ef8b420e92edbc841c352d \ - --hash=sha256:d8e03406cac8513435335dbab54c0d385e4a49e4945d2909a581c83647ca0290 \ - --hash=sha256:e10ce637b18caea04431ce14fabcf5c64a1c61ec9c56b071a4b7ca131ca52d44 \ - --hash=sha256:ec031d5d2feb36d1d1a24380e4db6d43695f3748343d99434e6f5f9156aaa2ed \ - --hash=sha256:ef6107725bd54b262d6dedcc2af448a266975032bc85ef0172c5f059da6325b4 \ - --hash=sha256:efdca5630322a10774e8e98e1af481aad470dd62c3170801852d752aa7a783ba \ - --hash=sha256:f753120cb8181e736c57ef7636e83f31b9c0d1722c516f7e86cf15b7aa57ff12 \ - --hash=sha256:ff3824dc5261f50c9b0dfb3be22b4567a6f938ccce4587b38952d85fd9e9afe4 - # via myst-parser -readthedocs-sphinx-ext==2.2.5 \ - --hash=sha256:ee5fd5b99db9f0c180b2396cbce528aa36671951b9526bb0272dbfce5517bd27 \ - --hash=sha256:f8c56184ea011c972dd45a90122568587cc85b0127bc9cf064d17c68bc809daa - # via rules-python-docs (docs/pyproject.toml) -requests==2.32.3 \ - --hash=sha256:55365417734eb18255590a9ff9eb97e9e1da868d4ccd6402399eaf68af20a760 \ - --hash=sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6 - # via - # readthedocs-sphinx-ext - # sphinx -snowballstemmer==2.2.0 \ - --hash=sha256:09b16deb8547d3412ad7b590689584cd0fe25ec8db3be37788be3810cbf19cb1 \ - --hash=sha256:c8e1716e83cc398ae16824e5572ae04e0d9fc2c6b985fb0f900f5f0c96ecba1a - # via sphinx -sphinx==8.1.3 \ - --hash=sha256:09719015511837b76bf6e03e42eb7595ac8c2e41eeb9c29c5b755c6b677992a2 \ - --hash=sha256:43c1911eecb0d3e161ad78611bc905d1ad0e523e4ddc202a58a821773dc4c927 - # via - # rules-python-docs (docs/pyproject.toml) - # myst-parser - # sphinx-reredirects - # sphinx-rtd-theme - # sphinxcontrib-jquery -sphinx-autodoc2==0.5.0 \ - --hash=sha256:7d76044aa81d6af74447080182b6868c7eb066874edc835e8ddf810735b6565a \ - --hash=sha256:e867013b1512f9d6d7e6f6799f8b537d6884462acd118ef361f3f619a60b5c9e - # via rules-python-docs (docs/pyproject.toml) -sphinx-reredirects==0.1.5 \ - --hash=sha256:444ae1438fba4418242ca76d6a6de3eaee82aaf0d8f2b0cac71a15d32ce6eba2 \ - --hash=sha256:cfa753b441020a22708ce8eb17d4fd553a28fc87a609330092917ada2a6da0d8 - # via rules-python-docs (docs/pyproject.toml) -sphinx-rtd-theme==3.0.1 \ - --hash=sha256:921c0ece75e90633ee876bd7b148cfaad136b481907ad154ac3669b6fc957916 \ - --hash=sha256:a4c5745d1b06dfcb80b7704fe532eb765b44065a8fad9851e4258c8804140703 - # via rules-python-docs (docs/pyproject.toml) -sphinxcontrib-applehelp==2.0.0 \ - --hash=sha256:2f29ef331735ce958efa4734873f084941970894c6090408b079c61b2e1c06d1 \ - --hash=sha256:4cd3f0ec4ac5dd9c17ec65e9ab272c9b867ea77425228e68ecf08d6b28ddbdb5 - # via sphinx -sphinxcontrib-devhelp==2.0.0 \ - --hash=sha256:411f5d96d445d1d73bb5d52133377b4248ec79db5c793ce7dbe59e074b4dd1ad \ - --hash=sha256:aefb8b83854e4b0998877524d1029fd3e6879210422ee3780459e28a1f03a8a2 - # via sphinx -sphinxcontrib-htmlhelp==2.1.0 \ - --hash=sha256:166759820b47002d22914d64a075ce08f4c46818e17cfc9470a9786b759b19f8 \ - --hash=sha256:c9e2916ace8aad64cc13a0d233ee22317f2b9025b9cf3295249fa985cc7082e9 - # via sphinx -sphinxcontrib-jquery==4.1 \ - --hash=sha256:1620739f04e36a2c779f1a131a2dfd49b2fd07351bf1968ced074365933abc7a \ - --hash=sha256:f936030d7d0147dd026a4f2b5a57343d233f1fc7b363f68b3d4f1cb0993878ae - # via sphinx-rtd-theme -sphinxcontrib-jsmath==1.0.1 \ - --hash=sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178 \ - --hash=sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8 - # via sphinx -sphinxcontrib-qthelp==2.0.0 \ - --hash=sha256:4fe7d0ac8fc171045be623aba3e2a8f613f8682731f9153bb2e40ece16b9bbab \ - --hash=sha256:b18a828cdba941ccd6ee8445dbe72ffa3ef8cbe7505d8cd1fa0d42d3f2d5f3eb - # via sphinx -sphinxcontrib-serializinghtml==2.0.0 \ - --hash=sha256:6e2cb0eef194e10c27ec0023bfeb25badbbb5868244cf5bc5bdc04e4464bf331 \ - --hash=sha256:e9d912827f872c029017a53f0ef2180b327c3f7fd23c87229f7a8e8b70031d4d - # via sphinx -typing-extensions==4.12.2 \ - --hash=sha256:04e5ca0351e0f3f85c6853954072df659d0d13fac324d0072316b67d7794700d \ - --hash=sha256:1a7ead55c7e559dd4dee8856e3a88b41225abfe1ce8df57b7c13915fe121ffb8 - # via - # rules-python-docs (docs/pyproject.toml) - # sphinx-autodoc2 -urllib3==2.2.3 \ - --hash=sha256:ca899ca043dcb1bafa3e262d73aa25c465bfb49e0bd9dd5d59f1d0acba2f8fac \ - --hash=sha256:e7d814a81dad81e6caf2ec9fdedb284ecc9c73076b62654547cc64ccdcae26e9 - # via requests diff --git a/docs/support.md b/docs/support.md index ea099650bd..088466d1bb 100644 --- a/docs/support.md +++ b/docs/support.md @@ -1,19 +1,35 @@ # Support Policy -The Bazel community maintains this repository. Neither Google nor the Bazel team -provides support for the code. However, this repository is part of the test -suite used to vet new Bazel releases. See the -page for information on our development workflow. +This is a community maintained project run by volunteers. What that means in +practice is: + +* Responses to issues and PRs depend on available time and energy. +* If you care, contribute (see ) and get involved. ## Supported rules_python Versions In general, only the latest version is supported. Backporting changes is -done on a best effort basis based on severity, risk of regressions, and +done on a best-effort basis based on severity, risk of regressions, and the willingness of volunteers. If you want or need particular functionality backported, then the best way is to open a PR to demonstrate the feasibility of the backport. +### Backports and Patch Releases + +Backports and patch releases are provided on a best-effort basis. Only fixes are +backported. Features are not backported. + +Backports can be done to older releases, but only if newer releases also have +the fix backported. For example, if the current release is 1.5, in order to +patch 1.4, version 1.5 must be patched first. + +Backports can be requested by [creating an issue with the patch release +template][patch-release-issue] or by sending a pull request performing the backport. +See the dev guide for [how to create a backport PR](creating-backport-prs). + +[patch-release-issue]: https://github.com/bazelbuild/rules_python/issues/new?template=patch_release_request.md + ## Supported Bazel Versions The supported Bazel versions are: @@ -31,11 +47,36 @@ minor/patch versions. See [Bazel's release support matrix](https://bazel.build/release#support-matrix) for what versions are the rolling, active, and prior releases. +## Supported Python versions + +As a general rule, we test all released non-EOL Python versions. Different +interpreter versions may work but are not guaranteed. We are interested in +staying compatible with upcoming unreleased versions, so if you see that things +stop working, please create tickets or, more preferably, pull requests. + ## Supported Platforms -We only support the platforms that our continuous integration jobs run, which -is Linux, Mac, and Windows. Code to support other platforms is allowed, but -can only be on a best-effort basis. +We only support the platforms that our continuous integration jobs run on, which +are Linux, Mac, and Windows. + +In order to better describe different support levels, the following acts as a rough +guideline for different platform tiers: +* Tier 0 - The platforms that our CI runs on: `linux_x86_64`, `osx_arm64`, `RBE linux_x86_64`. +* Tier 1 - The platforms that are similar enough to what the CI runs on: `linux_aarch64`, + `osx_x86_64`. + What is more, `windows_x86_64` is in this list, as we run tests in CI, but + developing for Windows is more challenging, and features may come later to + this platform. +* Tier 2 - The rest of the platforms that may have a varying level of support, e.g., + `linux_s390x`, `linux_ppc64le`, `windows_arm64`. + +:::{note} +Code to support Tier 2 platforms is allowed, but regressions will be fixed on a +best-effort basis, so feel free to contribute by creating PRs. + +If you would like to provide/sponsor CI setup for a platform that is not Tier 0, +please create a ticket or contact the maintainers on Slack. +::: ## Compatibility Policy @@ -51,7 +92,7 @@ a series of releases to so users can still incrementally upgrade. See the ## Experimental Features -An experimental features is functionality that may not be ready for general +An experimental feature is functionality that may not be ready for general use and may change quickly and/or significantly. Such features are denoted in their name or API docs as "experimental". They may have breaking changes made at any time. diff --git a/docs/toolchains.md b/docs/toolchains.md index d6c59544a8..f154fc29ed 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -1,15 +1,16 @@ :::{default-domain} bzl ::: +(configuring-toolchains)= # Configuring Python toolchains and runtimes -This documents how to configure the Python toolchain and runtimes for different +This document explains how to configure the Python toolchain and runtimes for different use cases. ## Bzlmod MODULE configuration -How to configure `rules_python` in your MODULE.bazel file depends on how and why -you're using Python. There are 4 basic use cases: +How to configure `rules_python` in your `MODULE.bazel` file depends on how and why +you're using Python. There are four basic use cases: 1. A root module that always uses Python. For example, you're building a Python application. @@ -43,13 +44,14 @@ you should read the dev-only library module section. bazel_dep(name="rules_python", version=...) python = use_extension("@rules_python//python/extensions:python.bzl", "python") -python.toolchain(python_version = "3.12", is_default = True) +python.defaults(python_version = "3.12") +python.toolchain(python_version = "3.12") ``` ### Library modules A library module is a module that can show up in arbitrary locations in the -bzlmod module graph -- it's unknown where in the breadth-first search order the +Bzlmod module graph -- it's unknown where in the breadth-first search order the module will be relative to other modules. For example, `rules_python` is a library module. @@ -71,7 +73,8 @@ python = use_extension( dev_dependency = True ) -python.toolchain(python_version = "3.12", is_default=True) +python.defaults(python_version = "3.12") +python.toolchain(python_version = "3.12") ``` #### Library modules without version constraints @@ -81,9 +84,9 @@ used for the Python programs it runs isn't chosen by the module itself. Instead, it's up to the root module to pick an appropriate version of Python. For this case, configuration is simple: just depend on `rules_python` and use -the normal `//python:py_binary.bzl` et al rules. There is no need to call -`python.toolchain` -- rules_python ensures _some_ Python version is available, -but more often the root module will specify some version. +the normal `//python:py_binary.bzl` et al. rules. There is no need to call +`python.toolchain` -- `rules_python` ensures _some_ Python version is available, +but more often, the root module will specify some version. ``` # MODULE.bazel @@ -105,7 +108,7 @@ specific Python version be used with its tools. This has some pros/cons: * It has higher build overhead because additional runtimes and libraries need to be downloaded, and Bazel has to keep additional configuration state. -To configure this, request the Python versions needed in MODULE.bazel and use +To configure this, request the Python versions needed in `MODULE.bazel` and use the version-aware rules for `py_binary`. ``` @@ -116,9 +119,9 @@ python = use_extension("@rules_python//python/extensions:python.bzl", "python") python.toolchain(python_version = "3.12") # BUILD.bazel -load("@python_versions//3.12:defs.bzl", "py_binary") +load("@rules_python//python:py_binary.bzl", "py_binary") -py_binary(...) +py_binary(..., python_version="3.12") ``` ### Pinning to a Python version @@ -129,24 +132,66 @@ is most useful for two cases: 1. For submodules to ensure they run with the appropriate Python version 2. To allow incremental, per-target, upgrading to newer Python versions, - typically in a mono-repo situation. + typically in a monorepo situation. To configure a submodule with the version-aware rules, request the particular -version you need, then use the `@python_versions` repo to use the rules that -force specific versions: +version you need when defining the toolchain: ```starlark +# MODULE.bazel python = use_extension("@rules_python//python/extensions:python.bzl", "python") python.toolchain( python_version = "3.11", ) -use_repo(python, "python_versions") +use_repo(python) +``` + +Then use the `@rules_python` repo in your `BUILD` file to explicitly pin the Python version when calling the rule: + +```starlark +# BUILD.bazel +load("@rules_python//python:py_binary.bzl", "py_binary") + +py_binary(..., python_version = "3.11") +py_test(..., python_version = "3.11") ``` -Then use e.g. `load("@python_versions//3.11:defs.bzl", "py_binary")` to use -the rules that force that particular version. Multiple versions can be specified -and use within a single build. +Multiple versions can be specified and used within a single build. + +```starlark +# MODULE.bazel +python = use_extension("@rules_python//python/extensions:python.bzl", "python") + +python.defaults( + # The environment variable takes precedence if set. + python_version = "3.11", + python_version_env = "BAZEL_PYTHON_VERSION", +) +python.toolchain( + python_version = "3.11", +) + +python.toolchain( + python_version = "3.12", +) + +# BUILD.bazel +load("@rules_python//python:py_binary.bzl", "py_binary") +load("@rules_python//python:py_test.bzl", "py_test") + +# Defaults to 3.11 +py_binary(...) +py_test(...) + +# Explicitly use Python 3.11 +py_binary(..., python_version = "3.11") +py_test(..., python_version = "3.11") + +# Explicitly use Python 3.12 +py_binary(..., python_version = "3.12") +py_test(..., python_version = "3.12") +``` For more documentation, see the bzlmod examples under the {gh-path}`examples` folder. Look for the examples that contain a `MODULE.bazel` file. @@ -157,17 +202,31 @@ The `python.toolchain()` call makes its contents available under a repo named `python_X_Y`, where X and Y are the major and minor versions. For example, `python.toolchain(python_version="3.11")` creates the repo `@python_3_11`. Remember to call `use_repo()` to make repos visible to your module: -`use_repo(python, "python_3_11")` +`use_repo(python, "python_3_11")`. + + +:::{deprecated} 1.1.0 +The toolchain-specific `py_binary` and `py_test` symbols are aliases to the regular rules. +For example, `load("@python_versions//3.11:defs.bzl", "py_binary")` & `load("@python_versions//3.11:defs.bzl", "py_test")` are deprecated. + +Usages of them should be changed to load the regular rules directly. +For example, use `load("@rules_python//python:py_binary.bzl", "py_binary")` & `load("@rules_python//python:py_test.bzl", "py_test")` and then specify the `python_version` when using the rules corresponding to the Python version you defined in your toolchain. {ref}`Library modules with version constraints` +::: + #### Toolchain usage in other rules -Python toolchains can be utilized in other bazel rules, such as `genrule()`, by +Python toolchains can be utilized in other Bazel rules, such as `genrule()`, by adding the `toolchains=["@rules_python//python:current_py_toolchain"]` attribute. You can obtain the path to the Python interpreter using the `$(PYTHON2)` and `$(PYTHON3)` ["Make" Variables](https://bazel.build/reference/be/make-variables). See the {gh-path}`test_current_py_toolchain ` target -for an example. +for an example. We also make available `$(PYTHON2_ROOTPATH)` and `$(PYTHON3_ROOTPATH)`, +which are Make Variable equivalents of `$(PYTHON2)` and `$(PYTHON3)` but for runfiles +locations. These will be helpful if you need to set environment variables of binary/test rules +while using [`--nolegacy_external_runfiles`](https://bazel.build/reference/command-line-reference#flag--legacy_external_runfiles). +The original make variables still work in exec contexts such as genrules. ### Overriding toolchain defaults and adding more versions @@ -183,12 +242,187 @@ existing attributes: {attr}`python.single_version_platform_override.coverage_tool`. * Adding additional Python versions via {bzl:obj}`python.single_version_override` or {bzl:obj}`python.single_version_platform_override`. +* Adding additional Python versions dynamically from a manifest file or URL + via {attr}`python.override.add_runtime_manifest_files` or + {attr}`python.override.add_runtime_manifest_urls`. + +### Registering custom runtimes + +Because the python-build-standalone project has _thousands_ of prebuilt runtimes +available, `rules_python` only includes popular runtimes in its built-in +configurations. If you want to use a runtime that isn't already known to +`rules_python`, then {obj}`single_version_platform_override()` can be used to do +so. In short, it allows specifying an arbitrary URL and using custom flags +to control when a runtime is used. + +In the example below, we register a particular python-build-standalone runtime +that is activated for Linux x86 builds when the custom flag +`--//:runtime=my-custom-runtime` is set. + +``` +# File: MODULE.bazel +bazel_dep(name = "bazel_skylib", version = "1.7.1.") +bazel_dep(name = "rules_python", version = "1.5.0") +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.single_version_platform_override( + platform = "my-platform", + python_version = "3.13.3", + sha256 = "01d08b9bc8a96698b9d64c2fc26da4ecc4fa9e708ce0a34fb88f11ab7e552cbd", + os_name = "linux", + arch = "x86_64", + target_settings = [ + "@@//:runtime=my-custom-runtime", + ], + urls = ["https://github.com/astral-sh/python-build-standalone/releases/download/20250409/cpython-3.13.3+20250409-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz"], +) +# File: //:BUILD.bazel +load("@bazel_skylib//rules:common_settings.bzl", "string_flag") +string_flag( + name = "custom_runtime", + build_setting_default = "", +) +config_setting( + name = "is_custom_runtime_linux-x86-install-only-stripped", + flag_values = { + ":custom_runtime": "linux-x86-install-only-stripped", + }, +) +``` + +Notes: +- While any URL and archive can be used, it's assumed their content looks like + a python-build-standalone archive. +- A "version-aware" toolchain is registered, which means the Python version flag + must also match (e.g., `--@rules_python//python/config_settings:python_version=3.13.3` + must be set -- see `minor_mapping` and `is_default` for controls and docs + about version matching and selection). +- The `target_compatible_with` attribute can be used to entirely specify the + argument of the same name that the toolchain uses. +- The labels in `target_settings` must be absolute; `@@` refers to the main repo. +- The `target_settings` are `config_setting` targets, which means you can + customize how matching occurs. + +:::{seealso} +See {obj}`//python/config_settings` for flags `rules_python` already defines +that can be used with `target_settings`. Some particular ones of note are +{flag}`--py_linux_libc` and {flag}`--py_freethreaded`, among others. +::: + +:::{versionadded} 1.5.0 +Added support for custom platform names, `target_compatible_with`, and +`target_settings` with `single_version_platform_override`. +::: + +### Registering runtimes from a manifest + +If you want to register multiple custom runtimes or versions at once, you can +use a python-build-standalone manifest file. This is useful if you want to +adopt new versions that are not yet built into `rules_python` without having +to manually define each one using `single_version_platform_override`. + +To do this, specify the `add_runtime_manifest_files` or +`add_runtime_manifest_urls` (and `runtime_manifest_sha`) attributes in +`python.override` in your `MODULE.bazel`. + +In the example below, we register all runtimes available in a specific local +or remote PBS release manifest: + +```starlark +# File: MODULE.bazel +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.override( + add_runtime_manifest_files = [ + "@//:SHA256SUMS", + ], + add_runtime_manifest_urls = [ + "https://github.com/astral-sh/python-build-standalone/releases/download/20260414/SHA256SUMS", + ], + base_url = "https://example.com/downloads", + runtime_manifest_sha = "ce18fdfd47c66830a40ea9b9e314a14b1636bbfd684501bc5ca1fc6d55a7933f", +) +``` + +#### Manifest file format + +The manifest must be a plain text file where each line contains the SHA256 hash +and the location of a runtime archive, separated by whitespace: + +``` + +``` + +The `` can be either: +- A relative filename (e.g., + `cpython-3.10.20+20260414-x86_64-unknown-linux-gnu-install_only.tar.zst`). + In this case, the download URL is constructed by appending the filename to + the `base_url` attribute (if using `add_runtime_manifest_files`) or to the + parent directory of each URL in `add_runtime_manifest_urls` (treating them + as mirrors). +- An absolute URL (e.g., + `https://example.com/downloads/cpython-3.10.20+20260414-x86_64-unknown-linux-gnu-install_only.tar.zst`). + In this case, the URL is used directly to download the archive. + +In both cases, the filename or the last path segment of the URL must follow +the standard python-build-standalone naming convention. `rules_python` parses +this name to extract runtime metadata (such as Python version, target +architecture, operating system, and libc). + +Notes: +- `rules_python` will read or download the manifest, parse it, and + automatically register toolchains for all valid Python runtimes found in it + that match supported platforms. +- Only runtimes matching known platforms in `rules_python` will be registered. + +:::{versionadded} 2.1.0 +Added support for registering runtimes from a manifest using +`add_runtime_manifest_files`, `add_runtime_manifest_urls`, and +`runtime_manifest_sha` in `python.override`. +::: + + +### Using defined toolchains from WORKSPACE + +It is possible to use toolchains defined in `MODULE.bazel` in `WORKSPACE`. For example, +the following `MODULE.bazel` and `WORKSPACE` provides a working {bzl:obj}`pip_parse` setup: +```starlark +# File: WORKSPACE +load("@rules_python//python:repositories.bzl", "py_repositories") + +py_repositories() + +load("@rules_python//python:pip.bzl", "pip_parse") + +pip_parse( + name = "third_party", + requirements_lock = "//:requirements.txt", + python_interpreter_target = "@python_3_10_host//:python", +) + +load("@third_party//:requirements.bzl", "install_deps") + +install_deps() + +# File: MODULE.bazel +bazel_dep(name = "rules_python", version = "0.40.0") + +python = use_extension("@rules_python//python/extensions:python.bzl", "python") + +python.defaults(python_version = "3.10") +python.toolchain(python_version = "3.10") + +use_repo(python, "python_3_10", "python_3_10_host") +``` + +Note, the user has to import the `*_host` repository to use the Python interpreter in the +{bzl:obj}`pip_parse` and `whl_library` repository rules, and once that is done, +users should be able to ensure the setting of the default toolchain even during the +transition period when some of the code is still defined in `WORKSPACE`. ## Workspace configuration -To import rules_python in your project, you first need to add it to your +To import `rules_python` in your project, you first need to add it to your `WORKSPACE` file, using the snippet provided in the -[release you choose](https://github.com/bazelbuild/rules_python/releases) +[release you choose](https://github.com/bazel-contrib/rules_python/releases). To depend on a particular unreleased version, you can do the following: @@ -197,7 +431,7 @@ load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") # Update the SHA and VERSION to the lastest version available here: -# https://github.com/bazelbuild/rules_python/releases. +# https://github.com/bazel-contrib/rules_python/releases. SHA="84aec9e21cc56fbc7f1335035a71c850d1b9b5cc6ff497306f84cced9a769841" @@ -207,7 +441,7 @@ http_archive( name = "rules_python", sha256 = SHA, strip_prefix = "rules_python-{}".format(VERSION), - url = "https://github.com/bazelbuild/rules_python/releases/download/{}/rules_python-{}.tar.gz".format(VERSION,VERSION), + url = "https://github.com/bazel-contrib/rules_python/releases/download/{}/rules_python-{}.tar.gz".format(VERSION,VERSION), ) load("@rules_python//python:repositories.bzl", "py_repositories") @@ -229,45 +463,202 @@ python_register_toolchains( python_version = "3.11", ) -load("@python_3_11//:defs.bzl", "interpreter") - load("@rules_python//python:pip.bzl", "pip_parse") pip_parse( ... - python_interpreter_target = interpreter, + python_interpreter_target = "@python_3_11_host//:python", ... ) ``` After registration, your Python targets will use the toolchain's interpreter during execution, but a system-installed interpreter -is still used to 'bootstrap' Python targets (see https://github.com/bazelbuild/rules_python/issues/691). +is still used to "bootstrap" Python targets (see {gh-issue}`691`). You may also find some quirks while using this toolchain. Please refer to [python-build-standalone documentation's _Quirks_ section](https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html). -## Autodetecting toolchain +## Local toolchain + +It's possible to use a locally installed Python runtime instead of the regular +prebuilt, remotely downloaded ones. A local toolchain contains the Python +runtime metadata (Python version, headers, ABI flags, etc.) that the regular +remotely downloaded runtimes contain, which makes it possible to build, e.g., C +extensions (unlike the autodetecting and runtime environment toolchains). + +For simple cases, the {obj}`local_runtime_repo` and +{obj}`local_runtime_toolchains_repo` rules are provided that will introspect a +Python installation and create an appropriate Bazel definition from it. To do +this, three pieces need to be wired together: + +1. Specify a path or command to a Python interpreter (multiple can be defined). +2. Create toolchains for the runtimes in (1). +3. Register the toolchains created by (2). + +The following is an example that will use `python3` from `PATH` to find the +interpreter, then introspect its installation to generate a full toolchain. + +```starlark +# File: MODULE.bazel + +local_runtime_repo = use_repo_rule( + "@rules_python//python/local_toolchains:repos.bzl", + "local_runtime_repo", +) + +local_runtime_toolchains_repo = use_repo_rule( + "@rules_python//python/local_toolchains:repos.bzl", + "local_runtime_toolchains_repo", +) + +# Step 1: Define the Python runtime +local_runtime_repo( + name = "local_python3", + interpreter_path = "python3", + on_failure = "fail", + dev_dependency = True +) + +# Step 2: Create toolchains for the runtimes +local_runtime_toolchains_repo( + name = "local_toolchains", + runtimes = ["local_python3"], + # TIP: The `target_settings` arg can be used to activate them based on + # command line flags; see docs below. + dev_dependency = True +) + +# Step 3: Register the toolchains +register_toolchains("@local_toolchains//:all", dev_dependency = True) +``` + +In the example above, `interpreter_path` is used to find Python via `PATH` +lookups. Alternatively, {obj}`interpreter_target` can be set, which can +refer to a Python in an arbitrary Bazel repository. + +:::{important} +Be sure to set `dev_dependency = True`. Using a local toolchain only makes sense +for the root module. + +If an intermediate module does it, then the `register_toolchains()` call will +take precedence over the default rules_python toolchains and cause problems for +downstream modules. +::: + +Multiple runtimes and/or toolchains can be defined, which allows for multiple +Python versions and/or platforms to be configured in a single `MODULE.bazel`. +Note that `register_toolchains` will insert the local toolchain earlier in the +toolchain ordering, so it will take precedence over other registered toolchains. +To better control when the toolchain is used, see [Conditionally using local +toolchains]. + +### Conditionally using local toolchains + +By default, a local toolchain has few constraints and is early in the toolchain +ordering, which means it will usually be used no matter what. This can be +problematic for CI (where it shouldn't be used), expensive for CI (CI must +initialize/download the repository to determine its Python version), and +annoying for iterative development (enabling/disabling it requires modifying +`MODULE.bazel`). + +These behaviors can be mitigated, but it requires additional configuration +to avoid triggering the local toolchain repository to initialize (i.e., run +local commands and perform downloads). + +The two settings to change are +{obj}`local_runtime_toolchains_repo.target_compatible_with` and +{obj}`local_runtime_toolchains_repo.target_settings`, which control how Bazel +decides if a toolchain should match. By default, they point to targets *within* +the local runtime repository (triggering repo initialization). We have to override +them to *not* reference the local runtime repository at all. + +In the example below, we reconfigure the local toolchains so they are only +activated if the custom flag `--//:py=local` is set and the target platform +matches the Bazel host platform. The net effect is that CI won't use the local +toolchain (nor initialize its repository), and developers can easily +enable/disable the local toolchain with a command line flag. + +``` +# File: MODULE.bazel +bazel_dep(name = "bazel_skylib", version = "1.7.1") + +local_runtime_toolchains_repo( + name = "local_toolchains", + runtimes = ["local_python3"], + target_compatible_with = { + "local_python3": ["HOST_CONSTRAINTS"], + }, + target_settings = { + "local_python3": ["@//:is_py_local"] + } +) + +# File: BUILD.bazel +load("@bazel_skylib//rules:common_settings.bzl", "string_flag") + +config_setting( + name = "is_py_local", + flag_values = {":py": "local"}, +) + +string_flag( + name = "py", + build_setting_default = "", +) +``` + +:::{tip} +Easily switching between *multiple* local toolchains can be accomplished by +adding additional `:is_py_X` targets and setting `--//:py` to match. +to easily switch between different local toolchains. +::: + + +## Runtime environment toolchain + +The runtime environment toolchain is a minimal toolchain that doesn't provide +information about Python at build time. In particular, this means it is not able +to build C extensions -- doing so requires knowing, at build time, what Python +headers to use. + +In effect, all it does is generate a small wrapper script that simply calls, e.g., +`/usr/bin/env python3` to run a program. This makes it easy to change what +Python is used to run a program but also makes it easy to use a Python version +that isn't compatible with build-time assumptions. + +``` +`register_toolchains`("@rules_python//python/runtime_env_toolchains:all") +``` + +Note that this toolchain has no constraints, i.e. it will match any platform, +Python version, etc. + +:::{seealso} +[Local `toolchain`], which creates a more full featured toolchain from a +locally installed Python. +::: + +### Autodetecting toolchain The autodetecting toolchain is a deprecated toolchain that is built into Bazel. -It's name is a bit misleading: it doesn't autodetect anything. All it does is +**Its name is a bit misleading: it doesn't autodetect anything.** All it does is use `python3` from the environment a binary runs within. This provides extremely limited functionality to the rules (at build time, nothing is knowable about the Python runtime). Bazel itself automatically registers `@bazel_tools//tools/python:autodetecting_toolchain` -as the lowest priority toolchain. For WORKSPACE builds, if no other toolchain -is registered, that toolchain will be used. For bzlmod builds, rules_python +as the lowest priority toolchain. For `WORKSPACE` builds, if no other toolchain +is registered, that toolchain will be used. For Bzlmod builds, `rules_python` automatically registers a higher-priority toolchain; it won't be used unless there is a toolchain misconfiguration somewhere. -To aid migration off the Bazel-builtin toolchain, rules_python provides +To aid migration off the Bazel-builtin toolchain, `rules_python` provides {bzl:obj}`@rules_python//python/runtime_env_toolchains:all`. This is an equivalent -toolchain, but is implemented using rules_python's objects. - +toolchain but is implemented using `rules_python`'s objects. ## Custom toolchains -While rules_python provides toolchains by default, it is not required to use +While `rules_python` provides toolchains by default, it is not required to use them, and you can define your own toolchains to use instead. This section -gives an introduction for how to define them yourself. +gives an introduction to how to define them yourself. :::{note} * Defining your own toolchains is an advanced feature. @@ -281,8 +672,8 @@ toolchains a "toolchain suite". One of the underlying design goals of the toolchains is to support complex and bespoke environments. Such environments may use an arbitrary combination of -{obj}`RBE`, cross-platform building, multiple Python versions, -building Python from source, embeding Python (as opposed to building separate +{bzl:obj}`RBE`, cross-platform building, multiple Python versions, +building Python from source, embedding Python (as opposed to building separate interpreters), using prebuilt binaries, or using binaries built from source. To that end, many of the attributes they accept, and fields they provide, are optional. @@ -293,7 +684,7 @@ The target toolchain type is {obj}`//python:toolchain_type`, and it is for _target configuration_ runtime information, e.g., the Python version and interpreter binary that a program will use. -The is typically implemented using {obj}`py_runtime()`, which +This is typically implemented using {obj}`py_runtime()`, which provides the {obj}`PyRuntimeInfo` provider. For historical reasons from the Python 2 transition, `py_runtime` is wrapped in {obj}`py_runtime_pair`, which provides {obj}`ToolchainInfo` with the field `py3_runtime`, which is an @@ -308,12 +699,12 @@ set {external:bzl:obj}`toolchain.exec_compatible_with`. ### Python C toolchain type The Python C toolchain type ("py cc") is {obj}`//python/cc:toolchain_type`, and -it has C/C++ information for the _target configuration_, e.g. the C headers that +it has C/C++ information for the _target configuration_, e.g., the C headers that provide `Python.h`. This is typically implemented using {obj}`py_cc_toolchain()`, which provides {obj}`ToolchainInfo` with the field `py_cc_toolchain` set, which is a -{obj}`PyCcToolchainInfo` provider instance. +{obj}`PyCcToolchainInfo` provider instance. This toolchain type is intended to hold only _target configuration_ values relating to the C/C++ information for the Python runtime. As such, when defining @@ -325,7 +716,7 @@ set {external:bzl:obj}`toolchain.exec_compatible_with`. ### Exec tools toolchain type The exec tools toolchain type is {obj}`//python:exec_tools_toolchain_type`, -and it is for supporting tools for _building_ programs, e.g. the binary to +and it is for supporting tools for _building_ programs, e.g., the binary to precompile code at build time. This toolchain type is intended to hold only _exec configuration_ values -- @@ -344,7 +735,7 @@ target configuration (e.g. Python version), then for one to be chosen based on finding one compatible with the available host platforms to run the tool on. However, what `target_compatible_with`/`target_settings` and -`exec_compatible_with` values to use depend on details of the tools being used. +`exec_compatible_with` values to use depends on the details of the tools being used. For example: * If you had a precompiler that supported any version of Python, then putting the Python version in `target_settings` is unnecessary. @@ -355,9 +746,9 @@ This can work because, when the rules invoke these build tools, they pass along all necessary information so that the tool can be entirely independent of the target configuration being built for. -Alternatively, if you had a precompiler that only ran on linux, and only -produced valid output for programs intended to run on linux, then _both_ -`exec_compatible_with` and `target_compatible_with` must be set to linux. +Alternatively, if you had a precompiler that only ran on Linux and only +produced valid output for programs intended to run on Linux, then _both_ +`exec_compatible_with` and `target_compatible_with` must be set to Linux. ### Custom toolchain example @@ -367,16 +758,19 @@ Here, we show an example for a semi-complicated toolchain suite, one that is: * For Python version 3.12.0 * Using an in-build interpreter built from source * That only runs on Linux -* Using a prebuilt precompiler that only runs on Linux, and only produces byte - code valid for 3.12 -* With the exec tools interpreter disabled (unnecessary with a prebuild +* Using a prebuilt precompiler that only runs on Linux and only produces + bytecode valid for 3.12 +* With the exec tools interpreter disabled (unnecessary with a prebuilt precompiler) * Providing C headers and libraries Defining toolchains for this might look something like this: ``` -# File: toolchain_impls/BUILD +# ------------------------------------------------------- +# File: toolchain_impl/BUILD +# Contains the tool definitions (runtime, headers, libs). +# ------------------------------------------------------- load("@rules_python//python:py_cc_toolchain.bzl", "py_cc_toolchain") load("@rules_python//python:py_exec_tools_toolchain.bzl", "py_exec_tools_toolchain") load("@rules_python//python:py_runtime.bzl", "py_runtime") @@ -418,21 +812,23 @@ cc_binary(name = "python3.12", ...) cc_library(name = "headers", ...) cc_library(name = "libs", ...) +# ------------------------------------------------------------------ # File: toolchains/BUILD # Putting toolchain() calls in a separate package from the toolchain -# implementations minimizes Bazel loading overhead +# implementations minimizes Bazel loading overhead. +# ------------------------------------------------------------------ toolchain( name = "runtime_toolchain", toolchain = "//toolchain_impl:runtime_pair", toolchain_type = "@rules_python//python:toolchain_type", - target_compatible_with = ["@platforms/os:linux"] + target_compatible_with = ["@platforms/os:linux"], ) toolchain( name = "py_cc_toolchain", toolchain = "//toolchain_impl:py_cc_toolchain_impl", toolchain_type = "@rules_python//python/cc:toolchain_type", - target_compatible_with = ["@platforms/os:linux"] + target_compatible_with = ["@platforms/os:linux"], ) toolchain( @@ -442,10 +838,21 @@ toolchain( target_settings = [ "@rules_python//python/config_settings:is_python_3.12", ], - exec_comaptible_with = ["@platforms/os:linux"] + exec_compatible_with = ["@platforms/os:linux"], ) + +# ----------------------------------------------- +# File: MODULE.bazel or WORKSPACE.bazel +# These toolchains will be considered before others. +# ----------------------------------------------- +register_toolchains("//toolchains:all") ``` +When registering custom toolchains, be aware of the [toolchain registration +order](https://bazel.build/extending/toolchains#toolchain-resolution). In brief, +toolchain order is the BFS-order of the modules; see the Bazel docs for a more +detailed description. + :::{note} The toolchain() calls should be in a separate BUILD file from everything else. This avoids Bazel having to perform unnecessary work when it discovers the list @@ -458,3 +865,81 @@ Currently the following flags are used to influence toolchain selection: * {obj}`--@rules_python//python/config_settings:py_linux_libc` for selecting the Linux libc variant. * {obj}`--@rules_python//python/config_settings:py_freethreaded` for selecting the freethreaded experimental Python builds available from `3.13.0` onwards. + +## Running the underlying interpreter + +To run the interpreter that Bazel will use, you can use the +`@rules_python//python/bin:python` target. This is a binary target with +the executable pointing at the `python3` binary plus its relevant runfiles. + +```console +$ bazel run @rules_python//python/bin:python +Python 3.11.1 (main, Jan 16 2023, 22:41:20) [Clang 15.0.7 ] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> +$ bazel run @rules_python//python/bin:python --@rules_python//python/config_settings:python_version=3.12 +Python 3.12.0 (main, Oct 3 2023, 01:27:23) [Clang 17.0.1 ] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> +``` + +You can also access a specific binary's interpreter this way by using the +`@rules_python//python/bin:python_src` target. In the example below, it is +assumed that the `@rules_python//tools/publish:twine` binary is fixed at Python +3.11. + +```console +$ bazel run @rules_python//python/bin:python --@rules_python//python/bin:interpreter_src=@rules_python//tools/publish:twine +Python 3.11.1 (main, Jan 16 2023, 22:41:20) [Clang 15.0.7 ] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> +$ bazel run @rules_python//python/bin:python --@rules_python//python/bin:interpreter_src=@rules_python//tools/publish:twine --@rules_python//python/config_settings:python_version=3.12 +Python 3.11.1 (main, Jan 16 2023, 22:41:20) [Clang 15.0.7 ] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> +``` +Despite setting the Python version explicitly to 3.12 in the example above, the +interpreter comes from the `@rules_python//tools/publish:twine` binary. That is +a fixed version. + +:::{note} +The `python` target does not provide access to any modules from `py_*` +targets on its own. Please file a feature request if this is desired. +::: + +### Differences from `//python/bin:repl` + +The `//python/bin:python` target provides access to the underlying interpreter +without any hermeticity guarantees. + +The [`//python/bin:repl` target](repl) provides an environment identical to +what `py_binary` provides. That means it handles things like the +[`PYTHONSAFEPATH`](https://docs.python.org/3/using/cmdline.html#envvar-PYTHONSAFEPATH) +environment variable automatically. The `//python/bin:python` target will not. + +## Consuming Python C headers and libraries + +The following targets expose the headers and libraries from the +currently selected Python C toolchain: + +- {obj}`@rules_python//python/cc:current_py_cc_headers` +- {obj}`@rules_python//python/cc:current_py_cc_headers_abi3` +- {obj}`@rules_python//python/cc:current_py_cc_libs` + +These targets behave similarly to a `cc_library`, but instead of defining +their own sources, they forward providers from the underlying toolchain- +selected `cc_library`. + +A Python C toolchain must be registered for these targets to work. +Under bzlmod, a toolchain is registered automatically. In non-bzlmod +setups, users must ensure that a toolchain is explicitly registered. + +Users should depend on these targets instead of legacy alias targets +when embedding Python or building C extensions, as this ensures +compatibility across different toolchain configurations. + + +:::{seealso} +The _How to get Python headres for C extensions_ how-to guide, and the +{obj}`//python/cc:BUILD.bazel` package API documentation. +::: diff --git a/downloader_config.cfg b/downloader_config.cfg new file mode 100644 index 0000000000..3fa6264eda --- /dev/null +++ b/downloader_config.cfg @@ -0,0 +1,21 @@ +# Try GitHub first (primary) +rewrite ^github\.com/bazel-contrib/bazel_features/(.*) github.com/bazel-contrib/bazel_features/$1 +rewrite ^github\.com/bazel-contrib/rules_go/(.*) github.com/bazel-contrib/rules_go/$1 +rewrite ^github\.com/bazelbuild/bazel-skylib/(.*) github.com/bazelbuild/bazel-skylib/$1 +rewrite ^github\.com/bazelbuild/platforms/(.*) github.com/bazelbuild/platforms/$1 +rewrite ^github\.com/bazelbuild/rules_kotlin/(.*) github.com/bazelbuild/rules_kotlin/$1 +rewrite ^github\.com/bazelbuild/rules_shell/(.*) github.com/bazelbuild/rules_shell/$1 +rewrite ^github\.com/bazelbuild/rules_java/(.*) github.com/bazelbuild/rules_java/$1 +rewrite ^github\.com/bazelbuild/stardoc/(.*) github.com/bazelbuild/stardoc/$1 + + +# Fall back to mirror (secondary) +# Tracking upstream BCR mirror addition: https://github.com/bazelbuild/platforms/issues/139 +rewrite ^github\.com/bazel-contrib/bazel_features/(.*) mirror.bazel.build/github.com/bazel-contrib/bazel_features/$1 +rewrite ^github\.com/bazel-contrib/rules_go/(.*) mirror.bazel.build/github.com/bazel-contrib/rules_go/$1 +rewrite ^github\.com/bazelbuild/bazel-skylib/(.*) mirror.bazel.build/github.com/bazelbuild/bazel-skylib/$1 +rewrite ^github\.com/bazelbuild/platforms/(.*) mirror.bazel.build/github.com/bazelbuild/platforms/$1 +rewrite ^github\.com/bazelbuild/rules_kotlin/(.*) mirror.bazel.build/github.com/bazelbuild/rules_kotlin/$1 +rewrite ^github\.com/bazelbuild/rules_shell/(.*) mirror.bazel.build/github.com/bazelbuild/rules_shell/$1 +rewrite ^github\.com/bazelbuild/rules_java/(.*) mirror.bazel.build/github.com/bazelbuild/rules_java/$1 +rewrite ^github\.com/bazelbuild/stardoc/(.*) mirror.bazel.build/github.com/bazelbuild/stardoc/$1 diff --git a/examples/BUILD.bazel b/examples/BUILD.bazel index 92ca8e7199..a08e5e64ae 100644 --- a/examples/BUILD.bazel +++ b/examples/BUILD.bazel @@ -21,5 +21,35 @@ lock( name = "bzlmod_requirements_3_9", srcs = ["bzlmod/requirements.in"], out = "bzlmod/requirements_lock_3_9.txt", - python_version = "3.9.19", + args = [ + "--emit-index-url", + "--universal", + "--python-version=3.9", + ], + python_version = "3.9", +) + +lock( + name = "bzlmod_requirements_3_11", + srcs = ["bzlmod/requirements.in"], + out = "bzlmod/requirements_lock_3_11.txt", + args = [ + "--emit-index-url", + "--universal", + "--python-version=3.11", + ], + python_version = "3.11", +) + +lock( + name = "bzlmod_requirements_3_11_windows", + srcs = ["bzlmod/requirements.in"], + out = "bzlmod/requirements_windows_3_11.txt", + args = [ + "--emit-index-url", + "--python-platform", + "windows", + "--python-version=3.11", + ], + python_version = "3.11", ) diff --git a/examples/build_file_generation/.bazelrc b/examples/build_file_generation/.bazelrc index fd0f731d2f..f1ae44fac8 100644 --- a/examples/build_file_generation/.bazelrc +++ b/examples/build_file_generation/.bazelrc @@ -3,8 +3,8 @@ test --test_output=errors --enable_runfiles # Windows requires these for multi-python support: build --enable_runfiles -# The bzlmod version of this example is in examples/bzlmod_build_file_generation +# The bzlmod version of this example is in gazelle/examples/bzlmod_build_file_generation # Once WORKSPACE support is dropped, this example can be entirely deleted. -build --experimental_enable_bzlmod=false - -common:bazel7.x --incompatible_python_disallow_native_rules +common --noenable_bzlmod +common --enable_workspace +common --incompatible_python_disallow_native_rules diff --git a/examples/build_file_generation/BUILD.bazel b/examples/build_file_generation/BUILD.bazel index 4d270dd850..a378775968 100644 --- a/examples/build_file_generation/BUILD.bazel +++ b/examples/build_file_generation/BUILD.bazel @@ -4,8 +4,10 @@ # ruleset. When the symbol is loaded you can use the rule. load("@bazel_gazelle//:def.bzl", "gazelle") load("@pip//:requirements.bzl", "all_whl_requirements") -load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test") load("@rules_python//python:pip.bzl", "compile_pip_requirements") +load("@rules_python//python:py_binary.bzl", "py_binary") +load("@rules_python//python:py_library.bzl", "py_library") +load("@rules_python//python:py_test.bzl", "py_test") load("@rules_python_gazelle_plugin//manifest:defs.bzl", "gazelle_python_manifest") load("@rules_python_gazelle_plugin//modules_mapping:def.bzl", "modules_mapping") diff --git a/examples/build_file_generation/WORKSPACE b/examples/build_file_generation/WORKSPACE index 3f1fad8a8d..27d0d13b7c 100644 --- a/examples/build_file_generation/WORKSPACE +++ b/examples/build_file_generation/WORKSPACE @@ -7,59 +7,12 @@ workspace(name = "build_file_generation_example") # file. When the symbol is loaded you can use the rule. load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") -###################################################################### -# We need rules_go and bazel_gazelle, to build the gazelle plugin from source. -# Setup instructions for this section are at -# https://github.com/bazelbuild/bazel-gazelle#running-gazelle-with-bazel -# You may need to update the version of the rule, which is listed in the above -# documentation. -###################################################################### - -# Define an http_archive rule that will download the below ruleset, -# test the sha, and extract the ruleset to you local bazel cache. - -http_archive( - name = "io_bazel_rules_go", - sha256 = "278b7ff5a826f3dc10f04feaf0b70d48b68748ccd512d7f98bf442077f043fe3", - urls = [ - "https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.41.0/rules_go-v0.41.0.zip", - "https://github.com/bazelbuild/rules_go/releases/download/v0.41.0/rules_go-v0.41.0.zip", - ], -) - -# Download the bazel_gazelle ruleset. -http_archive( - name = "bazel_gazelle", - sha256 = "d3fa66a39028e97d76f9e2db8f1b0c11c099e8e01bf363a923074784e451f809", - urls = [ - "https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.33.0/bazel-gazelle-v0.33.0.tar.gz", - "https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.33.0/bazel-gazelle-v0.33.0.tar.gz", - ], -) - -# Load rules_go ruleset and expose the toolchain and dep rules. -load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies") -load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies") - -# go_rules_dependencies is a function that registers external dependencies -# needed by the Go rules. -# See: https://github.com/bazelbuild/rules_go/blob/master/go/dependencies.rst#go_rules_dependencies -go_rules_dependencies() - -# go_rules_dependencies is a function that registers external dependencies -# needed by the Go rules. -# See: https://github.com/bazelbuild/rules_go/blob/master/go/dependencies.rst#go_rules_dependencies -go_register_toolchains(version = "1.19.4") - -# The following call configured the gazelle dependencies, Go environment and Go SDK. -gazelle_dependencies() - -# Remaining setup is for rules_python. +# Setup rules_python. # DON'T COPY_PASTE THIS. # Our example uses `local_repository` to point to the HEAD version of rules_python. # Users should instead use the installation instructions from the release they use. -# See https://github.com/bazelbuild/rules_python/releases +# See https://github.com/bazel-contrib/rules_python/releases local_repository( name = "rules_python", path = "../..", @@ -124,11 +77,58 @@ load("@pip//:requirements.bzl", "install_deps") # Initialize repositories for all packages in requirements_lock.txt. install_deps() +###################################################################### +# We need rules_go and bazel_gazelle, to build the gazelle plugin from source. +# Setup instructions for this section are at +# https://github.com/bazelbuild/bazel-gazelle#running-gazelle-with-bazel +# You may need to update the version of the rule, which is listed in the above +# documentation. +###################################################################### + +# Define an http_archive rule that will download the below ruleset, +# test the sha, and extract the ruleset to you local bazel cache. + +http_archive( + name = "io_bazel_rules_go", + sha256 = "9d72f7b8904128afb98d46bbef82ad7223ec9ff3718d419afb355fddd9f9484a", + urls = [ + "https://mirror.bazel.build/github.com/bazel-contrib/rules_go/releases/download/v0.55.1/rules_go-v0.55.1.zip", + "https://github.com/bazel-contrib/rules_go/releases/download/v0.55.1/rules_go-v0.55.1.zip", + ], +) + +# Download the bazel_gazelle ruleset. +http_archive( + name = "bazel_gazelle", + sha256 = "75df288c4b31c81eb50f51e2e14f4763cb7548daae126817247064637fd9ea62", + urls = [ + "https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.36.0/bazel-gazelle-v0.36.0.tar.gz", + "https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.36.0/bazel-gazelle-v0.36.0.tar.gz", + ], +) + +# Load rules_go ruleset and expose the toolchain and dep rules. +load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies") +load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies") + +# go_rules_dependencies is a function that registers external dependencies +# needed by the Go rules. +# See: https://github.com/bazelbuild/rules_go/blob/master/go/dependencies.rst#go_rules_dependencies +go_rules_dependencies() + +# go_rules_dependencies is a function that registers external dependencies +# needed by the Go rules. +# See: https://github.com/bazelbuild/rules_go/blob/master/go/dependencies.rst#go_rules_dependencies +go_register_toolchains(version = "1.21.13") + +# The following call configured the gazelle dependencies, Go environment and Go SDK. +gazelle_dependencies() + # The rules_python gazelle extension has some third-party go dependencies # which we need to fetch in order to compile it. load("@rules_python_gazelle_plugin//:deps.bzl", _py_gazelle_deps = "gazelle_deps") -# See: https://github.com/bazelbuild/rules_python/blob/main/gazelle/README.md +# See: https://github.com/bazel-contrib/rules_python/blob/main/gazelle/README.md # This rule loads and compiles various go dependencies that running gazelle # for python requirements. _py_gazelle_deps() diff --git a/examples/build_file_generation/gazelle_python.yaml b/examples/build_file_generation/gazelle_python.yaml index cd5904dcba..6b34f3c688 100644 --- a/examples/build_file_generation/gazelle_python.yaml +++ b/examples/build_file_generation/gazelle_python.yaml @@ -3,6 +3,7 @@ # To update this file, run: # bazel run //:gazelle_python_manifest.update +--- manifest: modules_mapping: alabaster: alabaster diff --git a/examples/build_file_generation/random_number_generator/BUILD.bazel b/examples/build_file_generation/random_number_generator/BUILD.bazel index 28370b418f..c77550084f 100644 --- a/examples/build_file_generation/random_number_generator/BUILD.bazel +++ b/examples/build_file_generation/random_number_generator/BUILD.bazel @@ -1,4 +1,5 @@ -load("@rules_python//python:defs.bzl", "py_library", "py_test") +load("@rules_python//python:py_library.bzl", "py_library") +load("@rules_python//python:py_test.bzl", "py_test") py_library( name = "random_number_generator", diff --git a/examples/bzlmod/.bazelignore b/examples/bzlmod/.bazelignore index ab3eb1635c..a59e740c96 100644 --- a/examples/bzlmod/.bazelignore +++ b/examples/bzlmod/.bazelignore @@ -1 +1,2 @@ other_module +vendor diff --git a/examples/bzlmod/.bazelrc b/examples/bzlmod/.bazelrc index ca83047ccc..28a44a7523 100644 --- a/examples/bzlmod/.bazelrc +++ b/examples/bzlmod/.bazelrc @@ -1,5 +1,21 @@ +# Starting with Bazel 8, Windows requires this flag in order +# for symlinks to work properly (namely, so that sh_test with +# py_binary as a data dependency gets symlinks that are executable) +startup --windows_enable_symlinks + common --enable_bzlmod common --lockfile_mode=update +# This adds an implicit --config= +# See docs for osname values +# https://bazel.build/reference/command-line-reference#common_options-flag--enable_platform_specific_config +common --enable_platform_specific_config + +common:windows --cxxopt=/std:c++17 +common:windows --host_cxxopt=/std:c++17 +common:linux --cxxopt=-std=c++17 +common:linux --host_cxxopt=-std=c++17 +common:macos --cxxopt=-std=c++17 +common:macos --host_cxxopt=-std=c++17 coverage --java_runtime_version=remotejdk_11 diff --git a/examples/bzlmod/.bazelversion b/examples/bzlmod/.bazelversion index 35907cd9ca..512e4c889e 100644 --- a/examples/bzlmod/.bazelversion +++ b/examples/bzlmod/.bazelversion @@ -1 +1 @@ -7.x +9.x diff --git a/examples/bzlmod/.gitignore b/examples/bzlmod/.gitignore index ac51a054d2..0f6c6316dd 100644 --- a/examples/bzlmod/.gitignore +++ b/examples/bzlmod/.gitignore @@ -1 +1,2 @@ bazel-* +vendor/ diff --git a/examples/bzlmod/.python_version b/examples/bzlmod/.python_version new file mode 100644 index 0000000000..bd28b9c5c2 --- /dev/null +++ b/examples/bzlmod/.python_version @@ -0,0 +1 @@ +3.9 diff --git a/examples/bzlmod/BUILD.bazel b/examples/bzlmod/BUILD.bazel index 054b957b3b..df07385690 100644 --- a/examples/bzlmod/BUILD.bazel +++ b/examples/bzlmod/BUILD.bazel @@ -9,7 +9,9 @@ load("@bazel_skylib//rules:build_test.bzl", "build_test") load("@pip//:requirements.bzl", "all_data_requirements", "all_requirements", "all_whl_requirements", "requirement") load("@python_3_9//:defs.bzl", py_test_with_transition = "py_test") load("@python_versions//3.10:defs.bzl", compile_pip_requirements_3_10 = "compile_pip_requirements") -load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test") +load("@rules_python//python:py_binary.bzl", "py_binary") +load("@rules_python//python:py_library.bzl", "py_library") +load("@rules_python//python:py_test.bzl", "py_test") # This stanza calls a rule that generates targets for managing pip dependencies # with pip-compile for a particular python version. diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel index 0a31c3beb8..106f25134e 100644 --- a/examples/bzlmod/MODULE.bazel +++ b/examples/bzlmod/MODULE.bazel @@ -5,35 +5,33 @@ module( ) bazel_dep(name = "bazel_skylib", version = "1.7.1") +bazel_dep(name = "platforms", version = "0.0.4") bazel_dep(name = "rules_python", version = "0.0.0") local_path_override( module_name = "rules_python", path = "../..", ) -# (py_proto_library specific) We are using rules_proto to define rules_proto targets to be consumed by py_proto_library. -bazel_dep(name = "rules_proto", version = "6.0.0-rc1") - -# (py_proto_library specific) Add the protobuf library for well-known types (e.g. `Any`, `Timestamp`, etc) -bazel_dep(name = "protobuf", version = "27.0", repo_name = "com_google_protobuf") - # Only needed to make rules_python's CI happy. rules_java 8.3.0+ is needed so # that --java_runtime_version=remotejdk_11 works with Bazel 8. -bazel_dep(name = "rules_java", version = "8.3.1") +bazel_dep(name = "rules_java", version = "8.16.1") # Only needed to make rules_python's CI happy. A test verifies that # MODULE.bazel.lock is cross-platform friendly, and there are transitive # dependencies on rules_rust, so we need rules_rust 0.54.1+ where such issues # were fixed. -bazel_dep(name = "rules_rust", version = "0.54.1") +bazel_dep(name = "rules_rust", version = "0.67.0") # We next initialize the python toolchain using the extension. # You can set different Python versions in this block. python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.defaults( + # Use python.defaults if you have defined multiple toolchain versions. + python_version = "3.9", + python_version_env = "BAZEL_PYTHON_VERSION", +) python.toolchain( configure_coverage_tool = True, - # Only set when you have multiple toolchain versions. - is_default = True, python_version = "3.9", ) @@ -54,7 +52,6 @@ python.override( # require versions not listed here. # available_python_versions = [ # "3.10.9", - # "3.9.18", # "3.9.19", # # The following is used by the `other_module` and we need to include it here # # as well. @@ -64,7 +61,7 @@ python.override( # instead of rules_python's defaulting to the latest available version, # controls what full version is used when `3.x` is requested. minor_mapping = { - "3.9": "3.9.19", + "3.9": "3.9.25", }, ) @@ -103,12 +100,15 @@ python.single_version_platform_override( # rules based on the `python_version` arg values. use_repo(python, "python_3_10", "python_3_9", "python_versions", "pythons_hub") -# EXPERIMENTAL: This is experimental and may be removed without notice -uv = use_extension("@rules_python//python/uv:extensions.bzl", "uv") -uv.toolchain(uv_version = "0.4.25") -use_repo(uv, "uv_toolchains") - -register_toolchains("@uv_toolchains//:all") +# EXPERIMENTAL: This is experimental and may be changed or removed without notice +uv = use_extension( + "@rules_python//python/uv:uv.bzl", + "uv", + # Use `dev_dependency` so that the toolchains are not defined pulled when your + # module is used elsewhere. + dev_dependency = True, +) +uv.configure(version = "0.6.2") # This extension allows a user to create modifications to how rules_python # creates different wheel repositories. Different attributes allow the user @@ -154,6 +154,26 @@ pip.whl_mods( ) use_repo(pip, "whl_mods_hub") +# Because below we are using `windows_aarch64` platform, we have to define various +# properties for it. +pip.default( + arch_name = "aarch64", + config_settings = [ + "@platforms//os:windows", + "@platforms//cpu:aarch64", + ], + env = { + "platform_version": "0", + }, + # Windows ARM64 support has been added only on 3.11 and above, hence, constrain + # the availability of the platform for those python versions. + marker = "python_version >= '3.11'", + os_name = "windows", + platform = "windows_aarch64", + whl_abi_tags = [], # default to all ABIs + whl_platform_tags = ["win_arm64"], +) + # To fetch pip dependencies, use pip.parse. We can pass in various options, # but typically we pass requirements and the Python version. The Python # version must have been configured by a corresponding `python.toolchain()` @@ -163,18 +183,6 @@ use_repo(pip, "whl_mods_hub") pip.parse( # We can use `envsubst in the above envsubst = ["PIP_INDEX_URL"], - # Use the bazel downloader to query the simple API for downloading the sources - # Note, that we can use envsubst for this value. - experimental_index_url = "${PIP_INDEX_URL:-https://pypi.org/simple}", - # One can also select a particular index for a particular package. - # This ensures that the setup is resistant against confusion attacks. - # experimental_index_url_overrides = { - # "my_package": "https://different-index-url.com", - # }, - # Or you can specify extra indexes like with `pip`: - # experimental_extra_index_urls = [ - # "https://different-index-url.com", - # ], experimental_requirement_cycles = { "sphinx": [ "sphinx", @@ -185,17 +193,19 @@ pip.parse( "sphinxcontrib-serializinghtml", ], }, - # You can use one of the values below to specify the target platform - # to generate the dependency graph for. - experimental_target_platforms = [ - # Specifying the target platforms explicitly - "cp39_linux_x86_64", - "cp39_linux_*", - "cp39_*", - ], extra_hub_aliases = { "wheel": ["generated_file"], }, + extra_pip_args = [ + # Use the bazel downloader to query the simple API for downloading the sources + # Note, that we can use envsubst for this value. + # One can also select a particular index for a particular package. + # This ensures that the setup is resistant against confusion attacks. + # experimental_index_url_overrides = { + # "my_package": "https://different-index-url.com", + # }, + "--index-url=${PIP_INDEX_URL:-https://pypi.org/simple}", + ], hub_name = "pip", python_version = "3.9", requirements_lock = "requirements_lock_3_9.txt", @@ -218,30 +228,34 @@ pip.parse( "sphinxcontrib-serializinghtml", ], }, - # You can use one of the values below to specify the target platform - # to generate the dependency graph for. - experimental_target_platforms = [ - # Using host python version - "linux_*", - "osx_*", - "windows_*", - # Or specifying an exact platform - "linux_x86_64", - # Or the following to get the `host` platform only - "host", - ], hub_name = "pip", python_version = "3.10", # The requirements files for each platform that we want to support. requirements_by_platform = { # Default requirements file for needs to explicitly provide the platforms "//:requirements_lock_3_10.txt": "linux_*,osx_*", + "//:requirements_windows_3_10.txt": "windows_x86_64", + }, + # These modifications were created above and we + # are providing pip.parse with the label of the mod + # and the name of the wheel. + whl_modifications = { + "@whl_mods_hub//:requests.json": "requests", + "@whl_mods_hub//:wheel.json": "wheel", + }, +) +pip.parse( + hub_name = "pip", + python_version = "3.11", + requirements_by_platform = { + # Default requirements file for needs to explicitly provide the platforms + "//:requirements_lock_3_11.txt": "linux_*,osx_*", # This API allows one to specify additional platforms that the users # configure the toolchains for themselves. In this example we add # `windows_aarch64` to illustrate that `rules_python` won't fail to # process the value, but it does not mean that this example will work # on Windows ARM. - "//:requirements_windows_3_10.txt": "windows_x86_64,windows_aarch64", + "//:requirements_windows_3_11.txt": "windows_x86_64,windows_aarch64", }, # These modifications were created above and we # are providing pip.parse with the label of the mod diff --git a/examples/bzlmod/entry_points/BUILD.bazel b/examples/bzlmod/entry_points/BUILD.bazel index a0939cb65b..4ca5b53568 100644 --- a/examples/bzlmod/entry_points/BUILD.bazel +++ b/examples/bzlmod/entry_points/BUILD.bazel @@ -1,4 +1,3 @@ -load("@python_versions//3.9:defs.bzl", py_console_script_binary_3_9 = "py_console_script_binary") load("@rules_python//python/entry_points:py_console_script_binary.bzl", "py_console_script_binary") # This is how you can define a `pylint` entrypoint which uses the default python version. @@ -24,10 +23,11 @@ py_console_script_binary( ], ) -# A specific Python version can be forced by using the generated version-aware -# wrappers, e.g. to force Python 3.9: -py_console_script_binary_3_9( +# A specific Python version can be forced by passing `python_version` +# attribute, e.g. to force Python 3.9: +py_console_script_binary( name = "yamllint", pkg = "@pip//yamllint:pkg", + python_version = "3.9", visibility = ["//entry_points:__subpackages__"], ) diff --git a/examples/bzlmod/entry_points/tests/BUILD.bazel b/examples/bzlmod/entry_points/tests/BUILD.bazel index 5a65e9e1a3..3c6e02a3c4 100644 --- a/examples/bzlmod/entry_points/tests/BUILD.bazel +++ b/examples/bzlmod/entry_points/tests/BUILD.bazel @@ -1,5 +1,5 @@ load("@bazel_skylib//rules:run_binary.bzl", "run_binary") -load("@rules_python//python:defs.bzl", "py_test") +load("@rules_python//python:py_test.bzl", "py_test") # Below are targets for testing the `py_console_script_binary` feature and are # not part of the example how to use the feature. diff --git a/examples/bzlmod/entry_points/tests/pylint_deps_test.py b/examples/bzlmod/entry_points/tests/pylint_deps_test.py index f6743ce9b5..54826a137c 100644 --- a/examples/bzlmod/entry_points/tests/pylint_deps_test.py +++ b/examples/bzlmod/entry_points/tests/pylint_deps_test.py @@ -15,7 +15,6 @@ import os import pathlib import subprocess -import tempfile import unittest from python.runfiles import runfiles @@ -29,9 +28,9 @@ def __init__(self, *args, **kwargs): def test_pylint_entry_point(self): rlocation_path = os.environ.get("ENTRY_POINT") - assert ( - rlocation_path is not None - ), "expected 'ENTRY_POINT' env variable to be set to rlocation of the tool" + assert rlocation_path is not None, ( + "expected 'ENTRY_POINT' env variable to be set to rlocation of the tool" + ) entry_point = pathlib.Path(runfiles.Create().Rlocation(rlocation_path)) self.assertTrue(entry_point.exists(), f"'{entry_point}' does not exist") @@ -51,20 +50,20 @@ def test_pylint_entry_point(self): "", proc.stderr.decode("utf-8").strip(), ) - self.assertRegex(proc.stdout.decode("utf-8").strip(), "^pylint 2\.15\.9") + self.assertRegex(proc.stdout.decode("utf-8").strip(), r"^pylint 2\.15\.9") def test_pylint_report_has_expected_warnings(self): rlocation_path = os.environ.get("PYLINT_REPORT") - assert ( - rlocation_path is not None - ), "expected 'PYLINT_REPORT' env variable to be set to rlocation of the report" + assert rlocation_path is not None, ( + "expected 'PYLINT_REPORT' env variable to be set to rlocation of the report" + ) pylint_report = pathlib.Path(runfiles.Create().Rlocation(rlocation_path)) self.assertTrue(pylint_report.exists(), f"'{pylint_report}' does not exist") self.assertRegex( pylint_report.read_text().strip(), - "W8201: Logging should be used instead of the print\(\) function\. \(print-function\)", + r"W8201: Logging should be used instead of the print\(\) function\. \(print-function\)", ) diff --git a/examples/bzlmod/entry_points/tests/pylint_test.py b/examples/bzlmod/entry_points/tests/pylint_test.py index c2532938d8..5a13e61920 100644 --- a/examples/bzlmod/entry_points/tests/pylint_test.py +++ b/examples/bzlmod/entry_points/tests/pylint_test.py @@ -28,9 +28,9 @@ def __init__(self, *args, **kwargs): def test_pylint_entry_point(self): rlocation_path = os.environ.get("ENTRY_POINT") - assert ( - rlocation_path is not None - ), "expected 'ENTRY_POINT' env variable to be set to rlocation of the tool" + assert rlocation_path is not None, ( + "expected 'ENTRY_POINT' env variable to be set to rlocation of the tool" + ) entry_point = pathlib.Path(runfiles.Create().Rlocation(rlocation_path)) self.assertTrue(entry_point.exists(), f"'{entry_point}' does not exist") @@ -50,7 +50,7 @@ def test_pylint_entry_point(self): "", proc.stderr.decode("utf-8").strip(), ) - self.assertRegex(proc.stdout.decode("utf-8").strip(), "^pylint 2\.15\.9") + self.assertRegex(proc.stdout.decode("utf-8").strip(), r"^pylint 2\.15\.9") if __name__ == "__main__": diff --git a/examples/bzlmod/entry_points/tests/yamllint_test.py b/examples/bzlmod/entry_points/tests/yamllint_test.py index 0a0235793b..29b5ebf9b8 100644 --- a/examples/bzlmod/entry_points/tests/yamllint_test.py +++ b/examples/bzlmod/entry_points/tests/yamllint_test.py @@ -28,9 +28,9 @@ def __init__(self, *args, **kwargs): def test_yamllint_entry_point(self): rlocation_path = os.environ.get("ENTRY_POINT") - assert ( - rlocation_path is not None - ), "expected 'ENTRY_POINT' env variable to be set to rlocation of the tool" + assert rlocation_path is not None, ( + "expected 'ENTRY_POINT' env variable to be set to rlocation of the tool" + ) entry_point = pathlib.Path(runfiles.Create().Rlocation(rlocation_path)) self.assertTrue(entry_point.exists(), f"'{entry_point}' does not exist") diff --git a/examples/bzlmod/libs/my_lib/BUILD.bazel b/examples/bzlmod/libs/my_lib/BUILD.bazel index 2679d0e4a0..77a059574d 100644 --- a/examples/bzlmod/libs/my_lib/BUILD.bazel +++ b/examples/bzlmod/libs/my_lib/BUILD.bazel @@ -1,5 +1,5 @@ load("@pip//:requirements.bzl", "requirement") -load("@rules_python//python:defs.bzl", "py_library") +load("@rules_python//python:py_library.bzl", "py_library") py_library( name = "my_lib", diff --git a/examples/bzlmod/other_module/BUILD.bazel b/examples/bzlmod/other_module/BUILD.bazel index a93b92aaed..6294c5b0ae 100644 --- a/examples/bzlmod/other_module/BUILD.bazel +++ b/examples/bzlmod/other_module/BUILD.bazel @@ -1,9 +1,10 @@ -load("@python_versions//3.11:defs.bzl", compile_pip_requirements_311 = "compile_pip_requirements") +load("@rules_python//python:pip.bzl", "compile_pip_requirements") # NOTE: To update the requirements, you need to uncomment the rules_python # override in the MODULE.bazel. -compile_pip_requirements_311( +compile_pip_requirements( name = "requirements", src = "requirements.in", + python_version = "3.11", requirements_txt = "requirements_lock_3_11.txt", ) diff --git a/examples/bzlmod/other_module/MODULE.bazel b/examples/bzlmod/other_module/MODULE.bazel index 959501abc2..a128c39ca0 100644 --- a/examples/bzlmod/other_module/MODULE.bazel +++ b/examples/bzlmod/other_module/MODULE.bazel @@ -5,45 +5,25 @@ module( # This module is using the same version of rules_python # that the parent module uses. bazel_dep(name = "rules_python", version = "") - -# The story behind this commented out override: -# This override is necessary to generate/update the requirements file -# for this module. This is because running it via the outer -# module doesn't work -- the `requirements.update` target can't find -# the correct file to update. -# Running in the submodule itself works, but submodules using overrides -# is considered an error until Bazel 6.3, which prevents the outer module -# from depending on this module. -# So until 6.3 and higher is the minimum, we leave this commented out. -# local_path_override( -# module_name = "rules_python", -# path = "../../..", -# ) - -PYTHON_NAME_39 = "python_3_9" - -PYTHON_NAME_311 = "python_3_11" +local_path_override( + module_name = "rules_python", + path = "../../..", +) python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.defaults( + # In a submodule this is ignored + python_version = "3.11", +) python.toolchain( configure_coverage_tool = True, - python_version = "3.9", + python_version = "3.12", ) python.toolchain( configure_coverage_tool = True, - # In a submodule this is ignored - is_default = True, python_version = "3.11", ) -# created by the above python.toolchain calls. -use_repo( - python, - "python_versions", - PYTHON_NAME_39, - PYTHON_NAME_311, -) - pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") pip.parse( hub_name = "other_module_pip", diff --git a/examples/bzlmod/other_module/other_module/pkg/BUILD.bazel b/examples/bzlmod/other_module/other_module/pkg/BUILD.bazel index 021c969802..318822d888 100644 --- a/examples/bzlmod/other_module/other_module/pkg/BUILD.bazel +++ b/examples/bzlmod/other_module/other_module/pkg/BUILD.bazel @@ -1,8 +1,6 @@ -load( - "@python_3_11//:defs.bzl", - py_binary_311 = "py_binary", -) -load("@rules_python//python:defs.bzl", "py_library") +load("@rules_python//python:py_binary.bzl", "py_binary") +load("@rules_python//python:py_library.bzl", "py_library") +load("@rules_python//python/zipapp:py_zipapp_binary.bzl", "py_zipapp_binary") py_library( name = "lib", @@ -15,11 +13,12 @@ py_library( # This is used for testing mulitple versions of Python. This is # used only when you need to support multiple versions of Python # in the same project. -py_binary_311( +py_binary( name = "bin", srcs = ["bin.py"], data = ["data/data.txt"], main = "bin.py", + python_version = "3.11", visibility = ["//visibility:public"], deps = [ ":lib", @@ -28,4 +27,13 @@ py_binary_311( ], ) +# This is used for regression testing runfiles paths in submodules. +# https://github.com/bazel-contrib/rules_python/issues/3563. +py_zipapp_binary( + name = "bin_zipapp", + testonly = True, + binary = ":bin", + visibility = ["//visibility:public"], +) + exports_files(["data/data.txt"]) diff --git a/examples/bzlmod/other_module/other_module/rule_builder/BUILD.bazel b/examples/bzlmod/other_module/other_module/rule_builder/BUILD.bazel new file mode 100644 index 0000000000..40f66b3322 --- /dev/null +++ b/examples/bzlmod/other_module/other_module/rule_builder/BUILD.bazel @@ -0,0 +1,13 @@ +load(":rule.bzl", "custom_py_binary") + +# This target's mere existence is the regression test: analyzing it exercises +# the implicit attr defaults (build_data_writer, debugger_if_target_config, +# uncachable_version_file) that py_binary_rule_builder() bundles from +# rules_python's private package, from a rule() call made in this external +# module. +custom_py_binary( + name = "app", + srcs = ["app.py"], + main = "app.py", + visibility = ["//visibility:public"], +) diff --git a/examples/bzlmod/other_module/other_module/rule_builder/app.py b/examples/bzlmod/other_module/other_module/rule_builder/app.py new file mode 100644 index 0000000000..db51494ed6 --- /dev/null +++ b/examples/bzlmod/other_module/other_module/rule_builder/app.py @@ -0,0 +1 @@ +print("hello from a rule built via py_binary_rule_builder()") diff --git a/examples/bzlmod/other_module/other_module/rule_builder/rule.bzl b/examples/bzlmod/other_module/other_module/rule_builder/rule.bzl new file mode 100644 index 0000000000..7b1505987a --- /dev/null +++ b/examples/bzlmod/other_module/other_module/rule_builder/rule.bzl @@ -0,0 +1,17 @@ +"""A minimal custom py_binary built via the executables rule builder API. + +Regression coverage for https://github.com/bazel-contrib/rules_python/pull/3919: +py_binary_rule_builder()'s implicit attr defaults (build_data_writer, +debugger_if_target_config, uncachable_version_file) previously had no +visibility outside of rules_python, so any external module (like this one) +constructing a rule via the builder failed at analysis time with a +visibility error. +""" + +load("@rules_python//python/api:executables.bzl", "executables") + +def _make_rule(): + builder = executables.py_binary_rule_builder() + return builder.build() + +custom_py_binary = _make_rule() diff --git a/examples/bzlmod/py_proto_library/BUILD.bazel b/examples/bzlmod/py_proto_library/BUILD.bazel deleted file mode 100644 index d0bc683021..0000000000 --- a/examples/bzlmod/py_proto_library/BUILD.bazel +++ /dev/null @@ -1,18 +0,0 @@ -load("@rules_python//python:py_test.bzl", "py_test") - -py_test( - name = "pricetag_test", - srcs = ["test.py"], - main = "test.py", - deps = [ - "//py_proto_library/example.com/proto:pricetag_proto_py_pb2", - ], -) - -py_test( - name = "message_test", - srcs = ["message_test.py"], - deps = [ - "//py_proto_library/example.com/another_proto:message_proto_py_pb2", - ], -) diff --git a/examples/bzlmod/py_proto_library/example.com/another_proto/BUILD.bazel b/examples/bzlmod/py_proto_library/example.com/another_proto/BUILD.bazel deleted file mode 100644 index 806fcb9dcc..0000000000 --- a/examples/bzlmod/py_proto_library/example.com/another_proto/BUILD.bazel +++ /dev/null @@ -1,16 +0,0 @@ -load("@rules_proto//proto:defs.bzl", "proto_library") -load("@rules_python//python:proto.bzl", "py_proto_library") - -py_proto_library( - name = "message_proto_py_pb2", - visibility = ["//visibility:public"], - deps = [":message_proto"], -) - -proto_library( - name = "message_proto", - srcs = ["message.proto"], - # https://bazel.build/reference/be/protocol-buffer#proto_library.strip_import_prefix - strip_import_prefix = "/py_proto_library/example.com", - deps = ["//py_proto_library/example.com/proto:pricetag_proto"], -) diff --git a/examples/bzlmod/py_proto_library/example.com/another_proto/message.proto b/examples/bzlmod/py_proto_library/example.com/another_proto/message.proto deleted file mode 100644 index 6e7dcc5793..0000000000 --- a/examples/bzlmod/py_proto_library/example.com/another_proto/message.proto +++ /dev/null @@ -1,10 +0,0 @@ -syntax = "proto3"; - -package rules_python; - -import "proto/pricetag.proto"; - -message TestMessage { - uint32 index = 1; - PriceTag pricetag = 2; -} diff --git a/examples/bzlmod/py_proto_library/example.com/proto/BUILD.bazel b/examples/bzlmod/py_proto_library/example.com/proto/BUILD.bazel deleted file mode 100644 index fa20f2ce94..0000000000 --- a/examples/bzlmod/py_proto_library/example.com/proto/BUILD.bazel +++ /dev/null @@ -1,17 +0,0 @@ -load("@rules_proto//proto:defs.bzl", "proto_library") -load("@rules_python//python:proto.bzl", "py_proto_library") - -py_proto_library( - name = "pricetag_proto_py_pb2", - visibility = ["//visibility:public"], - deps = [":pricetag_proto"], -) - -proto_library( - name = "pricetag_proto", - srcs = ["pricetag.proto"], - # https://bazel.build/reference/be/protocol-buffer#proto_library.strip_import_prefix - strip_import_prefix = "/py_proto_library/example.com", - visibility = ["//visibility:public"], - deps = ["@com_google_protobuf//:any_proto"], -) diff --git a/examples/bzlmod/py_proto_library/example.com/proto/pricetag.proto b/examples/bzlmod/py_proto_library/example.com/proto/pricetag.proto deleted file mode 100644 index 3fa68de84b..0000000000 --- a/examples/bzlmod/py_proto_library/example.com/proto/pricetag.proto +++ /dev/null @@ -1,11 +0,0 @@ -syntax = "proto3"; - -import "google/protobuf/any.proto"; - -package rules_python; - -message PriceTag { - string name = 2; - double cost = 1; - google.protobuf.Any metadata = 3; -} diff --git a/examples/bzlmod/py_proto_library/message_test.py b/examples/bzlmod/py_proto_library/message_test.py deleted file mode 100644 index b1a6942a54..0000000000 --- a/examples/bzlmod/py_proto_library/message_test.py +++ /dev/null @@ -1,16 +0,0 @@ -import sys -import unittest - -from another_proto import message_pb2 - - -class TestCase(unittest.TestCase): - def test_message(self): - got = message_pb2.TestMessage( - index=5, - ) - self.assertIsNotNone(got) - - -if __name__ == "__main__": - sys.exit(unittest.main()) diff --git a/examples/bzlmod/py_proto_library/test.py b/examples/bzlmod/py_proto_library/test.py deleted file mode 100644 index 24ab8ddc70..0000000000 --- a/examples/bzlmod/py_proto_library/test.py +++ /dev/null @@ -1,21 +0,0 @@ -import json -import unittest - -from proto import pricetag_pb2 - - -class TestCase(unittest.TestCase): - def test_pricetag(self): - got = pricetag_pb2.PriceTag( - name="dollar", - cost=5.00, - ) - - metadata = {"description": "some text..."} - got.metadata.value = json.dumps(metadata).encode("utf-8") - - self.assertIsNotNone(got) - - -if __name__ == "__main__": - unittest.main() diff --git a/examples/bzlmod/requirements_lock_3_10.txt b/examples/bzlmod/requirements_lock_3_10.txt index ace879f38e..c7e35a2b2c 100644 --- a/examples/bzlmod/requirements_lock_3_10.txt +++ b/examples/bzlmod/requirements_lock_3_10.txt @@ -50,9 +50,9 @@ isort==5.12.0 \ --hash=sha256:8bef7dde241278824a6d83f44a544709b065191b95b6e50894bdc722fcba0504 \ --hash=sha256:f84c2818376e66cf843d497486ea8fed8700b340f308f076c6fb1229dff318b6 # via pylint -jinja2==3.1.4 \ - --hash=sha256:4a3aee7acbbe7303aede8e9648d13b8bf88a429282aa6122a993f0ac800cb369 \ - --hash=sha256:bc5dd2abb727a5319567b7a813e6a2e7318c39f4f487cfe6c89c6f9c7d25197d +jinja2==3.1.6 \ + --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ + --hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67 # via sphinx lazy-object-proxy==1.9.0 \ --hash=sha256:09763491ce220c0299688940f8dc2c5d05fd1f45af1e42e636b2e8b2303e4382 \ diff --git a/examples/bzlmod/requirements_lock_3_11.txt b/examples/bzlmod/requirements_lock_3_11.txt new file mode 100644 index 0000000000..dd6ec4d29e --- /dev/null +++ b/examples/bzlmod/requirements_lock_3_11.txt @@ -0,0 +1,530 @@ +# This file was autogenerated by uv via the following command: +# bazel run //examples:bzlmod_requirements_3_11.update +--index-url https://pypi.org/simple +--extra-index-url https://pypi.org/simple/ + +alabaster==0.7.16 \ + --hash=sha256:75a8b99c28a5dad50dd7f8ccdd447a121ddb3892da9e53d1ca5cca3106d58d65 \ + --hash=sha256:b46733c07dce03ae4e150330b975c75737fa60f0a7c591b6c8bf4928a28e2c92 + # via sphinx +astroid==2.13.5 \ + --hash=sha256:6891f444625b6edb2ac798829b689e95297e100ddf89dbed5a8c610e34901501 \ + --hash=sha256:df164d5ac811b9f44105a72b8f9d5edfb7b5b2d7e979b04ea377a77b3229114a + # via pylint +babel==2.17.0 \ + --hash=sha256:0c54cffb19f690cdcc52a3b50bcbf71e07a808d1c80d549f2459b9d2cf0afb9d \ + --hash=sha256:4d0b53093fdfb4b21c92b5213dba5a1b23885afa8383709427046b21c366e5f2 + # via sphinx +certifi==2025.10.5 \ + --hash=sha256:0f212c2744a9bb6de0c56639a6f68afe01ecd92d91f14ae897c4fe7bbeeef0de \ + --hash=sha256:47c09d31ccf2acf0be3f701ea53595ee7e0b8fa08801c6624be771df09ae7b43 + # via requests +chardet==4.0.0 \ + --hash=sha256:0d6f53a15db4120f2b08c94f11e7d93d2c911ee118b6b30a04ec3ee8310179fa \ + --hash=sha256:f864054d66fd9118f2e67044ac8981a54775ec5b67aed0441892edb553d21da5 + # via requests +colorama==0.4.6 \ + --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \ + --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6 + # via + # -r examples/bzlmod/requirements.in + # pylint + # sphinx +dill==0.4.0 \ + --hash=sha256:0633f1d2df477324f53a895b02c901fb961bdbf65a17122586ea7019292cbcf0 \ + --hash=sha256:44f54bf6412c2c8464c14e8243eb163690a9800dbe2c367330883b19c7561049 + # via pylint +docutils==0.21.2 \ + --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ + --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2 + # via sphinx +idna==2.10 \ + --hash=sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6 \ + --hash=sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0 + # via requests +imagesize==1.4.1 \ + --hash=sha256:0d8d18d08f840c19d0ee7ca1fd82490fdc3729b7ac93f49870406ddde8ef8d8b \ + --hash=sha256:69150444affb9cb0d5cc5a92b3676f0b2fb7cd9ae39e947a5e11a36b4497cd4a + # via sphinx +isort==5.13.2 \ + --hash=sha256:48fdfcb9face5d58a4f6dde2e72a1fb8dcaf8ab26f95ab49fab84c2ddefb0109 \ + --hash=sha256:8ca5e72a8d85860d5a3fa69b8745237f2939afe12dbf656afbcb47fe72d947a6 + # via pylint +jinja2==3.1.6 \ + --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ + --hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67 + # via sphinx +lazy-object-proxy==1.12.0 \ + --hash=sha256:029d2b355076710505c9545aef5ab3f750d89779310e26ddf2b7b23f6ea03cd8 \ + --hash=sha256:08c465fb5cd23527512f9bd7b4c7ba6cec33e28aad36fbbe46bf7b858f9f3f7f \ + --hash=sha256:0a83c6f7a6b2bfc11ef3ed67f8cbe99f8ff500b05655d8e7df9aab993a6abc95 \ + --hash=sha256:1192e8c2f1031a6ff453ee40213afa01ba765b3dc861302cd91dbdb2e2660b00 \ + --hash=sha256:14e348185adbd03ec17d051e169ec45686dcd840a3779c9d4c10aabe2ca6e1c0 \ + --hash=sha256:15400b18893f345857b9e18b9bd87bd06aba84af6ed086187add70aeaa3f93f1 \ + --hash=sha256:1cf69cd1a6c7fe2dbcc3edaa017cf010f4192e53796538cc7d5e1fedbfa4bcff \ + --hash=sha256:1f5a462d92fd0cfb82f1fab28b51bfb209fabbe6aabf7f0d51472c0c124c0c61 \ + --hash=sha256:256262384ebd2a77b023ad02fbcc9326282bcfd16484d5531154b02bc304f4c5 \ + --hash=sha256:31020c84005d3daa4cc0fa5a310af2066efe6b0d82aeebf9ab199292652ff036 \ + --hash=sha256:338ab2f132276203e404951205fe80c3fd59429b3a724e7b662b2eb539bb1be9 \ + --hash=sha256:3605b632e82a1cbc32a1e5034278a64db555b3496e0795723ee697006b980508 \ + --hash=sha256:3d3964fbd326578bcdfffd017ef101b6fb0484f34e731fe060ba9b8816498c36 \ + --hash=sha256:424a8ab6695400845c39f13c685050eab69fa0bbac5790b201cd27375e5e41d7 \ + --hash=sha256:4a79b909aa16bde8ae606f06e6bbc9d3219d2e57fb3e0076e17879072b742c65 \ + --hash=sha256:4ab2c584e3cc8be0dfca422e05ad30a9abe3555ce63e9ab7a559f62f8dbc6ff9 \ + --hash=sha256:53c7fd99eb156bbb82cbc5d5188891d8fdd805ba6c1e3b92b90092da2a837073 \ + --hash=sha256:563d2ec8e4d4b68ee7848c5ab4d6057a6d703cb7963b342968bb8758dda33a23 \ + --hash=sha256:61d5e3310a4aa5792c2b599a7a78ccf8687292c8eb09cf187cca8f09cf6a7519 \ + --hash=sha256:6763941dbf97eea6b90f5b06eb4da9418cc088fce0e3883f5816090f9afcde4a \ + --hash=sha256:67f07ab742f1adfb3966c40f630baaa7902be4222a17941f3d85fd1dae5565ff \ + --hash=sha256:717484c309df78cedf48396e420fa57fc8a2b1f06ea889df7248fdd156e58847 \ + --hash=sha256:75ba769017b944fcacbf6a80c18b2761a1795b03f8899acdad1f1c39db4409be \ + --hash=sha256:7601ec171c7e8584f8ff3f4e440aa2eebf93e854f04639263875b8c2971f819f \ + --hash=sha256:7b22c2bbfb155706b928ac4d74c1a63ac8552a55ba7fff4445155523ea4067e1 \ + --hash=sha256:800f32b00a47c27446a2b767df7538e6c66a3488632c402b4fb2224f9794f3c0 \ + --hash=sha256:81d1852fb30fab81696f93db1b1e55a5d1ff7940838191062f5f56987d5fcc3e \ + --hash=sha256:86fd61cb2ba249b9f436d789d1356deae69ad3231dc3c0f17293ac535162672e \ + --hash=sha256:8c40b3c9faee2e32bfce0df4ae63f4e73529766893258eca78548bac801c8f66 \ + --hash=sha256:8ee0d6027b760a11cc18281e702c0309dd92da458a74b4c15025d7fc490deede \ + --hash=sha256:997b1d6e10ecc6fb6fe0f2c959791ae59599f41da61d652f6c903d1ee58b7370 \ + --hash=sha256:a61095f5d9d1a743e1e20ec6d6db6c2ca511961777257ebd9b288951b23b44fa \ + --hash=sha256:a6b7ea5ea1ffe15059eb44bcbcb258f97bcb40e139b88152c40d07b1a1dfc9ac \ + --hash=sha256:ae575ad9b674d0029fc077c5231b3bc6b433a3d1a62a8c363df96974b5534728 \ + --hash=sha256:be5fe974e39ceb0d6c9db0663c0464669cf866b2851c73971409b9566e880eab \ + --hash=sha256:be9045646d83f6c2664c1330904b245ae2371b5c57a3195e4028aedc9f999655 \ + --hash=sha256:c1ca33565f698ac1aece152a10f432415d1a2aa9a42dfe23e5ba2bc255ab91f6 \ + --hash=sha256:c3b2e0af1f7f77c4263759c4824316ce458fabe0fceadcd24ef8ca08b2d1e402 \ + --hash=sha256:c4fcbe74fb85df8ba7825fa05eddca764138da752904b378f0ae5ab33a36c308 \ + --hash=sha256:c9defba70ab943f1df98a656247966d7729da2fe9c2d5d85346464bf320820a3 \ + --hash=sha256:cc6e3614eca88b1c8a625fc0a47d0d745e7c3255b21dac0e30b3037c5e3deeb8 \ + --hash=sha256:d01c7819a410f7c255b20799b65d36b414379a30c6f1684c7bd7eb6777338c1b \ + --hash=sha256:efff4375a8c52f55a145dc8487a2108c2140f0bec4151ab4e1843e52eb9987ad \ + --hash=sha256:fdc70d81235fc586b9e3d1aeef7d1553259b62ecaae9db2167a5d2550dcc391a + # via astroid +markupsafe==3.0.3 \ + --hash=sha256:0303439a41979d9e74d18ff5e2dd8c43ed6c6001fd40e5bf2e43f7bd9bbc523f \ + --hash=sha256:068f375c472b3e7acbe2d5318dea141359e6900156b5b2ba06a30b169086b91a \ + --hash=sha256:0bf2a864d67e76e5c9a34dc26ec616a66b9888e25e7b9460e1c76d3293bd9dbf \ + --hash=sha256:0db14f5dafddbb6d9208827849fad01f1a2609380add406671a26386cdf15a19 \ + --hash=sha256:0eb9ff8191e8498cca014656ae6b8d61f39da5f95b488805da4bb029cccbfbaf \ + --hash=sha256:0f4b68347f8c5eab4a13419215bdfd7f8c9b19f2b25520968adfad23eb0ce60c \ + --hash=sha256:1085e7fbddd3be5f89cc898938f42c0b3c711fdcb37d75221de2666af647c175 \ + --hash=sha256:116bb52f642a37c115f517494ea5feb03889e04df47eeff5b130b1808ce7c219 \ + --hash=sha256:12c63dfb4a98206f045aa9563db46507995f7ef6d83b2f68eda65c307c6829eb \ + --hash=sha256:133a43e73a802c5562be9bbcd03d090aa5a1fe899db609c29e8c8d815c5f6de6 \ + --hash=sha256:1353ef0c1b138e1907ae78e2f6c63ff67501122006b0f9abad68fda5f4ffc6ab \ + --hash=sha256:15d939a21d546304880945ca1ecb8a039db6b4dc49b2c5a400387cdae6a62e26 \ + --hash=sha256:177b5253b2834fe3678cb4a5f0059808258584c559193998be2601324fdeafb1 \ + --hash=sha256:1872df69a4de6aead3491198eaf13810b565bdbeec3ae2dc8780f14458ec73ce \ + --hash=sha256:1b4b79e8ebf6b55351f0d91fe80f893b4743f104bff22e90697db1590e47a218 \ + --hash=sha256:1b52b4fb9df4eb9ae465f8d0c228a00624de2334f216f178a995ccdcf82c4634 \ + --hash=sha256:1ba88449deb3de88bd40044603fafffb7bc2b055d626a330323a9ed736661695 \ + --hash=sha256:1cc7ea17a6824959616c525620e387f6dd30fec8cb44f649e31712db02123dad \ + --hash=sha256:218551f6df4868a8d527e3062d0fb968682fe92054e89978594c28e642c43a73 \ + --hash=sha256:26a5784ded40c9e318cfc2bdb30fe164bdb8665ded9cd64d500a34fb42067b1c \ + --hash=sha256:2713baf880df847f2bece4230d4d094280f4e67b1e813eec43b4c0e144a34ffe \ + --hash=sha256:2a15a08b17dd94c53a1da0438822d70ebcd13f8c3a95abe3a9ef9f11a94830aa \ + --hash=sha256:2f981d352f04553a7171b8e44369f2af4055f888dfb147d55e42d29e29e74559 \ + --hash=sha256:32001d6a8fc98c8cb5c947787c5d08b0a50663d139f1305bac5885d98d9b40fa \ + --hash=sha256:3524b778fe5cfb3452a09d31e7b5adefeea8c5be1d43c4f810ba09f2ceb29d37 \ + --hash=sha256:3537e01efc9d4dccdf77221fb1cb3b8e1a38d5428920e0657ce299b20324d758 \ + --hash=sha256:35add3b638a5d900e807944a078b51922212fb3dedb01633a8defc4b01a3c85f \ + --hash=sha256:38664109c14ffc9e7437e86b4dceb442b0096dfe3541d7864d9cbe1da4cf36c8 \ + --hash=sha256:3a7e8ae81ae39e62a41ec302f972ba6ae23a5c5396c8e60113e9066ef893da0d \ + --hash=sha256:3b562dd9e9ea93f13d53989d23a7e775fdfd1066c33494ff43f5418bc8c58a5c \ + --hash=sha256:457a69a9577064c05a97c41f4e65148652db078a3a509039e64d3467b9e7ef97 \ + --hash=sha256:4bd4cd07944443f5a265608cc6aab442e4f74dff8088b0dfc8238647b8f6ae9a \ + --hash=sha256:4e885a3d1efa2eadc93c894a21770e4bc67899e3543680313b09f139e149ab19 \ + --hash=sha256:4faffd047e07c38848ce017e8725090413cd80cbc23d86e55c587bf979e579c9 \ + --hash=sha256:509fa21c6deb7a7a273d629cf5ec029bc209d1a51178615ddf718f5918992ab9 \ + --hash=sha256:5678211cb9333a6468fb8d8be0305520aa073f50d17f089b5b4b477ea6e67fdc \ + --hash=sha256:591ae9f2a647529ca990bc681daebdd52c8791ff06c2bfa05b65163e28102ef2 \ + --hash=sha256:5a7d5dc5140555cf21a6fefbdbf8723f06fcd2f63ef108f2854de715e4422cb4 \ + --hash=sha256:69c0b73548bc525c8cb9a251cddf1931d1db4d2258e9599c28c07ef3580ef354 \ + --hash=sha256:6b5420a1d9450023228968e7e6a9ce57f65d148ab56d2313fcd589eee96a7a50 \ + --hash=sha256:722695808f4b6457b320fdc131280796bdceb04ab50fe1795cd540799ebe1698 \ + --hash=sha256:729586769a26dbceff69f7a7dbbf59ab6572b99d94576a5592625d5b411576b9 \ + --hash=sha256:77f0643abe7495da77fb436f50f8dab76dbc6e5fd25d39589a0f1fe6548bfa2b \ + --hash=sha256:795e7751525cae078558e679d646ae45574b47ed6e7771863fcc079a6171a0fc \ + --hash=sha256:7be7b61bb172e1ed687f1754f8e7484f1c8019780f6f6b0786e76bb01c2ae115 \ + --hash=sha256:7c3fb7d25180895632e5d3148dbdc29ea38ccb7fd210aa27acbd1201a1902c6e \ + --hash=sha256:7e68f88e5b8799aa49c85cd116c932a1ac15caaa3f5db09087854d218359e485 \ + --hash=sha256:83891d0e9fb81a825d9a6d61e3f07550ca70a076484292a70fde82c4b807286f \ + --hash=sha256:8485f406a96febb5140bfeca44a73e3ce5116b2501ac54fe953e488fb1d03b12 \ + --hash=sha256:8709b08f4a89aa7586de0aadc8da56180242ee0ada3999749b183aa23df95025 \ + --hash=sha256:8f71bc33915be5186016f675cd83a1e08523649b0e33efdb898db577ef5bb009 \ + --hash=sha256:915c04ba3851909ce68ccc2b8e2cd691618c4dc4c4232fb7982bca3f41fd8c3d \ + --hash=sha256:949b8d66bc381ee8b007cd945914c721d9aba8e27f71959d750a46f7c282b20b \ + --hash=sha256:94c6f0bb423f739146aec64595853541634bde58b2135f27f61c1ffd1cd4d16a \ + --hash=sha256:9a1abfdc021a164803f4d485104931fb8f8c1efd55bc6b748d2f5774e78b62c5 \ + --hash=sha256:9b79b7a16f7fedff2495d684f2b59b0457c3b493778c9eed31111be64d58279f \ + --hash=sha256:a320721ab5a1aba0a233739394eb907f8c8da5c98c9181d1161e77a0c8e36f2d \ + --hash=sha256:a4afe79fb3de0b7097d81da19090f4df4f8d3a2b3adaa8764138aac2e44f3af1 \ + --hash=sha256:ad2cf8aa28b8c020ab2fc8287b0f823d0a7d8630784c31e9ee5edea20f406287 \ + --hash=sha256:b8512a91625c9b3da6f127803b166b629725e68af71f8184ae7e7d54686a56d6 \ + --hash=sha256:bc51efed119bc9cfdf792cdeaa4d67e8f6fcccab66ed4bfdd6bde3e59bfcbb2f \ + --hash=sha256:bdc919ead48f234740ad807933cdf545180bfbe9342c2bb451556db2ed958581 \ + --hash=sha256:bdd37121970bfd8be76c5fb069c7751683bdf373db1ed6c010162b2a130248ed \ + --hash=sha256:be8813b57049a7dc738189df53d69395eba14fb99345e0a5994914a3864c8a4b \ + --hash=sha256:c0c0b3ade1c0b13b936d7970b1d37a57acde9199dc2aecc4c336773e1d86049c \ + --hash=sha256:c47a551199eb8eb2121d4f0f15ae0f923d31350ab9280078d1e5f12b249e0026 \ + --hash=sha256:c4ffb7ebf07cfe8931028e3e4c85f0357459a3f9f9490886198848f4fa002ec8 \ + --hash=sha256:ccfcd093f13f0f0b7fdd0f198b90053bf7b2f02a3927a30e63f3ccc9df56b676 \ + --hash=sha256:d2ee202e79d8ed691ceebae8e0486bd9a2cd4794cec4824e1c99b6f5009502f6 \ + --hash=sha256:d53197da72cc091b024dd97249dfc7794d6a56530370992a5e1a08983ad9230e \ + --hash=sha256:d6dd0be5b5b189d31db7cda48b91d7e0a9795f31430b7f271219ab30f1d3ac9d \ + --hash=sha256:d88b440e37a16e651bda4c7c2b930eb586fd15ca7406cb39e211fcff3bf3017d \ + --hash=sha256:de8a88e63464af587c950061a5e6a67d3632e36df62b986892331d4620a35c01 \ + --hash=sha256:df2449253ef108a379b8b5d6b43f4b1a8e81a061d6537becd5582fba5f9196d7 \ + --hash=sha256:e1c1493fb6e50ab01d20a22826e57520f1284df32f2d8601fdd90b6304601419 \ + --hash=sha256:e1cf1972137e83c5d4c136c43ced9ac51d0e124706ee1c8aa8532c1287fa8795 \ + --hash=sha256:e2103a929dfa2fcaf9bb4e7c091983a49c9ac3b19c9061b6d5427dd7d14d81a1 \ + --hash=sha256:e56b7d45a839a697b5eb268c82a71bd8c7f6c94d6fd50c3d577fa39a9f1409f5 \ + --hash=sha256:e8afc3f2ccfa24215f8cb28dcf43f0113ac3c37c2f0f0806d8c70e4228c5cf4d \ + --hash=sha256:e8fc20152abba6b83724d7ff268c249fa196d8259ff481f3b1476383f8f24e42 \ + --hash=sha256:eaa9599de571d72e2daf60164784109f19978b327a3910d3e9de8c97b5b70cfe \ + --hash=sha256:ec15a59cf5af7be74194f7ab02d0f59a62bdcf1a537677ce67a2537c9b87fcda \ + --hash=sha256:f190daf01f13c72eac4efd5c430a8de82489d9cff23c364c3ea822545032993e \ + --hash=sha256:f34c41761022dd093b4b6896d4810782ffbabe30f2d443ff5f083e0cbbb8c737 \ + --hash=sha256:f3e98bb3798ead92273dc0e5fd0f31ade220f59a266ffd8a4f6065e0a3ce0523 \ + --hash=sha256:f42d0984e947b8adf7dd6dde396e720934d12c506ce84eea8476409563607591 \ + --hash=sha256:f71a396b3bf33ecaa1626c255855702aca4d3d9fea5e051b41ac59a9c1c41edc \ + --hash=sha256:f9e130248f4462aaa8e2552d547f36ddadbeaa573879158d721bbd33dfe4743a \ + --hash=sha256:fed51ac40f757d41b7c48425901843666a6677e3e8eb0abcff09e4ba6e664f50 + # via jinja2 +mccabe==0.7.0 \ + --hash=sha256:348e0240c33b60bbdf4e523192ef919f28cb2c3d7d5c7794f74009290f236325 \ + --hash=sha256:6c2d30ab6be0e4a46919781807b4f0d834ebdd6c6e3dca0bda5a15f863427b6e + # via pylint +packaging==25.0 \ + --hash=sha256:29572ef2b1f17581046b3a2227d5c611fb25ec70ca1ba8554b24b0e69331a484 \ + --hash=sha256:d443872c98d677bf60f6a1f2f8c1cb748e8fe762d2bf9d3148b5599295b0fc4f + # via sphinx +pathspec==0.12.1 \ + --hash=sha256:a0d503e138a4c123b27490a4f7beda6a01c6f288df0e4a8b79c7eb0dc7b4cc08 \ + --hash=sha256:a482d51503a1ab33b1c67a6c3813a26953dbdc71c31dacaef9a838c4e29f5712 + # via yamllint +platformdirs==4.4.0 \ + --hash=sha256:abd01743f24e5287cd7a5db3752faf1a2d65353f38ec26d98e25a6db65958c85 \ + --hash=sha256:ca753cf4d81dc309bc67b0ea38fd15dc97bc30ce419a7f58d13eb3bf14c4febf + # via pylint +pygments==2.19.2 \ + --hash=sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887 \ + --hash=sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b + # via sphinx +pylint==2.15.10 \ + --hash=sha256:9df0d07e8948a1c3ffa3b6e2d7e6e63d9fb457c5da5b961ed63106594780cc7e \ + --hash=sha256:b3dc5ef7d33858f297ac0d06cc73862f01e4f2e74025ec3eff347ce0bc60baf5 + # via + # -r examples/bzlmod/requirements.in + # pylint-print +pylint-print==1.0.1 \ + --hash=sha256:30aa207e9718ebf4ceb47fb87012092e6d8743aab932aa07aa14a73e750ad3d0 \ + --hash=sha256:a2b2599e7887b93e551db2624c523c1e6e9e58c3be8416cd98d41e4427e2669b + # via -r examples/bzlmod/requirements.in +python-dateutil==2.9.0.post0 \ + --hash=sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3 \ + --hash=sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427 + # via + # -r examples/bzlmod/requirements.in + # s3cmd +python-magic==0.4.27 \ + --hash=sha256:c1ba14b08e4a5f5c31a302b7721239695b2f0f058d125bd5ce1ee36b9d9d3c3b \ + --hash=sha256:c212960ad306f700aa0d01e5d7a325d20548ff97eb9920dcd29513174f0294d3 + # via s3cmd +pyyaml==6.0.3 \ + --hash=sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c \ + --hash=sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a \ + --hash=sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3 \ + --hash=sha256:02ea2dfa234451bbb8772601d7b8e426c2bfa197136796224e50e35a78777956 \ + --hash=sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6 \ + --hash=sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c \ + --hash=sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65 \ + --hash=sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a \ + --hash=sha256:1ebe39cb5fc479422b83de611d14e2c0d3bb2a18bbcb01f229ab3cfbd8fee7a0 \ + --hash=sha256:214ed4befebe12df36bcc8bc2b64b396ca31be9304b8f59e25c11cf94a4c033b \ + --hash=sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1 \ + --hash=sha256:22ba7cfcad58ef3ecddc7ed1db3409af68d023b7f940da23c6c2a1890976eda6 \ + --hash=sha256:27c0abcb4a5dac13684a37f76e701e054692a9b2d3064b70f5e4eb54810553d7 \ + --hash=sha256:28c8d926f98f432f88adc23edf2e6d4921ac26fb084b028c733d01868d19007e \ + --hash=sha256:2e71d11abed7344e42a8849600193d15b6def118602c4c176f748e4583246007 \ + --hash=sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310 \ + --hash=sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4 \ + --hash=sha256:3c5677e12444c15717b902a5798264fa7909e41153cdf9ef7ad571b704a63dd9 \ + --hash=sha256:3ff07ec89bae51176c0549bc4c63aa6202991da2d9a6129d7aef7f1407d3f295 \ + --hash=sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea \ + --hash=sha256:418cf3f2111bc80e0933b2cd8cd04f286338bb88bdc7bc8e6dd775ebde60b5e0 \ + --hash=sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e \ + --hash=sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac \ + --hash=sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9 \ + --hash=sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7 \ + --hash=sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35 \ + --hash=sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb \ + --hash=sha256:5cf4e27da7e3fbed4d6c3d8e797387aaad68102272f8f9752883bc32d61cb87b \ + --hash=sha256:5e0b74767e5f8c593e8c9b5912019159ed0533c70051e9cce3e8b6aa699fcd69 \ + --hash=sha256:5ed875a24292240029e4483f9d4a4b8a1ae08843b9c54f43fcc11e404532a8a5 \ + --hash=sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b \ + --hash=sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c \ + --hash=sha256:6344df0d5755a2c9a276d4473ae6b90647e216ab4757f8426893b5dd2ac3f369 \ + --hash=sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd \ + --hash=sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824 \ + --hash=sha256:66291b10affd76d76f54fad28e22e51719ef9ba22b29e1d7d03d6777a9174198 \ + --hash=sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065 \ + --hash=sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c \ + --hash=sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c \ + --hash=sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764 \ + --hash=sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196 \ + --hash=sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b \ + --hash=sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00 \ + --hash=sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac \ + --hash=sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8 \ + --hash=sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e \ + --hash=sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28 \ + --hash=sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3 \ + --hash=sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5 \ + --hash=sha256:9c57bb8c96f6d1808c030b1687b9b5fb476abaa47f0db9c0101f5e9f394e97f4 \ + --hash=sha256:9c7708761fccb9397fe64bbc0395abcae8c4bf7b0eac081e12b809bf47700d0b \ + --hash=sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf \ + --hash=sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5 \ + --hash=sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702 \ + --hash=sha256:b30236e45cf30d2b8e7b3e85881719e98507abed1011bf463a8fa23e9c3e98a8 \ + --hash=sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788 \ + --hash=sha256:b865addae83924361678b652338317d1bd7e79b1f4596f96b96c77a5a34b34da \ + --hash=sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d \ + --hash=sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc \ + --hash=sha256:bdb2c67c6c1390b63c6ff89f210c8fd09d9a1217a465701eac7316313c915e4c \ + --hash=sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba \ + --hash=sha256:c2514fceb77bc5e7a2f7adfaa1feb2fb311607c9cb518dbc378688ec73d8292f \ + --hash=sha256:c3355370a2c156cffb25e876646f149d5d68f5e0a3ce86a5084dd0b64a994917 \ + --hash=sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5 \ + --hash=sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26 \ + --hash=sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f \ + --hash=sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b \ + --hash=sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be \ + --hash=sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c \ + --hash=sha256:efd7b85f94a6f21e4932043973a7ba2613b059c4a000551892ac9f1d11f5baf3 \ + --hash=sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6 \ + --hash=sha256:fa160448684b4e94d80416c0fa4aac48967a969efe22931448d853ada8baf926 \ + --hash=sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0 + # via yamllint +requests==2.25.1 \ + --hash=sha256:27973dd4a904a4f13b263a19c866c13b92a39ed1c964655f025f3f8d3d75b804 \ + --hash=sha256:c210084e36a42ae6b9219e00e48287def368a26d03a048ddad7bfee44f75871e + # via + # -r examples/bzlmod/requirements.in + # sphinx +s3cmd==2.1.0 \ + --hash=sha256:49cd23d516b17974b22b611a95ce4d93fe326feaa07320bd1d234fed68cbccfa \ + --hash=sha256:966b0a494a916fc3b4324de38f089c86c70ee90e8e1cae6d59102103a4c0cc03 + # via -r examples/bzlmod/requirements.in +six==1.17.0 \ + --hash=sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274 \ + --hash=sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81 + # via python-dateutil +snowballstemmer==3.0.1 \ + --hash=sha256:6cd7b3897da8d6c9ffb968a6781fa6532dce9c3618a4b127d920dab764a19064 \ + --hash=sha256:6d5eeeec8e9f84d4d56b847692bacf79bc2c8e90c7f80ca4444ff8b6f2e52895 + # via sphinx +sphinx==7.3.7 \ + --hash=sha256:413f75440be4cacf328f580b4274ada4565fb2187d696a84970c23f77b64d8c3 \ + --hash=sha256:a4a7db75ed37531c05002d56ed6948d4c42f473a36f46e1382b0bd76ca9627bc + # via -r examples/bzlmod/requirements.in +sphinxcontrib-applehelp==2.0.0 \ + --hash=sha256:2f29ef331735ce958efa4734873f084941970894c6090408b079c61b2e1c06d1 \ + --hash=sha256:4cd3f0ec4ac5dd9c17ec65e9ab272c9b867ea77425228e68ecf08d6b28ddbdb5 + # via sphinx +sphinxcontrib-devhelp==2.0.0 \ + --hash=sha256:411f5d96d445d1d73bb5d52133377b4248ec79db5c793ce7dbe59e074b4dd1ad \ + --hash=sha256:aefb8b83854e4b0998877524d1029fd3e6879210422ee3780459e28a1f03a8a2 + # via sphinx +sphinxcontrib-htmlhelp==2.1.0 \ + --hash=sha256:166759820b47002d22914d64a075ce08f4c46818e17cfc9470a9786b759b19f8 \ + --hash=sha256:c9e2916ace8aad64cc13a0d233ee22317f2b9025b9cf3295249fa985cc7082e9 + # via sphinx +sphinxcontrib-jsmath==1.0.1 \ + --hash=sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178 \ + --hash=sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8 + # via sphinx +sphinxcontrib-qthelp==2.0.0 \ + --hash=sha256:4fe7d0ac8fc171045be623aba3e2a8f613f8682731f9153bb2e40ece16b9bbab \ + --hash=sha256:b18a828cdba941ccd6ee8445dbe72ffa3ef8cbe7505d8cd1fa0d42d3f2d5f3eb + # via sphinx +sphinxcontrib-serializinghtml==2.0.0 \ + --hash=sha256:6e2cb0eef194e10c27ec0023bfeb25badbbb5868244cf5bc5bdc04e4464bf331 \ + --hash=sha256:e9d912827f872c029017a53f0ef2180b327c3f7fd23c87229f7a8e8b70031d4d + # via + # -r examples/bzlmod/requirements.in + # sphinx +tabulate==0.9.0 \ + --hash=sha256:0095b12bf5966de529c0feb1fa08671671b3368eec77d7ef7ab114be2c068b3c \ + --hash=sha256:024ca478df22e9340661486f85298cff5f6dcdba14f3813e8830015b9ed1948f + # via -r examples/bzlmod/requirements.in +tomlkit==0.13.3 \ + --hash=sha256:430cf247ee57df2b94ee3fbe588e71d362a941ebb545dec29b53961d61add2a1 \ + --hash=sha256:c89c649d79ee40629a9fda55f8ace8c6a1b42deb912b2a8fd8d942ddadb606b0 + # via pylint +urllib3==1.26.20 \ + --hash=sha256:0ed14ccfbf1c30a9072c7ca157e4319b70d65f623e91e7b32fadb2853431016e \ + --hash=sha256:40c2dc0c681e47eb8f90e7e27bf6ff7df2e677421fd46756da1161c39ca70d32 + # via requests +websockets==15.0.1 \ + --hash=sha256:0701bc3cfcb9164d04a14b149fd74be7347a530ad3bbf15ab2c678a2cd3dd9a2 \ + --hash=sha256:0a34631031a8f05657e8e90903e656959234f3a04552259458aac0b0f9ae6fd9 \ + --hash=sha256:0af68c55afbd5f07986df82831c7bff04846928ea8d1fd7f30052638788bc9b5 \ + --hash=sha256:0c9e74d766f2818bb95f84c25be4dea09841ac0f734d1966f415e4edfc4ef1c3 \ + --hash=sha256:0f3c1e2ab208db911594ae5b4f79addeb3501604a165019dd221c0bdcabe4db8 \ + --hash=sha256:0fdfe3e2a29e4db3659dbd5bbf04560cea53dd9610273917799f1cde46aa725e \ + --hash=sha256:1009ee0c7739c08a0cd59de430d6de452a55e42d6b522de7aa15e6f67db0b8e1 \ + --hash=sha256:1234d4ef35db82f5446dca8e35a7da7964d02c127b095e172e54397fb6a6c256 \ + --hash=sha256:16b6c1b3e57799b9d38427dda63edcbe4926352c47cf88588c0be4ace18dac85 \ + --hash=sha256:2034693ad3097d5355bfdacfffcbd3ef5694f9718ab7f29c29689a9eae841880 \ + --hash=sha256:21c1fa28a6a7e3cbdc171c694398b6df4744613ce9b36b1a498e816787e28123 \ + --hash=sha256:229cf1d3ca6c1804400b0a9790dc66528e08a6a1feec0d5040e8b9eb14422375 \ + --hash=sha256:27ccee0071a0e75d22cb35849b1db43f2ecd3e161041ac1ee9d2352ddf72f065 \ + --hash=sha256:363c6f671b761efcb30608d24925a382497c12c506b51661883c3e22337265ed \ + --hash=sha256:39c1fec2c11dc8d89bba6b2bf1556af381611a173ac2b511cf7231622058af41 \ + --hash=sha256:3b1ac0d3e594bf121308112697cf4b32be538fb1444468fb0a6ae4feebc83411 \ + --hash=sha256:3be571a8b5afed347da347bfcf27ba12b069d9d7f42cb8c7028b5e98bbb12597 \ + --hash=sha256:3c714d2fc58b5ca3e285461a4cc0c9a66bd0e24c5da9911e30158286c9b5be7f \ + --hash=sha256:3d00075aa65772e7ce9e990cab3ff1de702aa09be3940d1dc88d5abf1ab8a09c \ + --hash=sha256:3e90baa811a5d73f3ca0bcbf32064d663ed81318ab225ee4f427ad4e26e5aff3 \ + --hash=sha256:47819cea040f31d670cc8d324bb6435c6f133b8c7a19ec3d61634e62f8d8f9eb \ + --hash=sha256:47b099e1f4fbc95b701b6e85768e1fcdaf1630f3cbe4765fa216596f12310e2e \ + --hash=sha256:4a9fac8e469d04ce6c25bb2610dc535235bd4aa14996b4e6dbebf5e007eba5ee \ + --hash=sha256:4b826973a4a2ae47ba357e4e82fa44a463b8f168e1ca775ac64521442b19e87f \ + --hash=sha256:4c2529b320eb9e35af0fa3016c187dffb84a3ecc572bcee7c3ce302bfeba52bf \ + --hash=sha256:54479983bd5fb469c38f2f5c7e3a24f9a4e70594cd68cd1fa6b9340dadaff7cf \ + --hash=sha256:558d023b3df0bffe50a04e710bc87742de35060580a293c2a984299ed83bc4e4 \ + --hash=sha256:5756779642579d902eed757b21b0164cd6fe338506a8083eb58af5c372e39d9a \ + --hash=sha256:592f1a9fe869c778694f0aa806ba0374e97648ab57936f092fd9d87f8bc03665 \ + --hash=sha256:595b6c3969023ecf9041b2936ac3827e4623bfa3ccf007575f04c5a6aa318c22 \ + --hash=sha256:5a939de6b7b4e18ca683218320fc67ea886038265fd1ed30173f5ce3f8e85675 \ + --hash=sha256:5d54b09eba2bada6011aea5375542a157637b91029687eb4fdb2dab11059c1b4 \ + --hash=sha256:5df592cd503496351d6dc14f7cdad49f268d8e618f80dce0cd5a36b93c3fc08d \ + --hash=sha256:5f4c04ead5aed67c8a1a20491d54cdfba5884507a48dd798ecaf13c74c4489f5 \ + --hash=sha256:64dee438fed052b52e4f98f76c5790513235efaa1ef7f3f2192c392cd7c91b65 \ + --hash=sha256:66dd88c918e3287efc22409d426c8f729688d89a0c587c88971a0faa2c2f3792 \ + --hash=sha256:678999709e68425ae2593acf2e3ebcbcf2e69885a5ee78f9eb80e6e371f1bf57 \ + --hash=sha256:67f2b6de947f8c757db2db9c71527933ad0019737ec374a8a6be9a956786aaf9 \ + --hash=sha256:693f0192126df6c2327cce3baa7c06f2a117575e32ab2308f7f8216c29d9e2e3 \ + --hash=sha256:746ee8dba912cd6fc889a8147168991d50ed70447bf18bcda7039f7d2e3d9151 \ + --hash=sha256:756c56e867a90fb00177d530dca4b097dd753cde348448a1012ed6c5131f8b7d \ + --hash=sha256:76d1f20b1c7a2fa82367e04982e708723ba0e7b8d43aa643d3dcd404d74f1475 \ + --hash=sha256:7f493881579c90fc262d9cdbaa05a6b54b3811c2f300766748db79f098db9940 \ + --hash=sha256:823c248b690b2fd9303ba00c4f66cd5e2d8c3ba4aa968b2779be9532a4dad431 \ + --hash=sha256:82544de02076bafba038ce055ee6412d68da13ab47f0c60cab827346de828dee \ + --hash=sha256:8dd8327c795b3e3f219760fa603dcae1dcc148172290a8ab15158cf85a953413 \ + --hash=sha256:8fdc51055e6ff4adeb88d58a11042ec9a5eae317a0a53d12c062c8a8865909e8 \ + --hash=sha256:a625e06551975f4b7ea7102bc43895b90742746797e2e14b70ed61c43a90f09b \ + --hash=sha256:abdc0c6c8c648b4805c5eacd131910d2a7f6455dfd3becab248ef108e89ab16a \ + --hash=sha256:ac017dd64572e5c3bd01939121e4d16cf30e5d7e110a119399cf3133b63ad054 \ + --hash=sha256:ac1e5c9054fe23226fb11e05a6e630837f074174c4c2f0fe442996112a6de4fb \ + --hash=sha256:ac60e3b188ec7574cb761b08d50fcedf9d77f1530352db4eef1707fe9dee7205 \ + --hash=sha256:b359ed09954d7c18bbc1680f380c7301f92c60bf924171629c5db97febb12f04 \ + --hash=sha256:b7643a03db5c95c799b89b31c036d5f27eeb4d259c798e878d6937d71832b1e4 \ + --hash=sha256:ba9e56e8ceeeedb2e080147ba85ffcd5cd0711b89576b83784d8605a7df455fa \ + --hash=sha256:c338ffa0520bdb12fbc527265235639fb76e7bc7faafbb93f6ba80d9c06578a9 \ + --hash=sha256:cad21560da69f4ce7658ca2cb83138fb4cf695a2ba3e475e0559e05991aa8122 \ + --hash=sha256:d08eb4c2b7d6c41da6ca0600c077e93f5adcfd979cd777d747e9ee624556da4b \ + --hash=sha256:d50fd1ee42388dcfb2b3676132c78116490976f1300da28eb629272d5d93e905 \ + --hash=sha256:d591f8de75824cbb7acad4e05d2d710484f15f29d4a915092675ad3456f11770 \ + --hash=sha256:d5f6b181bb38171a8ad1d6aa58a67a6aa9d4b38d0f8c5f496b9e42561dfc62fe \ + --hash=sha256:d63efaa0cd96cf0c5fe4d581521d9fa87744540d4bc999ae6e08595a1014b45b \ + --hash=sha256:d99e5546bf73dbad5bf3547174cd6cb8ba7273062a23808ffea025ecb1cf8562 \ + --hash=sha256:e09473f095a819042ecb2ab9465aee615bd9c2028e4ef7d933600a8401c79561 \ + --hash=sha256:e8b56bdcdb4505c8078cb6c7157d9811a85790f2f2b3632c7d1462ab5783d215 \ + --hash=sha256:ee443ef070bb3b6ed74514f5efaa37a252af57c90eb33b956d35c8e9c10a1931 \ + --hash=sha256:f29d80eb9a9263b8d109135351caf568cc3f80b9928bccde535c235de55c22d9 \ + --hash=sha256:f7a866fbc1e97b5c617ee4116daaa09b722101d4a3c170c787450ba409f9736f \ + --hash=sha256:fcd5cf9e305d7b8338754470cf69cf81f420459dbae8a3b40cee57417f4614a7 + # via -r examples/bzlmod/requirements.in +wheel==0.45.1 \ + --hash=sha256:661e1abd9198507b1409a20c02106d9670b2576e916d58f520316666abca6729 \ + --hash=sha256:708e7481cc80179af0e556bbf0cc00b8444c7321e2700b8d8580231d13017248 + # via -r examples/bzlmod/requirements.in +wrapt==1.17.3 \ + --hash=sha256:02b551d101f31694fc785e58e0720ef7d9a10c4e62c1c9358ce6f63f23e30a56 \ + --hash=sha256:042ec3bb8f319c147b1301f2393bc19dba6e176b7da446853406d041c36c7828 \ + --hash=sha256:0610b46293c59a3adbae3dee552b648b984176f8562ee0dba099a56cfbe4df1f \ + --hash=sha256:0b02e424deef65c9f7326d8c19220a2c9040c51dc165cddb732f16198c168396 \ + --hash=sha256:0b1831115c97f0663cb77aa27d381237e73ad4f721391a9bfb2fe8bc25fa6e77 \ + --hash=sha256:0ed61b7c2d49cee3c027372df5809a59d60cf1b6c2f81ee980a091f3afed6a2d \ + --hash=sha256:0f5f51a6466667a5a356e6381d362d259125b57f059103dd9fdc8c0cf1d14139 \ + --hash=sha256:16ecf15d6af39246fe33e507105d67e4b81d8f8d2c6598ff7e3ca1b8a37213f7 \ + --hash=sha256:1f0b2f40cf341ee8cc1a97d51ff50dddb9fcc73241b9143ec74b30fc4f44f6cb \ + --hash=sha256:1f23fa283f51c890eda8e34e4937079114c74b4c81d2b2f1f1d94948f5cc3d7f \ + --hash=sha256:223db574bb38637e8230eb14b185565023ab624474df94d2af18f1cdb625216f \ + --hash=sha256:249f88ed15503f6492a71f01442abddd73856a0032ae860de6d75ca62eed8067 \ + --hash=sha256:24c2ed34dc222ed754247a2702b1e1e89fdbaa4016f324b4b8f1a802d4ffe87f \ + --hash=sha256:273a736c4645e63ac582c60a56b0acb529ef07f78e08dc6bfadf6a46b19c0da7 \ + --hash=sha256:281262213373b6d5e4bb4353bc36d1ba4084e6d6b5d242863721ef2bf2c2930b \ + --hash=sha256:30ce38e66630599e1193798285706903110d4f057aab3168a34b7fdc85569afc \ + --hash=sha256:33486899acd2d7d3066156b03465b949da3fd41a5da6e394ec49d271baefcf05 \ + --hash=sha256:343e44b2a8e60e06a7e0d29c1671a0d9951f59174f3709962b5143f60a2a98bd \ + --hash=sha256:373342dd05b1d07d752cecbec0c41817231f29f3a89aa8b8843f7b95992ed0c7 \ + --hash=sha256:3af60380ba0b7b5aeb329bc4e402acd25bd877e98b3727b0135cb5c2efdaefe9 \ + --hash=sha256:3e62d15d3cfa26e3d0788094de7b64efa75f3a53875cdbccdf78547aed547a81 \ + --hash=sha256:41b1d2bc74c2cac6f9074df52b2efbef2b30bdfe5f40cb78f8ca22963bc62977 \ + --hash=sha256:423ed5420ad5f5529db9ce89eac09c8a2f97da18eb1c870237e84c5a5c2d60aa \ + --hash=sha256:46acc57b331e0b3bcb3e1ca3b421d65637915cfcd65eb783cb2f78a511193f9b \ + --hash=sha256:4da9f45279fff3543c371d5ababc57a0384f70be244de7759c85a7f989cb4ebe \ + --hash=sha256:507553480670cab08a800b9463bdb881b2edeed77dc677b0a5915e6106e91a58 \ + --hash=sha256:53e5e39ff71b3fc484df8a522c933ea2b7cdd0d5d15ae82e5b23fde87d44cbd8 \ + --hash=sha256:54a30837587c6ee3cd1a4d1c2ec5d24e77984d44e2f34547e2323ddb4e22eb77 \ + --hash=sha256:5531d911795e3f935a9c23eb1c8c03c211661a5060aab167065896bbf62a5f85 \ + --hash=sha256:55cbbc356c2842f39bcc553cf695932e8b30e30e797f961860afb308e6b1bb7c \ + --hash=sha256:59923aa12d0157f6b82d686c3fd8e1166fa8cdfb3e17b42ce3b6147ff81528df \ + --hash=sha256:5a03a38adec8066d5a37bea22f2ba6bbf39fcdefbe2d91419ab864c3fb515454 \ + --hash=sha256:5a7b3c1ee8265eb4c8f1b7d29943f195c00673f5ab60c192eba2d4a7eae5f46a \ + --hash=sha256:5d4478d72eb61c36e5b446e375bbc49ed002430d17cdec3cecb36993398e1a9e \ + --hash=sha256:5ea5eb3c0c071862997d6f3e02af1d055f381b1d25b286b9d6644b79db77657c \ + --hash=sha256:604d076c55e2fdd4c1c03d06dc1a31b95130010517b5019db15365ec4a405fc6 \ + --hash=sha256:656873859b3b50eeebe6db8b1455e99d90c26ab058db8e427046dbc35c3140a5 \ + --hash=sha256:65d1d00fbfb3ea5f20add88bbc0f815150dbbde3b026e6c24759466c8b5a9ef9 \ + --hash=sha256:6b538e31eca1a7ea4605e44f81a48aa24c4632a277431a6ed3f328835901f4fd \ + --hash=sha256:6fd1ad24dc235e4ab88cda009e19bf347aabb975e44fd5c2fb22a3f6e4141277 \ + --hash=sha256:70d86fa5197b8947a2fa70260b48e400bf2ccacdcab97bb7de47e3d1e6312225 \ + --hash=sha256:7171ae35d2c33d326ac19dd8facb1e82e5fd04ef8c6c0e394d7af55a55051c22 \ + --hash=sha256:73d496de46cd2cdbdbcce4ae4bcdb4afb6a11234a1df9c085249d55166b95116 \ + --hash=sha256:7425ac3c54430f5fc5e7b6f41d41e704db073309acfc09305816bc6a0b26bb16 \ + --hash=sha256:74afa28374a3c3a11b3b5e5fca0ae03bef8450d6aa3ab3a1e2c30e3a75d023dc \ + --hash=sha256:758895b01d546812d1f42204bd443b8c433c44d090248bf22689df673ccafe00 \ + --hash=sha256:79573c24a46ce11aab457b472efd8d125e5a51da2d1d24387666cd85f54c05b2 \ + --hash=sha256:7e18f01b0c3e4a07fe6dfdb00e29049ba17eadbc5e7609a2a3a4af83ab7d710a \ + --hash=sha256:88547535b787a6c9ce4086917b6e1d291aa8ed914fdd3a838b3539dc95c12804 \ + --hash=sha256:88bbae4d40d5a46142e70d58bf664a89b6b4befaea7b2ecc14e03cedb8e06c04 \ + --hash=sha256:8cccf4f81371f257440c88faed6b74f1053eef90807b77e31ca057b2db74edb1 \ + --hash=sha256:9baa544e6acc91130e926e8c802a17f3b16fbea0fd441b5a60f5cf2cc5c3deba \ + --hash=sha256:a36692b8491d30a8c75f1dfee65bef119d6f39ea84ee04d9f9311f83c5ad9390 \ + --hash=sha256:a47681378a0439215912ef542c45a783484d4dd82bac412b71e59cf9c0e1cea0 \ + --hash=sha256:a7c06742645f914f26c7f1fa47b8bc4c91d222f76ee20116c43d5ef0912bba2d \ + --hash=sha256:a9a2203361a6e6404f80b99234fe7fb37d1fc73487b5a78dc1aa5b97201e0f22 \ + --hash=sha256:ab232e7fdb44cdfbf55fc3afa31bcdb0d8980b9b95c38b6405df2acb672af0e0 \ + --hash=sha256:ad85e269fe54d506b240d2d7b9f5f2057c2aa9a2ea5b32c66f8902f768117ed2 \ + --hash=sha256:af338aa93554be859173c39c85243970dc6a289fa907402289eeae7543e1ae18 \ + --hash=sha256:afd964fd43b10c12213574db492cb8f73b2f0826c8df07a68288f8f19af2ebe6 \ + --hash=sha256:b32888aad8b6e68f83a8fdccbf3165f5469702a7544472bdf41f582970ed3311 \ + --hash=sha256:c31eebe420a9a5d2887b13000b043ff6ca27c452a9a22fa71f35f118e8d4bf89 \ + --hash=sha256:caea3e9c79d5f0d2c6d9ab96111601797ea5da8e6d0723f77eabb0d4068d2b2f \ + --hash=sha256:cf30f6e3c077c8e6a9a7809c94551203c8843e74ba0c960f4a98cd80d4665d39 \ + --hash=sha256:d40770d7c0fd5cbed9d84b2c3f2e156431a12c9a37dc6284060fb4bec0b7ffd4 \ + --hash=sha256:d8a210b158a34164de8bb68b0e7780041a903d7b00c87e906fb69928bf7890d5 \ + --hash=sha256:dc4a8d2b25efb6681ecacad42fca8859f88092d8732b170de6a5dddd80a1c8fa \ + --hash=sha256:df7d30371a2accfe4013e90445f6388c570f103d61019b6b7c57e0265250072a \ + --hash=sha256:e01375f275f010fcbf7f643b4279896d04e571889b8a5b3f848423d91bf07050 \ + --hash=sha256:e1a4120ae5705f673727d3253de3ed0e016f7cd78dc463db1b31e2463e1f3cf6 \ + --hash=sha256:e228514a06843cae89621384cfe3a80418f3c04aadf8a3b14e46a7be704e4235 \ + --hash=sha256:e405adefb53a435f01efa7ccdec012c016b5a1d3f35459990afc39b6be4d5056 \ + --hash=sha256:e6b13af258d6a9ad602d57d889f83b9d5543acd471eee12eb51f5b01f8eb1bc2 \ + --hash=sha256:e6f40a8aa5a92f150bdb3e1c44b7e98fb7113955b2e5394122fa5532fec4b418 \ + --hash=sha256:e71d5c6ebac14875668a1e90baf2ea0ef5b7ac7918355850c0908ae82bcb297c \ + --hash=sha256:ed7c635ae45cfbc1a7371f708727bf74690daedc49b4dba310590ca0bd28aa8a \ + --hash=sha256:f38e60678850c42461d4202739f9bf1e3a737c7ad283638251e79cc49effb6b6 \ + --hash=sha256:f66eb08feaa410fe4eebd17f2a2c8e2e46d3476e9f8c783daa8e09e0faa666d0 \ + --hash=sha256:f9b2601381be482f70e5d1051a5965c25fb3625455a2bf520b5a077b22afb775 \ + --hash=sha256:fbd3c8319de8e1dc79d346929cd71d523622da527cca14e0c1d257e31c2b8b10 \ + --hash=sha256:fd341868a4b6714a5962c1af0bd44f7c404ef78720c7de4892901e540417111c + # via astroid +yamllint==1.37.1 \ + --hash=sha256:364f0d79e81409f591e323725e6a9f4504c8699ddf2d7263d8d2b539cd66a583 \ + --hash=sha256:81f7c0c5559becc8049470d86046b36e96113637bcbe4753ecef06977c00245d + # via -r examples/bzlmod/requirements.in diff --git a/examples/bzlmod/requirements_lock_3_9.txt b/examples/bzlmod/requirements_lock_3_9.txt index bfabfd5fa5..8a6d41441a 100644 --- a/examples/bzlmod/requirements_lock_3_9.txt +++ b/examples/bzlmod/requirements_lock_3_9.txt @@ -46,7 +46,7 @@ imagesize==1.4.1 \ --hash=sha256:0d8d18d08f840c19d0ee7ca1fd82490fdc3729b7ac93f49870406ddde8ef8d8b \ --hash=sha256:69150444affb9cb0d5cc5a92b3676f0b2fb7cd9ae39e947a5e11a36b4497cd4a # via sphinx -importlib-metadata==8.4.0 ; python_version < '3.10' \ +importlib-metadata==8.4.0 ; python_full_version < '3.10' \ --hash=sha256:66f342cc6ac9818fc6ff340576acd24d65ba0b3efabb2b4ac08b598965a4a2f1 \ --hash=sha256:9a547d3bc3608b025f93d403fdd1aae741c24fbb8314df4b155675742ce303c5 # via sphinx @@ -54,9 +54,9 @@ isort==5.11.4 \ --hash=sha256:6db30c5ded9815d813932c04c2f85a360bcdd35fed496f4d8f35495ef0a261b6 \ --hash=sha256:c033fd0edb91000a7f09527fe5c75321878f98322a77ddcc81adbd83724afb7b # via pylint -jinja2==3.1.4 \ - --hash=sha256:4a3aee7acbbe7303aede8e9648d13b8bf88a429282aa6122a993f0ac800cb369 \ - --hash=sha256:bc5dd2abb727a5319567b7a813e6a2e7318c39f4f487cfe6c89c6f9c7d25197d +jinja2==3.1.6 \ + --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ + --hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67 # via sphinx lazy-object-proxy==1.10.0 \ --hash=sha256:009e6bb1f1935a62889ddc8541514b6a9e1fcf302667dcb049a0be5c8f613e56 \ @@ -262,9 +262,9 @@ s3cmd==2.1.0 \ --hash=sha256:49cd23d516b17974b22b611a95ce4d93fe326feaa07320bd1d234fed68cbccfa \ --hash=sha256:966b0a494a916fc3b4324de38f089c86c70ee90e8e1cae6d59102103a4c0cc03 # via -r examples/bzlmod/requirements.in -setuptools==65.6.3 \ - --hash=sha256:57f6f22bde4e042978bcd50176fdb381d7c21a9efa4041202288d3737a0c6a54 \ - --hash=sha256:a7620757bf984b58deaf32fc8a4577a9bbc0850cf92c20e1ce41c38c19e5fb75 +setuptools==78.1.1 \ + --hash=sha256:c3a9c4211ff4c309edb8b8c4f1cbfa7ae324c4ba9f91ff254e3d305b9fd54561 \ + --hash=sha256:fcc17fd9cd898242f6b4adfaca46137a9edef687f43e6f78469692a5e70d851d # via # babel # yamllint @@ -316,7 +316,7 @@ tabulate==0.9.0 \ --hash=sha256:0095b12bf5966de529c0feb1fa08671671b3368eec77d7ef7ab114be2c068b3c \ --hash=sha256:024ca478df22e9340661486f85298cff5f6dcdba14f3813e8830015b9ed1948f # via -r examples/bzlmod/requirements.in -tomli==2.0.1 ; python_version < '3.11' \ +tomli==2.0.1 ; python_full_version < '3.11' \ --hash=sha256:939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc \ --hash=sha256:de526c12914f0c550d15924c62d72abc48d6fe7364aa87328337a31007fe8a4f # via pylint @@ -324,7 +324,7 @@ tomlkit==0.11.6 \ --hash=sha256:07de26b0d8cfc18f871aec595fda24d95b08fef89d147caa861939f37230bf4b \ --hash=sha256:71b952e5721688937fb02cf9d354dbcf0785066149d2855e44531ebdd2b65d73 # via pylint -typing-extensions==4.12.2 ; python_version < '3.10' \ +typing-extensions==4.12.2 ; python_full_version < '3.10' \ --hash=sha256:04e5ca0351e0f3f85c6853954072df659d0d13fac324d0072316b67d7794700d \ --hash=sha256:1a7ead55c7e559dd4dee8856e3a88b41225abfe1ce8df57b7c13915fe121ffb8 # via @@ -480,7 +480,7 @@ yamllint==1.28.0 \ --hash=sha256:89bb5b5ac33b1ade059743cf227de73daa34d5e5a474b06a5e17fc16583b0cf2 \ --hash=sha256:9e3d8ddd16d0583214c5fdffe806c9344086721f107435f68bad990e5a88826b # via -r examples/bzlmod/requirements.in -zipp==3.20.0 ; python_version < '3.10' \ +zipp==3.20.0 ; python_full_version < '3.10' \ --hash=sha256:0145e43d89664cfe1a2e533adc75adafed82fe2da404b4bbb6b026c0157bdb31 \ --hash=sha256:58da6168be89f0be59beb194da1250516fdaa062ccebd30127ac65d30045e10d # via importlib-metadata diff --git a/examples/bzlmod/requirements_windows_3_10.txt b/examples/bzlmod/requirements_windows_3_10.txt index e4373c1682..0e43dbfe6b 100644 --- a/examples/bzlmod/requirements_windows_3_10.txt +++ b/examples/bzlmod/requirements_windows_3_10.txt @@ -53,9 +53,9 @@ isort==5.12.0 \ --hash=sha256:8bef7dde241278824a6d83f44a544709b065191b95b6e50894bdc722fcba0504 \ --hash=sha256:f84c2818376e66cf843d497486ea8fed8700b340f308f076c6fb1229dff318b6 # via pylint -jinja2==3.1.4 \ - --hash=sha256:4a3aee7acbbe7303aede8e9648d13b8bf88a429282aa6122a993f0ac800cb369 \ - --hash=sha256:bc5dd2abb727a5319567b7a813e6a2e7318c39f4f487cfe6c89c6f9c7d25197d +jinja2==3.1.6 \ + --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ + --hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67 # via sphinx lazy-object-proxy==1.9.0 \ --hash=sha256:09763491ce220c0299688940f8dc2c5d05fd1f45af1e42e636b2e8b2303e4382 \ diff --git a/examples/bzlmod/requirements_windows_3_11.txt b/examples/bzlmod/requirements_windows_3_11.txt new file mode 100644 index 0000000000..fd7895b273 --- /dev/null +++ b/examples/bzlmod/requirements_windows_3_11.txt @@ -0,0 +1,530 @@ +# This file was autogenerated by uv via the following command: +# bazel run //examples:bzlmod_requirements_3_11_windows.update +--index-url https://pypi.org/simple +--extra-index-url https://pypi.org/simple/ + +alabaster==0.7.16 \ + --hash=sha256:75a8b99c28a5dad50dd7f8ccdd447a121ddb3892da9e53d1ca5cca3106d58d65 \ + --hash=sha256:b46733c07dce03ae4e150330b975c75737fa60f0a7c591b6c8bf4928a28e2c92 + # via sphinx +astroid==2.13.5 \ + --hash=sha256:6891f444625b6edb2ac798829b689e95297e100ddf89dbed5a8c610e34901501 \ + --hash=sha256:df164d5ac811b9f44105a72b8f9d5edfb7b5b2d7e979b04ea377a77b3229114a + # via pylint +babel==2.17.0 \ + --hash=sha256:0c54cffb19f690cdcc52a3b50bcbf71e07a808d1c80d549f2459b9d2cf0afb9d \ + --hash=sha256:4d0b53093fdfb4b21c92b5213dba5a1b23885afa8383709427046b21c366e5f2 + # via sphinx +certifi==2025.10.5 \ + --hash=sha256:0f212c2744a9bb6de0c56639a6f68afe01ecd92d91f14ae897c4fe7bbeeef0de \ + --hash=sha256:47c09d31ccf2acf0be3f701ea53595ee7e0b8fa08801c6624be771df09ae7b43 + # via requests +chardet==4.0.0 \ + --hash=sha256:0d6f53a15db4120f2b08c94f11e7d93d2c911ee118b6b30a04ec3ee8310179fa \ + --hash=sha256:f864054d66fd9118f2e67044ac8981a54775ec5b67aed0441892edb553d21da5 + # via requests +colorama==0.4.6 \ + --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \ + --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6 + # via + # -r examples/bzlmod/requirements.in + # pylint + # sphinx +dill==0.4.0 \ + --hash=sha256:0633f1d2df477324f53a895b02c901fb961bdbf65a17122586ea7019292cbcf0 \ + --hash=sha256:44f54bf6412c2c8464c14e8243eb163690a9800dbe2c367330883b19c7561049 + # via pylint +docutils==0.21.2 \ + --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ + --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2 + # via sphinx +idna==2.10 \ + --hash=sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6 \ + --hash=sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0 + # via requests +imagesize==1.4.1 \ + --hash=sha256:0d8d18d08f840c19d0ee7ca1fd82490fdc3729b7ac93f49870406ddde8ef8d8b \ + --hash=sha256:69150444affb9cb0d5cc5a92b3676f0b2fb7cd9ae39e947a5e11a36b4497cd4a + # via sphinx +isort==5.13.2 \ + --hash=sha256:48fdfcb9face5d58a4f6dde2e72a1fb8dcaf8ab26f95ab49fab84c2ddefb0109 \ + --hash=sha256:8ca5e72a8d85860d5a3fa69b8745237f2939afe12dbf656afbcb47fe72d947a6 + # via pylint +jinja2==3.1.6 \ + --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ + --hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67 + # via sphinx +lazy-object-proxy==1.12.0 \ + --hash=sha256:029d2b355076710505c9545aef5ab3f750d89779310e26ddf2b7b23f6ea03cd8 \ + --hash=sha256:08c465fb5cd23527512f9bd7b4c7ba6cec33e28aad36fbbe46bf7b858f9f3f7f \ + --hash=sha256:0a83c6f7a6b2bfc11ef3ed67f8cbe99f8ff500b05655d8e7df9aab993a6abc95 \ + --hash=sha256:1192e8c2f1031a6ff453ee40213afa01ba765b3dc861302cd91dbdb2e2660b00 \ + --hash=sha256:14e348185adbd03ec17d051e169ec45686dcd840a3779c9d4c10aabe2ca6e1c0 \ + --hash=sha256:15400b18893f345857b9e18b9bd87bd06aba84af6ed086187add70aeaa3f93f1 \ + --hash=sha256:1cf69cd1a6c7fe2dbcc3edaa017cf010f4192e53796538cc7d5e1fedbfa4bcff \ + --hash=sha256:1f5a462d92fd0cfb82f1fab28b51bfb209fabbe6aabf7f0d51472c0c124c0c61 \ + --hash=sha256:256262384ebd2a77b023ad02fbcc9326282bcfd16484d5531154b02bc304f4c5 \ + --hash=sha256:31020c84005d3daa4cc0fa5a310af2066efe6b0d82aeebf9ab199292652ff036 \ + --hash=sha256:338ab2f132276203e404951205fe80c3fd59429b3a724e7b662b2eb539bb1be9 \ + --hash=sha256:3605b632e82a1cbc32a1e5034278a64db555b3496e0795723ee697006b980508 \ + --hash=sha256:3d3964fbd326578bcdfffd017ef101b6fb0484f34e731fe060ba9b8816498c36 \ + --hash=sha256:424a8ab6695400845c39f13c685050eab69fa0bbac5790b201cd27375e5e41d7 \ + --hash=sha256:4a79b909aa16bde8ae606f06e6bbc9d3219d2e57fb3e0076e17879072b742c65 \ + --hash=sha256:4ab2c584e3cc8be0dfca422e05ad30a9abe3555ce63e9ab7a559f62f8dbc6ff9 \ + --hash=sha256:53c7fd99eb156bbb82cbc5d5188891d8fdd805ba6c1e3b92b90092da2a837073 \ + --hash=sha256:563d2ec8e4d4b68ee7848c5ab4d6057a6d703cb7963b342968bb8758dda33a23 \ + --hash=sha256:61d5e3310a4aa5792c2b599a7a78ccf8687292c8eb09cf187cca8f09cf6a7519 \ + --hash=sha256:6763941dbf97eea6b90f5b06eb4da9418cc088fce0e3883f5816090f9afcde4a \ + --hash=sha256:67f07ab742f1adfb3966c40f630baaa7902be4222a17941f3d85fd1dae5565ff \ + --hash=sha256:717484c309df78cedf48396e420fa57fc8a2b1f06ea889df7248fdd156e58847 \ + --hash=sha256:75ba769017b944fcacbf6a80c18b2761a1795b03f8899acdad1f1c39db4409be \ + --hash=sha256:7601ec171c7e8584f8ff3f4e440aa2eebf93e854f04639263875b8c2971f819f \ + --hash=sha256:7b22c2bbfb155706b928ac4d74c1a63ac8552a55ba7fff4445155523ea4067e1 \ + --hash=sha256:800f32b00a47c27446a2b767df7538e6c66a3488632c402b4fb2224f9794f3c0 \ + --hash=sha256:81d1852fb30fab81696f93db1b1e55a5d1ff7940838191062f5f56987d5fcc3e \ + --hash=sha256:86fd61cb2ba249b9f436d789d1356deae69ad3231dc3c0f17293ac535162672e \ + --hash=sha256:8c40b3c9faee2e32bfce0df4ae63f4e73529766893258eca78548bac801c8f66 \ + --hash=sha256:8ee0d6027b760a11cc18281e702c0309dd92da458a74b4c15025d7fc490deede \ + --hash=sha256:997b1d6e10ecc6fb6fe0f2c959791ae59599f41da61d652f6c903d1ee58b7370 \ + --hash=sha256:a61095f5d9d1a743e1e20ec6d6db6c2ca511961777257ebd9b288951b23b44fa \ + --hash=sha256:a6b7ea5ea1ffe15059eb44bcbcb258f97bcb40e139b88152c40d07b1a1dfc9ac \ + --hash=sha256:ae575ad9b674d0029fc077c5231b3bc6b433a3d1a62a8c363df96974b5534728 \ + --hash=sha256:be5fe974e39ceb0d6c9db0663c0464669cf866b2851c73971409b9566e880eab \ + --hash=sha256:be9045646d83f6c2664c1330904b245ae2371b5c57a3195e4028aedc9f999655 \ + --hash=sha256:c1ca33565f698ac1aece152a10f432415d1a2aa9a42dfe23e5ba2bc255ab91f6 \ + --hash=sha256:c3b2e0af1f7f77c4263759c4824316ce458fabe0fceadcd24ef8ca08b2d1e402 \ + --hash=sha256:c4fcbe74fb85df8ba7825fa05eddca764138da752904b378f0ae5ab33a36c308 \ + --hash=sha256:c9defba70ab943f1df98a656247966d7729da2fe9c2d5d85346464bf320820a3 \ + --hash=sha256:cc6e3614eca88b1c8a625fc0a47d0d745e7c3255b21dac0e30b3037c5e3deeb8 \ + --hash=sha256:d01c7819a410f7c255b20799b65d36b414379a30c6f1684c7bd7eb6777338c1b \ + --hash=sha256:efff4375a8c52f55a145dc8487a2108c2140f0bec4151ab4e1843e52eb9987ad \ + --hash=sha256:fdc70d81235fc586b9e3d1aeef7d1553259b62ecaae9db2167a5d2550dcc391a + # via astroid +markupsafe==3.0.3 \ + --hash=sha256:0303439a41979d9e74d18ff5e2dd8c43ed6c6001fd40e5bf2e43f7bd9bbc523f \ + --hash=sha256:068f375c472b3e7acbe2d5318dea141359e6900156b5b2ba06a30b169086b91a \ + --hash=sha256:0bf2a864d67e76e5c9a34dc26ec616a66b9888e25e7b9460e1c76d3293bd9dbf \ + --hash=sha256:0db14f5dafddbb6d9208827849fad01f1a2609380add406671a26386cdf15a19 \ + --hash=sha256:0eb9ff8191e8498cca014656ae6b8d61f39da5f95b488805da4bb029cccbfbaf \ + --hash=sha256:0f4b68347f8c5eab4a13419215bdfd7f8c9b19f2b25520968adfad23eb0ce60c \ + --hash=sha256:1085e7fbddd3be5f89cc898938f42c0b3c711fdcb37d75221de2666af647c175 \ + --hash=sha256:116bb52f642a37c115f517494ea5feb03889e04df47eeff5b130b1808ce7c219 \ + --hash=sha256:12c63dfb4a98206f045aa9563db46507995f7ef6d83b2f68eda65c307c6829eb \ + --hash=sha256:133a43e73a802c5562be9bbcd03d090aa5a1fe899db609c29e8c8d815c5f6de6 \ + --hash=sha256:1353ef0c1b138e1907ae78e2f6c63ff67501122006b0f9abad68fda5f4ffc6ab \ + --hash=sha256:15d939a21d546304880945ca1ecb8a039db6b4dc49b2c5a400387cdae6a62e26 \ + --hash=sha256:177b5253b2834fe3678cb4a5f0059808258584c559193998be2601324fdeafb1 \ + --hash=sha256:1872df69a4de6aead3491198eaf13810b565bdbeec3ae2dc8780f14458ec73ce \ + --hash=sha256:1b4b79e8ebf6b55351f0d91fe80f893b4743f104bff22e90697db1590e47a218 \ + --hash=sha256:1b52b4fb9df4eb9ae465f8d0c228a00624de2334f216f178a995ccdcf82c4634 \ + --hash=sha256:1ba88449deb3de88bd40044603fafffb7bc2b055d626a330323a9ed736661695 \ + --hash=sha256:1cc7ea17a6824959616c525620e387f6dd30fec8cb44f649e31712db02123dad \ + --hash=sha256:218551f6df4868a8d527e3062d0fb968682fe92054e89978594c28e642c43a73 \ + --hash=sha256:26a5784ded40c9e318cfc2bdb30fe164bdb8665ded9cd64d500a34fb42067b1c \ + --hash=sha256:2713baf880df847f2bece4230d4d094280f4e67b1e813eec43b4c0e144a34ffe \ + --hash=sha256:2a15a08b17dd94c53a1da0438822d70ebcd13f8c3a95abe3a9ef9f11a94830aa \ + --hash=sha256:2f981d352f04553a7171b8e44369f2af4055f888dfb147d55e42d29e29e74559 \ + --hash=sha256:32001d6a8fc98c8cb5c947787c5d08b0a50663d139f1305bac5885d98d9b40fa \ + --hash=sha256:3524b778fe5cfb3452a09d31e7b5adefeea8c5be1d43c4f810ba09f2ceb29d37 \ + --hash=sha256:3537e01efc9d4dccdf77221fb1cb3b8e1a38d5428920e0657ce299b20324d758 \ + --hash=sha256:35add3b638a5d900e807944a078b51922212fb3dedb01633a8defc4b01a3c85f \ + --hash=sha256:38664109c14ffc9e7437e86b4dceb442b0096dfe3541d7864d9cbe1da4cf36c8 \ + --hash=sha256:3a7e8ae81ae39e62a41ec302f972ba6ae23a5c5396c8e60113e9066ef893da0d \ + --hash=sha256:3b562dd9e9ea93f13d53989d23a7e775fdfd1066c33494ff43f5418bc8c58a5c \ + --hash=sha256:457a69a9577064c05a97c41f4e65148652db078a3a509039e64d3467b9e7ef97 \ + --hash=sha256:4bd4cd07944443f5a265608cc6aab442e4f74dff8088b0dfc8238647b8f6ae9a \ + --hash=sha256:4e885a3d1efa2eadc93c894a21770e4bc67899e3543680313b09f139e149ab19 \ + --hash=sha256:4faffd047e07c38848ce017e8725090413cd80cbc23d86e55c587bf979e579c9 \ + --hash=sha256:509fa21c6deb7a7a273d629cf5ec029bc209d1a51178615ddf718f5918992ab9 \ + --hash=sha256:5678211cb9333a6468fb8d8be0305520aa073f50d17f089b5b4b477ea6e67fdc \ + --hash=sha256:591ae9f2a647529ca990bc681daebdd52c8791ff06c2bfa05b65163e28102ef2 \ + --hash=sha256:5a7d5dc5140555cf21a6fefbdbf8723f06fcd2f63ef108f2854de715e4422cb4 \ + --hash=sha256:69c0b73548bc525c8cb9a251cddf1931d1db4d2258e9599c28c07ef3580ef354 \ + --hash=sha256:6b5420a1d9450023228968e7e6a9ce57f65d148ab56d2313fcd589eee96a7a50 \ + --hash=sha256:722695808f4b6457b320fdc131280796bdceb04ab50fe1795cd540799ebe1698 \ + --hash=sha256:729586769a26dbceff69f7a7dbbf59ab6572b99d94576a5592625d5b411576b9 \ + --hash=sha256:77f0643abe7495da77fb436f50f8dab76dbc6e5fd25d39589a0f1fe6548bfa2b \ + --hash=sha256:795e7751525cae078558e679d646ae45574b47ed6e7771863fcc079a6171a0fc \ + --hash=sha256:7be7b61bb172e1ed687f1754f8e7484f1c8019780f6f6b0786e76bb01c2ae115 \ + --hash=sha256:7c3fb7d25180895632e5d3148dbdc29ea38ccb7fd210aa27acbd1201a1902c6e \ + --hash=sha256:7e68f88e5b8799aa49c85cd116c932a1ac15caaa3f5db09087854d218359e485 \ + --hash=sha256:83891d0e9fb81a825d9a6d61e3f07550ca70a076484292a70fde82c4b807286f \ + --hash=sha256:8485f406a96febb5140bfeca44a73e3ce5116b2501ac54fe953e488fb1d03b12 \ + --hash=sha256:8709b08f4a89aa7586de0aadc8da56180242ee0ada3999749b183aa23df95025 \ + --hash=sha256:8f71bc33915be5186016f675cd83a1e08523649b0e33efdb898db577ef5bb009 \ + --hash=sha256:915c04ba3851909ce68ccc2b8e2cd691618c4dc4c4232fb7982bca3f41fd8c3d \ + --hash=sha256:949b8d66bc381ee8b007cd945914c721d9aba8e27f71959d750a46f7c282b20b \ + --hash=sha256:94c6f0bb423f739146aec64595853541634bde58b2135f27f61c1ffd1cd4d16a \ + --hash=sha256:9a1abfdc021a164803f4d485104931fb8f8c1efd55bc6b748d2f5774e78b62c5 \ + --hash=sha256:9b79b7a16f7fedff2495d684f2b59b0457c3b493778c9eed31111be64d58279f \ + --hash=sha256:a320721ab5a1aba0a233739394eb907f8c8da5c98c9181d1161e77a0c8e36f2d \ + --hash=sha256:a4afe79fb3de0b7097d81da19090f4df4f8d3a2b3adaa8764138aac2e44f3af1 \ + --hash=sha256:ad2cf8aa28b8c020ab2fc8287b0f823d0a7d8630784c31e9ee5edea20f406287 \ + --hash=sha256:b8512a91625c9b3da6f127803b166b629725e68af71f8184ae7e7d54686a56d6 \ + --hash=sha256:bc51efed119bc9cfdf792cdeaa4d67e8f6fcccab66ed4bfdd6bde3e59bfcbb2f \ + --hash=sha256:bdc919ead48f234740ad807933cdf545180bfbe9342c2bb451556db2ed958581 \ + --hash=sha256:bdd37121970bfd8be76c5fb069c7751683bdf373db1ed6c010162b2a130248ed \ + --hash=sha256:be8813b57049a7dc738189df53d69395eba14fb99345e0a5994914a3864c8a4b \ + --hash=sha256:c0c0b3ade1c0b13b936d7970b1d37a57acde9199dc2aecc4c336773e1d86049c \ + --hash=sha256:c47a551199eb8eb2121d4f0f15ae0f923d31350ab9280078d1e5f12b249e0026 \ + --hash=sha256:c4ffb7ebf07cfe8931028e3e4c85f0357459a3f9f9490886198848f4fa002ec8 \ + --hash=sha256:ccfcd093f13f0f0b7fdd0f198b90053bf7b2f02a3927a30e63f3ccc9df56b676 \ + --hash=sha256:d2ee202e79d8ed691ceebae8e0486bd9a2cd4794cec4824e1c99b6f5009502f6 \ + --hash=sha256:d53197da72cc091b024dd97249dfc7794d6a56530370992a5e1a08983ad9230e \ + --hash=sha256:d6dd0be5b5b189d31db7cda48b91d7e0a9795f31430b7f271219ab30f1d3ac9d \ + --hash=sha256:d88b440e37a16e651bda4c7c2b930eb586fd15ca7406cb39e211fcff3bf3017d \ + --hash=sha256:de8a88e63464af587c950061a5e6a67d3632e36df62b986892331d4620a35c01 \ + --hash=sha256:df2449253ef108a379b8b5d6b43f4b1a8e81a061d6537becd5582fba5f9196d7 \ + --hash=sha256:e1c1493fb6e50ab01d20a22826e57520f1284df32f2d8601fdd90b6304601419 \ + --hash=sha256:e1cf1972137e83c5d4c136c43ced9ac51d0e124706ee1c8aa8532c1287fa8795 \ + --hash=sha256:e2103a929dfa2fcaf9bb4e7c091983a49c9ac3b19c9061b6d5427dd7d14d81a1 \ + --hash=sha256:e56b7d45a839a697b5eb268c82a71bd8c7f6c94d6fd50c3d577fa39a9f1409f5 \ + --hash=sha256:e8afc3f2ccfa24215f8cb28dcf43f0113ac3c37c2f0f0806d8c70e4228c5cf4d \ + --hash=sha256:e8fc20152abba6b83724d7ff268c249fa196d8259ff481f3b1476383f8f24e42 \ + --hash=sha256:eaa9599de571d72e2daf60164784109f19978b327a3910d3e9de8c97b5b70cfe \ + --hash=sha256:ec15a59cf5af7be74194f7ab02d0f59a62bdcf1a537677ce67a2537c9b87fcda \ + --hash=sha256:f190daf01f13c72eac4efd5c430a8de82489d9cff23c364c3ea822545032993e \ + --hash=sha256:f34c41761022dd093b4b6896d4810782ffbabe30f2d443ff5f083e0cbbb8c737 \ + --hash=sha256:f3e98bb3798ead92273dc0e5fd0f31ade220f59a266ffd8a4f6065e0a3ce0523 \ + --hash=sha256:f42d0984e947b8adf7dd6dde396e720934d12c506ce84eea8476409563607591 \ + --hash=sha256:f71a396b3bf33ecaa1626c255855702aca4d3d9fea5e051b41ac59a9c1c41edc \ + --hash=sha256:f9e130248f4462aaa8e2552d547f36ddadbeaa573879158d721bbd33dfe4743a \ + --hash=sha256:fed51ac40f757d41b7c48425901843666a6677e3e8eb0abcff09e4ba6e664f50 + # via jinja2 +mccabe==0.7.0 \ + --hash=sha256:348e0240c33b60bbdf4e523192ef919f28cb2c3d7d5c7794f74009290f236325 \ + --hash=sha256:6c2d30ab6be0e4a46919781807b4f0d834ebdd6c6e3dca0bda5a15f863427b6e + # via pylint +packaging==25.0 \ + --hash=sha256:29572ef2b1f17581046b3a2227d5c611fb25ec70ca1ba8554b24b0e69331a484 \ + --hash=sha256:d443872c98d677bf60f6a1f2f8c1cb748e8fe762d2bf9d3148b5599295b0fc4f + # via sphinx +pathspec==0.12.1 \ + --hash=sha256:a0d503e138a4c123b27490a4f7beda6a01c6f288df0e4a8b79c7eb0dc7b4cc08 \ + --hash=sha256:a482d51503a1ab33b1c67a6c3813a26953dbdc71c31dacaef9a838c4e29f5712 + # via yamllint +platformdirs==4.4.0 \ + --hash=sha256:abd01743f24e5287cd7a5db3752faf1a2d65353f38ec26d98e25a6db65958c85 \ + --hash=sha256:ca753cf4d81dc309bc67b0ea38fd15dc97bc30ce419a7f58d13eb3bf14c4febf + # via pylint +pygments==2.19.2 \ + --hash=sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887 \ + --hash=sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b + # via sphinx +pylint==2.15.10 \ + --hash=sha256:9df0d07e8948a1c3ffa3b6e2d7e6e63d9fb457c5da5b961ed63106594780cc7e \ + --hash=sha256:b3dc5ef7d33858f297ac0d06cc73862f01e4f2e74025ec3eff347ce0bc60baf5 + # via + # -r examples/bzlmod/requirements.in + # pylint-print +pylint-print==1.0.1 \ + --hash=sha256:30aa207e9718ebf4ceb47fb87012092e6d8743aab932aa07aa14a73e750ad3d0 \ + --hash=sha256:a2b2599e7887b93e551db2624c523c1e6e9e58c3be8416cd98d41e4427e2669b + # via -r examples/bzlmod/requirements.in +python-dateutil==2.9.0.post0 \ + --hash=sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3 \ + --hash=sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427 + # via + # -r examples/bzlmod/requirements.in + # s3cmd +python-magic==0.4.27 \ + --hash=sha256:c1ba14b08e4a5f5c31a302b7721239695b2f0f058d125bd5ce1ee36b9d9d3c3b \ + --hash=sha256:c212960ad306f700aa0d01e5d7a325d20548ff97eb9920dcd29513174f0294d3 + # via s3cmd +pyyaml==6.0.3 \ + --hash=sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c \ + --hash=sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a \ + --hash=sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3 \ + --hash=sha256:02ea2dfa234451bbb8772601d7b8e426c2bfa197136796224e50e35a78777956 \ + --hash=sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6 \ + --hash=sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c \ + --hash=sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65 \ + --hash=sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a \ + --hash=sha256:1ebe39cb5fc479422b83de611d14e2c0d3bb2a18bbcb01f229ab3cfbd8fee7a0 \ + --hash=sha256:214ed4befebe12df36bcc8bc2b64b396ca31be9304b8f59e25c11cf94a4c033b \ + --hash=sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1 \ + --hash=sha256:22ba7cfcad58ef3ecddc7ed1db3409af68d023b7f940da23c6c2a1890976eda6 \ + --hash=sha256:27c0abcb4a5dac13684a37f76e701e054692a9b2d3064b70f5e4eb54810553d7 \ + --hash=sha256:28c8d926f98f432f88adc23edf2e6d4921ac26fb084b028c733d01868d19007e \ + --hash=sha256:2e71d11abed7344e42a8849600193d15b6def118602c4c176f748e4583246007 \ + --hash=sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310 \ + --hash=sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4 \ + --hash=sha256:3c5677e12444c15717b902a5798264fa7909e41153cdf9ef7ad571b704a63dd9 \ + --hash=sha256:3ff07ec89bae51176c0549bc4c63aa6202991da2d9a6129d7aef7f1407d3f295 \ + --hash=sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea \ + --hash=sha256:418cf3f2111bc80e0933b2cd8cd04f286338bb88bdc7bc8e6dd775ebde60b5e0 \ + --hash=sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e \ + --hash=sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac \ + --hash=sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9 \ + --hash=sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7 \ + --hash=sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35 \ + --hash=sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb \ + --hash=sha256:5cf4e27da7e3fbed4d6c3d8e797387aaad68102272f8f9752883bc32d61cb87b \ + --hash=sha256:5e0b74767e5f8c593e8c9b5912019159ed0533c70051e9cce3e8b6aa699fcd69 \ + --hash=sha256:5ed875a24292240029e4483f9d4a4b8a1ae08843b9c54f43fcc11e404532a8a5 \ + --hash=sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b \ + --hash=sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c \ + --hash=sha256:6344df0d5755a2c9a276d4473ae6b90647e216ab4757f8426893b5dd2ac3f369 \ + --hash=sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd \ + --hash=sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824 \ + --hash=sha256:66291b10affd76d76f54fad28e22e51719ef9ba22b29e1d7d03d6777a9174198 \ + --hash=sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065 \ + --hash=sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c \ + --hash=sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c \ + --hash=sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764 \ + --hash=sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196 \ + --hash=sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b \ + --hash=sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00 \ + --hash=sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac \ + --hash=sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8 \ + --hash=sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e \ + --hash=sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28 \ + --hash=sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3 \ + --hash=sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5 \ + --hash=sha256:9c57bb8c96f6d1808c030b1687b9b5fb476abaa47f0db9c0101f5e9f394e97f4 \ + --hash=sha256:9c7708761fccb9397fe64bbc0395abcae8c4bf7b0eac081e12b809bf47700d0b \ + --hash=sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf \ + --hash=sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5 \ + --hash=sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702 \ + --hash=sha256:b30236e45cf30d2b8e7b3e85881719e98507abed1011bf463a8fa23e9c3e98a8 \ + --hash=sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788 \ + --hash=sha256:b865addae83924361678b652338317d1bd7e79b1f4596f96b96c77a5a34b34da \ + --hash=sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d \ + --hash=sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc \ + --hash=sha256:bdb2c67c6c1390b63c6ff89f210c8fd09d9a1217a465701eac7316313c915e4c \ + --hash=sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba \ + --hash=sha256:c2514fceb77bc5e7a2f7adfaa1feb2fb311607c9cb518dbc378688ec73d8292f \ + --hash=sha256:c3355370a2c156cffb25e876646f149d5d68f5e0a3ce86a5084dd0b64a994917 \ + --hash=sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5 \ + --hash=sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26 \ + --hash=sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f \ + --hash=sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b \ + --hash=sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be \ + --hash=sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c \ + --hash=sha256:efd7b85f94a6f21e4932043973a7ba2613b059c4a000551892ac9f1d11f5baf3 \ + --hash=sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6 \ + --hash=sha256:fa160448684b4e94d80416c0fa4aac48967a969efe22931448d853ada8baf926 \ + --hash=sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0 + # via yamllint +requests==2.25.1 \ + --hash=sha256:27973dd4a904a4f13b263a19c866c13b92a39ed1c964655f025f3f8d3d75b804 \ + --hash=sha256:c210084e36a42ae6b9219e00e48287def368a26d03a048ddad7bfee44f75871e + # via + # -r examples/bzlmod/requirements.in + # sphinx +s3cmd==2.1.0 \ + --hash=sha256:49cd23d516b17974b22b611a95ce4d93fe326feaa07320bd1d234fed68cbccfa \ + --hash=sha256:966b0a494a916fc3b4324de38f089c86c70ee90e8e1cae6d59102103a4c0cc03 + # via -r examples/bzlmod/requirements.in +six==1.17.0 \ + --hash=sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274 \ + --hash=sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81 + # via python-dateutil +snowballstemmer==3.0.1 \ + --hash=sha256:6cd7b3897da8d6c9ffb968a6781fa6532dce9c3618a4b127d920dab764a19064 \ + --hash=sha256:6d5eeeec8e9f84d4d56b847692bacf79bc2c8e90c7f80ca4444ff8b6f2e52895 + # via sphinx +sphinx==7.3.7 \ + --hash=sha256:413f75440be4cacf328f580b4274ada4565fb2187d696a84970c23f77b64d8c3 \ + --hash=sha256:a4a7db75ed37531c05002d56ed6948d4c42f473a36f46e1382b0bd76ca9627bc + # via -r examples/bzlmod/requirements.in +sphinxcontrib-applehelp==2.0.0 \ + --hash=sha256:2f29ef331735ce958efa4734873f084941970894c6090408b079c61b2e1c06d1 \ + --hash=sha256:4cd3f0ec4ac5dd9c17ec65e9ab272c9b867ea77425228e68ecf08d6b28ddbdb5 + # via sphinx +sphinxcontrib-devhelp==2.0.0 \ + --hash=sha256:411f5d96d445d1d73bb5d52133377b4248ec79db5c793ce7dbe59e074b4dd1ad \ + --hash=sha256:aefb8b83854e4b0998877524d1029fd3e6879210422ee3780459e28a1f03a8a2 + # via sphinx +sphinxcontrib-htmlhelp==2.1.0 \ + --hash=sha256:166759820b47002d22914d64a075ce08f4c46818e17cfc9470a9786b759b19f8 \ + --hash=sha256:c9e2916ace8aad64cc13a0d233ee22317f2b9025b9cf3295249fa985cc7082e9 + # via sphinx +sphinxcontrib-jsmath==1.0.1 \ + --hash=sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178 \ + --hash=sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8 + # via sphinx +sphinxcontrib-qthelp==2.0.0 \ + --hash=sha256:4fe7d0ac8fc171045be623aba3e2a8f613f8682731f9153bb2e40ece16b9bbab \ + --hash=sha256:b18a828cdba941ccd6ee8445dbe72ffa3ef8cbe7505d8cd1fa0d42d3f2d5f3eb + # via sphinx +sphinxcontrib-serializinghtml==2.0.0 \ + --hash=sha256:6e2cb0eef194e10c27ec0023bfeb25badbbb5868244cf5bc5bdc04e4464bf331 \ + --hash=sha256:e9d912827f872c029017a53f0ef2180b327c3f7fd23c87229f7a8e8b70031d4d + # via + # -r examples/bzlmod/requirements.in + # sphinx +tabulate==0.9.0 \ + --hash=sha256:0095b12bf5966de529c0feb1fa08671671b3368eec77d7ef7ab114be2c068b3c \ + --hash=sha256:024ca478df22e9340661486f85298cff5f6dcdba14f3813e8830015b9ed1948f + # via -r examples/bzlmod/requirements.in +tomlkit==0.13.3 \ + --hash=sha256:430cf247ee57df2b94ee3fbe588e71d362a941ebb545dec29b53961d61add2a1 \ + --hash=sha256:c89c649d79ee40629a9fda55f8ace8c6a1b42deb912b2a8fd8d942ddadb606b0 + # via pylint +urllib3==1.26.20 \ + --hash=sha256:0ed14ccfbf1c30a9072c7ca157e4319b70d65f623e91e7b32fadb2853431016e \ + --hash=sha256:40c2dc0c681e47eb8f90e7e27bf6ff7df2e677421fd46756da1161c39ca70d32 + # via requests +websockets==15.0.1 \ + --hash=sha256:0701bc3cfcb9164d04a14b149fd74be7347a530ad3bbf15ab2c678a2cd3dd9a2 \ + --hash=sha256:0a34631031a8f05657e8e90903e656959234f3a04552259458aac0b0f9ae6fd9 \ + --hash=sha256:0af68c55afbd5f07986df82831c7bff04846928ea8d1fd7f30052638788bc9b5 \ + --hash=sha256:0c9e74d766f2818bb95f84c25be4dea09841ac0f734d1966f415e4edfc4ef1c3 \ + --hash=sha256:0f3c1e2ab208db911594ae5b4f79addeb3501604a165019dd221c0bdcabe4db8 \ + --hash=sha256:0fdfe3e2a29e4db3659dbd5bbf04560cea53dd9610273917799f1cde46aa725e \ + --hash=sha256:1009ee0c7739c08a0cd59de430d6de452a55e42d6b522de7aa15e6f67db0b8e1 \ + --hash=sha256:1234d4ef35db82f5446dca8e35a7da7964d02c127b095e172e54397fb6a6c256 \ + --hash=sha256:16b6c1b3e57799b9d38427dda63edcbe4926352c47cf88588c0be4ace18dac85 \ + --hash=sha256:2034693ad3097d5355bfdacfffcbd3ef5694f9718ab7f29c29689a9eae841880 \ + --hash=sha256:21c1fa28a6a7e3cbdc171c694398b6df4744613ce9b36b1a498e816787e28123 \ + --hash=sha256:229cf1d3ca6c1804400b0a9790dc66528e08a6a1feec0d5040e8b9eb14422375 \ + --hash=sha256:27ccee0071a0e75d22cb35849b1db43f2ecd3e161041ac1ee9d2352ddf72f065 \ + --hash=sha256:363c6f671b761efcb30608d24925a382497c12c506b51661883c3e22337265ed \ + --hash=sha256:39c1fec2c11dc8d89bba6b2bf1556af381611a173ac2b511cf7231622058af41 \ + --hash=sha256:3b1ac0d3e594bf121308112697cf4b32be538fb1444468fb0a6ae4feebc83411 \ + --hash=sha256:3be571a8b5afed347da347bfcf27ba12b069d9d7f42cb8c7028b5e98bbb12597 \ + --hash=sha256:3c714d2fc58b5ca3e285461a4cc0c9a66bd0e24c5da9911e30158286c9b5be7f \ + --hash=sha256:3d00075aa65772e7ce9e990cab3ff1de702aa09be3940d1dc88d5abf1ab8a09c \ + --hash=sha256:3e90baa811a5d73f3ca0bcbf32064d663ed81318ab225ee4f427ad4e26e5aff3 \ + --hash=sha256:47819cea040f31d670cc8d324bb6435c6f133b8c7a19ec3d61634e62f8d8f9eb \ + --hash=sha256:47b099e1f4fbc95b701b6e85768e1fcdaf1630f3cbe4765fa216596f12310e2e \ + --hash=sha256:4a9fac8e469d04ce6c25bb2610dc535235bd4aa14996b4e6dbebf5e007eba5ee \ + --hash=sha256:4b826973a4a2ae47ba357e4e82fa44a463b8f168e1ca775ac64521442b19e87f \ + --hash=sha256:4c2529b320eb9e35af0fa3016c187dffb84a3ecc572bcee7c3ce302bfeba52bf \ + --hash=sha256:54479983bd5fb469c38f2f5c7e3a24f9a4e70594cd68cd1fa6b9340dadaff7cf \ + --hash=sha256:558d023b3df0bffe50a04e710bc87742de35060580a293c2a984299ed83bc4e4 \ + --hash=sha256:5756779642579d902eed757b21b0164cd6fe338506a8083eb58af5c372e39d9a \ + --hash=sha256:592f1a9fe869c778694f0aa806ba0374e97648ab57936f092fd9d87f8bc03665 \ + --hash=sha256:595b6c3969023ecf9041b2936ac3827e4623bfa3ccf007575f04c5a6aa318c22 \ + --hash=sha256:5a939de6b7b4e18ca683218320fc67ea886038265fd1ed30173f5ce3f8e85675 \ + --hash=sha256:5d54b09eba2bada6011aea5375542a157637b91029687eb4fdb2dab11059c1b4 \ + --hash=sha256:5df592cd503496351d6dc14f7cdad49f268d8e618f80dce0cd5a36b93c3fc08d \ + --hash=sha256:5f4c04ead5aed67c8a1a20491d54cdfba5884507a48dd798ecaf13c74c4489f5 \ + --hash=sha256:64dee438fed052b52e4f98f76c5790513235efaa1ef7f3f2192c392cd7c91b65 \ + --hash=sha256:66dd88c918e3287efc22409d426c8f729688d89a0c587c88971a0faa2c2f3792 \ + --hash=sha256:678999709e68425ae2593acf2e3ebcbcf2e69885a5ee78f9eb80e6e371f1bf57 \ + --hash=sha256:67f2b6de947f8c757db2db9c71527933ad0019737ec374a8a6be9a956786aaf9 \ + --hash=sha256:693f0192126df6c2327cce3baa7c06f2a117575e32ab2308f7f8216c29d9e2e3 \ + --hash=sha256:746ee8dba912cd6fc889a8147168991d50ed70447bf18bcda7039f7d2e3d9151 \ + --hash=sha256:756c56e867a90fb00177d530dca4b097dd753cde348448a1012ed6c5131f8b7d \ + --hash=sha256:76d1f20b1c7a2fa82367e04982e708723ba0e7b8d43aa643d3dcd404d74f1475 \ + --hash=sha256:7f493881579c90fc262d9cdbaa05a6b54b3811c2f300766748db79f098db9940 \ + --hash=sha256:823c248b690b2fd9303ba00c4f66cd5e2d8c3ba4aa968b2779be9532a4dad431 \ + --hash=sha256:82544de02076bafba038ce055ee6412d68da13ab47f0c60cab827346de828dee \ + --hash=sha256:8dd8327c795b3e3f219760fa603dcae1dcc148172290a8ab15158cf85a953413 \ + --hash=sha256:8fdc51055e6ff4adeb88d58a11042ec9a5eae317a0a53d12c062c8a8865909e8 \ + --hash=sha256:a625e06551975f4b7ea7102bc43895b90742746797e2e14b70ed61c43a90f09b \ + --hash=sha256:abdc0c6c8c648b4805c5eacd131910d2a7f6455dfd3becab248ef108e89ab16a \ + --hash=sha256:ac017dd64572e5c3bd01939121e4d16cf30e5d7e110a119399cf3133b63ad054 \ + --hash=sha256:ac1e5c9054fe23226fb11e05a6e630837f074174c4c2f0fe442996112a6de4fb \ + --hash=sha256:ac60e3b188ec7574cb761b08d50fcedf9d77f1530352db4eef1707fe9dee7205 \ + --hash=sha256:b359ed09954d7c18bbc1680f380c7301f92c60bf924171629c5db97febb12f04 \ + --hash=sha256:b7643a03db5c95c799b89b31c036d5f27eeb4d259c798e878d6937d71832b1e4 \ + --hash=sha256:ba9e56e8ceeeedb2e080147ba85ffcd5cd0711b89576b83784d8605a7df455fa \ + --hash=sha256:c338ffa0520bdb12fbc527265235639fb76e7bc7faafbb93f6ba80d9c06578a9 \ + --hash=sha256:cad21560da69f4ce7658ca2cb83138fb4cf695a2ba3e475e0559e05991aa8122 \ + --hash=sha256:d08eb4c2b7d6c41da6ca0600c077e93f5adcfd979cd777d747e9ee624556da4b \ + --hash=sha256:d50fd1ee42388dcfb2b3676132c78116490976f1300da28eb629272d5d93e905 \ + --hash=sha256:d591f8de75824cbb7acad4e05d2d710484f15f29d4a915092675ad3456f11770 \ + --hash=sha256:d5f6b181bb38171a8ad1d6aa58a67a6aa9d4b38d0f8c5f496b9e42561dfc62fe \ + --hash=sha256:d63efaa0cd96cf0c5fe4d581521d9fa87744540d4bc999ae6e08595a1014b45b \ + --hash=sha256:d99e5546bf73dbad5bf3547174cd6cb8ba7273062a23808ffea025ecb1cf8562 \ + --hash=sha256:e09473f095a819042ecb2ab9465aee615bd9c2028e4ef7d933600a8401c79561 \ + --hash=sha256:e8b56bdcdb4505c8078cb6c7157d9811a85790f2f2b3632c7d1462ab5783d215 \ + --hash=sha256:ee443ef070bb3b6ed74514f5efaa37a252af57c90eb33b956d35c8e9c10a1931 \ + --hash=sha256:f29d80eb9a9263b8d109135351caf568cc3f80b9928bccde535c235de55c22d9 \ + --hash=sha256:f7a866fbc1e97b5c617ee4116daaa09b722101d4a3c170c787450ba409f9736f \ + --hash=sha256:fcd5cf9e305d7b8338754470cf69cf81f420459dbae8a3b40cee57417f4614a7 + # via -r examples/bzlmod/requirements.in +wheel==0.45.1 \ + --hash=sha256:661e1abd9198507b1409a20c02106d9670b2576e916d58f520316666abca6729 \ + --hash=sha256:708e7481cc80179af0e556bbf0cc00b8444c7321e2700b8d8580231d13017248 + # via -r examples/bzlmod/requirements.in +wrapt==1.17.3 \ + --hash=sha256:02b551d101f31694fc785e58e0720ef7d9a10c4e62c1c9358ce6f63f23e30a56 \ + --hash=sha256:042ec3bb8f319c147b1301f2393bc19dba6e176b7da446853406d041c36c7828 \ + --hash=sha256:0610b46293c59a3adbae3dee552b648b984176f8562ee0dba099a56cfbe4df1f \ + --hash=sha256:0b02e424deef65c9f7326d8c19220a2c9040c51dc165cddb732f16198c168396 \ + --hash=sha256:0b1831115c97f0663cb77aa27d381237e73ad4f721391a9bfb2fe8bc25fa6e77 \ + --hash=sha256:0ed61b7c2d49cee3c027372df5809a59d60cf1b6c2f81ee980a091f3afed6a2d \ + --hash=sha256:0f5f51a6466667a5a356e6381d362d259125b57f059103dd9fdc8c0cf1d14139 \ + --hash=sha256:16ecf15d6af39246fe33e507105d67e4b81d8f8d2c6598ff7e3ca1b8a37213f7 \ + --hash=sha256:1f0b2f40cf341ee8cc1a97d51ff50dddb9fcc73241b9143ec74b30fc4f44f6cb \ + --hash=sha256:1f23fa283f51c890eda8e34e4937079114c74b4c81d2b2f1f1d94948f5cc3d7f \ + --hash=sha256:223db574bb38637e8230eb14b185565023ab624474df94d2af18f1cdb625216f \ + --hash=sha256:249f88ed15503f6492a71f01442abddd73856a0032ae860de6d75ca62eed8067 \ + --hash=sha256:24c2ed34dc222ed754247a2702b1e1e89fdbaa4016f324b4b8f1a802d4ffe87f \ + --hash=sha256:273a736c4645e63ac582c60a56b0acb529ef07f78e08dc6bfadf6a46b19c0da7 \ + --hash=sha256:281262213373b6d5e4bb4353bc36d1ba4084e6d6b5d242863721ef2bf2c2930b \ + --hash=sha256:30ce38e66630599e1193798285706903110d4f057aab3168a34b7fdc85569afc \ + --hash=sha256:33486899acd2d7d3066156b03465b949da3fd41a5da6e394ec49d271baefcf05 \ + --hash=sha256:343e44b2a8e60e06a7e0d29c1671a0d9951f59174f3709962b5143f60a2a98bd \ + --hash=sha256:373342dd05b1d07d752cecbec0c41817231f29f3a89aa8b8843f7b95992ed0c7 \ + --hash=sha256:3af60380ba0b7b5aeb329bc4e402acd25bd877e98b3727b0135cb5c2efdaefe9 \ + --hash=sha256:3e62d15d3cfa26e3d0788094de7b64efa75f3a53875cdbccdf78547aed547a81 \ + --hash=sha256:41b1d2bc74c2cac6f9074df52b2efbef2b30bdfe5f40cb78f8ca22963bc62977 \ + --hash=sha256:423ed5420ad5f5529db9ce89eac09c8a2f97da18eb1c870237e84c5a5c2d60aa \ + --hash=sha256:46acc57b331e0b3bcb3e1ca3b421d65637915cfcd65eb783cb2f78a511193f9b \ + --hash=sha256:4da9f45279fff3543c371d5ababc57a0384f70be244de7759c85a7f989cb4ebe \ + --hash=sha256:507553480670cab08a800b9463bdb881b2edeed77dc677b0a5915e6106e91a58 \ + --hash=sha256:53e5e39ff71b3fc484df8a522c933ea2b7cdd0d5d15ae82e5b23fde87d44cbd8 \ + --hash=sha256:54a30837587c6ee3cd1a4d1c2ec5d24e77984d44e2f34547e2323ddb4e22eb77 \ + --hash=sha256:5531d911795e3f935a9c23eb1c8c03c211661a5060aab167065896bbf62a5f85 \ + --hash=sha256:55cbbc356c2842f39bcc553cf695932e8b30e30e797f961860afb308e6b1bb7c \ + --hash=sha256:59923aa12d0157f6b82d686c3fd8e1166fa8cdfb3e17b42ce3b6147ff81528df \ + --hash=sha256:5a03a38adec8066d5a37bea22f2ba6bbf39fcdefbe2d91419ab864c3fb515454 \ + --hash=sha256:5a7b3c1ee8265eb4c8f1b7d29943f195c00673f5ab60c192eba2d4a7eae5f46a \ + --hash=sha256:5d4478d72eb61c36e5b446e375bbc49ed002430d17cdec3cecb36993398e1a9e \ + --hash=sha256:5ea5eb3c0c071862997d6f3e02af1d055f381b1d25b286b9d6644b79db77657c \ + --hash=sha256:604d076c55e2fdd4c1c03d06dc1a31b95130010517b5019db15365ec4a405fc6 \ + --hash=sha256:656873859b3b50eeebe6db8b1455e99d90c26ab058db8e427046dbc35c3140a5 \ + --hash=sha256:65d1d00fbfb3ea5f20add88bbc0f815150dbbde3b026e6c24759466c8b5a9ef9 \ + --hash=sha256:6b538e31eca1a7ea4605e44f81a48aa24c4632a277431a6ed3f328835901f4fd \ + --hash=sha256:6fd1ad24dc235e4ab88cda009e19bf347aabb975e44fd5c2fb22a3f6e4141277 \ + --hash=sha256:70d86fa5197b8947a2fa70260b48e400bf2ccacdcab97bb7de47e3d1e6312225 \ + --hash=sha256:7171ae35d2c33d326ac19dd8facb1e82e5fd04ef8c6c0e394d7af55a55051c22 \ + --hash=sha256:73d496de46cd2cdbdbcce4ae4bcdb4afb6a11234a1df9c085249d55166b95116 \ + --hash=sha256:7425ac3c54430f5fc5e7b6f41d41e704db073309acfc09305816bc6a0b26bb16 \ + --hash=sha256:74afa28374a3c3a11b3b5e5fca0ae03bef8450d6aa3ab3a1e2c30e3a75d023dc \ + --hash=sha256:758895b01d546812d1f42204bd443b8c433c44d090248bf22689df673ccafe00 \ + --hash=sha256:79573c24a46ce11aab457b472efd8d125e5a51da2d1d24387666cd85f54c05b2 \ + --hash=sha256:7e18f01b0c3e4a07fe6dfdb00e29049ba17eadbc5e7609a2a3a4af83ab7d710a \ + --hash=sha256:88547535b787a6c9ce4086917b6e1d291aa8ed914fdd3a838b3539dc95c12804 \ + --hash=sha256:88bbae4d40d5a46142e70d58bf664a89b6b4befaea7b2ecc14e03cedb8e06c04 \ + --hash=sha256:8cccf4f81371f257440c88faed6b74f1053eef90807b77e31ca057b2db74edb1 \ + --hash=sha256:9baa544e6acc91130e926e8c802a17f3b16fbea0fd441b5a60f5cf2cc5c3deba \ + --hash=sha256:a36692b8491d30a8c75f1dfee65bef119d6f39ea84ee04d9f9311f83c5ad9390 \ + --hash=sha256:a47681378a0439215912ef542c45a783484d4dd82bac412b71e59cf9c0e1cea0 \ + --hash=sha256:a7c06742645f914f26c7f1fa47b8bc4c91d222f76ee20116c43d5ef0912bba2d \ + --hash=sha256:a9a2203361a6e6404f80b99234fe7fb37d1fc73487b5a78dc1aa5b97201e0f22 \ + --hash=sha256:ab232e7fdb44cdfbf55fc3afa31bcdb0d8980b9b95c38b6405df2acb672af0e0 \ + --hash=sha256:ad85e269fe54d506b240d2d7b9f5f2057c2aa9a2ea5b32c66f8902f768117ed2 \ + --hash=sha256:af338aa93554be859173c39c85243970dc6a289fa907402289eeae7543e1ae18 \ + --hash=sha256:afd964fd43b10c12213574db492cb8f73b2f0826c8df07a68288f8f19af2ebe6 \ + --hash=sha256:b32888aad8b6e68f83a8fdccbf3165f5469702a7544472bdf41f582970ed3311 \ + --hash=sha256:c31eebe420a9a5d2887b13000b043ff6ca27c452a9a22fa71f35f118e8d4bf89 \ + --hash=sha256:caea3e9c79d5f0d2c6d9ab96111601797ea5da8e6d0723f77eabb0d4068d2b2f \ + --hash=sha256:cf30f6e3c077c8e6a9a7809c94551203c8843e74ba0c960f4a98cd80d4665d39 \ + --hash=sha256:d40770d7c0fd5cbed9d84b2c3f2e156431a12c9a37dc6284060fb4bec0b7ffd4 \ + --hash=sha256:d8a210b158a34164de8bb68b0e7780041a903d7b00c87e906fb69928bf7890d5 \ + --hash=sha256:dc4a8d2b25efb6681ecacad42fca8859f88092d8732b170de6a5dddd80a1c8fa \ + --hash=sha256:df7d30371a2accfe4013e90445f6388c570f103d61019b6b7c57e0265250072a \ + --hash=sha256:e01375f275f010fcbf7f643b4279896d04e571889b8a5b3f848423d91bf07050 \ + --hash=sha256:e1a4120ae5705f673727d3253de3ed0e016f7cd78dc463db1b31e2463e1f3cf6 \ + --hash=sha256:e228514a06843cae89621384cfe3a80418f3c04aadf8a3b14e46a7be704e4235 \ + --hash=sha256:e405adefb53a435f01efa7ccdec012c016b5a1d3f35459990afc39b6be4d5056 \ + --hash=sha256:e6b13af258d6a9ad602d57d889f83b9d5543acd471eee12eb51f5b01f8eb1bc2 \ + --hash=sha256:e6f40a8aa5a92f150bdb3e1c44b7e98fb7113955b2e5394122fa5532fec4b418 \ + --hash=sha256:e71d5c6ebac14875668a1e90baf2ea0ef5b7ac7918355850c0908ae82bcb297c \ + --hash=sha256:ed7c635ae45cfbc1a7371f708727bf74690daedc49b4dba310590ca0bd28aa8a \ + --hash=sha256:f38e60678850c42461d4202739f9bf1e3a737c7ad283638251e79cc49effb6b6 \ + --hash=sha256:f66eb08feaa410fe4eebd17f2a2c8e2e46d3476e9f8c783daa8e09e0faa666d0 \ + --hash=sha256:f9b2601381be482f70e5d1051a5965c25fb3625455a2bf520b5a077b22afb775 \ + --hash=sha256:fbd3c8319de8e1dc79d346929cd71d523622da527cca14e0c1d257e31c2b8b10 \ + --hash=sha256:fd341868a4b6714a5962c1af0bd44f7c404ef78720c7de4892901e540417111c + # via astroid +yamllint==1.37.1 \ + --hash=sha256:364f0d79e81409f591e323725e6a9f4504c8699ddf2d7263d8d2b539cd66a583 \ + --hash=sha256:81f7c0c5559becc8049470d86046b36e96113637bcbe4753ecef06977c00245d + # via -r examples/bzlmod/requirements.in diff --git a/examples/bzlmod/runfiles/BUILD.bazel b/examples/bzlmod/runfiles/BUILD.bazel index add56b3bd0..11a8ce0bb7 100644 --- a/examples/bzlmod/runfiles/BUILD.bazel +++ b/examples/bzlmod/runfiles/BUILD.bazel @@ -1,4 +1,4 @@ -load("@rules_python//python:defs.bzl", "py_test") +load("@rules_python//python:py_test.bzl", "py_test") py_test( name = "runfiles_test", diff --git a/examples/bzlmod/runfiles/runfiles_test.py b/examples/bzlmod/runfiles/runfiles_test.py index e1ba14e569..7b7e87726a 100644 --- a/examples/bzlmod/runfiles/runfiles_test.py +++ b/examples/bzlmod/runfiles/runfiles_test.py @@ -27,36 +27,36 @@ def testCurrentRepository(self): def testRunfilesWithRepoMapping(self): data_path = runfiles.Create().Rlocation("example_bzlmod/runfiles/data/data.txt") - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, example_bzlmod!") def testRunfileWithRlocationpath(self): data_rlocationpath = os.getenv("DATA_RLOCATIONPATH") data_path = runfiles.Create().Rlocation(data_rlocationpath) - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, example_bzlmod!") def testRunfileInOtherModuleWithOurRepoMapping(self): data_path = runfiles.Create().Rlocation( "our_other_module/other_module/pkg/data/data.txt" ) - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, other_module!") def testRunfileInOtherModuleWithItsRepoMapping(self): data_path = lib.GetRunfilePathWithRepoMapping() - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, other_module!") def testRunfileInOtherModuleWithCurrentRepository(self): data_path = lib.GetRunfilePathWithCurrentRepository() - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, other_module!") def testRunfileInOtherModuleWithRlocationpath(self): data_rlocationpath = os.getenv("OTHER_MODULE_DATA_RLOCATIONPATH") data_path = runfiles.Create().Rlocation(data_rlocationpath) - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, other_module!") diff --git a/examples/bzlmod/test.py b/examples/bzlmod/test.py index 24be3ba3fe..3febed7585 100644 --- a/examples/bzlmod/test.py +++ b/examples/bzlmod/test.py @@ -13,7 +13,6 @@ # limitations under the License. import os -import pathlib import re import sys import unittest @@ -59,7 +58,7 @@ def test_coverage_sys_path(self): f"sys.path has {len(sys.path)} items:\n {all_paths}", ) - first_item, last_item = sys.path[0], sys.path[-1] + first_item, _ = sys.path[0], sys.path[-1] self.assertFalse( first_item.endswith("coverage"), f"Expected the first item in sys.path '{first_item}' to not be related to coverage", diff --git a/examples/bzlmod/tests/BUILD.bazel b/examples/bzlmod/tests/BUILD.bazel index 96e4cdde25..4650fb8788 100644 --- a/examples/bzlmod/tests/BUILD.bazel +++ b/examples/bzlmod/tests/BUILD.bazel @@ -1,9 +1,6 @@ -load("@python_versions//3.10:defs.bzl", py_binary_3_10 = "py_binary", py_test_3_10 = "py_test") -load("@python_versions//3.11:defs.bzl", py_binary_3_11 = "py_binary", py_test_3_11 = "py_test") -load("@python_versions//3.9:defs.bzl", py_binary_3_9 = "py_binary", py_test_3_9 = "py_test") load("@pythons_hub//:versions.bzl", "MINOR_MAPPING") -load("@rules_python//python:defs.bzl", "py_binary", "py_test") -load("@rules_python//python/config_settings:transition.bzl", py_versioned_binary = "py_binary", py_versioned_test = "py_test") +load("@rules_python//python:py_binary.bzl", "py_binary") +load("@rules_python//python:py_test.bzl", "py_test") load("@rules_shell//shell:sh_test.bzl", "sh_test") py_binary( @@ -12,25 +9,28 @@ py_binary( main = "version.py", ) -py_binary_3_9( +py_binary( name = "version_3_9", srcs = ["version.py"], main = "version.py", + python_version = "3.9", ) -py_binary_3_10( +py_binary( name = "version_3_10", srcs = ["version.py"], main = "version.py", + python_version = "3.10", ) -py_binary_3_11( +py_binary( name = "version_3_11", srcs = ["version.py"], main = "version.py", + python_version = "3.11", ) -py_versioned_binary( +py_binary( name = "version_3_10_versioned", srcs = ["version.py"], main = "version.py", @@ -48,21 +48,23 @@ py_test( deps = ["//libs/my_lib"], ) -py_test_3_9( +py_test( name = "my_lib_3_9_test", srcs = ["my_lib_test.py"], main = "my_lib_test.py", + python_version = "3.9", deps = ["//libs/my_lib"], ) -py_test_3_10( +py_test( name = "my_lib_3_10_test", srcs = ["my_lib_test.py"], main = "my_lib_test.py", + python_version = "3.10", deps = ["//libs/my_lib"], ) -py_versioned_test( +py_test( name = "my_lib_versioned_test", srcs = ["my_lib_test.py"], main = "my_lib_test.py", @@ -91,21 +93,23 @@ py_test( main = "version_test.py", ) -py_test_3_9( +py_test( name = "version_3_9_test", srcs = ["version_test.py"], env = {"VERSION_CHECK": "3.9"}, main = "version_test.py", + python_version = "3.9", ) -py_test_3_10( +py_test( name = "version_3_10_test", srcs = ["version_test.py"], env = {"VERSION_CHECK": "3.10"}, main = "version_test.py", + python_version = "3.10", ) -py_versioned_test( +py_test( name = "version_versioned_test", srcs = ["version_test.py"], env = {"VERSION_CHECK": "3.10"}, @@ -113,11 +117,12 @@ py_versioned_test( python_version = "3.10", ) -py_test_3_11( +py_test( name = "version_3_11_test", srcs = ["version_test.py"], env = {"VERSION_CHECK": "3.11"}, main = "version_test.py", + python_version = "3.11", ) py_test( @@ -132,7 +137,7 @@ py_test( main = "cross_version_test.py", ) -py_test_3_10( +py_test( name = "version_3_10_takes_3_9_subprocess_test", srcs = ["cross_version_test.py"], data = [":version_3_9"], @@ -142,9 +147,10 @@ py_test_3_10( "VERSION_CHECK": "3.10", }, main = "cross_version_test.py", + python_version = "3.10", ) -py_versioned_test( +py_test( name = "version_3_10_takes_3_9_subprocess_test_2", srcs = ["cross_version_test.py"], data = [":version_3_9"], diff --git a/examples/bzlmod/tests/my_lib_test.py b/examples/bzlmod/tests/my_lib_test.py index b06374c983..019d29e31f 100644 --- a/examples/bzlmod/tests/my_lib_test.py +++ b/examples/bzlmod/tests/my_lib_test.py @@ -12,7 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. -import os import sys import libs.my_lib as my_lib diff --git a/examples/bzlmod/tests/other_module/BUILD.bazel b/examples/bzlmod/tests/other_module/BUILD.bazel index 1bd8a900a9..f4756e7c3e 100644 --- a/examples/bzlmod/tests/other_module/BUILD.bazel +++ b/examples/bzlmod/tests/other_module/BUILD.bazel @@ -5,6 +5,7 @@ # in the root module. load("@bazel_skylib//rules:build_test.bzl", "build_test") +load("@rules_python//python:py_test.bzl", "py_test") build_test( name = "other_module_bin_build_test", @@ -12,3 +13,27 @@ build_test( "@our_other_module//other_module/pkg:bin", ], ) + +# Regression test for https://github.com/bazel-contrib/rules_python/pull/3919: py_binary_rule_builder()'s implicit +# attr defaults previously had no visibility outside of rules_python, so +# building this target (defined via the builder from an external module) +# failed at analysis time with a visibility error. +build_test( + name = "other_module_rule_builder_build_test", + targets = [ + "@our_other_module//other_module/rule_builder:app", + ], +) + +py_test( + name = "other_module_import_test", + srcs = ["other_module_import_test.py"], + data = ["@our_other_module//other_module/pkg:bin_zipapp"], + env = {"ZIPAPP_PATH": "$(location @our_other_module//other_module/pkg:bin_zipapp)"}, + # For now, skip this test on Windows because it fails for reasons + # other than the code path being tested. + target_compatible_with = select({ + "@platforms//os:windows": ["@platforms//:incompatible"], + "//conditions:default": [], + }), +) diff --git a/examples/bzlmod/tests/other_module/other_module_import_test.py b/examples/bzlmod/tests/other_module/other_module_import_test.py new file mode 100644 index 0000000000..b5b15c383d --- /dev/null +++ b/examples/bzlmod/tests/other_module/other_module_import_test.py @@ -0,0 +1,25 @@ +"""Regression test for https://github.com/bazel-contrib/rules_python/issues/3563""" + +import os +import subprocess +import sys + + +def main(): + # The rlocation path for the bin_zipapp. It is in the "our_other_module" repository. + zipapp_path = os.environ.get("ZIPAPP_PATH") + print(f"Running bin_zipapp at: {zipapp_path}") + + result = subprocess.run([zipapp_path], capture_output=True, text=True) + print("--- bin_zippapp stdout ---") + print(result.stdout) + print("--- bin_zippapp stderr ---") + print(result.stderr) + + if result.returncode != 0: + print(f"bin_zippapp failed with return code {result.returncode}") + sys.exit(result.returncode) + + +if __name__ == "__main__": + main() diff --git a/examples/bzlmod/whl_mods/BUILD.bazel b/examples/bzlmod/whl_mods/BUILD.bazel index 241d9c1073..7c5ab5056e 100644 --- a/examples/bzlmod/whl_mods/BUILD.bazel +++ b/examples/bzlmod/whl_mods/BUILD.bazel @@ -1,4 +1,4 @@ -load("@rules_python//python:defs.bzl", "py_test") +load("@rules_python//python:py_test.bzl", "py_test") exports_files( glob(["data/**"]), diff --git a/examples/bzlmod_build_file_generation/.bazelrc b/examples/bzlmod_build_file_generation/.bazelrc deleted file mode 100644 index 0289886d4d..0000000000 --- a/examples/bzlmod_build_file_generation/.bazelrc +++ /dev/null @@ -1,9 +0,0 @@ -test --test_output=errors --enable_runfiles - -# Windows requires these for multi-python support: -build --enable_runfiles - -common --experimental_enable_bzlmod - -coverage --java_runtime_version=remotejdk_11 -common:bazel7.x --incompatible_python_disallow_native_rules diff --git a/examples/bzlmod_build_file_generation/other_module/MODULE.bazel b/examples/bzlmod_build_file_generation/other_module/MODULE.bazel deleted file mode 100644 index 992e120760..0000000000 --- a/examples/bzlmod_build_file_generation/other_module/MODULE.bazel +++ /dev/null @@ -1,5 +0,0 @@ -module( - name = "other_module", -) - -bazel_dep(name = "rules_python", version = "") diff --git a/examples/multi_python_versions/MODULE.bazel b/examples/multi_python_versions/MODULE.bazel index 578315741f..82faaf8214 100644 --- a/examples/multi_python_versions/MODULE.bazel +++ b/examples/multi_python_versions/MODULE.bazel @@ -10,23 +10,30 @@ local_path_override( ) python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.defaults( + # The environment variable takes precedence if set. + python_version = "3.10", + python_version_env = "BAZEL_PYTHON_VERSION", +) python.toolchain( configure_coverage_tool = True, - python_version = "3.8", + python_version = "3.10", ) python.toolchain( configure_coverage_tool = True, - # Only set when you have mulitple toolchain versions. - is_default = True, - python_version = "3.9", + python_version = "3.11", ) python.toolchain( configure_coverage_tool = True, - python_version = "3.10", + python_version = "3.12", ) python.toolchain( configure_coverage_tool = True, - python_version = "3.11", + python_version = "3.13", +) +python.toolchain( + configure_coverage_tool = True, + python_version = "3.14", ) use_repo( python, @@ -36,25 +43,31 @@ use_repo( pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") use_repo(pip, "pypi") + pip.parse( hub_name = "pypi", - python_version = "3.8", - requirements_lock = "//requirements:requirements_lock_3_8.txt", + python_version = "3.10", + requirements_lock = "//requirements:requirements_lock_3_10.txt", ) pip.parse( hub_name = "pypi", - python_version = "3.9", - requirements_lock = "//requirements:requirements_lock_3_9.txt", + python_version = "3.11", + requirements_lock = "//requirements:requirements_lock_3_11.txt", ) pip.parse( hub_name = "pypi", - python_version = "3.10", - requirements_lock = "//requirements:requirements_lock_3_10.txt", + python_version = "3.12", + requirements_lock = "//requirements:requirements_lock_3_12.txt", ) pip.parse( hub_name = "pypi", - python_version = "3.11", - requirements_lock = "//requirements:requirements_lock_3_11.txt", + python_version = "3.13", + requirements_lock = "//requirements:requirements_lock_3_13.txt", +) +pip.parse( + hub_name = "pypi", + python_version = "3.14", + requirements_lock = "//requirements:requirements_lock_3_14.txt", ) # example test dependencies @@ -62,4 +75,4 @@ bazel_dep(name = "rules_shell", version = "0.2.0", dev_dependency = True) # Only needed to make rules_python's CI happy. rules_java 8.3.0+ is needed so # that --java_runtime_version=remotejdk_11 works with Bazel 8. -bazel_dep(name = "rules_java", version = "8.3.1") +bazel_dep(name = "rules_java", version = "8.16.1") diff --git a/examples/multi_python_versions/WORKSPACE b/examples/multi_python_versions/WORKSPACE index 48d2065282..0b6b8a0cbf 100644 --- a/examples/multi_python_versions/WORKSPACE +++ b/examples/multi_python_versions/WORKSPACE @@ -9,16 +9,17 @@ load("@rules_python//python:repositories.bzl", "py_repositories", "python_regist py_repositories() -default_python_version = "3.9" +default_python_version = "3.10" python_register_multi_toolchains( name = "python", default_version = default_python_version, python_versions = [ - "3.8", - "3.9", "3.10", "3.11", + "3.12", + "3.13", + "3.14", ], register_coverage_tool = True, ) @@ -31,14 +32,16 @@ multi_pip_parse( python_interpreter_target = { "3.10": "@python_3_10_host//:python", "3.11": "@python_3_11_host//:python", - "3.8": "@python_3_8_host//:python", - "3.9": "@python_3_9_host//:python", + "3.12": "@python_3_12_host//:python", + "3.13": "@python_3_13_host//:python", + "3.14": "@python_3_14_host//:python", }, requirements_lock = { "3.10": "//requirements:requirements_lock_3_10.txt", "3.11": "//requirements:requirements_lock_3_11.txt", - "3.8": "//requirements:requirements_lock_3_8.txt", - "3.9": "//requirements:requirements_lock_3_9.txt", + "3.12": "//requirements:requirements_lock_3_12.txt", + "3.13": "//requirements:requirements_lock_3_13.txt", + "3.14": "//requirements:requirements_lock_3_14.txt", }, ) diff --git a/examples/multi_python_versions/libs/my_lib/BUILD.bazel b/examples/multi_python_versions/libs/my_lib/BUILD.bazel index 8c29f6083c..7ff62249c4 100644 --- a/examples/multi_python_versions/libs/my_lib/BUILD.bazel +++ b/examples/multi_python_versions/libs/my_lib/BUILD.bazel @@ -1,5 +1,5 @@ load("@pypi//:requirements.bzl", "requirement") -load("@rules_python//python:defs.bzl", "py_library") +load("@rules_python//python:py_library.bzl", "py_library") py_library( name = "my_lib", diff --git a/examples/multi_python_versions/requirements/BUILD.bazel b/examples/multi_python_versions/requirements/BUILD.bazel index f67333a657..ee8ff029f8 100644 --- a/examples/multi_python_versions/requirements/BUILD.bazel +++ b/examples/multi_python_versions/requirements/BUILD.bazel @@ -1,28 +1,36 @@ -load("@python//3.10:defs.bzl", compile_pip_requirements_3_10 = "compile_pip_requirements") -load("@python//3.11:defs.bzl", compile_pip_requirements_3_11 = "compile_pip_requirements") -load("@python//3.8:defs.bzl", compile_pip_requirements_3_8 = "compile_pip_requirements") -load("@python//3.9:defs.bzl", compile_pip_requirements_3_9 = "compile_pip_requirements") +load("@rules_python//python:pip.bzl", "compile_pip_requirements") -compile_pip_requirements_3_8( - name = "requirements_3_8", +compile_pip_requirements( + name = "requirements_3_10", src = "requirements.in", - requirements_txt = "requirements_lock_3_8.txt", + python_version = "3.10", + requirements_txt = "requirements_lock_3_10.txt", ) -compile_pip_requirements_3_9( - name = "requirements_3_9", +compile_pip_requirements( + name = "requirements_3_11", src = "requirements.in", - requirements_txt = "requirements_lock_3_9.txt", + python_version = "3.11", + requirements_txt = "requirements_lock_3_11.txt", ) -compile_pip_requirements_3_10( - name = "requirements_3_10", +compile_pip_requirements( + name = "requirements_3_12", src = "requirements.in", - requirements_txt = "requirements_lock_3_10.txt", + python_version = "3.12", + requirements_txt = "requirements_lock_3_12.txt", ) -compile_pip_requirements_3_11( - name = "requirements_3_11", +compile_pip_requirements( + name = "requirements_3_13", src = "requirements.in", - requirements_txt = "requirements_lock_3_11.txt", + python_version = "3.13", + requirements_txt = "requirements_lock_3_13.txt", +) + +compile_pip_requirements( + name = "requirements_3_14", + src = "requirements.in", + python_version = "3.14", + requirements_txt = "requirements_lock_3_14.txt", ) diff --git a/examples/multi_python_versions/requirements/requirements.in b/examples/multi_python_versions/requirements/requirements.in index 14774b465e..4d1474b9a2 100644 --- a/examples/multi_python_versions/requirements/requirements.in +++ b/examples/multi_python_versions/requirements/requirements.in @@ -1 +1 @@ -websockets +websockets ; python_full_version > "3.9.1" diff --git a/examples/multi_python_versions/requirements/requirements_lock_3_10.txt b/examples/multi_python_versions/requirements/requirements_lock_3_10.txt index 4910d13844..3a8453223f 100644 --- a/examples/multi_python_versions/requirements/requirements_lock_3_10.txt +++ b/examples/multi_python_versions/requirements/requirements_lock_3_10.txt @@ -4,7 +4,7 @@ # # bazel run //requirements:requirements_3_10.update # -websockets==11.0.3 \ +websockets==11.0.3 ; python_full_version > "3.9.1" \ --hash=sha256:01f5567d9cf6f502d655151645d4e8b72b453413d3819d2b6f1185abc23e82dd \ --hash=sha256:03aae4edc0b1c68498f41a6772d80ac7c1e33c06c6ffa2ac1c27a07653e79d6f \ --hash=sha256:0ac56b661e60edd453585f4bd68eb6a29ae25b5184fd5ba51e97652580458998 \ diff --git a/examples/multi_python_versions/requirements/requirements_lock_3_11.txt b/examples/multi_python_versions/requirements/requirements_lock_3_11.txt index 35666b54b1..f1fa8f56f5 100644 --- a/examples/multi_python_versions/requirements/requirements_lock_3_11.txt +++ b/examples/multi_python_versions/requirements/requirements_lock_3_11.txt @@ -4,7 +4,7 @@ # # bazel run //requirements:requirements_3_11.update # -websockets==11.0.3 \ +websockets==11.0.3 ; python_full_version > "3.9.1" \ --hash=sha256:01f5567d9cf6f502d655151645d4e8b72b453413d3819d2b6f1185abc23e82dd \ --hash=sha256:03aae4edc0b1c68498f41a6772d80ac7c1e33c06c6ffa2ac1c27a07653e79d6f \ --hash=sha256:0ac56b661e60edd453585f4bd68eb6a29ae25b5184fd5ba51e97652580458998 \ diff --git a/examples/multi_python_versions/requirements/requirements_lock_3_8.txt b/examples/multi_python_versions/requirements/requirements_lock_3_12.txt similarity index 97% rename from examples/multi_python_versions/requirements/requirements_lock_3_8.txt rename to examples/multi_python_versions/requirements/requirements_lock_3_12.txt index 10b5df4830..818b20e14c 100644 --- a/examples/multi_python_versions/requirements/requirements_lock_3_8.txt +++ b/examples/multi_python_versions/requirements/requirements_lock_3_12.txt @@ -1,10 +1,10 @@ # -# This file is autogenerated by pip-compile with Python 3.8 +# This file is autogenerated by pip-compile with Python 3.12 # by the following command: # -# bazel run //requirements:requirements_3_8.update +# bazel run //requirements:requirements_3_12.update # -websockets==11.0.3 \ +websockets==11.0.3 ; python_full_version > "3.9.1" \ --hash=sha256:01f5567d9cf6f502d655151645d4e8b72b453413d3819d2b6f1185abc23e82dd \ --hash=sha256:03aae4edc0b1c68498f41a6772d80ac7c1e33c06c6ffa2ac1c27a07653e79d6f \ --hash=sha256:0ac56b661e60edd453585f4bd68eb6a29ae25b5184fd5ba51e97652580458998 \ diff --git a/examples/multi_python_versions/requirements/requirements_lock_3_9.txt b/examples/multi_python_versions/requirements/requirements_lock_3_13.txt similarity index 97% rename from examples/multi_python_versions/requirements/requirements_lock_3_9.txt rename to examples/multi_python_versions/requirements/requirements_lock_3_13.txt index 0001f88d48..8dc44b8a07 100644 --- a/examples/multi_python_versions/requirements/requirements_lock_3_9.txt +++ b/examples/multi_python_versions/requirements/requirements_lock_3_13.txt @@ -1,10 +1,10 @@ # -# This file is autogenerated by pip-compile with Python 3.9 +# This file is autogenerated by pip-compile with Python 3.13 # by the following command: # -# bazel run //requirements:requirements_3_9.update +# bazel run //requirements:requirements_3_13.update # -websockets==11.0.3 \ +websockets==11.0.3 ; python_full_version > "3.9.1" \ --hash=sha256:01f5567d9cf6f502d655151645d4e8b72b453413d3819d2b6f1185abc23e82dd \ --hash=sha256:03aae4edc0b1c68498f41a6772d80ac7c1e33c06c6ffa2ac1c27a07653e79d6f \ --hash=sha256:0ac56b661e60edd453585f4bd68eb6a29ae25b5184fd5ba51e97652580458998 \ diff --git a/examples/multi_python_versions/requirements/requirements_lock_3_14.txt b/examples/multi_python_versions/requirements/requirements_lock_3_14.txt new file mode 100644 index 0000000000..f0aaaa90af --- /dev/null +++ b/examples/multi_python_versions/requirements/requirements_lock_3_14.txt @@ -0,0 +1,78 @@ +# +# This file is autogenerated by pip-compile with Python 3.14 +# by the following command: +# +# bazel run //requirements:requirements_3_14.update +# +websockets==11.0.3 ; python_full_version > "3.9.1" \ + --hash=sha256:01f5567d9cf6f502d655151645d4e8b72b453413d3819d2b6f1185abc23e82dd \ + --hash=sha256:03aae4edc0b1c68498f41a6772d80ac7c1e33c06c6ffa2ac1c27a07653e79d6f \ + --hash=sha256:0ac56b661e60edd453585f4bd68eb6a29ae25b5184fd5ba51e97652580458998 \ + --hash=sha256:0ee68fe502f9031f19d495dae2c268830df2760c0524cbac5d759921ba8c8e82 \ + --hash=sha256:1553cb82942b2a74dd9b15a018dce645d4e68674de2ca31ff13ebc2d9f283788 \ + --hash=sha256:1a073fc9ab1c8aff37c99f11f1641e16da517770e31a37265d2755282a5d28aa \ + --hash=sha256:1d2256283fa4b7f4c7d7d3e84dc2ece74d341bce57d5b9bf385df109c2a1a82f \ + --hash=sha256:1d5023a4b6a5b183dc838808087033ec5df77580485fc533e7dab2567851b0a4 \ + --hash=sha256:1fdf26fa8a6a592f8f9235285b8affa72748dc12e964a5518c6c5e8f916716f7 \ + --hash=sha256:2529338a6ff0eb0b50c7be33dc3d0e456381157a31eefc561771ee431134a97f \ + --hash=sha256:279e5de4671e79a9ac877427f4ac4ce93751b8823f276b681d04b2156713b9dd \ + --hash=sha256:2d903ad4419f5b472de90cd2d40384573b25da71e33519a67797de17ef849b69 \ + --hash=sha256:332d126167ddddec94597c2365537baf9ff62dfcc9db4266f263d455f2f031cb \ + --hash=sha256:34fd59a4ac42dff6d4681d8843217137f6bc85ed29722f2f7222bd619d15e95b \ + --hash=sha256:3580dd9c1ad0701169e4d6fc41e878ffe05e6bdcaf3c412f9d559389d0c9e016 \ + --hash=sha256:3ccc8a0c387629aec40f2fc9fdcb4b9d5431954f934da3eaf16cdc94f67dbfac \ + --hash=sha256:41f696ba95cd92dc047e46b41b26dd24518384749ed0d99bea0a941ca87404c4 \ + --hash=sha256:42cc5452a54a8e46a032521d7365da775823e21bfba2895fb7b77633cce031bb \ + --hash=sha256:4841ed00f1026dfbced6fca7d963c4e7043aa832648671b5138008dc5a8f6d99 \ + --hash=sha256:4b253869ea05a5a073ebfdcb5cb3b0266a57c3764cf6fe114e4cd90f4bfa5f5e \ + --hash=sha256:54c6e5b3d3a8936a4ab6870d46bdd6ec500ad62bde9e44462c32d18f1e9a8e54 \ + --hash=sha256:619d9f06372b3a42bc29d0cd0354c9bb9fb39c2cbc1a9c5025b4538738dbffaf \ + --hash=sha256:6505c1b31274723ccaf5f515c1824a4ad2f0d191cec942666b3d0f3aa4cb4007 \ + --hash=sha256:660e2d9068d2bedc0912af508f30bbeb505bbbf9774d98def45f68278cea20d3 \ + --hash=sha256:6681ba9e7f8f3b19440921e99efbb40fc89f26cd71bf539e45d8c8a25c976dc6 \ + --hash=sha256:68b977f21ce443d6d378dbd5ca38621755f2063d6fdb3335bda981d552cfff86 \ + --hash=sha256:69269f3a0b472e91125b503d3c0b3566bda26da0a3261c49f0027eb6075086d1 \ + --hash=sha256:6f1a3f10f836fab6ca6efa97bb952300b20ae56b409414ca85bff2ad241d2a61 \ + --hash=sha256:7622a89d696fc87af8e8d280d9b421db5133ef5b29d3f7a1ce9f1a7bf7fcfa11 \ + --hash=sha256:777354ee16f02f643a4c7f2b3eff8027a33c9861edc691a2003531f5da4f6bc8 \ + --hash=sha256:84d27a4832cc1a0ee07cdcf2b0629a8a72db73f4cf6de6f0904f6661227f256f \ + --hash=sha256:8531fdcad636d82c517b26a448dcfe62f720e1922b33c81ce695d0edb91eb931 \ + --hash=sha256:86d2a77fd490ae3ff6fae1c6ceaecad063d3cc2320b44377efdde79880e11526 \ + --hash=sha256:88fc51d9a26b10fc331be344f1781224a375b78488fc343620184e95a4b27016 \ + --hash=sha256:8a34e13a62a59c871064dfd8ffb150867e54291e46d4a7cf11d02c94a5275bae \ + --hash=sha256:8c82f11964f010053e13daafdc7154ce7385ecc538989a354ccc7067fd7028fd \ + --hash=sha256:92b2065d642bf8c0a82d59e59053dd2fdde64d4ed44efe4870fa816c1232647b \ + --hash=sha256:97b52894d948d2f6ea480171a27122d77af14ced35f62e5c892ca2fae9344311 \ + --hash=sha256:9d9acd80072abcc98bd2c86c3c9cd4ac2347b5a5a0cae7ed5c0ee5675f86d9af \ + --hash=sha256:9f59a3c656fef341a99e3d63189852be7084c0e54b75734cde571182c087b152 \ + --hash=sha256:aa5003845cdd21ac0dc6c9bf661c5beddd01116f6eb9eb3c8e272353d45b3288 \ + --hash=sha256:b16fff62b45eccb9c7abb18e60e7e446998093cdcb50fed33134b9b6878836de \ + --hash=sha256:b30c6590146e53149f04e85a6e4fcae068df4289e31e4aee1fdf56a0dead8f97 \ + --hash=sha256:b58cbf0697721120866820b89f93659abc31c1e876bf20d0b3d03cef14faf84d \ + --hash=sha256:b67c6f5e5a401fc56394f191f00f9b3811fe843ee93f4a70df3c389d1adf857d \ + --hash=sha256:bceab846bac555aff6427d060f2fcfff71042dba6f5fca7dc4f75cac815e57ca \ + --hash=sha256:bee9fcb41db2a23bed96c6b6ead6489702c12334ea20a297aa095ce6d31370d0 \ + --hash=sha256:c114e8da9b475739dde229fd3bc6b05a6537a88a578358bc8eb29b4030fac9c9 \ + --hash=sha256:c1f0524f203e3bd35149f12157438f406eff2e4fb30f71221c8a5eceb3617b6b \ + --hash=sha256:c792ea4eabc0159535608fc5658a74d1a81020eb35195dd63214dcf07556f67e \ + --hash=sha256:c7f3cb904cce8e1be667c7e6fef4516b98d1a6a0635a58a57528d577ac18a128 \ + --hash=sha256:d67ac60a307f760c6e65dad586f556dde58e683fab03323221a4e530ead6f74d \ + --hash=sha256:dcacf2c7a6c3a84e720d1bb2b543c675bf6c40e460300b628bab1b1efc7c034c \ + --hash=sha256:de36fe9c02995c7e6ae6efe2e205816f5f00c22fd1fbf343d4d18c3d5ceac2f5 \ + --hash=sha256:def07915168ac8f7853812cc593c71185a16216e9e4fa886358a17ed0fd9fcf6 \ + --hash=sha256:df41b9bc27c2c25b486bae7cf42fccdc52ff181c8c387bfd026624a491c2671b \ + --hash=sha256:e052b8467dd07d4943936009f46ae5ce7b908ddcac3fda581656b1b19c083d9b \ + --hash=sha256:e063b1865974611313a3849d43f2c3f5368093691349cf3c7c8f8f75ad7cb280 \ + --hash=sha256:e1459677e5d12be8bbc7584c35b992eea142911a6236a3278b9b5ce3326f282c \ + --hash=sha256:e1a99a7a71631f0efe727c10edfba09ea6bee4166a6f9c19aafb6c0b5917d09c \ + --hash=sha256:e590228200fcfc7e9109509e4d9125eace2042fd52b595dd22bbc34bb282307f \ + --hash=sha256:e6316827e3e79b7b8e7d8e3b08f4e331af91a48e794d5d8b099928b6f0b85f20 \ + --hash=sha256:e7837cb169eca3b3ae94cc5787c4fed99eef74c0ab9506756eea335e0d6f3ed8 \ + --hash=sha256:e848f46a58b9fcf3d06061d17be388caf70ea5b8cc3466251963c8345e13f7eb \ + --hash=sha256:ed058398f55163a79bb9f06a90ef9ccc063b204bb346c4de78efc5d15abfe602 \ + --hash=sha256:f2e58f2c36cc52d41f2659e4c0cbf7353e28c8c9e63e30d8c6d3494dc9fdedcf \ + --hash=sha256:f467ba0050b7de85016b43f5a22b46383ef004c4f672148a8abf32bc999a87f0 \ + --hash=sha256:f61bdb1df43dc9c131791fbc2355535f9024b9a04398d3bd0684fc16ab07df74 \ + --hash=sha256:fb06eea71a00a7af0ae6aefbb932fb8a7df3cb390cc217d51a9ad7343de1b8d0 \ + --hash=sha256:ffd7dcaf744f25f82190856bc26ed81721508fc5cbf2a330751e135ff1283564 + # via -r requirements/requirements.in diff --git a/examples/multi_python_versions/tests/BUILD.bazel b/examples/multi_python_versions/tests/BUILD.bazel index d5c66e026f..607058d992 100644 --- a/examples/multi_python_versions/tests/BUILD.bazel +++ b/examples/multi_python_versions/tests/BUILD.bazel @@ -1,12 +1,9 @@ load("@bazel_skylib//rules:copy_file.bzl", "copy_file") load("@bazel_skylib//rules:diff_test.bzl", "diff_test") load("@bazel_skylib//rules:write_file.bzl", "write_file") -load("@python//3.10:defs.bzl", py_binary_3_10 = "py_binary", py_test_3_10 = "py_test") -load("@python//3.11:defs.bzl", py_binary_3_11 = "py_binary", py_test_3_11 = "py_test") -load("@python//3.8:defs.bzl", py_binary_3_8 = "py_binary", py_test_3_8 = "py_test") -load("@python//3.9:defs.bzl", py_binary_3_9 = "py_binary", py_test_3_9 = "py_test") load("@pythons_hub//:versions.bzl", "MINOR_MAPPING", "PYTHON_VERSIONS") -load("@rules_python//python:defs.bzl", "py_binary", "py_test") +load("@rules_python//python:py_binary.bzl", "py_binary") +load("@rules_python//python:py_test.bzl", "py_test") load("@rules_python//python:versions.bzl", DEFAULT_MINOR_MAPPING = "MINOR_MAPPING", DEFAULT_TOOL_VERSIONS = "TOOL_VERSIONS") load("@rules_python//python/private:text_util.bzl", "render") # buildifier: disable=bzl-visibility load("@rules_shell//shell:sh_test.bzl", "sh_test") @@ -25,28 +22,39 @@ py_binary( srcs = ["version_default.py"], ) -py_binary_3_8( - name = "version_3_8", +py_binary( + name = "version_3_10", srcs = ["version.py"], main = "version.py", + python_version = "3.10", ) -py_binary_3_9( - name = "version_3_9", +py_binary( + name = "version_3_11", srcs = ["version.py"], main = "version.py", + python_version = "3.11", ) -py_binary_3_10( - name = "version_3_10", +py_binary( + name = "version_3_12", srcs = ["version.py"], main = "version.py", + python_version = "3.12", ) -py_binary_3_11( - name = "version_3_11", +py_binary( + name = "version_3_13", srcs = ["version.py"], main = "version.py", + python_version = "3.13", +) + +py_binary( + name = "version_3_14", + srcs = ["version.py"], + main = "version.py", + python_version = "3.14", ) py_test( @@ -56,31 +64,43 @@ py_test( deps = ["//libs/my_lib"], ) -py_test_3_8( - name = "my_lib_3_8_test", +py_test( + name = "my_lib_3_10_test", + srcs = ["my_lib_test.py"], + main = "my_lib_test.py", + python_version = "3.10", + deps = ["//libs/my_lib"], +) + +py_test( + name = "my_lib_3_11_test", srcs = ["my_lib_test.py"], main = "my_lib_test.py", + python_version = "3.11", deps = ["//libs/my_lib"], ) -py_test_3_9( - name = "my_lib_3_9_test", +py_test( + name = "my_lib_3_12_test", srcs = ["my_lib_test.py"], main = "my_lib_test.py", + python_version = "3.12", deps = ["//libs/my_lib"], ) -py_test_3_10( - name = "my_lib_3_10_test", +py_test( + name = "my_lib_3_13_test", srcs = ["my_lib_test.py"], main = "my_lib_test.py", + python_version = "3.13", deps = ["//libs/my_lib"], ) -py_test_3_11( - name = "my_lib_3_11_test", +py_test( + name = "my_lib_3_14_test", srcs = ["my_lib_test.py"], main = "my_lib_test.py", + python_version = "3.14", deps = ["//libs/my_lib"], ) @@ -94,59 +114,72 @@ copy_file( py_test( name = "version_default_test", srcs = ["version_default_test.py"], - env = {"VERSION_CHECK": "3.9"}, # The default defined in the WORKSPACE. + env = {"VERSION_CHECK": "3.10"}, # The default defined in the WORKSPACE/MODULE ) -py_test_3_8( - name = "version_3_8_test", +py_test( + name = "version_3_10_test", srcs = ["version_test.py"], - env = {"VERSION_CHECK": "3.8"}, + env = {"VERSION_CHECK": "3.10"}, main = "version_test.py", + python_version = "3.10", ) -py_test_3_9( - name = "version_3_9_test", +py_test( + name = "version_3_11_test", srcs = ["version_test.py"], - env = {"VERSION_CHECK": "3.9"}, + env = {"VERSION_CHECK": "3.11"}, main = "version_test.py", + python_version = "3.11", ) -py_test_3_10( - name = "version_3_10_test", +py_test( + name = "version_3_12_test", srcs = ["version_test.py"], - env = {"VERSION_CHECK": "3.10"}, + env = {"VERSION_CHECK": "3.12"}, main = "version_test.py", + python_version = "3.12", ) -py_test_3_11( - name = "version_3_11_test", +py_test( + name = "version_3_13_test", srcs = ["version_test.py"], - env = {"VERSION_CHECK": "3.11"}, + env = {"VERSION_CHECK": "3.13"}, + main = "version_test.py", + python_version = "3.13", +) + +py_test( + name = "version_3_14_test", + srcs = ["version_test.py"], + env = {"VERSION_CHECK": "3.14"}, main = "version_test.py", + python_version = "3.14", ) py_test( - name = "version_default_takes_3_10_subprocess_test", + name = "version_default_takes_3_11_subprocess_test", srcs = ["cross_version_test.py"], - data = [":version_3_10"], + data = [":version_3_11"], env = { - "SUBPROCESS_VERSION_CHECK": "3.10", - "SUBPROCESS_VERSION_PY_BINARY": "$(rootpath :version_3_10)", - "VERSION_CHECK": "3.9", + "SUBPROCESS_VERSION_CHECK": "3.11", + "SUBPROCESS_VERSION_PY_BINARY": "$(rootpaths :version_3_11)", + "VERSION_CHECK": "3.10", }, main = "cross_version_test.py", ) -py_test_3_10( - name = "version_3_10_takes_3_9_subprocess_test", +py_test( + name = "version_3_11_takes_3_10_subprocess_test", srcs = ["cross_version_test.py"], - data = [":version_3_9"], + data = [":version_3_10"], env = { - "SUBPROCESS_VERSION_CHECK": "3.9", - "SUBPROCESS_VERSION_PY_BINARY": "$(rootpath :version_3_9)", - "VERSION_CHECK": "3.10", + "SUBPROCESS_VERSION_CHECK": "3.10", + "SUBPROCESS_VERSION_PY_BINARY": "$(rootpaths :version_3_10)", + "VERSION_CHECK": "3.11", }, main = "cross_version_test.py", + python_version = "3.11", ) sh_test( @@ -154,38 +187,28 @@ sh_test( srcs = ["version_test.sh"], data = [":version_default"], env = { - "VERSION_CHECK": "3.9", # The default defined in the WORKSPACE. - "VERSION_PY_BINARY": "$(rootpath :version_default)", + "VERSION_CHECK": "3.10", # The default defined in the WORKSPACE/MODULE. + "VERSION_PY_BINARY": "$(rootpaths :version_default)", }, ) sh_test( - name = "version_test_binary_3_8", - srcs = ["version_test.sh"], - data = [":version_3_8"], - env = { - "VERSION_CHECK": "3.8", - "VERSION_PY_BINARY": "$(rootpath :version_3_8)", - }, -) - -sh_test( - name = "version_test_binary_3_9", + name = "version_test_binary_3_10", srcs = ["version_test.sh"], - data = [":version_3_9"], + data = [":version_3_10"], env = { - "VERSION_CHECK": "3.9", - "VERSION_PY_BINARY": "$(rootpath :version_3_9)", + "VERSION_CHECK": "3.10", + "VERSION_PY_BINARY": "$(rootpaths :version_3_10)", }, ) sh_test( - name = "version_test_binary_3_10", + name = "version_test_binary_3_11", srcs = ["version_test.sh"], - data = [":version_3_10"], + data = [":version_3_11"], env = { - "VERSION_CHECK": "3.10", - "VERSION_PY_BINARY": "$(rootpath :version_3_10)", + "VERSION_CHECK": "3.11", + "VERSION_PY_BINARY": "$(rootpaths :version_3_11)", }, ) diff --git a/examples/multi_python_versions/tests/my_lib_test.py b/examples/multi_python_versions/tests/my_lib_test.py index 449cb8473c..b6c577c55a 100644 --- a/examples/multi_python_versions/tests/my_lib_test.py +++ b/examples/multi_python_versions/tests/my_lib_test.py @@ -12,7 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. -import os import sys import libs.my_lib as my_lib diff --git a/examples/multi_python_versions/tests/version_test.sh b/examples/multi_python_versions/tests/version_test.sh index 3bedb95ef9..3f5fd960cb 100755 --- a/examples/multi_python_versions/tests/version_test.sh +++ b/examples/multi_python_versions/tests/version_test.sh @@ -16,7 +16,11 @@ set -o errexit -o nounset -o pipefail -version_py_binary=$("${VERSION_PY_BINARY}") +# VERSION_PY_BINARY is a space separate list of the executable and its main +# py file. We just want the executable. +bin=($VERSION_PY_BINARY) +bin="${bin[@]//*.py}" +version_py_binary=$($bin) if [[ "${version_py_binary}" != "${VERSION_CHECK}" ]]; then echo >&2 "expected version '${VERSION_CHECK}' is different than returned '${version_py_binary}'" diff --git a/examples/pip_parse/.bazelrc b/examples/pip_parse/.bazelrc index a56904803c..f263a1744d 100644 --- a/examples/pip_parse/.bazelrc +++ b/examples/pip_parse/.bazelrc @@ -1,3 +1,3 @@ # https://docs.bazel.build/versions/main/best-practices.html#using-the-bazelrc-file try-import %workspace%/user.bazelrc -common:bazel7.x --incompatible_python_disallow_native_rules +common --incompatible_python_disallow_native_rules diff --git a/examples/pip_parse/BUILD.bazel b/examples/pip_parse/BUILD.bazel index fd744a2836..2dc9d61127 100644 --- a/examples/pip_parse/BUILD.bazel +++ b/examples/pip_parse/BUILD.bazel @@ -1,11 +1,12 @@ -load("@rules_python//python:defs.bzl", "py_binary", "py_test") load("@rules_python//python:pip.bzl", "compile_pip_requirements") +load("@rules_python//python:py_binary.bzl", "py_binary") +load("@rules_python//python:py_test.bzl", "py_test") load("@rules_python//python/entry_points:py_console_script_binary.bzl", "py_console_script_binary") # Toolchain setup, this is optional. # Demonstrate that we can use the same python interpreter for the toolchain and executing pip in pip install (see WORKSPACE). # -#load("@rules_python//python:defs.bzl", "py_runtime_pair") +#load("@rules_python//python:py_runtime_pair.bzl", "py_runtime_pair") # #py_runtime( # name = "python3_runtime", @@ -44,9 +45,6 @@ py_test( deps = [":main"], ) -# For pip dependencies which have entry points, the `entry_point` macro can be -# used from the generated `pip_parse` repository to access a runnable binary. - py_console_script_binary( name = "yamllint", pkg = "@pypi//yamllint", @@ -56,6 +54,10 @@ py_console_script_binary( compile_pip_requirements( name = "requirements", src = "requirements.in", + constraints = [ + "constraints_certifi.txt", + "constraints_urllib3.txt", + ], requirements_txt = "requirements_lock.txt", requirements_windows = "requirements_windows.txt", ) @@ -74,5 +76,8 @@ py_test( "WHEEL_DIST_INFO_CONTENTS": "$(rootpaths @pypi//requests:dist_info)", "YAMLLINT_ENTRY_POINT": "$(rlocationpath :yamllint)", }, - deps = ["@rules_python//python/runfiles"], + deps = [ + "@pypi//libclang", + "@rules_python//python/runfiles", + ], ) diff --git a/examples/pip_parse/MODULE.bazel b/examples/pip_parse/MODULE.bazel index f9ca90833f..ead5a06e29 100644 --- a/examples/pip_parse/MODULE.bazel +++ b/examples/pip_parse/MODULE.bazel @@ -9,14 +9,14 @@ local_path_override( python = use_extension("@rules_python//python/extensions:python.bzl", "python") python.toolchain( # We can specify the exact version. - python_version = "3.9.13", + python_version = "3.9.25", ) # You can use this repo mapping to ensure that your BUILD.bazel files don't need # to be updated when the python version changes to a different `3.9` version. use_repo( python, - python_3_9 = "python_3_9_13", + python_3_9 = "python_3_9_25", ) pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") @@ -34,7 +34,7 @@ pip.parse( }, hub_name = "pypi", # We need to use the same version here as in the `python.toolchain` call. - python_version = "3.9.13", + python_version = "3.9.25", requirements_lock = "//:requirements_lock.txt", requirements_windows = "//:requirements_windows.txt", ) diff --git a/examples/pip_parse/WORKSPACE b/examples/pip_parse/WORKSPACE index bb4714d941..e0d60af9ff 100644 --- a/examples/pip_parse/WORKSPACE +++ b/examples/pip_parse/WORKSPACE @@ -11,7 +11,7 @@ py_repositories() python_register_toolchains( name = "python_3_9", - python_version = "3.9.13", + python_version = "3.9.25", ) load("@rules_python//python:pip.bzl", "pip_parse") diff --git a/examples/pip_parse/constraints_certifi.txt b/examples/pip_parse/constraints_certifi.txt new file mode 100644 index 0000000000..7dc4eac259 --- /dev/null +++ b/examples/pip_parse/constraints_certifi.txt @@ -0,0 +1 @@ +certifi>=2025.1.31 \ No newline at end of file diff --git a/examples/pip_parse/constraints_urllib3.txt b/examples/pip_parse/constraints_urllib3.txt new file mode 100644 index 0000000000..3818262552 --- /dev/null +++ b/examples/pip_parse/constraints_urllib3.txt @@ -0,0 +1 @@ +urllib3>1.26.18 diff --git a/examples/pip_parse/pip_parse_test.py b/examples/pip_parse/pip_parse_test.py index 2fdd45477e..89e5eca254 100644 --- a/examples/pip_parse/pip_parse_test.py +++ b/examples/pip_parse/pip_parse_test.py @@ -50,18 +50,22 @@ def test_entry_point(self): def test_data(self): actual = os.environ.get("WHEEL_DATA_CONTENTS") self.assertIsNotNone(actual) - actual = self._remove_leading_dirs(actual.split(" ")) + actual = set(self._remove_leading_dirs(actual.split(" "))) - self.assertListEqual( - actual, - [ - "data/share/doc/packages/s3cmd/INSTALL.md", - "data/share/doc/packages/s3cmd/LICENSE", - "data/share/doc/packages/s3cmd/NEWS", - "data/share/doc/packages/s3cmd/README.md", - "data/share/man/man1/s3cmd.1", - ], - ) + s3cmd_bin = "bin/s3cmd" + if os.name == "nt": + s3cmd_bin += ".bat" + + expected = { + s3cmd_bin, + "data/share/doc/packages/s3cmd/INSTALL.md", + "data/share/doc/packages/s3cmd/LICENSE", + "data/share/doc/packages/s3cmd/NEWS", + "data/share/doc/packages/s3cmd/README.md", + "data/share/man/man1/s3cmd.1", + } + + self.assertEqual(actual, expected) def test_dist_info(self): actual = os.environ.get("WHEEL_DIST_INFO_CONTENTS") diff --git a/examples/pip_parse/requirements.in b/examples/pip_parse/requirements.in index 9d9e766d21..e4af3b1efe 100644 --- a/examples/pip_parse/requirements.in +++ b/examples/pip_parse/requirements.in @@ -3,3 +3,4 @@ s3cmd~=2.1.0 yamllint~=1.28.0 sphinx sphinxcontrib-serializinghtml +libclang diff --git a/examples/pip_parse/requirements_lock.txt b/examples/pip_parse/requirements_lock.txt index 5e7a198c38..2b9d8fcd47 100644 --- a/examples/pip_parse/requirements_lock.txt +++ b/examples/pip_parse/requirements_lock.txt @@ -12,10 +12,12 @@ babel==2.13.1 \ --hash=sha256:33e0952d7dd6374af8dbf6768cc4ddf3ccfefc244f9986d4074704f2fbd18900 \ --hash=sha256:7077a4984b02b6727ac10f1f7294484f737443d7e2e66c5e4380e41a3ae0b4ed # via sphinx -certifi==2024.7.4 \ - --hash=sha256:5a1e7645bc0ec61a09e26c36f6106dd4cf40c6db3a1fb6352b0244e7fb057c7b \ - --hash=sha256:c198e21b1289c2ab85ee4e67bb4b4ef3ead0892059901a8d5b622f24a1101e90 - # via requests +certifi==2025.4.26 \ + --hash=sha256:0a816057ea3cdefcef70270d2c515e4506bbc954f417fa5ade2021213bb8f0c6 \ + --hash=sha256:30350364dfe371162649852c63336a15c70c6510c2ad5015b21c2345311805f3 + # via + # -c constraints_certifi.txt + # requests chardet==4.0.0 \ --hash=sha256:0d6f53a15db4120f2b08c94f11e7d93d2c911ee118b6b30a04ec3ee8310179fa \ --hash=sha256:f864054d66fd9118f2e67044ac8981a54775ec5b67aed0441892edb553d21da5 @@ -36,10 +38,22 @@ importlib-metadata==6.8.0 \ --hash=sha256:3ebb78df84a805d7698245025b975d9d67053cd94c79245ba4b3eb694abe68bb \ --hash=sha256:dbace7892d8c0c4ac1ad096662232f831d4e64f4c4545bd53016a3e9d4654743 # via sphinx -jinja2==3.1.4 \ - --hash=sha256:4a3aee7acbbe7303aede8e9648d13b8bf88a429282aa6122a993f0ac800cb369 \ - --hash=sha256:bc5dd2abb727a5319567b7a813e6a2e7318c39f4f487cfe6c89c6f9c7d25197d +jinja2==3.1.6 \ + --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ + --hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67 # via sphinx +libclang==18.1.1 \ + --hash=sha256:0b2e143f0fac830156feb56f9231ff8338c20aecfe72b4ffe96f19e5a1dbb69a \ + --hash=sha256:3f0e1f49f04d3cd198985fea0511576b0aee16f9ff0e0f0cad7f9c57ec3c20e8 \ + --hash=sha256:4dd2d3b82fab35e2bf9ca717d7b63ac990a3519c7e312f19fa8e86dcc712f7fb \ + --hash=sha256:54dda940a4a0491a9d1532bf071ea3ef26e6dbaf03b5000ed94dd7174e8f9592 \ + --hash=sha256:69f8eb8f65c279e765ffd28aaa7e9e364c776c17618af8bff22a8df58677ff4f \ + --hash=sha256:6f14c3f194704e5d09769108f03185fce7acaf1d1ae4bbb2f30a72c2400cb7c5 \ + --hash=sha256:83ce5045d101b669ac38e6da8e58765f12da2d3aafb3b9b98d88b286a60964d8 \ + --hash=sha256:a1214966d08d73d971287fc3ead8dfaf82eb07fb197680d8b3859dbbbbf78250 \ + --hash=sha256:c533091d8a3bbf7460a00cb6c1a71da93bffe148f172c7d03b1c31fbf8aa2a0b \ + --hash=sha256:cf4a99b05376513717ab5d82a0db832c56ccea4fd61a69dbb7bccf2dfb207dbe + # via -r requirements.in markupsafe==2.1.3 \ --hash=sha256:05fb21170423db021895e1ea1e1f3ab3adb85d1c2333cbc2310f2a26bc77272e \ --hash=sha256:0a4e4a1aff6c7ac4cd55792abf96c915634c2b97e3cc1c7129578aa68ebd754e \ @@ -218,10 +232,12 @@ sphinxcontrib-serializinghtml==1.1.9 \ # via # -r requirements.in # sphinx -urllib3==1.26.18 \ - --hash=sha256:34b97092d7e0a3a8cf7cd10e386f401b3737364026c45e622aa02903dffe0f07 \ - --hash=sha256:f8ecc1bba5667413457c529ab955bf8c67b45db799d159066261719e328580a0 - # via requests +urllib3==1.26.20 \ + --hash=sha256:0ed14ccfbf1c30a9072c7ca157e4319b70d65f623e91e7b32fadb2853431016e \ + --hash=sha256:40c2dc0c681e47eb8f90e7e27bf6ff7df2e677421fd46756da1161c39ca70d32 + # via + # -c constraints_urllib3.txt + # requests yamllint==1.28.0 \ --hash=sha256:89bb5b5ac33b1ade059743cf227de73daa34d5e5a474b06a5e17fc16583b0cf2 \ --hash=sha256:9e3d8ddd16d0583214c5fdffe806c9344086721f107435f68bad990e5a88826b diff --git a/examples/pip_parse/requirements_windows.txt b/examples/pip_parse/requirements_windows.txt index 4b1969255a..c407d0c8cb 100644 --- a/examples/pip_parse/requirements_windows.txt +++ b/examples/pip_parse/requirements_windows.txt @@ -12,10 +12,12 @@ babel==2.13.1 \ --hash=sha256:33e0952d7dd6374af8dbf6768cc4ddf3ccfefc244f9986d4074704f2fbd18900 \ --hash=sha256:7077a4984b02b6727ac10f1f7294484f737443d7e2e66c5e4380e41a3ae0b4ed # via sphinx -certifi==2024.7.4 \ - --hash=sha256:5a1e7645bc0ec61a09e26c36f6106dd4cf40c6db3a1fb6352b0244e7fb057c7b \ - --hash=sha256:c198e21b1289c2ab85ee4e67bb4b4ef3ead0892059901a8d5b622f24a1101e90 - # via requests +certifi==2025.4.26 \ + --hash=sha256:0a816057ea3cdefcef70270d2c515e4506bbc954f417fa5ade2021213bb8f0c6 \ + --hash=sha256:30350364dfe371162649852c63336a15c70c6510c2ad5015b21c2345311805f3 + # via + # -c constraints_certifi.txt + # requests chardet==4.0.0 \ --hash=sha256:0d6f53a15db4120f2b08c94f11e7d93d2c911ee118b6b30a04ec3ee8310179fa \ --hash=sha256:f864054d66fd9118f2e67044ac8981a54775ec5b67aed0441892edb553d21da5 @@ -40,10 +42,22 @@ importlib-metadata==6.8.0 \ --hash=sha256:3ebb78df84a805d7698245025b975d9d67053cd94c79245ba4b3eb694abe68bb \ --hash=sha256:dbace7892d8c0c4ac1ad096662232f831d4e64f4c4545bd53016a3e9d4654743 # via sphinx -jinja2==3.1.4 \ - --hash=sha256:4a3aee7acbbe7303aede8e9648d13b8bf88a429282aa6122a993f0ac800cb369 \ - --hash=sha256:bc5dd2abb727a5319567b7a813e6a2e7318c39f4f487cfe6c89c6f9c7d25197d +jinja2==3.1.6 \ + --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ + --hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67 # via sphinx +libclang==18.1.1 \ + --hash=sha256:0b2e143f0fac830156feb56f9231ff8338c20aecfe72b4ffe96f19e5a1dbb69a \ + --hash=sha256:3f0e1f49f04d3cd198985fea0511576b0aee16f9ff0e0f0cad7f9c57ec3c20e8 \ + --hash=sha256:4dd2d3b82fab35e2bf9ca717d7b63ac990a3519c7e312f19fa8e86dcc712f7fb \ + --hash=sha256:54dda940a4a0491a9d1532bf071ea3ef26e6dbaf03b5000ed94dd7174e8f9592 \ + --hash=sha256:69f8eb8f65c279e765ffd28aaa7e9e364c776c17618af8bff22a8df58677ff4f \ + --hash=sha256:6f14c3f194704e5d09769108f03185fce7acaf1d1ae4bbb2f30a72c2400cb7c5 \ + --hash=sha256:83ce5045d101b669ac38e6da8e58765f12da2d3aafb3b9b98d88b286a60964d8 \ + --hash=sha256:a1214966d08d73d971287fc3ead8dfaf82eb07fb197680d8b3859dbbbbf78250 \ + --hash=sha256:c533091d8a3bbf7460a00cb6c1a71da93bffe148f172c7d03b1c31fbf8aa2a0b \ + --hash=sha256:cf4a99b05376513717ab5d82a0db832c56ccea4fd61a69dbb7bccf2dfb207dbe + # via -r requirements.in markupsafe==2.1.3 \ --hash=sha256:05fb21170423db021895e1ea1e1f3ab3adb85d1c2333cbc2310f2a26bc77272e \ --hash=sha256:0a4e4a1aff6c7ac4cd55792abf96c915634c2b97e3cc1c7129578aa68ebd754e \ @@ -222,10 +236,12 @@ sphinxcontrib-serializinghtml==1.1.9 \ # via # -r requirements.in # sphinx -urllib3==1.26.18 \ - --hash=sha256:34b97092d7e0a3a8cf7cd10e386f401b3737364026c45e622aa02903dffe0f07 \ - --hash=sha256:f8ecc1bba5667413457c529ab955bf8c67b45db799d159066261719e328580a0 - # via requests +urllib3==1.26.20 \ + --hash=sha256:0ed14ccfbf1c30a9072c7ca157e4319b70d65f623e91e7b32fadb2853431016e \ + --hash=sha256:40c2dc0c681e47eb8f90e7e27bf6ff7df2e677421fd46756da1161c39ca70d32 + # via + # -c constraints_urllib3.txt + # requests yamllint==1.28.0 \ --hash=sha256:89bb5b5ac33b1ade059743cf227de73daa34d5e5a474b06a5e17fc16583b0cf2 \ --hash=sha256:9e3d8ddd16d0583214c5fdffe806c9344086721f107435f68bad990e5a88826b diff --git a/examples/pip_parse_vendored/.bazelrc b/examples/pip_parse_vendored/.bazelrc index be3555d1eb..a6ea2d9138 100644 --- a/examples/pip_parse_vendored/.bazelrc +++ b/examples/pip_parse_vendored/.bazelrc @@ -5,5 +5,6 @@ build --enable_runfiles # Vendoring requirements.bzl files isn't necessary under bzlmod # When workspace support is dropped, this example can be removed. -build --noexperimental_enable_bzlmod -common:bazel7.x --incompatible_python_disallow_native_rules +common --noenable_bzlmod +common --enable_workspace +common --incompatible_python_disallow_native_rules diff --git a/examples/pip_parse_vendored/.bazelversion b/examples/pip_parse_vendored/.bazelversion new file mode 100644 index 0000000000..35907cd9ca --- /dev/null +++ b/examples/pip_parse_vendored/.bazelversion @@ -0,0 +1 @@ +7.x diff --git a/examples/pip_parse_vendored/BUILD.bazel b/examples/pip_parse_vendored/BUILD.bazel index e2b1f5d49b..74b9286359 100644 --- a/examples/pip_parse_vendored/BUILD.bazel +++ b/examples/pip_parse_vendored/BUILD.bazel @@ -1,8 +1,9 @@ load("@bazel_skylib//rules:build_test.bzl", "build_test") load("@bazel_skylib//rules:diff_test.bzl", "diff_test") load("@bazel_skylib//rules:write_file.bzl", "write_file") -load("@rules_python//python:defs.bzl", "py_test") load("@rules_python//python:pip.bzl", "compile_pip_requirements") +load("@rules_python//python:py_test.bzl", "py_test") +load("@rules_shell//shell:sh_binary.bzl", "sh_binary") load("//:requirements.bzl", "all_data_requirements", "all_requirements", "all_whl_requirements", "requirement") # This rule adds a convenient way to update the requirements.txt diff --git a/examples/pip_parse_vendored/README.md b/examples/pip_parse_vendored/README.md index f53260a175..baa51f5729 100644 --- a/examples/pip_parse_vendored/README.md +++ b/examples/pip_parse_vendored/README.md @@ -1,7 +1,7 @@ # pip_parse vendored This example is like pip_parse, however we avoid loading from the generated file. -See https://github.com/bazelbuild/rules_python/issues/608 +See https://github.com/bazel-contrib/rules_python/issues/608 and https://blog.aspect.dev/avoid-eager-fetches. The requirements now form a triple: @@ -20,12 +20,11 @@ python_register_toolchains( name = "python39", python_version = "3.9", ) -load("@python39//:defs.bzl", "interpreter") # Load dependencies vendored by some other ruleset. load("@some_rules//:py_deps.bzl", "install_deps") install_deps( - python_interpreter_target = interpreter, + python_interpreter_target = "@python39_host//:python", ) ``` diff --git a/examples/pip_parse_vendored/WORKSPACE b/examples/pip_parse_vendored/WORKSPACE index d7a11ea596..5e80b4116b 100644 --- a/examples/pip_parse_vendored/WORKSPACE +++ b/examples/pip_parse_vendored/WORKSPACE @@ -39,3 +39,19 @@ pip_parse( load("//:requirements.bzl", "install_deps") install_deps() + +load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") + +# See https://github.com/bazelbuild/rules_shell/releases/tag/v0.2.0 +http_archive( + name = "rules_shell", + sha256 = "410e8ff32e018b9efd2743507e7595c26e2628567c42224411ff533b57d27c28", + strip_prefix = "rules_shell-0.2.0", + url = "https://github.com/bazelbuild/rules_shell/releases/download/v0.2.0/rules_shell-v0.2.0.tar.gz", +) + +load("@rules_shell//shell:repositories.bzl", "rules_shell_dependencies", "rules_shell_toolchains") + +rules_shell_dependencies() + +rules_shell_toolchains() diff --git a/examples/pip_parse_vendored/requirements.bzl b/examples/pip_parse_vendored/requirements.bzl index 50bfe9fe8e..f5551573fb 100644 --- a/examples/pip_parse_vendored/requirements.bzl +++ b/examples/pip_parse_vendored/requirements.bzl @@ -4,7 +4,7 @@ """ load("@rules_python//python:pip.bzl", "pip_utils") -load("@rules_python//python/pip_install:pip_repository.bzl", "group_library", "whl_library") +load("@rules_python//python/pip_install:pip_repository.bzl", "whl_config_repo", "whl_library") all_requirements = [ "@my_project_pip_deps_vendored_certifi//:pkg", @@ -33,11 +33,11 @@ all_data_requirements = [ ] _packages = [ - ("my_project_pip_deps_vendored_certifi", "certifi==2023.7.22 --hash=sha256:539cc1d13202e33ca466e88b2807e29f4c13049d6d87031a3c110744495cb082 --hash=sha256:92d6037539857d8206b8f6ae472e8b77db8058fec5937a1ef3f54304089edbb9"), - ("my_project_pip_deps_vendored_charset_normalizer", "charset-normalizer==2.1.1 --hash=sha256:5a3d016c7c547f69d6f81fb0db9449ce888b418b5b9952cc5e6e66843e9dd845 --hash=sha256:83e9a75d1911279afd89352c68b45348559d1fc0506b054b346651b5e7fee29f"), - ("my_project_pip_deps_vendored_idna", "idna==3.4 --hash=sha256:814f528e8dead7d329833b91c5faa87d60bf71824cd12a7530b5526063d02cb4 --hash=sha256:90b77e79eaa3eba6de819a0c442c0b4ceefc341a7a2ab77d7562bf49f425c5c2"), - ("my_project_pip_deps_vendored_requests", "requests==2.28.1 --hash=sha256:7c5599b102feddaa661c826c56ab4fee28bfd17f5abca1ebbe3e7f19d7c97983 --hash=sha256:8fefa2a1a1365bf5520aac41836fbee479da67864514bdb821f31ce07ce65349"), - ("my_project_pip_deps_vendored_urllib3", "urllib3==1.26.13 --hash=sha256:47cc05d99aaa09c9e72ed5809b60e7ba354e64b59c9c173ac3018642d8bb41fc --hash=sha256:c083dd0dce68dbfbe1129d5271cb90f9447dea7d52097c6e0126120c521ddea8"), + ("my_project_pip_deps_vendored_certifi", "certifi==2023.7.22 --hash=sha256:539cc1d13202e33ca466e88b2807e29f4c13049d6d87031a3c110744495cb082 --hash=sha256:92d6037539857d8206b8f6ae472e8b77db8058fec5937a1ef3f54304089edbb9"), + ("my_project_pip_deps_vendored_charset_normalizer", "charset-normalizer==2.1.1 --hash=sha256:5a3d016c7c547f69d6f81fb0db9449ce888b418b5b9952cc5e6e66843e9dd845 --hash=sha256:83e9a75d1911279afd89352c68b45348559d1fc0506b054b346651b5e7fee29f"), + ("my_project_pip_deps_vendored_idna", "idna==3.4 --hash=sha256:814f528e8dead7d329833b91c5faa87d60bf71824cd12a7530b5526063d02cb4 --hash=sha256:90b77e79eaa3eba6de819a0c442c0b4ceefc341a7a2ab77d7562bf49f425c5c2"), + ("my_project_pip_deps_vendored_requests", "requests==2.28.1 --hash=sha256:7c5599b102feddaa661c826c56ab4fee28bfd17f5abca1ebbe3e7f19d7c97983 --hash=sha256:8fefa2a1a1365bf5520aac41836fbee479da67864514bdb821f31ce07ce65349"), + ("my_project_pip_deps_vendored_urllib3", "urllib3==1.26.13 --hash=sha256:47cc05d99aaa09c9e72ed5809b60e7ba354e64b59c9c173ac3018642d8bb41fc --hash=sha256:c083dd0dce68dbfbe1129d5271cb90f9447dea7d52097c6e0126120c521ddea8"), ] _config = { "download_only": False, @@ -91,12 +91,17 @@ def install_deps(**whl_library_kwargs): for requirement in group_requirements } - group_repo = "my_project_pip_deps_vendored__groups" - group_library( - name = group_repo, + config_repo = "my_project_pip_deps_vendored__config" + whl_config_repo( + name = "my_project_pip_deps_vendored__config", repo_prefix = "my_project_pip_deps_vendored_", groups = all_requirement_groups, + whl_map = { + p: "" + for p in all_whl_requirements_by_package + }, ) + config_load = "@{}//:config.bzl".format(config_repo) # Install wheels which may be participants in a group whl_config = dict(_config) @@ -112,5 +117,6 @@ def install_deps(**whl_library_kwargs): group_name = group_name, group_deps = group_deps, annotation = _get_annotation(requirement), + config_load = config_load, **whl_config ) diff --git a/examples/pip_repository_annotations/.bazelrc b/examples/pip_repository_annotations/.bazelrc deleted file mode 100644 index 4f62c6e76f..0000000000 --- a/examples/pip_repository_annotations/.bazelrc +++ /dev/null @@ -1,7 +0,0 @@ -# https://docs.bazel.build/versions/main/best-practices.html#using-the-bazelrc-file -try-import %workspace%/user.bazelrc - -# This example is WORKSPACE specific. The equivalent functionality -# is in examples/bzlmod as the `whl_mods` feature. -build --experimental_enable_bzlmod=false -common:bazel7.x --incompatible_python_disallow_native_rules diff --git a/examples/pip_repository_annotations/.gitignore b/examples/pip_repository_annotations/.gitignore deleted file mode 100644 index a6ef824c1f..0000000000 --- a/examples/pip_repository_annotations/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/bazel-* diff --git a/examples/pip_repository_annotations/BUILD.bazel b/examples/pip_repository_annotations/BUILD.bazel deleted file mode 100644 index bdf9df1274..0000000000 --- a/examples/pip_repository_annotations/BUILD.bazel +++ /dev/null @@ -1,28 +0,0 @@ -load("@rules_python//python:defs.bzl", "py_test") -load("@rules_python//python:pip.bzl", "compile_pip_requirements") - -exports_files( - glob(["data/**"]), - visibility = ["//visibility:public"], -) - -# This rule adds a convenient way to update the requirements file. -compile_pip_requirements( - name = "requirements", - src = "requirements.in", -) - -py_test( - name = "pip_parse_annotations_test", - srcs = ["pip_repository_annotations_test.py"], - env = { - "REQUESTS_PKG_DIR": "pip_requests", - "WHEEL_PKG_DIR": "pip_wheel", - }, - main = "pip_repository_annotations_test.py", - deps = [ - "@pip_requests//:pkg", - "@pip_wheel//:pkg", - "@rules_python//python/runfiles", - ], -) diff --git a/examples/pip_repository_annotations/WORKSPACE b/examples/pip_repository_annotations/WORKSPACE deleted file mode 100644 index 8540555084..0000000000 --- a/examples/pip_repository_annotations/WORKSPACE +++ /dev/null @@ -1,62 +0,0 @@ -workspace(name = "pip_repository_annotations_example") - -local_repository( - name = "rules_python", - path = "../..", -) - -load("@rules_python//python:repositories.bzl", "py_repositories", "python_register_toolchains") - -py_repositories() - -python_register_toolchains( - name = "python39", - python_version = "3.9", -) - -load("@rules_python//python:pip.bzl", "package_annotation", "pip_parse") - -# Here we can see an example of annotations being applied to an arbitrary -# package. For details on `package_annotation` and it's uses, see the -# docs at @rules_python//docs:pip.md`. -ANNOTATIONS = { - # This annotation verifies that annotations work correctly for pip packages with extras - # specified, in this case requests[security]. - "requests": package_annotation( - additive_build_content = """\ -load("@bazel_skylib//rules:write_file.bzl", "write_file") -write_file( - name = "generated_file", - out = "generated_file.txt", - content = ["Hello world from requests"], -) -""", - data = [":generated_file"], - ), - "wheel": package_annotation( - additive_build_content = """\ -load("@bazel_skylib//rules:write_file.bzl", "write_file") -write_file( - name = "generated_file", - out = "generated_file.txt", - content = ["Hello world from build content file"], -) -""", - copy_executables = {"@pip_repository_annotations_example//:data/copy_executable.py": "copied_content/executable.py"}, - copy_files = {"@pip_repository_annotations_example//:data/copy_file.txt": "copied_content/file.txt"}, - data = [":generated_file"], - data_exclude_glob = ["site-packages/*.dist-info/WHEEL"], - ), -} - -# For a more thorough example of `pip_parse`. See `@rules_python//examples/pip_parse` -pip_parse( - name = "pip", - annotations = ANNOTATIONS, - python_interpreter_target = "@python39_host//:python", - requirements_lock = "//:requirements.txt", -) - -load("@pip//:requirements.bzl", "install_deps") - -install_deps() diff --git a/examples/pip_repository_annotations/data/copy_file.txt b/examples/pip_repository_annotations/data/copy_file.txt deleted file mode 100644 index b1020f7b95..0000000000 --- a/examples/pip_repository_annotations/data/copy_file.txt +++ /dev/null @@ -1 +0,0 @@ -Hello world from copied file diff --git a/examples/pip_repository_annotations/pip_repository_annotations_test.py b/examples/pip_repository_annotations/pip_repository_annotations_test.py deleted file mode 100644 index e41dd4f0f6..0000000000 --- a/examples/pip_repository_annotations/pip_repository_annotations_test.py +++ /dev/null @@ -1,129 +0,0 @@ -#!/usr/bin/env python3 -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - - -import os -import platform -import subprocess -import sys -import unittest -from pathlib import Path - -from rules_python.python.runfiles import runfiles - - -class PipRepositoryAnnotationsTest(unittest.TestCase): - maxDiff = None - - def wheel_pkg_dir(self) -> str: - env = os.environ.get("WHEEL_PKG_DIR") - self.assertIsNotNone(env) - return env - - def test_build_content_and_data(self): - r = runfiles.Create() - rpath = r.Rlocation( - "pip_repository_annotations_example/external/{}/generated_file.txt".format( - self.wheel_pkg_dir() - ) - ) - generated_file = Path(rpath) - self.assertTrue(generated_file.exists()) - - content = generated_file.read_text().rstrip() - self.assertEqual(content, "Hello world from build content file") - - def test_copy_files(self): - r = runfiles.Create() - rpath = r.Rlocation( - "pip_repository_annotations_example/external/{}/copied_content/file.txt".format( - self.wheel_pkg_dir() - ) - ) - copied_file = Path(rpath) - self.assertTrue(copied_file.exists()) - - content = copied_file.read_text().rstrip() - self.assertEqual(content, "Hello world from copied file") - - def test_copy_executables(self): - r = runfiles.Create() - rpath = r.Rlocation( - "pip_repository_annotations_example/external/{}/copied_content/executable{}".format( - self.wheel_pkg_dir(), - ".exe" if platform.system() == "windows" else ".py", - ) - ) - executable = Path(rpath) - self.assertTrue(executable.exists()) - - proc = subprocess.run( - [sys.executable, str(executable)], - check=True, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - ) - stdout = proc.stdout.decode("utf-8").strip() - self.assertEqual(stdout, "Hello world from copied executable") - - def test_data_exclude_glob(self): - current_wheel_version = "0.38.4" - - r = runfiles.Create() - dist_info_dir = "pip_repository_annotations_example/external/{}/site-packages/wheel-{}.dist-info".format( - self.wheel_pkg_dir(), - current_wheel_version, - ) - - # Note: `METADATA` is important as it's consumed by https://docs.python.org/3/library/importlib.metadata.html - # `METADATA` is expected to be there to show dist-info files are included in the runfiles. - metadata_path = r.Rlocation("{}/METADATA".format(dist_info_dir)) - - # However, `WHEEL` was explicitly excluded, so it should be missing - wheel_path = r.Rlocation("{}/WHEEL".format(dist_info_dir)) - - # Because windows does not have `--enable_runfiles` on by default, the - # `runfiles.Rlocation` results will be different on this platform vs - # unix platforms. See `@rules_python//python/runfiles` for more details. - if platform.system() == "Windows": - self.assertIsNotNone(metadata_path) - self.assertIsNone(wheel_path) - else: - self.assertTrue(Path(metadata_path).exists()) - self.assertFalse(Path(wheel_path).exists()) - - def requests_pkg_dir(self) -> str: - env = os.environ.get("REQUESTS_PKG_DIR") - self.assertIsNotNone(env) - return env - - def test_extra(self): - # This test verifies that annotations work correctly for pip packages with extras - # specified, in this case requests[security]. - r = runfiles.Create() - rpath = r.Rlocation( - "pip_repository_annotations_example/external/{}/generated_file.txt".format( - self.requests_pkg_dir() - ) - ) - generated_file = Path(rpath) - self.assertTrue(generated_file.exists()) - - content = generated_file.read_text().rstrip() - self.assertEqual(content, "Hello world from requests") - - -if __name__ == "__main__": - unittest.main() diff --git a/examples/pip_repository_annotations/requirements.in b/examples/pip_repository_annotations/requirements.in deleted file mode 100644 index c9afafc6f5..0000000000 --- a/examples/pip_repository_annotations/requirements.in +++ /dev/null @@ -1,7 +0,0 @@ -# This flag allows for regression testing requirements arguments in -# `pip_repository` rules. ---extra-index-url https://pypi.org/simple/ - -certifi>=2023.7.22 # https://security.snyk.io/vuln/SNYK-PYTHON-CERTIFI-5805047 -wheel -requests[security]>=2.8.1 diff --git a/examples/pip_repository_annotations/requirements.txt b/examples/pip_repository_annotations/requirements.txt deleted file mode 100644 index f1069a7452..0000000000 --- a/examples/pip_repository_annotations/requirements.txt +++ /dev/null @@ -1,34 +0,0 @@ -# -# This file is autogenerated by pip-compile with Python 3.9 -# by the following command: -# -# bazel run //:requirements.update -# ---extra-index-url https://pypi.org/simple/ - -certifi==2023.7.22 \ - --hash=sha256:539cc1d13202e33ca466e88b2807e29f4c13049d6d87031a3c110744495cb082 \ - --hash=sha256:92d6037539857d8206b8f6ae472e8b77db8058fec5937a1ef3f54304089edbb9 - # via - # -r requirements.in - # requests -charset-normalizer==2.1.1 \ - --hash=sha256:5a3d016c7c547f69d6f81fb0db9449ce888b418b5b9952cc5e6e66843e9dd845 \ - --hash=sha256:83e9a75d1911279afd89352c68b45348559d1fc0506b054b346651b5e7fee29f - # via requests -idna==3.7 \ - --hash=sha256:028ff3aadf0609c1fd278d8ea3089299412a7a8b9bd005dd08b9f8285bcb5cfc \ - --hash=sha256:82fee1fc78add43492d3a1898bfa6d8a904cc97d8427f683ed8e798d07761aa0 - # via requests -requests[security]==2.28.1 \ - --hash=sha256:7c5599b102feddaa661c826c56ab4fee28bfd17f5abca1ebbe3e7f19d7c97983 \ - --hash=sha256:8fefa2a1a1365bf5520aac41836fbee479da67864514bdb821f31ce07ce65349 - # via -r requirements.in -urllib3==1.26.18 \ - --hash=sha256:34b97092d7e0a3a8cf7cd10e386f401b3737364026c45e622aa02903dffe0f07 \ - --hash=sha256:f8ecc1bba5667413457c529ab955bf8c67b45db799d159066261719e328580a0 - # via requests -wheel==0.38.4 \ - --hash=sha256:965f5259b566725405b05e7cf774052044b1ed30119b5d586b2703aafe8719ac \ - --hash=sha256:b60533f3f5d530e971d6737ca6d58681ee434818fab630c83a734bb10c083ce8 - # via -r requirements.in diff --git a/examples/py_proto_library/.bazelrc b/examples/py_proto_library/.bazelrc deleted file mode 100644 index 65d8a0a2f6..0000000000 --- a/examples/py_proto_library/.bazelrc +++ /dev/null @@ -1,3 +0,0 @@ -# The equivalent bzlmod behavior is covered by examples/bzlmod/py_proto_library -common --noenable_bzlmod -common:bazel7.x --incompatible_python_disallow_native_rules diff --git a/examples/py_proto_library/.gitignore b/examples/py_proto_library/.gitignore deleted file mode 100644 index e5ae073b3c..0000000000 --- a/examples/py_proto_library/.gitignore +++ /dev/null @@ -1,4 +0,0 @@ -# git ignore patterns - -/bazel-* -user.bazelrc diff --git a/examples/py_proto_library/BUILD.bazel b/examples/py_proto_library/BUILD.bazel deleted file mode 100644 index 0158aa2d37..0000000000 --- a/examples/py_proto_library/BUILD.bazel +++ /dev/null @@ -1,18 +0,0 @@ -load("@rules_python//python:defs.bzl", "py_test") - -py_test( - name = "pricetag_test", - srcs = ["test.py"], - main = "test.py", - deps = [ - "//example.com/proto:pricetag_proto_py_pb2", - ], -) - -py_test( - name = "message_test", - srcs = ["message_test.py"], - deps = [ - "//example.com/another_proto:message_proto_py_pb2", - ], -) diff --git a/examples/py_proto_library/WORKSPACE b/examples/py_proto_library/WORKSPACE deleted file mode 100644 index 81f189dbbf..0000000000 --- a/examples/py_proto_library/WORKSPACE +++ /dev/null @@ -1,49 +0,0 @@ -# NB: short workspace name is required to workaround PATH length limitation, see -# https://github.com/bazelbuild/bazel/issues/18683#issuecomment-1843857373 -workspace(name = "p") - -# The following local_path_override is only needed to run this example as part of our CI. -local_repository( - name = "rules_python", - path = "../..", -) - -# When not using this example in the rules_python git repo you would load the python -# rules using http_archive(), as documented in the release notes. - -load("@rules_python//python:repositories.bzl", "py_repositories", "python_register_toolchains") - -# We install the rules_python dependencies using the function below. -py_repositories() - -python_register_toolchains( - name = "python39", - python_version = "3.9", -) - -# Then we need to setup dependencies in order to use py_proto_library -load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") - -http_archive( - name = "rules_proto", - sha256 = "904a8097fae42a690c8e08d805210e40cccb069f5f9a0f6727cf4faa7bed2c9c", - strip_prefix = "rules_proto-6.0.0-rc1", - url = "https://github.com/bazelbuild/rules_proto/releases/download/6.0.0-rc1/rules_proto-6.0.0-rc1.tar.gz", -) - -load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains") - -rules_proto_dependencies() - -rules_proto_toolchains() - -http_archive( - name = "com_google_protobuf", - sha256 = "4fc5ff1b2c339fb86cd3a25f0b5311478ab081e65ad258c6789359cd84d421f8", - strip_prefix = "protobuf-26.1", - urls = ["https://github.com/protocolbuffers/protobuf/archive/v26.1.tar.gz"], -) - -load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") - -protobuf_deps() diff --git a/examples/py_proto_library/example.com/another_proto/BUILD.bazel b/examples/py_proto_library/example.com/another_proto/BUILD.bazel deleted file mode 100644 index dd58265bc9..0000000000 --- a/examples/py_proto_library/example.com/another_proto/BUILD.bazel +++ /dev/null @@ -1,16 +0,0 @@ -load("@rules_proto//proto:defs.bzl", "proto_library") -load("@rules_python//python:proto.bzl", "py_proto_library") - -py_proto_library( - name = "message_proto_py_pb2", - visibility = ["//visibility:public"], - deps = [":message_proto"], -) - -proto_library( - name = "message_proto", - srcs = ["message.proto"], - # https://bazel.build/reference/be/protocol-buffer#proto_library.strip_import_prefix - strip_import_prefix = "/example.com", - deps = ["//example.com/proto:pricetag_proto"], -) diff --git a/examples/py_proto_library/example.com/another_proto/message.proto b/examples/py_proto_library/example.com/another_proto/message.proto deleted file mode 100644 index 6e7dcc5793..0000000000 --- a/examples/py_proto_library/example.com/another_proto/message.proto +++ /dev/null @@ -1,10 +0,0 @@ -syntax = "proto3"; - -package rules_python; - -import "proto/pricetag.proto"; - -message TestMessage { - uint32 index = 1; - PriceTag pricetag = 2; -} diff --git a/examples/py_proto_library/example.com/proto/BUILD.bazel b/examples/py_proto_library/example.com/proto/BUILD.bazel deleted file mode 100644 index dc91162aa6..0000000000 --- a/examples/py_proto_library/example.com/proto/BUILD.bazel +++ /dev/null @@ -1,17 +0,0 @@ -load("@rules_proto//proto:defs.bzl", "proto_library") -load("@rules_python//python:proto.bzl", "py_proto_library") - -py_proto_library( - name = "pricetag_proto_py_pb2", - visibility = ["//visibility:public"], - deps = [":pricetag_proto"], -) - -proto_library( - name = "pricetag_proto", - srcs = ["pricetag.proto"], - # https://bazel.build/reference/be/protocol-buffer#proto_library.strip_import_prefix - strip_import_prefix = "/example.com", - visibility = ["//visibility:public"], - deps = ["@com_google_protobuf//:any_proto"], -) diff --git a/examples/py_proto_library/example.com/proto/pricetag.proto b/examples/py_proto_library/example.com/proto/pricetag.proto deleted file mode 100644 index 3fa68de84b..0000000000 --- a/examples/py_proto_library/example.com/proto/pricetag.proto +++ /dev/null @@ -1,11 +0,0 @@ -syntax = "proto3"; - -import "google/protobuf/any.proto"; - -package rules_python; - -message PriceTag { - string name = 2; - double cost = 1; - google.protobuf.Any metadata = 3; -} diff --git a/examples/py_proto_library/message_test.py b/examples/py_proto_library/message_test.py deleted file mode 100644 index b1a6942a54..0000000000 --- a/examples/py_proto_library/message_test.py +++ /dev/null @@ -1,16 +0,0 @@ -import sys -import unittest - -from another_proto import message_pb2 - - -class TestCase(unittest.TestCase): - def test_message(self): - got = message_pb2.TestMessage( - index=5, - ) - self.assertIsNotNone(got) - - -if __name__ == "__main__": - sys.exit(unittest.main()) diff --git a/examples/py_proto_library/test.py b/examples/py_proto_library/test.py deleted file mode 100644 index 24ab8ddc70..0000000000 --- a/examples/py_proto_library/test.py +++ /dev/null @@ -1,21 +0,0 @@ -import json -import unittest - -from proto import pricetag_pb2 - - -class TestCase(unittest.TestCase): - def test_pricetag(self): - got = pricetag_pb2.PriceTag( - name="dollar", - cost=5.00, - ) - - metadata = {"description": "some text..."} - got.metadata.value = json.dumps(metadata).encode("utf-8") - - self.assertIsNotNone(got) - - -if __name__ == "__main__": - unittest.main() diff --git a/examples/wheel/BUILD.bazel b/examples/wheel/BUILD.bazel index 1eaf03525a..01dc4fab41 100644 --- a/examples/wheel/BUILD.bazel +++ b/examples/wheel/BUILD.bazel @@ -15,9 +15,10 @@ load("@bazel_skylib//rules:build_test.bzl", "build_test") load("@bazel_skylib//rules:write_file.bzl", "write_file") load("//examples/wheel/private:wheel_utils.bzl", "directory_writer", "make_variable_tags") -load("//python:defs.bzl", "py_library", "py_test") load("//python:packaging.bzl", "py_package", "py_wheel") load("//python:pip.bzl", "compile_pip_requirements") +load("//python:py_library.bzl", "py_library") +load("//python:py_test.bzl", "py_test") load("//python:versions.bzl", "gen_python_config_settings") load("//python/entry_points:py_console_script_binary.bzl", "py_console_script_binary") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility @@ -32,6 +33,7 @@ py_library( deps = [ "//examples/wheel/lib:simple_module", "//examples/wheel/lib:module_with_data", + "//examples/wheel/lib:module_with_type_annotations", # Example dependency which is not packaged in the wheel # due to "packages" filter on py_package rule. "//tests/load_from_macro:foo", @@ -66,6 +68,7 @@ py_wheel( version = "0.0.1", deps = [ "//examples/wheel/lib:module_with_data", + "//examples/wheel/lib:module_with_type_annotations", "//examples/wheel/lib:simple_module", ], ) @@ -89,6 +92,7 @@ py_wheel( version = "$(VERSION)", deps = [ "//examples/wheel/lib:module_with_data", + "//examples/wheel/lib:module_with_type_annotations", "//examples/wheel/lib:simple_module", ], ) @@ -108,6 +112,7 @@ py_wheel( version = "0.1.{BUILD_TIMESTAMP}", deps = [ "//examples/wheel/lib:module_with_data", + "//examples/wheel/lib:module_with_type_annotations", "//examples/wheel/lib:simple_module", ], ) @@ -225,6 +230,25 @@ py_wheel( ], ) +# An example of how to change the wheel package root directory using 'add_path_prefix'. +py_wheel( + name = "custom_prefix_package_root", + add_path_prefix = "custom_prefix", + # Package data. We're building "examples_custom_prefix_package_root-0.0.1-py3-none-any.whl" + distribution = "examples_custom_prefix_package_root", + entry_points = { + "console_scripts": ["main = foo.bar:baz"], + }, + python_tag = "py3", + strip_path_prefixes = [ + "examples", + ], + version = "0.0.1", + deps = [ + ":example_pkg", + ], +) + py_wheel( name = "python_requires_in_a_package", distribution = "example_python_requires_in_a_package", @@ -289,6 +313,12 @@ starlark # Example comment """.splitlines(), ) +write_file( + name = "empty_requires_file", + out = "empty_requires.txt", + content = [""], +) + write_file( name = "extra_requires_file", out = "extra_requires.txt", @@ -302,6 +332,17 @@ wheel; python_version == "3.11" or python_version == "3.12" # Example comment """.splitlines(), ) +write_file( + name = "requires_dist_depends_on_extras_file", + out = "requires_dist_depends_on_extras.txt", + content = """\ +# Requirements file +--index-url https://pypi.com + +extra_requires[example]==0.0.1 +""".splitlines(), +) + # py_wheel can use text files to specify their requirements. This # can be convenient for users of `compile_pip_requirements` who have # granular `requirements.in` files per package. This target shows @@ -319,6 +360,15 @@ py_wheel( deps = [":example_pkg"], ) +py_wheel( + name = "empty_requires_files", + distribution = "empty_requires_files", + python_tag = "py3", + requires_file = ":empty_requires.txt", + version = "0.0.1", + deps = [":example_pkg"], +) + # Package just a specific py_libraries, without their dependencies py_wheel( name = "minimal_data_files", @@ -354,6 +404,45 @@ py_wheel( deps = [":example_pkg"], ) +py_wheel( + name = "requires_dist_depends_on_extras", + distribution = "requires_dist_depends_on_extras", + requires = [ + "extra_requires[example]==0.0.1", + ], + version = "0.0.1", +) + +py_wheel( + name = "requires_dist_depends_on_extras_using_file", + distribution = "requires_dist_depends_on_extras_using_file", + requires_file = ":requires_dist_depends_on_extras.txt", + version = "0.0.1", +) + +filegroup( + name = "data_files_test_group", + # Re-using some files already checked into the repo. + srcs = [ + "README.md", + "//examples/wheel:NOTICE", + ], +) + +py_wheel( + name = "data_files_installed_in_folder", + testonly = True, # Set this to verify the generated .dist target doesn't break things + # Re-using some files already checked into the repo. + data_files = { + # Single file + "//examples/wheel:NOTICE": "scripts/", + # Filegroup + ":data_files_test_group": "data/", + }, + distribution = "data_files_installed_in_folder", + version = "0.0.1", +) + py_test( name = "wheel_test", srcs = ["wheel_test.py"], @@ -362,6 +451,8 @@ py_test( ":custom_package_root_multi_prefix", ":custom_package_root_multi_prefix_reverse_order", ":customized", + ":data_files_installed_in_folder", + ":empty_requires_files", ":extra_requires", ":filename_escaping", ":minimal_data_files", @@ -370,6 +461,8 @@ py_test( ":minimal_with_py_package", ":python_abi3_binary_wheel", ":python_requires_in_a_package", + ":requires_dist_depends_on_extras", + ":requires_dist_depends_on_extras_using_file", ":requires_files", ":use_rule_with_dir_in_outs", ], diff --git a/examples/wheel/lib/BUILD.bazel b/examples/wheel/lib/BUILD.bazel index 755818daa1..7fcd8572cf 100644 --- a/examples/wheel/lib/BUILD.bazel +++ b/examples/wheel/lib/BUILD.bazel @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -load("//python:defs.bzl", "py_library") +load("//python:py_library.bzl", "py_library") package(default_visibility = ["//visibility:public"]) @@ -23,6 +23,12 @@ py_library( srcs = ["simple_module.py"], ) +py_library( + name = "module_with_type_annotations", + srcs = ["module_with_type_annotations.py"], + pyi_srcs = ["module_with_type_annotations.pyi"], +) + py_library( name = "module_with_data", srcs = ["module_with_data.py"], diff --git a/examples/wheel/lib/module_with_type_annotations.py b/examples/wheel/lib/module_with_type_annotations.py new file mode 100644 index 0000000000..eda57bae6a --- /dev/null +++ b/examples/wheel/lib/module_with_type_annotations.py @@ -0,0 +1,17 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +def function(): + return "qux" diff --git a/tests/integration/ignore_root_user_error/foo_test.py b/examples/wheel/lib/module_with_type_annotations.pyi similarity index 86% rename from tests/integration/ignore_root_user_error/foo_test.py rename to examples/wheel/lib/module_with_type_annotations.pyi index 724cdcb69a..b250cd01cf 100644 --- a/tests/integration/ignore_root_user_error/foo_test.py +++ b/examples/wheel/lib/module_with_type_annotations.pyi @@ -1,4 +1,4 @@ -# Copyright 2019 The Bazel Authors. All rights reserved. +# Copyright 2025 The Bazel Authors. All rights reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -11,3 +11,5 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. + +def function() -> str: ... diff --git a/examples/wheel/main.py b/examples/wheel/main.py index 7c4d323e87..37b4f69811 100644 --- a/examples/wheel/main.py +++ b/examples/wheel/main.py @@ -13,6 +13,7 @@ # limitations under the License. import examples.wheel.lib.module_with_data as module_with_data +import examples.wheel.lib.module_with_type_annotations as module_with_type_annotations import examples.wheel.lib.simple_module as simple_module @@ -23,6 +24,7 @@ def function(): def main(): print(function()) print(module_with_data.function()) + print(module_with_type_annotations.function()) print(simple_module.function()) diff --git a/examples/wheel/private/BUILD.bazel b/examples/wheel/private/BUILD.bazel index 3462d354d4..326fc3538c 100644 --- a/examples/wheel/private/BUILD.bazel +++ b/examples/wheel/private/BUILD.bazel @@ -1,4 +1,4 @@ -load("@rules_python//python:defs.bzl", "py_binary") +load("@rules_python//python:py_binary.bzl", "py_binary") py_binary( name = "directory_writer", diff --git a/examples/wheel/private/wheel_utils.bzl b/examples/wheel/private/wheel_utils.bzl index 037fed0175..4cd776a98a 100644 --- a/examples/wheel/private/wheel_utils.bzl +++ b/examples/wheel/private/wheel_utils.bzl @@ -28,6 +28,7 @@ def _directory_writer_impl(ctx): ctx.actions.run( outputs = [output], + mnemonic = "PyDirWriter", arguments = [args], executable = ctx.executable._writer, ) diff --git a/examples/wheel/requirements_server.in b/examples/wheel/requirements_server.in index d5d483d56a..631bdb8fb5 100644 --- a/examples/wheel/requirements_server.in +++ b/examples/wheel/requirements_server.in @@ -1,2 +1,2 @@ # This is for running publishing tests -pypiserver +pypiserver>=2.2.0 diff --git a/examples/wheel/requirements_server.txt b/examples/wheel/requirements_server.txt index eccab1271b..7324131a7c 100644 --- a/examples/wheel/requirements_server.txt +++ b/examples/wheel/requirements_server.txt @@ -4,9 +4,17 @@ # # bazel run //examples/wheel:requirements_server.update # -pypiserver==2.0.1 \ - --hash=sha256:1dd98fb99d2da4199fb44c7284e57d69a9f7fda2c6c8dc01975c151c592677bf \ - --hash=sha256:7b58fbd54468235f79e4de07c4f7a9ff829e7ac6869bef47ec11e0710138e162 +importlib-resources==7.1.0 \ + --hash=sha256:0722d4c6212489c530f2a145a34c0a7a3b4721bc96a15fada5930e2a0b760708 \ + --hash=sha256:1bd7b48b4088eddb2cd16382150bb515af0bd2c70128194392725f82ad2c96a1 + # via pypiserver +packaging==26.1 \ + --hash=sha256:5d9c0669c6285e491e0ced2eee587eaf67b670d94a19e94e3984a481aba6802f \ + --hash=sha256:f042152b681c4bfac5cae2742a55e103d27ab2ec0f3d88037136b6bfe7c9c5de + # via pypiserver +pypiserver==2.4.1 \ + --hash=sha256:156540f87ecfd6db06ae2c16e25ae5afe4fda6f510bd1c34e46fbb0c491bcd9e \ + --hash=sha256:45f116d0bff6aafcaed002cfad48a6832e62a82393e3a9b447d5c41a0e310fff # via -r examples/wheel/requirements_server.in # The following packages are considered to be unsafe in a requirements file: diff --git a/examples/wheel/test_publish.py b/examples/wheel/test_publish.py index 496642acb7..4bc657c52e 100644 --- a/examples/wheel/test_publish.py +++ b/examples/wheel/test_publish.py @@ -6,6 +6,7 @@ import unittest from contextlib import closing from pathlib import Path +from urllib.error import URLError from urllib.request import urlopen @@ -50,17 +51,16 @@ def setUp(self): ], ) - line = "Hit Ctrl-C to quit" interval = 0.1 wait_seconds = 40 for _ in range(int(wait_seconds / interval)): # 40 second timeout - current_logs = self.log_file.read_text() - if line in current_logs: - print(current_logs.strip()) - print("...") - break - - time.sleep(0.1) + try: + with urlopen(self.url, timeout=1) as response: + if response.status == 200: + break + except (URLError, OSError): + pass + time.sleep(interval) else: raise RuntimeError( f"Could not get the server running fast enough, waited for {wait_seconds}s" @@ -98,13 +98,15 @@ def test_upload_and_query_simple_api(self): got_content = response.read().decode("utf-8") want_content = """ - + + + Links for example-minimal-library

Links for example-minimal-library

- example_minimal_library-0.0.1-py3-none-any.whl
+ example_minimal_library-0.0.1-py3-none-any.whl
""" self.assertEqual( diff --git a/examples/wheel/wheel_test.py b/examples/wheel/wheel_test.py index 4494ee170d..8dcad42138 100644 --- a/examples/wheel/wheel_test.py +++ b/examples/wheel/wheel_test.py @@ -76,6 +76,8 @@ def test_py_library_wheel(self): zf.namelist(), [ "examples/wheel/lib/module_with_data.py", + "examples/wheel/lib/module_with_type_annotations.py", + "examples/wheel/lib/module_with_type_annotations.pyi", "examples/wheel/lib/simple_module.py", "example_minimal_library-0.0.1.dist-info/WHEEL", "example_minimal_library-0.0.1.dist-info/METADATA", @@ -83,7 +85,7 @@ def test_py_library_wheel(self): ], ) self.assertFileSha256Equal( - filename, "79a4e9c1838c0631d5d8fa49a26efd6e9a364f6b38d9597c0f6df112271a0e28" + filename, "ef5afd9f6c3ff569ef7e5b2799d3a2ec9675d029414f341e0abd7254d6b9a25d" ) def test_py_package_wheel(self): @@ -98,6 +100,8 @@ def test_py_package_wheel(self): "examples/wheel/lib/data,with,commas.txt", "examples/wheel/lib/data.txt", "examples/wheel/lib/module_with_data.py", + "examples/wheel/lib/module_with_type_annotations.py", + "examples/wheel/lib/module_with_type_annotations.pyi", "examples/wheel/lib/simple_module.py", "examples/wheel/main.py", "example_minimal_package-0.0.1.dist-info/WHEEL", @@ -106,7 +110,7 @@ def test_py_package_wheel(self): ], ) self.assertFileSha256Equal( - filename, "82370bf61310e2d3c7b1218368457dc7e161bf5dc1a280d7d45102b5e56acf43" + filename, "39bec133cf79431e8d057eae550cd91aa9dfbddfedb53d98ebd36e3ade2753d0" ) def test_customized_wheel(self): @@ -121,6 +125,8 @@ def test_customized_wheel(self): "examples/wheel/lib/data,with,commas.txt", "examples/wheel/lib/data.txt", "examples/wheel/lib/module_with_data.py", + "examples/wheel/lib/module_with_type_annotations.py", + "examples/wheel/lib/module_with_type_annotations.pyi", "examples/wheel/lib/simple_module.py", "examples/wheel/main.py", "example_customized-0.0.1.dist-info/WHEEL", @@ -138,6 +144,7 @@ def test_customized_wheel(self): "example_customized-0.0.1.dist-info/entry_points.txt" ) + print(record_contents) self.assertEqual( record_contents, # The entries are guaranteed to be sorted. @@ -145,8 +152,10 @@ def test_customized_wheel(self): "examples/wheel/lib/data,with,commas.txt",sha256=9vJKEdfLu8bZRArKLroPZJh1XKkK3qFMXiM79MBL2Sg,12 examples/wheel/lib/data.txt,sha256=9vJKEdfLu8bZRArKLroPZJh1XKkK3qFMXiM79MBL2Sg,12 examples/wheel/lib/module_with_data.py,sha256=8s0Khhcqz3yVsBKv2IB5u4l4TMKh7-c_V6p65WVHPms,637 +examples/wheel/lib/module_with_type_annotations.py,sha256=2p_0YFT0TBUufbGCAR_u2vtxF1nM0lf3dX4VGeUtYq0,637 +examples/wheel/lib/module_with_type_annotations.pyi,sha256=fja3ql_WRJ1qO8jyZjWWrTTMcg1J7EpOQivOHY_8vI4,630 examples/wheel/lib/simple_module.py,sha256=z2hwciab_XPNIBNH8B1Q5fYgnJvQTeYf0ZQJpY8yLLY,637 -examples/wheel/main.py,sha256=sgg5iWN_9inYBjm6_Zw27hYdmo-l24fA-2rfphT-IlY,909 +examples/wheel/main.py,sha256=mFiRfzQEDwCHr-WVNQhOH26M42bw1UMF6IoqvtuDTrw,1047 example_customized-0.0.1.dist-info/WHEEL,sha256=sobxWSyDDkdg_rinUth-jxhXHqoNqlmNMJY3aTZn2Us,91 example_customized-0.0.1.dist-info/METADATA,sha256=QYQcDJFQSIqan8eiXqL67bqsUfgEAwf2hoK_Lgi1S-0,559 example_customized-0.0.1.dist-info/entry_points.txt,sha256=pqzpbQ8MMorrJ3Jp0ntmpZcuvfByyqzMXXi2UujuXD0,137 @@ -197,7 +206,7 @@ def test_customized_wheel(self): second = second.main:s""", ) self.assertFileSha256Equal( - filename, "706e8dd45884d8cb26e92869f7d29ab7ed9f683b4e2d08f06c03dbdaa12191b8" + filename, "685f68fc6665f53c9b769fd1ba12cce9937ab7f40ef4e60c82ef2de8653935de" ) def test_filename_escaping(self): @@ -211,6 +220,8 @@ def test_filename_escaping(self): "examples/wheel/lib/data,with,commas.txt", "examples/wheel/lib/data.txt", "examples/wheel/lib/module_with_data.py", + "examples/wheel/lib/module_with_type_annotations.py", + "examples/wheel/lib/module_with_type_annotations.pyi", "examples/wheel/lib/simple_module.py", "examples/wheel/main.py", # PEP calls for replacing only in the archive filename. @@ -248,6 +259,8 @@ def test_custom_package_root_wheel(self): "wheel/lib/data,with,commas.txt", "wheel/lib/data.txt", "wheel/lib/module_with_data.py", + "wheel/lib/module_with_type_annotations.py", + "wheel/lib/module_with_type_annotations.pyi", "wheel/lib/simple_module.py", "wheel/main.py", "examples_custom_package_root-0.0.1.dist-info/WHEEL", @@ -265,7 +278,7 @@ def test_custom_package_root_wheel(self): for line in record_contents.splitlines(): self.assertFalse(line.startswith("/")) self.assertFileSha256Equal( - filename, "568922541703f6edf4b090a8413991f9fa625df2844e644dd30bdbe9deb660be" + filename, "2fbfc3baaf6fccca0f97d02316b8344507fe6c8136991a66ee5f162235adb19f" ) def test_custom_package_root_multi_prefix_wheel(self): @@ -281,6 +294,8 @@ def test_custom_package_root_multi_prefix_wheel(self): "data,with,commas.txt", "data.txt", "module_with_data.py", + "module_with_type_annotations.py", + "module_with_type_annotations.pyi", "simple_module.py", "main.py", "example_custom_package_root_multi_prefix-0.0.1.dist-info/WHEEL", @@ -297,7 +312,7 @@ def test_custom_package_root_multi_prefix_wheel(self): for line in record_contents.splitlines(): self.assertFalse(line.startswith("/")) self.assertFileSha256Equal( - filename, "a8b91ce9d6f570e97b40a357a292a6f595d3470f07c479cb08550257cc9c8306" + filename, "3e67971ca1e8a9ba36a143df7532e641f5661c56235e41d818309316c955ba58" ) def test_custom_package_root_multi_prefix_reverse_order_wheel(self): @@ -313,6 +328,8 @@ def test_custom_package_root_multi_prefix_reverse_order_wheel(self): "lib/data,with,commas.txt", "lib/data.txt", "lib/module_with_data.py", + "lib/module_with_type_annotations.py", + "lib/module_with_type_annotations.pyi", "lib/simple_module.py", "main.py", "example_custom_package_root_multi_prefix_reverse_order-0.0.1.dist-info/WHEEL", @@ -329,7 +346,7 @@ def test_custom_package_root_multi_prefix_reverse_order_wheel(self): for line in record_contents.splitlines(): self.assertFalse(line.startswith("/")) self.assertFileSha256Equal( - filename, "8f44e940731757c186079a42cfe7ea3d43cd96b526e3fb2ca2a3ea3048a9d489" + filename, "372ef9e11fb79f1952172993718a326b5adda192d94884b54377c34b44394982" ) def test_python_requires_wheel(self): @@ -354,7 +371,7 @@ def test_python_requires_wheel(self): """, ) self.assertFileSha256Equal( - filename, "ba32493f5e43e481346384aaab9e8fa09c23884276ad057c5f432096a0350101" + filename, "10a325ba8f77428b5cfcff6345d508f5eb77c140889eb62490d7382f60d4ebfe" ) def test_python_abi3_binary_wheel(self): @@ -419,7 +436,7 @@ def test_rule_creates_directory_and_is_included_in_wheel(self): ], ) self.assertFileSha256Equal( - filename, "ac9216bd54dcae1a6270c35fccf8a73b0be87c1b026c28e963b7c76b2f9b722b" + filename, "85e44c43cc19ccae9fe2e1d629230203aa11791bed1f7f68a069fb58d1c93cd2" ) def test_rule_expands_workspace_status_keys_in_wheel_metadata(self): @@ -467,7 +484,6 @@ def test_requires_file_and_extra_requires_files(self): if line.startswith(b"Requires-Dist:"): requires.append(line.decode("utf-8").strip()) - print(requires) self.assertEqual( [ "Requires-Dist: tomli>=2.0.0", @@ -479,12 +495,35 @@ def test_requires_file_and_extra_requires_files(self): requires, ) + def test_empty_requires_file(self): + filename = self._get_path("empty_requires_files-0.0.1-py3-none-any.whl") + + with zipfile.ZipFile(filename) as zf: + self.assertAllEntriesHasReproducibleMetadata(zf) + metadata_file = None + for f in zf.namelist(): + if os.path.basename(f) == "METADATA": + metadata_file = f + self.assertIsNotNone(metadata_file) + + metadata = zf.read(metadata_file).decode("utf-8") + metadata_lines = metadata.splitlines() + + requires = [] + for i, line in enumerate(metadata_lines): + if line.startswith("Name:"): + self.assertTrue(metadata_lines[i + 1].startswith("Version:")) + if line.startswith("Requires-Dist:"): + requires.append(line.strip()) + + self.assertEqual([], requires) + def test_minimal_data_files(self): filename = self._get_path("minimal_data_files-0.0.1-py3-none-any.whl") with zipfile.ZipFile(filename) as zf: self.assertAllEntriesHasReproducibleMetadata(zf) - metadata_file = None + metadata_file = None # noqa: F841 self.assertEqual( zf.namelist(), [ @@ -526,6 +565,79 @@ def test_extra_requires(self): requires, ) + def test_requires_dist_depends_on_extras(self): + filename = self._get_path( + "requires_dist_depends_on_extras-0.0.1-py3-none-any.whl" + ) + + with zipfile.ZipFile(filename) as zf: + self.assertAllEntriesHasReproducibleMetadata(zf) + metadata_file = None + for f in zf.namelist(): + if os.path.basename(f) == "METADATA": + metadata_file = f + self.assertIsNotNone(metadata_file) + + requires = [] + with zf.open(metadata_file) as fp: + for line in fp: + if line.startswith(b"Requires-Dist:"): + requires.append(line.decode("utf-8").strip()) + + print(requires) + self.assertEqual( + [ + "Requires-Dist: extra_requires[example]==0.0.1", + ], + requires, + ) + + def test_requires_dist_depends_on_extras_file(self): + filename = self._get_path( + "requires_dist_depends_on_extras_using_file-0.0.1-py3-none-any.whl" + ) + + with zipfile.ZipFile(filename) as zf: + self.assertAllEntriesHasReproducibleMetadata(zf) + metadata_file = None + for f in zf.namelist(): + if os.path.basename(f) == "METADATA": + metadata_file = f + self.assertIsNotNone(metadata_file) + + requires = [] + with zf.open(metadata_file) as fp: + for line in fp: + if line.startswith(b"Requires-Dist:"): + requires.append(line.decode("utf-8").strip()) + + print(requires) + self.assertEqual( + [ + "Requires-Dist: extra_requires[example]==0.0.1", + ], + requires, + ) + + def test_data_files_installed_in_folder(self): + filename = self._get_path( + "data_files_installed_in_folder-0.0.1-py3-none-any.whl" + ) + + with zipfile.ZipFile(filename) as zf: + self.assertAllEntriesHasReproducibleMetadata(zf) + self.assertEqual( + zf.namelist(), + [ + "data_files_installed_in_folder-0.0.1.dist-info/WHEEL", + "data_files_installed_in_folder-0.0.1.dist-info/METADATA", + "data_files_installed_in_folder-0.0.1.data/data/NOTICE", + "data_files_installed_in_folder-0.0.1.data/data/README.md", + "data_files_installed_in_folder-0.0.1.data/scripts/NOTICE", + "data_files_installed_in_folder-0.0.1.dist-info/RECORD", + ], + ) + if __name__ == "__main__": unittest.main() diff --git a/gazelle/.bazelignore b/gazelle/.bazelignore new file mode 100644 index 0000000000..5930a06190 --- /dev/null +++ b/gazelle/.bazelignore @@ -0,0 +1,8 @@ +bazel-bin +bazel-gazelle +bazel-out +bazel-testlogs +examples/bzlmod_build_file_generation/bazel-bin +examples/bzlmod_build_file_generation/bazel-bzlmod_build_file_generation +examples/bzlmod_build_file_generation/bazel-out +examples/bzlmod_build_file_generation/bazel-testlog diff --git a/gazelle/.bazelrc b/gazelle/.bazelrc index 97040903a6..e30216814a 100644 --- a/gazelle/.bazelrc +++ b/gazelle/.bazelrc @@ -1,3 +1,10 @@ +common --deleted_packages=examples/bzlmod_build_file_generation +common --deleted_packages=examples/bzlmod_build_file_generation/runfiles +common --experimental_downloader_config=downloader_config.cfg +common --http_timeout_scaling=10.0 +common --experimental_repository_downloader_retries=10 + + test --test_output=errors # Do NOT implicitly create empty __init__.py files in the runfiles tree. @@ -7,6 +14,7 @@ test --test_output=errors # creating (possibly empty) __init__.py files and adding them to the srcs of # Python targets as required. build --incompatible_default_to_explicit_init_py +build --@rules_python//python/config_settings:incompatible_default_to_explicit_init_py=True # Windows makes use of runfiles for some rules build --enable_runfiles diff --git a/gazelle/BUILD.bazel b/gazelle/BUILD.bazel index f74338d4b5..30758181ed 100644 --- a/gazelle/BUILD.bazel +++ b/gazelle/BUILD.bazel @@ -1,9 +1,12 @@ load("@bazel_gazelle//:def.bzl", "gazelle") +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") # Gazelle configuration options. # See https://github.com/bazelbuild/bazel-gazelle#running-gazelle-with-bazel -# gazelle:prefix github.com/bazelbuild/rules_python/gazelle +# gazelle:prefix github.com/bazel-contrib/rules_python/gazelle # gazelle:exclude bazel-out +# gazelle:exclude deps.bzl +# gazelle:exclude internal_dev_deps.bzl gazelle( name = "gazelle", ) @@ -36,3 +39,9 @@ filegroup( ], visibility = ["@rules_python//:__pkg__"], ) + +bzl_library( + name = "def", + srcs = ["def.bzl"], + visibility = ["//visibility:public"], +) diff --git a/gazelle/MODULE.bazel b/gazelle/MODULE.bazel index d216ad5dc1..b6eed23c6f 100644 --- a/gazelle/MODULE.bazel +++ b/gazelle/MODULE.bazel @@ -4,10 +4,10 @@ module( compatibility_level = 1, ) -bazel_dep(name = "bazel_skylib", version = "1.6.1") -bazel_dep(name = "rules_python", version = "0.18.0") -bazel_dep(name = "rules_go", version = "0.41.0", repo_name = "io_bazel_rules_go") -bazel_dep(name = "gazelle", version = "0.33.0", repo_name = "bazel_gazelle") +bazel_dep(name = "bazel_skylib", version = "1.8.2") +bazel_dep(name = "rules_python", version = "1.5.0") +bazel_dep(name = "rules_go", version = "0.59.0", repo_name = "io_bazel_rules_go") +bazel_dep(name = "gazelle", version = "0.47.0", repo_name = "bazel_gazelle") bazel_dep(name = "rules_cc", version = "0.0.16") local_path_override( @@ -34,3 +34,14 @@ use_repo( python_stdlib_list, "python_stdlib_list", ) + +internal_dev_deps = use_extension( + "//:internal_dev_deps.bzl", + "internal_dev_deps_extension", + dev_dependency = True, +) +use_repo( + internal_dev_deps, + "django-types", + "pytest", +) diff --git a/gazelle/README.md b/gazelle/README.md index c0494d141b..1cbef2d856 100644 --- a/gazelle/README.md +++ b/gazelle/README.md @@ -1,659 +1,6 @@ # Python Gazelle plugin -[Gazelle](https://github.com/bazelbuild/bazel-gazelle) -is a build file generator for Bazel projects. It can create new BUILD.bazel files for a project that follows language conventions, and it can update existing build files to include new sources, dependencies, and options. - -Gazelle may be run by Bazel using the gazelle rule, or it may be installed and run as a command line tool. - -This directory contains a plugin for -[Gazelle](https://github.com/bazelbuild/bazel-gazelle) -that generates BUILD files content for Python code. When Gazelle is run as a command line tool with this plugin, it embeds a Python interpreter resolved during the plugin build. -The behavior of the plugin is slightly different with different version of the interpreter as the Python `stdlib` changes with every minor version release. -Distributors of Gazelle binaries should, therefore, build a Gazelle binary for each OS+CPU architecture+Minor Python version combination they are targeting. - -The following instructions are for when you use [bzlmod](https://docs.bazel.build/versions/5.0.0/bzlmod.html). -Please refer to older documentation that includes instructions on how to use Gazelle -without using bzlmod as your dependency manager. - -## Example - -We have an example of using Gazelle with Python located [here](https://github.com/bazelbuild/rules_python/tree/main/examples/bzlmod). -A fully-working example without using bzlmod is in [`examples/build_file_generation`](../examples/build_file_generation). - -The following documentation covers using bzlmod. - -## Adding Gazelle to your project - -First, you'll need to add Gazelle to your `MODULES.bazel` file. -Get the current version of Gazelle from there releases here: https://github.com/bazelbuild/bazel-gazelle/releases/. - - -See the installation `MODULE.bazel` snippet on the Releases page: -https://github.com/bazelbuild/rules_python/releases in order to configure rules_python. - -You will also need to add the `bazel_dep` for configuration for `rules_python_gazelle_plugin`. - -Here is a snippet of a `MODULE.bazel` file. - -```starlark -# The following stanza defines the dependency rules_python. -bazel_dep(name = "rules_python", version = "0.22.0") - -# The following stanza defines the dependency rules_python_gazelle_plugin. -# For typical setups you set the version. -bazel_dep(name = "rules_python_gazelle_plugin", version = "0.22.0") - -# The following stanza defines the dependency gazelle. -bazel_dep(name = "gazelle", version = "0.31.0", repo_name = "bazel_gazelle") - -# Import the python repositories generated by the given module extension into the scope of the current module. -use_repo(python, "python3_9") -use_repo(python, "python3_9_toolchains") - -# Register an already-defined toolchain so that Bazel can use it during toolchain resolution. -register_toolchains( - "@python3_9_toolchains//:all", -) - -# Use the pip extension -pip = use_extension("@rules_python//python:extensions.bzl", "pip") - -# Use the extension to call the `pip_repository` rule that invokes `pip`, with `incremental` set. -# Accepts a locked/compiled requirements file and installs the dependencies listed within. -# Those dependencies become available in a generated `requirements.bzl` file. -# You can instead check this `requirements.bzl` file into your repo. -# Because this project has different requirements for windows vs other -# operating systems, we have requirements for each. -pip.parse( - name = "pip", - requirements_lock = "//:requirements_lock.txt", - requirements_windows = "//:requirements_windows.txt", -) - -# Imports the pip toolchain generated by the given module extension into the scope of the current module. -use_repo(pip, "pip") -``` -Next, we'll fetch metadata about your Python dependencies, so that gazelle can -determine which package a given import statement comes from. This is provided -by the `modules_mapping` rule. We'll make a target for consuming this -`modules_mapping`, and writing it as a manifest file for Gazelle to read. -This is checked into the repo for speed, as it takes some time to calculate -in a large monorepo. - -Gazelle will walk up the filesystem from a Python file to find this metadata, -looking for a file called `gazelle_python.yaml` in an ancestor folder of the Python code. -Create an empty file with this name. It might be next to your `requirements.txt` file. -(You can just use `touch` at this point, it just needs to exist.) - -To keep the metadata updated, put this in your `BUILD.bazel` file next to `gazelle_python.yaml`: - -```starlark -load("@pip//:requirements.bzl", "all_whl_requirements") -load("@rules_python_gazelle_plugin//manifest:defs.bzl", "gazelle_python_manifest") -load("@rules_python_gazelle_plugin//modules_mapping:def.bzl", "modules_mapping") - -# This rule fetches the metadata for python packages we depend on. That data is -# required for the gazelle_python_manifest rule to update our manifest file. -modules_mapping( - name = "modules_map", - wheels = all_whl_requirements, -) - -# Gazelle python extension needs a manifest file mapping from -# an import to the installed package that provides it. -# This macro produces two targets: -# - //:gazelle_python_manifest.update can be used with `bazel run` -# to recalculate the manifest -# - //:gazelle_python_manifest.test is a test target ensuring that -# the manifest doesn't need to be updated -gazelle_python_manifest( - name = "gazelle_python_manifest", - modules_mapping = ":modules_map", - # This is what we called our `pip_parse` rule, where third-party - # python libraries are loaded in BUILD files. - pip_repository_name = "pip", - # This should point to wherever we declare our python dependencies - # (the same as what we passed to the modules_mapping rule in WORKSPACE) - # This argument is optional. If provided, the `.test` target is very - # fast because it just has to check an integrity field. If not provided, - # the integrity field is not added to the manifest which can help avoid - # merge conflicts in large repos. - requirements = "//:requirements_lock.txt", -) -``` - -Finally, you create a target that you'll invoke to run the Gazelle tool -with the rules_python extension included. This typically goes in your root -`/BUILD.bazel` file: - -```starlark -load("@bazel_gazelle//:def.bzl", "gazelle") - -# Our gazelle target points to the python gazelle binary. -# This is the simple case where we only need one language supported. -# If you also had proto, go, or other gazelle-supported languages, -# you would also need a gazelle_binary rule. -# See https://github.com/bazelbuild/bazel-gazelle/blob/master/extend.rst#example -gazelle( - name = "gazelle", - gazelle = "@rules_python_gazelle_plugin//python:gazelle_binary", -) -``` - -That's it, now you can finally run `bazel run //:gazelle` anytime -you edit Python code, and it should update your `BUILD` files correctly. - -## Usage - -Gazelle is non-destructive. -It will try to leave your edits to BUILD files alone, only making updates to `py_*` targets. -However it will remove dependencies that appear to be unused, so it's a -good idea to check in your work before running Gazelle so you can easily -revert any changes it made. - -The rules_python extension assumes some conventions about your Python code. -These are noted below, and might require changes to your existing code. - -Note that the `gazelle` program has multiple commands. At present, only the `update` command (the default) does anything for Python code. - -### Directives - -You can configure the extension using directives, just like for other -languages. These are just comments in the `BUILD.bazel` file which -govern behavior of the extension when processing files under that -folder. - -See https://github.com/bazelbuild/bazel-gazelle#directives -for some general directives that may be useful. -In particular, the `resolve` directive is language-specific -and can be used with Python. -Examples of these directives in use can be found in the -/gazelle/testdata folder in the rules_python repo. - -Python-specific directives are as follows: - -| **Directive** | **Default value** | -|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------| -| `# gazelle:python_extension` | `enabled` | -| Controls whether the Python extension is enabled or not. Sub-packages inherit this value. Can be either "enabled" or "disabled". | | -| [`# gazelle:python_root`](#directive-python_root) | n/a | -| Sets a Bazel package as a Python root. This is used on monorepos with multiple Python projects that don't share the top-level of the workspace as the root. See [Directive: `python_root`](#directive-python_root) below. | | -| `# gazelle:python_manifest_file_name` | `gazelle_python.yaml` | -| Overrides the default manifest file name. | | -| `# gazelle:python_ignore_files` | n/a | -| Controls the files which are ignored from the generated targets. | | -| `# gazelle:python_ignore_dependencies` | n/a | -| Controls the ignored dependencies from the generated targets. | | -| `# gazelle:python_validate_import_statements` | `true` | -| Controls whether the Python import statements should be validated. Can be "true" or "false" | | -| `# gazelle:python_generation_mode` | `package` | -| Controls the target generation mode. Can be "file", "package", or "project" | | -| `# gazelle:python_generation_mode_per_file_include_init` | `false` | -| Controls whether `__init__.py` files are included as srcs in each generated target when target generation mode is "file". Can be "true", or "false" | | -| [`# gazelle:python_generation_mode_per_package_require_test_entry_point`](#directive-python_generation_mode_per_package_require_test_entry_point) | `true` | -| Controls whether a file called `__test__.py` or a target called `__test__` is required to generate one test target per package in package mode. || -| `# gazelle:python_library_naming_convention` | `$package_name$` | -| Controls the `py_library` naming convention. It interpolates `$package_name$` with the Bazel package name. E.g. if the Bazel package name is `foo`, setting this to `$package_name$_my_lib` would result in a generated target named `foo_my_lib`. | | -| `# gazelle:python_binary_naming_convention` | `$package_name$_bin` | -| Controls the `py_binary` naming convention. Follows the same interpolation rules as `python_library_naming_convention`. | | -| `# gazelle:python_test_naming_convention` | `$package_name$_test` | -| Controls the `py_test` naming convention. Follows the same interpolation rules as `python_library_naming_convention`. | | -| `# gazelle:resolve py ...` | n/a | -| Instructs the plugin what target to add as a dependency to satisfy a given import statement. The syntax is `# gazelle:resolve py import-string label` where `import-string` is the symbol in the python `import` statement, and `label` is the Bazel label that Gazelle should write in `deps`. | | -| [`# gazelle:python_default_visibility labels`](#directive-python_default_visibility) | | -| Instructs gazelle to use these visibility labels on all python targets. `labels` is a comma-separated list of labels (without spaces). | `//$python_root$:__subpackages__` | -| [`# gazelle:python_visibility label`](#directive-python_visibility) | | -| Appends additional visibility labels to each generated target. This directive can be set multiple times. | | -| [`# gazelle:python_test_file_pattern`](#directive-python_test_file_pattern) | `*_test.py,test_*.py` | -| Filenames matching these comma-separated `glob`s will be mapped to `py_test` targets. | -| `# gazelle:python_label_convention` | `$distribution_name$` | -| Defines the format of the distribution name in labels to third-party deps. Useful for using Gazelle plugin with other rules with different repository conventions (e.g. `rules_pycross`). Full label is always prepended with (pip) repository name, e.g. `@pip//numpy`. | -| `# gazelle:python_label_normalization` | `snake_case` | -| Controls how distribution names in labels to third-party deps are normalized. Useful for using Gazelle plugin with other rules with different label conventions (e.g. `rules_pycross` uses PEP-503). Can be "snake_case", "none", or "pep503". | - -#### Directive: `python_root`: - -Set this directive within the Bazel package that you want to use as the Python root. -For example, if using a `src` dir (as recommended by the [Python Packaging User -Guide][python-packaging-user-guide]), then set this directive in `src/BUILD.bazel`: - -```starlark -# ./src/BUILD.bazel -# Tell gazelle that are python root is the same dir as this Bazel package. -# gazelle:python_root -``` - -Note that the directive does not have any arguments. - -Gazelle will then add the necessary `imports` attribute to all targets that it -generates: - -```starlark -# in ./src/foo/BUILD.bazel -py_libary( - ... - imports = [".."], # Gazelle adds this - ... -) - -# in ./src/foo/bar/BUILD.bazel -py_libary( - ... - imports = ["../.."], # Gazelle adds this - ... -) -``` - -[python-packaging-user-guide]: https://github.com/pypa/packaging.python.org/blob/4c86169a/source/tutorials/packaging-projects.rst - - -#### Directive: `python_default_visibility`: - -Instructs gazelle to use these visibility labels on all _python_ targets -(typically `py_*`, but can be modified via the `map_kind` directive). The arg -to this directive is a a comma-separated list (without spaces) of labels. - -For example: - -```starlark -# gazelle:python_default_visibility //:__subpackages__,//tests:__subpackages__ -``` - -produces the following visibility attribute: - -```starlark -py_library( - ..., - visibility = [ - "//:__subpackages__", - "//tests:__subpackages__", - ], - ..., -) -``` - -You can also inject the `python_root` value by using the exact string -`$python_root$`. All instances of this string will be replaced by the `python_root` -value. - -```starlark -# gazelle:python_default_visibility //$python_root$:__pkg__,//foo/$python_root$/tests:__subpackages__ - -# Assuming the "# gazelle:python_root" directive is set in ./py/src/BUILD.bazel, -# the results will be: -py_library( - ..., - visibility = [ - "//foo/py/src/tests:__subpackages__", # sorted alphabetically - "//py/src:__pkg__", - ], - ..., -) -``` - -Two special values are also accepted as an argument to the directive: - -+ `NONE`: This removes all default visibility. Labels added by the - `python_visibility` directive are still included. -+ `DEFAULT`: This resets the default visibility. - -For example: - -```starlark -# gazelle:python_default_visibility NONE - -py_library( - name = "...", - srcs = [...], -) -``` - -```starlark -# gazelle:python_default_visibility //foo:bar -# gazelle:python_default_visibility DEFAULT - -py_library( - ..., - visibility = ["//:__subpackages__"], - ..., -) -``` - -These special values can be useful for sub-packages. - - -#### Directive: `python_visibility`: - -Appends additional `visibility` labels to each generated target. - -This directive can be set multiple times. The generated `visibility` attribute -will include the default visibility and all labels defined by this directive. -All labels will be ordered alphabetically. - -```starlark -# ./BUILD.bazel -# gazelle:python_visibility //tests:__pkg__ -# gazelle:python_visibility //bar:baz - -py_library( - ... - visibility = [ - "//:__subpackages__", # default visibility - "//bar:baz", - "//tests:__pkg__", - ], - ... -) -``` - -Child Bazel packages inherit values from parents: - -```starlark -# ./bar/BUILD.bazel -# gazelle:python_visibility //tests:__subpackages__ - -py_library( - ... - visibility = [ - "//:__subpackages__", # default visibility - "//bar:baz", # defined in ../BUILD.bazel - "//tests:__pkg__", # defined in ../BUILD.bazel - "//tests:__subpackages__", # defined in this ./BUILD.bazel - ], - ... -) - -``` - -This directive also supports the `$python_root$` placeholder that -`# gazelle:python_default_visibility` supports. - -```starlark -# gazlle:python_visibility //$python_root$/foo:bar - -py_library( - ... - visibility = ["//this_is_my_python_root/foo:bar"], - ... -) -``` - - -#### Directive: `python_test_file_pattern`: - -This directive adjusts which python files will be mapped to the `py_test` rule. - -+ The default is `*_test.py,test_*.py`: both `test_*.py` and `*_test.py` files - will generate `py_test` targets. -+ This directive must have a value. If no value is given, an error will be raised. -+ It is recommended, though not necessary, to include the `.py` extension in - the `glob`s: `foo*.py,?at.py`. -+ Like most directives, it applies to the current Bazel package and all subpackages - until the directive is set again. -+ This directive accepts multiple `glob` patterns, separated by commas without spaces: - -```starlark -# gazelle:python_test_file_pattern foo*.py,?at - -py_library( - name = "mylib", - srcs = ["mylib.py"], -) - -py_test( - name = "foo_bar", - srcs = ["foo_bar.py"], -) - -py_test( - name = "cat", - srcs = ["cat.py"], -) - -py_test( - name = "hat", - srcs = ["hat.py"], -) -``` - - -##### Notes - -Resetting to the default value (such as in a subpackage) is manual. Set: - -```starlark -# gazelle:python_test_file_pattern *_test.py,test_*.py -``` - -There currently is no way to tell gazelle that _no_ files in a package should -be mapped to `py_test` targets (see [Issue #1826][issue-1826]). The workaround -is to set this directive to a pattern that will never match a `.py` file, such -as `foo.bar`: - -```starlark -# No files in this package should be mapped to py_test targets. -# gazelle:python_test_file_pattern foo.bar - -py_library( - name = "my_test", - srcs = ["my_test.py"], -) -``` - -[issue-1826]: https://github.com/bazelbuild/rules_python/issues/1826 - -#### Directive: `python_generation_mode_per_package_require_test_entry_point`: -When `# gazelle:python_generation_mode package`, whether a file called `__test__.py` or a target called `__test__`, a.k.a., entry point, is required to generate one test target per package. If this is set to true but no entry point is found, Gazelle will fall back to file mode and generate one test target per file. Setting this directive to false forces Gazelle to generate one test target per package even without entry point. However, this means the `main` attribute of the `py_test` will not be set and the target will not be runnable unless either: -1. there happen to be a file in the `srcs` with the same name as the `py_test` target, or -2. a macro populating the `main` attribute of `py_test` is configured with `gazelle:map_kind` to replace `py_test` when Gazelle is generating Python test targets. For example, user can provide such a macro to Gazelle: - -```starlark -load("@rules_python//python:defs.bzl", _py_test="py_test") -load("@aspect_rules_py//py:defs.bzl", "py_pytest_main") - -def py_test(name, main=None, **kwargs): - deps = kwargs.pop("deps", []) - if not main: - py_pytest_main( - name = "__test__", - deps = ["@pip_pytest//:pkg"], # change this to the pytest target in your repo. - ) - - deps.append(":__test__") - main = ":__test__.py" - - _py_test( - name = name, - main = main, - deps = deps, - **kwargs, -) -``` - -### Annotations - -*Annotations* refer to comments found _within Python files_ that configure how -Gazelle acts for that particular file. - -Annotations have the form: - -```python -# gazelle:annotation_name value -``` - -and can reside anywhere within a Python file where comments are valid. For example: - -```python -import foo -# gazelle:annotation_name value - -def bar(): # gazelle:annotation_name value - pass -``` - -The annotations are: - -| **Annotation** | **Default value** | -|---------------------------------------------------------------|-------------------| -| [`# gazelle:ignore imports`](#annotation-ignore) | N/A | -| Tells Gazelle to ignore import statements. `imports` is a comma-separated list of imports to ignore. | | -| [`# gazelle:include_dep targets`](#annotation-include_dep) | N/A | -| Tells Gazelle to include a set of dependencies, even if they are not imported in a Python module. `targets` is a comma-separated list of target names to include as dependencies. | | - - -#### Annotation: `ignore` - -This annotation accepts a comma-separated string of values. Values are names of Python -imports that Gazelle should _not_ include in target dependencies. - -The annotation can be added multiple times, and all values are combined and -de-duplicated. - -For `python_generation_mode = "package"`, the `ignore` annotations -found across all files included in the generated target are removed from `deps`. - -Example: - -```python -import numpy # a pypi package - -# gazelle:ignore bar.baz.hello,foo -import bar.baz.hello -import foo - -# Ignore this import because _reasons_ -import baz # gazelle:ignore baz -``` - -will cause Gazelle to generate: - -```starlark -deps = ["@pypi//numpy"], -``` - - -#### Annotation: `include_dep` - -This annotation accepts a comma-separated string of values. Values _must_ -be Python targets, but _no validation is done_. If a value is not a Python -target, building will result in an error saying: - -``` - does not have mandatory providers: 'PyInfo' or 'CcInfo' or 'PyInfo'. -``` - -Adding non-Python targets to the generated target is a feature request being -tracked in [Issue #1865](https://github.com/bazelbuild/rules_python/issues/1865). - -The annotation can be added multiple times, and all values are combined -and de-duplicated. - -For `python_generation_mode = "package"`, the `include_dep` annotations -found across all files included in the generated target are included in `deps`. - -Example: - -```python -# gazelle:include_dep //foo:bar,:hello_world,//:abc -# gazelle:include_dep //:def,//foo:bar -import numpy # a pypi package -``` - -will cause Gazelle to generate: - -```starlark -deps = [ - ":hello_world", - "//:abc", - "//:def", - "//foo:bar", - "@pypi//numpy", -] -``` - - -### Libraries - -Python source files are those ending in `.py` but not ending in `_test.py`. - -First, we look for the nearest ancestor BUILD file starting from the folder -containing the Python source file. - -In package generation mode, if there is no `py_library` in this BUILD file, one -is created using the package name as the target's name. This makes it the -default target in the package. Next, all source files are collected into the -`srcs` of the `py_library`. - -In project generation mode, all source files in subdirectories (that don't have -BUILD files) are also collected. - -In file generation mode, each file is given its own target. - -Finally, the `import` statements in the source files are parsed, and -dependencies are added to the `deps` attribute. - -### Unit Tests - -A `py_test` target is added to the BUILD file when gazelle encounters -a file named `__test__.py`. -Often, Python unit test files are named with the suffix `_test`. -For example, if we had a folder that is a package named "foo" we could have a Python file named `foo_test.py` -and gazelle would create a `py_test` block for the file. - -The following is an example of a `py_test` target that gazelle would add when -it encounters a file named `__test__.py`. - -```starlark -py_test( - name = "build_file_generation_test", - srcs = ["__test__.py"], - main = "__test__.py", - deps = [":build_file_generation"], -) -``` - -You can control the naming convention for test targets by adding a gazelle directive named -`# gazelle:python_test_naming_convention`. See the instructions in the section above that -covers directives. - -### Binaries - -When a `__main__.py` file is encountered, this indicates the entry point -of a Python program. A `py_binary` target will be created, named `[package]_bin`. - -When no such entry point exists, Gazelle will look for a line like this in the top level in every module: - -```python -if __name == "__main__": -``` - -Gazelle will create a `py_binary` target for every module with such a line, with -the target name the same as the module name. - -If `python_generation_mode` is set to `file`, then instead of one `py_binary` -target per module, Gazelle will create one `py_binary` target for each file with -such a line, and the name of the target will match the name of the script. - -Note that it's possible for another script to depend on a `py_binary` target and -import from the `py_binary`'s scripts. This can have possible negative effects on -Bazel analysis time and runfiles size compared to depending on a `py_library` -target. The simplest way to avoid these negative effects is to extract library -code into a separate script without a `main` line. Gazelle will then create a -`py_library` target for that library code, and other scripts can depend on that -`py_library` target. - -## Developer Notes - -Gazelle extensions are written in Go. This gazelle plugin is a hybrid, as it uses Go to execute a -Python interpreter as a subprocess to parse Python source files. -See the gazelle documentation https://github.com/bazelbuild/bazel-gazelle/blob/master/extend.md -for more information on extending Gazelle. - -If you add new Go dependencies to the plugin source code, you need to "tidy" the go.mod file. -After changing that file, run `go mod tidy` or `bazel run @go_sdk//:bin/go -- mod tidy` -to update the go.mod and go.sum files. Then run `bazel run //:gazelle_update_repos` to have gazelle -add the new dependenies to the deps.bzl file. The deps.bzl file is used as defined in our /WORKSPACE -to include the external repos Bazel loads Go dependencies from. - -Then after editing Go code, run `bazel run //:gazelle` to generate/update the rules in the -BUILD.bazel files in our repo. +:::{note} +The gazelle plugin docs have been migrated to our primary documentation on +ReadTheDocs. Please see https://rules-python.readthedocs.io/en/latest/gazelle/docs/index.html. +::: diff --git a/gazelle/WORKSPACE b/gazelle/WORKSPACE index d9f0645071..ebb18e7a47 100644 --- a/gazelle/WORKSPACE +++ b/gazelle/WORKSPACE @@ -2,21 +2,34 @@ workspace(name = "rules_python_gazelle_plugin") load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") +http_archive( + name = "bazel_skylib", + sha256 = "6e78f0e57de26801f6f564fa7c4a48dc8b36873e416257a92bbb0937eeac8446", + urls = [ + "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.8.2/bazel-skylib-1.8.2.tar.gz", + "https://github.com/bazelbuild/bazel-skylib/releases/download/1.8.2/bazel-skylib-1.8.2.tar.gz", + ], +) + +load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace") + +bazel_skylib_workspace() + http_archive( name = "io_bazel_rules_go", - sha256 = "278b7ff5a826f3dc10f04feaf0b70d48b68748ccd512d7f98bf442077f043fe3", + sha256 = "9d72f7b8904128afb98d46bbef82ad7223ec9ff3718d419afb355fddd9f9484a", urls = [ - "https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.41.0/rules_go-v0.41.0.zip", - "https://github.com/bazelbuild/rules_go/releases/download/v0.41.0/rules_go-v0.41.0.zip", + "https://mirror.bazel.build/github.com/bazel-contrib/rules_go/releases/download/v0.55.1/rules_go-v0.55.1.zip", + "https://github.com/bazel-contrib/rules_go/releases/download/v0.55.1/rules_go-v0.55.1.zip", ], ) http_archive( name = "bazel_gazelle", - sha256 = "29d5dafc2a5582995488c6735115d1d366fcd6a0fc2e2a153f02988706349825", + sha256 = "675114d8b433d0a9f54d81171833be96ebc4113115664b791e6f204d58e93446", urls = [ - "https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.31.0/bazel-gazelle-v0.31.0.tar.gz", - "https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.31.0/bazel-gazelle-v0.31.0.tar.gz", + "https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.47.0/bazel-gazelle-v0.47.0.tar.gz", + "https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.47.0/bazel-gazelle-v0.47.0.tar.gz", ], ) @@ -25,7 +38,7 @@ load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_depe go_rules_dependencies() -go_register_toolchains(version = "1.19.4") +go_register_toolchains(version = "1.22.9") gazelle_dependencies() @@ -38,6 +51,12 @@ load("@rules_python//python:repositories.bzl", "py_repositories") py_repositories() +load("//:internal_dev_deps.bzl", "internal_dev_deps") + +internal_dev_deps() + +register_toolchains("@rules_python//python/runtime_env_toolchains:all") + load("//:deps.bzl", _py_gazelle_deps = "gazelle_deps") # gazelle:repository_macro deps.bzl%go_deps diff --git a/gazelle/deps.bzl b/gazelle/deps.bzl index 948d61e5ae..c57d5193b8 100644 --- a/gazelle/deps.bzl +++ b/gazelle/deps.bzl @@ -14,10 +14,7 @@ "This file managed by `bazel run //:gazelle_update_repos`" -load( - "@bazel_gazelle//:deps.bzl", - _go_repository = "go_repository", -) +load("@bazel_gazelle//:deps.bzl", _go_repository = "go_repository") load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") def go_repository(name, **kwargs): @@ -29,9 +26,9 @@ def python_stdlib_list_deps(): http_archive( name = "python_stdlib_list", build_file_content = """exports_files(glob(["stdlib_list/lists/*.txt"]))""", - sha256 = "3f6fc8fba0a99ce8fa76c1b794a24f38962f6275ea9d5cfb43a874abe472571e", - strip_prefix = "stdlib-list-0.10.0", - url = "https://github.com/pypi/stdlib-list/releases/download/v0.10.0/v0.10.0.tar.gz", + sha256 = "517824f27ee89e591d8ae7c1dd9ff34f672eae50ee886ea31bb8816d77535675", + strip_prefix = "stdlib_list-0.12.0", + url = "https://files.pythonhosted.org/packages/8c/25/f1540879c8815387980e56f973e54605bd924612399ace31487f7444171c/stdlib_list-0.12.0.tar.gz", ) def gazelle_deps(): @@ -40,75 +37,30 @@ def gazelle_deps(): def go_deps(): "Fetch go dependencies" - go_repository( - name = "co_honnef_go_tools", - importpath = "honnef.co/go/tools", - sum = "h1:/hemPrYIhOhy8zYrNj+069zDB68us2sMGsfkFJO0iZs=", - version = "v0.0.0-20190523083050-ea95bdfd59fc", - ) go_repository( name = "com_github_bazelbuild_bazel_gazelle", importpath = "github.com/bazelbuild/bazel-gazelle", - sum = "h1:ROyUyUHzoEdvoOs1e0haxJx1l5EjZX6AOqiKdVlaBbg=", - version = "v0.31.1", + sum = "h1:g3Rr1ZbkC1Pk20aOgBITxSD/efS1WbaSty5jC786Z3Q=", + version = "v0.47.0", ) - go_repository( name = "com_github_bazelbuild_buildtools", build_naming_convention = "go_default_library", importpath = "github.com/bazelbuild/buildtools", - sum = "h1:HTepWP/jhtWTC1gvK0RnvKCgjh4gLqiwaOwGozAXcbw=", - version = "v0.0.0-20231103205921-433ea8554e82", + sum = "h1:njQAmjTv/YHRm/0Lfv9DXHFZ4MdT2IA/RKHTnqZkgDw=", + version = "v0.0.0-20250930140053-2eb4fccefb52", ) go_repository( name = "com_github_bazelbuild_rules_go", importpath = "github.com/bazelbuild/rules_go", - sum = "h1:JzlRxsFNhlX+g4drDRPhIaU5H5LnI978wdMJ0vK4I+k=", - version = "v0.41.0", + sum = "h1:cQYGcunY8myOB+0Ym6PGQRhc/milkRcNv0my3XgxaDU=", + version = "v0.55.1", ) - go_repository( name = "com_github_bmatcuk_doublestar_v4", importpath = "github.com/bmatcuk/doublestar/v4", - sum = "h1:FH9SifrbvJhnlQpztAx++wlkk70QBf0iBWDwNy7PA4I=", - version = "v4.6.1", - ) - - go_repository( - name = "com_github_burntsushi_toml", - importpath = "github.com/BurntSushi/toml", - sum = "h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=", - version = "v0.3.1", - ) - go_repository( - name = "com_github_census_instrumentation_opencensus_proto", - importpath = "github.com/census-instrumentation/opencensus-proto", - sum = "h1:glEXhBS5PSLLv4IXzLA5yPRVX4bilULVyxxbrfOtDAk=", - version = "v0.2.1", - ) - go_repository( - name = "com_github_chzyer_logex", - importpath = "github.com/chzyer/logex", - sum = "h1:Swpa1K6QvQznwJRcfTfQJmTE72DqScAa40E+fbHEXEE=", - version = "v1.1.10", - ) - go_repository( - name = "com_github_chzyer_readline", - importpath = "github.com/chzyer/readline", - sum = "h1:fY5BOSpyZCqRo5OhCuC+XN+r/bBCmeuuJtjz+bCNIf8=", - version = "v0.0.0-20180603132655-2972be24d48e", - ) - go_repository( - name = "com_github_chzyer_test", - importpath = "github.com/chzyer/test", - sum = "h1:q763qf9huN11kDQavWsoZXJNW3xEE4JJyHa5Q25/sd8=", - version = "v0.0.0-20180213035817-a1ea475d72b1", - ) - go_repository( - name = "com_github_client9_misspell", - importpath = "github.com/client9/misspell", - sum = "h1:ta993UF76GwbvJcIo3Y68y/M3WxlpEHPWIGDkJYwzJI=", - version = "v0.3.4", + sum = "h1:X8jg9rRZmJd4yRy7ZeNDRnM+T3ZfHv15JiBJ/avrEXE=", + version = "v4.9.1", ) go_repository( name = "com_github_davecgh_go_spew", @@ -116,32 +68,18 @@ def go_deps(): sum = "h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=", version = "v1.1.1", ) - go_repository( name = "com_github_emirpasic_gods", importpath = "github.com/emirpasic/gods", sum = "h1:FXtiHYKDGKCW2KzwZKx0iC0PQmdlorYgdFG9jPXJ1Bc=", version = "v1.18.1", ) - go_repository( - name = "com_github_envoyproxy_go_control_plane", - importpath = "github.com/envoyproxy/go-control-plane", - sum = "h1:4cmBvAEBNJaGARUEs3/suWRyfyBfhf7I60WBZq+bv2w=", - version = "v0.9.1-0.20191026205805-5f8ba28d4473", - ) - go_repository( - name = "com_github_envoyproxy_protoc_gen_validate", - importpath = "github.com/envoyproxy/protoc-gen-validate", - sum = "h1:EQciDnbrYxy13PgWoY8AqoxGiPrpgBZ1R8UNe3ddc+A=", - version = "v0.1.0", - ) go_repository( name = "com_github_fsnotify_fsnotify", importpath = "github.com/fsnotify/fsnotify", - sum = "h1:n+5WquG0fcWoWp6xPWfHdbskMCQaFnG6PfBrh1Ky4HY=", - version = "v1.6.0", + sum = "h1:8JEhPFa5W2WU7YfeZzPNqzMP6Lwt7L2715Ggo0nosvA=", + version = "v1.7.0", ) - go_repository( name = "com_github_ghodss_yaml", importpath = "github.com/ghodss/yaml", @@ -149,28 +87,28 @@ def go_deps(): version = "v1.0.0", ) go_repository( - name = "com_github_golang_glog", - importpath = "github.com/golang/glog", - sum = "h1:VKtxabqXZkF25pY9ekfRL6a582T4P37/31XEstQ5p58=", - version = "v0.0.0-20160126235308-23def4e6c14b", + name = "com_github_gogo_protobuf", + importpath = "github.com/gogo/protobuf", + sum = "h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=", + version = "v1.3.2", ) go_repository( name = "com_github_golang_mock", importpath = "github.com/golang/mock", - sum = "h1:ErTB+efbowRARo13NNdxyJji2egdxLGQhRaY+DUumQc=", - version = "v1.6.0", + sum = "h1:YojYx61/OLFsiv6Rw1Z96LpldJIy31o+UHmwAUMJ6/U=", + version = "v1.7.0-rc.1", ) go_repository( name = "com_github_golang_protobuf", importpath = "github.com/golang/protobuf", - sum = "h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw=", - version = "v1.5.2", + sum = "h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=", + version = "v1.5.4", ) go_repository( name = "com_github_google_go_cmp", importpath = "github.com/google/go-cmp", - sum = "h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=", - version = "v0.5.9", + sum = "h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=", + version = "v0.6.0", ) go_repository( name = "com_github_pmezard_go_difflib", @@ -178,18 +116,11 @@ def go_deps(): sum = "h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=", version = "v1.0.0", ) - - go_repository( - name = "com_github_prometheus_client_model", - importpath = "github.com/prometheus/client_model", - sum = "h1:gQz4mCbXsO+nc9n1hCxHcGA3Zx3Eo+UHZoInFGUIXNM=", - version = "v0.0.0-20190812154241-14fe0d1b01d4", - ) go_repository( name = "com_github_smacker_go_tree_sitter", importpath = "github.com/smacker/go-tree-sitter", - sum = "h1:7QZKUmQfnxncZIJGyvX8M8YeMfn8kM10j3J/2KwVTN4=", - version = "v0.0.0-20240422154435-0628b34cbf9c", + sum = "h1:6C8qej6f1bStuePVkLSFxoU22XBS165D3klxlzRg8F4=", + version = "v0.0.0-20240827094217-dd81d9e9be82", ) go_repository( name = "com_github_stretchr_objx", @@ -203,19 +134,6 @@ def go_deps(): sum = "h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=", version = "v1.9.0", ) - - go_repository( - name = "com_github_yuin_goldmark", - importpath = "github.com/yuin/goldmark", - sum = "h1:fVcFKWvrslecOb/tg+Cc05dkeYx540o0FuFt3nUVDoE=", - version = "v1.4.13", - ) - go_repository( - name = "com_google_cloud_go", - importpath = "cloud.google.com/go", - sum = "h1:e0WKqKTd5BnrG8aKH3J3h+QvEIQtSUcf2n5UZ5ZgLtQ=", - version = "v0.26.0", - ) go_repository( name = "in_gopkg_check_v1", importpath = "gopkg.in/check.v1", @@ -234,90 +152,71 @@ def go_deps(): sum = "h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=", version = "v3.0.1", ) - go_repository( name = "net_starlark_go", importpath = "go.starlark.net", sum = "h1:xwwDQW5We85NaTk2APgoN9202w/l0DVGp+GZMfsrh7s=", version = "v0.0.0-20210223155950-e043a3d3c984", ) - go_repository( - name = "org_golang_google_appengine", - importpath = "google.golang.org/appengine", - sum = "h1:/wp5JvzpHIxhs/dumFmF7BXTf3Z+dd4uXta4kVyO508=", - version = "v1.4.0", - ) go_repository( name = "org_golang_google_genproto", importpath = "google.golang.org/genproto", - sum = "h1:+kGHl1aib/qcwaRi1CbqBZ1rk19r85MNUf8HaBghugY=", - version = "v0.0.0-20200526211855-cb27e3aa2013", + sum = "h1:387Y+JbxF52bmesc8kq1NyYIp33dnxCw6eiA7JMsTmw=", + version = "v0.0.0-20250115164207-1a7da9e5054f", ) go_repository( - name = "org_golang_google_grpc", - importpath = "google.golang.org/grpc", - sum = "h1:fPVVDxY9w++VjTZsYvXWqEf9Rqar/e+9zYfxKK+W+YU=", - version = "v1.50.0", - ) - go_repository( - name = "org_golang_google_protobuf", - importpath = "google.golang.org/protobuf", - sum = "h1:w43yiav+6bVFTBQFZX0r7ipe9JQ1QsbMgHwbBziscLw=", - version = "v1.28.0", + name = "org_golang_google_genproto_googleapis_rpc", + importpath = "google.golang.org/genproto/googleapis/rpc", + sum = "h1:3UsHvIr4Wc2aW4brOaSCmcxh9ksica6fHEr8P1XhkYw=", + version = "v0.0.0-20250106144421-5f5ef82da422", ) go_repository( - name = "org_golang_x_crypto", - importpath = "golang.org/x/crypto", - sum = "h1:VklqNMn3ovrHsnt90PveolxSbWFaJdECFbxSq0Mqo2M=", - version = "v0.0.0-20190308221718-c2843e01d9a2", + name = "org_golang_google_grpc", + importpath = "google.golang.org/grpc", + sum = "h1:OgPcDAFKHnH8X3O4WcO4XUc8GRDeKsKReqbQtiCj7N8=", + version = "v1.67.3", ) go_repository( - name = "org_golang_x_exp", - importpath = "golang.org/x/exp", - sum = "h1:c2HOrn5iMezYjSlGPncknSEr/8x5LELb/ilJbXi9DEA=", - version = "v0.0.0-20190121172915-509febef88a4", + name = "org_golang_google_grpc_cmd_protoc_gen_go_grpc", + importpath = "google.golang.org/grpc/cmd/protoc-gen-go-grpc", + sum = "h1:F29+wU6Ee6qgu9TddPgooOdaqsxTMunOoj8KA5yuS5A=", + version = "v1.5.1", ) go_repository( - name = "org_golang_x_lint", - importpath = "golang.org/x/lint", - sum = "h1:XQyxROzUlZH+WIQwySDgnISgOivlhjIEwaQaJEJrrN0=", - version = "v0.0.0-20190313153728-d0100b6bd8b3", + name = "org_golang_google_protobuf", + importpath = "google.golang.org/protobuf", + sum = "h1:82DV7MYdb8anAVi3qge1wSnMDrnKK7ebr+I0hHRN1BU=", + version = "v1.36.3", ) go_repository( name = "org_golang_x_mod", importpath = "golang.org/x/mod", - sum = "h1:lFO9qtOdlre5W1jxS3r/4szv2/6iXxScdzjoBMXNhYk=", - version = "v0.10.0", + sum = "h1:Zb7khfcRGKk+kqfxFaP5tZqCnDZMjC5VtUBs87Hr6QM=", + version = "v0.23.0", ) go_repository( name = "org_golang_x_net", importpath = "golang.org/x/net", - sum = "h1:X2//UzNDwYmtCLn7To6G58Wr6f5ahEAQgKNzv9Y951M=", - version = "v0.10.0", - ) - go_repository( - name = "org_golang_x_oauth2", - importpath = "golang.org/x/oauth2", - sum = "h1:vEDujvNQGv4jgYKudGeI/+DAX4Jffq6hpD55MmoEvKs=", - version = "v0.0.0-20180821212333-d2e6202438be", + sum = "h1:T5GQRQb2y08kTAByq9L4/bz8cipCdA8FbRTXewonqY8=", + version = "v0.35.0", ) go_repository( name = "org_golang_x_sync", importpath = "golang.org/x/sync", - sum = "h1:PUR+T4wwASmuSTYdKjYHI5TD22Wy5ogLU5qZCOLxBrI=", - version = "v0.2.0", + sum = "h1:GGz8+XQP4FvTTrjZPzNKTMFtSXH80RAzG+5ghFPgK9w=", + version = "v0.11.0", ) go_repository( name = "org_golang_x_sys", importpath = "golang.org/x/sys", - sum = "h1:EBmGv8NaZBZTWvrbjNoL6HVt+IVy3QDQpJs7VRIw3tU=", - version = "v0.8.0", + sum = "h1:QjkSwP/36a20jFYWkSue1YwXzLmsV5Gfq7Eiy72C1uc=", + version = "v0.30.0", ) go_repository( name = "org_golang_x_text", importpath = "golang.org/x/text", - sum = "h1:cokOdA+Jmi5PJGXLlLllQSgYigAEfHXJAERHVMaCc2k=", - version = "v0.3.3", + sum = "h1:bofq7m3/HAFvbF51jz3Q9wLg3jkvSPuiZu/pD1XwgtM=", + version = "v0.22.0", ) go_repository( name = "org_golang_x_tools", @@ -325,12 +224,12 @@ def go_deps(): "gazelle:exclude **/testdata/**/*", ], importpath = "golang.org/x/tools", - sum = "h1:8WMNJAz3zrtPmnYC7ISf5dEn3MT0gY7jBJfw27yrrLo=", - version = "v0.9.1", + sum = "h1:BgcpHewrV5AUp2G9MebG4XPFI1E2W41zU1SaqVA9vJY=", + version = "v0.30.0", ) go_repository( - name = "org_golang_x_xerrors", - importpath = "golang.org/x/xerrors", - sum = "h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=", - version = "v0.0.0-20200804184101-5ec99f83aff1", + name = "org_golang_x_tools_go_vcs", + importpath = "golang.org/x/tools/go/vcs", + sum = "h1:cOIJqWBl99H1dH5LWizPa+0ImeeJq3t3cJjaeOWUAL4=", + version = "v0.1.0-deprecated", ) diff --git a/gazelle/docs/BUILD.bazel b/gazelle/docs/BUILD.bazel new file mode 100644 index 0000000000..7c6b6fd56e --- /dev/null +++ b/gazelle/docs/BUILD.bazel @@ -0,0 +1,5 @@ +filegroup( + name = "docs", + srcs = glob(["*.md"]), + visibility = ["//visibility:public"], +) diff --git a/gazelle/docs/annotations.md b/gazelle/docs/annotations.md new file mode 100644 index 0000000000..b3f06b9991 --- /dev/null +++ b/gazelle/docs/annotations.md @@ -0,0 +1,200 @@ +# Annotations + +*Annotations* refer to comments found _within Python files_ that configure how +Gazelle acts for that particular file. + +Annotations have the form: + +```python +# gazelle:annotation_name value +``` + +and can reside anywhere within a Python file where comments are valid. For example: + +```python +import foo +# gazelle:annotation_name value + +def bar(): # gazelle:annotation_name value + pass +``` + +The annotations are: + +{.glossary} +[`# gazelle:ignore imports`](#annotation-ignore) +: Tells Gazelle to ignore import statements. `imports` is a comma-separated + list of imports to ignore. + * Default: n/a + * Allowed Values: A comma-separated string of python package names + +[`# gazelle:include_dep targets`](#annotation-include-dep) +: Tells Gazelle to include a set of dependencies, even if they are not imported + in a Python module. `targets` is a comma-separated list of target names + to include as dependencies. + * Default: n/a + * Allowed Values: A comma-separated string of targets + +[`# gazelle:include_pytest_conftest bool`](#annotation-include-pytest-conftest) +: Whether or not to include a sibling `:conftest` target in the `deps` + of a {bzl:obj}`py_test` target. The default behaviour is to include `:conftest` + (i.e.: `# gazelle:include_pytest_conftest true`). + * Default: n/a + * Allowed Values: `true`, `false` + + +(annotation-ignore)= +## `ignore` + +This annotation accepts a comma-separated string of values. Values are names of +Python imports that Gazelle should _not_ include in target dependencies. + +The annotation can be added multiple times, and all values are combined and +de-duplicated. + +For `python_generation_mode = "package"`, the `ignore` annotations +found across all files included in the generated target are removed from +`deps`. + +### Example: + +```python +import numpy # a pypi package + +# gazelle:ignore bar.baz.hello,foo +import bar.baz.hello +import foo + +# Ignore this import because _reasons_ +import baz # gazelle:ignore baz +``` + +will cause Gazelle to generate: + +```starlark +deps = ["@pypi//numpy"], +``` + + +(annotation-include-dep)= +## `include_dep` + +This annotation accepts a comma-separated string of values. Values _must_ +be Python targets, but _no validation is done_. If a value is not a Python +target, building will result in an error saying: + +``` + does not have mandatory providers: 'PyInfo' or 'CcInfo' or 'PyInfo'. +``` + +Adding non-Python targets to the generated target is a feature request being +tracked in {gh-issue}`1865`. + +The annotation can be added multiple times, and all values are combined +and de-duplicated. + +For `python_generation_mode = "package"`, the `include_dep` annotations +found across all files included in the generated target are included in +`deps`. + +### Example: + +```python +# gazelle:include_dep //foo:bar,:hello_world,//:abc +# gazelle:include_dep //:def,//foo:bar +import numpy # a pypi package +``` + +will cause Gazelle to generate: + +```starlark +deps = [ + ":hello_world", + "//:abc", + "//:def", + "//foo:bar", + "@pypi//numpy", +] +``` + + +(annotation-include-pytest-conftest)= +## `include_pytest_conftest` + +:::{versionadded} 1.6.0 +{gh-pr}`3080` +::: + +This annotation accepts any string that can be parsed by go's +[`strconv.ParseBool`][ParseBool]. If an unparsable string is passed, the +annotation is ignored. + +[ParseBool]: https://pkg.go.dev/strconv#ParseBool + +Starting with [`rules_python` 0.14.0][rules-python-0.14.0] (specifically +{gh-pr}`879`), Gazelle will include a `:conftest` dependency to a +{bzl:obj}`py_test` target that is in the same directory as `conftest.py`. + +[rules-python-0.14.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.14.0 + +This annotation allows users to adjust that behavior. To disable the behavior, +set the annotation value to `false`: + +``` +# some_file_test.py +# gazelle:include_pytest_conftest false +``` + +### Example: + +Given a directory tree like: + +``` +. +├── BUILD.bazel +├── conftest.py +└── some_file_test.py +``` + +The default Gazelle behavior would create: + +```starlark +py_library( + name = "conftest", + testonly = True, + srcs = ["conftest.py"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "some_file_test", + srcs = ["some_file_test.py"], + deps = [":conftest"], +) +``` + +When `# gazelle:include_pytest_conftest false` is found in +`some_file_test.py` + +```python +# some_file_test.py +# gazelle:include_pytest_conftest false +``` + +Gazelle will generate: + +```starlark +py_library( + name = "conftest", + testonly = True, + srcs = ["conftest.py"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "some_file_test", + srcs = ["some_file_test.py"], +) +``` + +See {gh-issue}`3076` for more information. diff --git a/gazelle/docs/development.md b/gazelle/docs/development.md new file mode 100644 index 0000000000..29ac7a0605 --- /dev/null +++ b/gazelle/docs/development.md @@ -0,0 +1,57 @@ +# Development + +Gazelle extensions are written in Go. + +See the [Gazelle documentation][gazelle-extend] for more information on +extending Gazelle. + +[gazelle-extend]: https://github.com/bazel-contrib/bazel-gazelle/blob/master/extend.md + + +## Dependencies + +If you add new Go dependencies to the plugin source code, you need to "tidy" +the go.mod file. After changing that file, run `go mod tidy` or +`bazel run @go_sdk//:bin/go -- mod tidy` to update the `go.mod` and `go.sum` +files. Then run `bazel run //:gazelle_update_repos` to have gazelle add the +new dependencies to the `deps.bzl` file. The `deps.bzl` file is used as +defined in our `/WORKSPACE` to include the external repos Bazel loads Go +dependencies from. + +Then after editing Go code, run `bazel run //:gazelle` to generate/update +the rules in the `BUILD.bazel` files in our repo. + + +## Tests + +:::{seealso} +{gh-path}`gazelle/python/testdata/README.md` +::: + +To run tests, {command}`cd` into the {gh-path}`gazelle` directory and run +`bazel test //...`. + +Test cases are found at {gh-path}`gazelle/python/testdata`. To make a new +test case, create a directory in that folder with the following files: + ++ `README.md` with a short blurb describing the test case(s). ++ `test.yaml`, either empty (with just the docstart `---` line) or with + the expected `stderr` and exit codes of the test case. ++ and empty `WORKSPACE` file + +You will also need `BUILD.in` and `BUILD.out` files somewhere within the test +case directory. These can be in the test case root, in subdirectories, or +both. + ++ `BUILD.in` files are populated with the "before" information - typically + things like Gazelle directives or pre-existing targets. This is how the + `BUILD.bazel` file looks before running Gazelle. ++ `BUILD.out` files are the expected result after running Gazelle within + the test case. + +:::{tip} +The easiest way to create a new test is to look at one of the existing test +cases. +::: + +The source code for running tests is {gh-path}`gazelle/python/python_test.go`. diff --git a/gazelle/docs/directives.md b/gazelle/docs/directives.md new file mode 100644 index 0000000000..60f948e4af --- /dev/null +++ b/gazelle/docs/directives.md @@ -0,0 +1,896 @@ +# Directives + +You can configure the extension using directives, just like for other +languages. These are just comments in the `BUILD.bazel` file which +govern behavior of the extension when processing files under that +folder. + +See the [Gazelle docs on directives][gazelle-directives] for some general +directives that may be useful. In particular, the `resolve` directive +is language-specific and can be used with Python. Examples of these and +the Python-specific directives in use can be found in the +{gh-path}`gazelle/testdata` folder in the `rules_python` repo. + +[gazelle-directives]: https://github.com/bazelbuild/bazel-gazelle#directives + +The Python-specific directives are: + +{.glossary} +[`# gazelle:python_extension value`](#directive-python-extension) +: Controls whether the Python extension is enabled or not. Sub-packages + inherit this value. + * Default: `enabled` + * Allowed Values: `enabled`, `disabled` + +[`# gazelle:python_root`](#directive-python-root) +: Sets a Bazel package as a Python root. This is used on monorepos with + multiple Python projects that don't share the top-level of the workspace + as the root. + * Default: n/a + * Allowed Values: None. This direcive does not consume values. + +[`# gazelle:python_manifest_file_name value`](#directive-python-manifest-file-name) +: Overrides the default manifest file name. + * Default: `gazelle_python.yaml` + * Allowed Values: A string + +[`# gazelle:python_ignore_files value`](#directive-python-ignore-files) +: Controls the files which are ignored from the generated targets. + * Default: n/a + * Allowed Values: A comma-separated list of strings. + +[`# gazelle:python_ignore_dependencies value`](#directive-python-ignore-dependencies) +: Controls the ignored dependencies from the generated targets. + * Default: n/a + * Allowed Values: A comma-separated list of strings. + +[`# gazelle:python_validate_import_statements bool`](#directive-python-validate-import-statements) +: Controls whether the Python import statements should be validated. + * Default: `true` + * Allowed Values: `true`, `false` + +[`# gazelle:python_generation_mode value`](#directive-python-generation-mode) +: Controls the target generation mode. + * Default: `package` + * Allowed Values: `file`, `package`, `project` + +[`# gazelle:python_generation_mode_per_file_include_init bool`](#directive-python-generation-mode-per-file-include-init) +: Controls whether `__init__.py` files are included as srcs in each + generated target when target generation mode is "file". + * Default: `false` + * Allowed Values: `true`, `false` + +[`# gazelle:python_generation_mode_per_package_require_test_entry_point bool`](#directive-python-generation-mode-per-package-require-test-entry-point) +: Controls whether a file called `__test__.py` or a target called + `__test__` is required to generate one test target per package in + package mode. + * Default: `true` + * Allowed Values: `true`, `false` + +[`# gazelle:python_library_naming_convention value`](#directive-python-library-naming-convention) +: Controls the {bzl:obj}`py_library` naming convention. It interpolates + `$package_name$` with the Bazel package name. E.g. if the Bazel package + name is `foo`, setting this to `$package_name$_my_lib` would result in a + generated target named `foo_my_lib`. + * Default: `$package_name$` + * Allowed Values: A string containing `"$package_name$"` + +[`# gazelle:python_binary_naming_convention value`](#directive-python-binary-naming-convention) +: Controls the {bzl:obj}`py_binary` naming convention. Follows the same interpolation + rules as `python_library_naming_convention`. + * Default: `$package_name$_bin` + * Allowed Values: A string containing `"$package_name$"` + +[`# gazelle:python_test_naming_convention value`](#directive-python-test-naming-convention) +: Controls the {bzl:obj}`py_test` naming convention. Follows the same interpolation + rules as `python_library_naming_convention`. + * Default: `$package_name$_test` + * Allowed Values: A string containing `"$package_name$"` + +[`# gazelle:python_proto_naming_convention value`](#directive-python-proto-naming-convention) +: Controls the {bzl:obj}`py_proto_library` naming convention. It interpolates + `$proto_name$` with the `proto_library` rule name, minus any trailing + `_proto`. E.g. if the `proto_library` name is `foo_proto`, setting this + to `$proto_name$_my_lib` would render to `foo_my_lib`. + * Default: `$proto_name$_py_pb2` + * Allowed Values: A string containing `"$proto_name$"` + +[`# gazelle:resolve py import-lang import-string label`](#directive-resolve-py) +: Instructs the plugin what target to add as a dependency to satisfy a given + import statement. The syntax is `# gazelle:resolve py import-string label` + where `import-string` is the symbol in the python `import` statement, + and `label` is the Bazel label that Gazelle should write in `deps`. + * Default: n/a + * Allowed Values: See the [bazel-gazelle docs][gazelle-directives] + +[`# gazelle:python_default_visibility labels`](#directive-python-default-visibility) +: Instructs gazelle to use these visibility labels on all python targets. + `labels` is a comma-separated list of labels (without spaces). + * Default: `//$python_root$:__subpackages__` + * Allowed Values: A string + +[`# gazelle:python_visibility label`](#directive-python-visibility) +: Appends additional visibility labels to each generated target. This r + directive can be set multiple times. + * Default: n/a + * Allowed Values: A string + +[`# gazelle:python_test_file_pattern value`](#directive-python-test-file-pattern) +: Filenames matching these comma-separated {command}`glob`s will be mapped to + {bzl:obj}`py_test` targets. + * Default: `*_test.py,test_*.py` + * Allowed Values: A glob string + +[`# gazelle:python_label_convention value`](#directive-python-label-convention) +: Defines the format of the distribution name in labels to third-party deps. + Useful for using Gazelle plugin with other rules with different repository + conventions (e.g. `rules_pycross`). Full label is always prepended with + the `pip` repository name, e.g. `@pip//numpy` if your + `MODULE.bazel` has `use_repo(pip, "pip")` or `@pypi//numpy` + if your `MODULE.bazel` has `use_repo(pip, "pypi")`. + * Default: `$distribution_name$` + * Allowed Values: A string + +[`# gazelle:python_label_normalization value`](#directive-python-label-normalization) +: Controls how distribution names in labels to third-party deps are + normalized. Useful for using Gazelle plugin with other rules with different + label conventions (e.g. `rules_pycross` uses PEP-503). + * Default: `snake_case` + * Allowed Values: `snake_case`, `none`, `pep503` + +[`# gazelle:python_experimental_allow_relative_imports bool`](#directive-python-experimental-allow-relative-imports) +: Controls whether Gazelle resolves dependencies for import statements that + use paths relative to the current package. + * Default: `false` + * Allowed Values: `true`, `false` + +[`# gazelle:python_generate_pyi_deps bool`](#directive-python-generate-pyi-deps) +: Controls whether to generate a separate `pyi_deps` attribute for + type-checking dependencies or merge them into the regular `deps` + attribute. When `true` (default), generates separate `pyi_deps`. When + `false`, type-checking dependencies are merged into `deps`. Imports in + blocks with the format + `if typing.TYPE_CHECKING:` or `if TYPE_CHECKING:` and type-only stub + packages (eg. boto3-stubs) are recognized as type-checking dependencies. + * Default: `true` + * Allowed Values: `true`, `false` + +[`# gazelle:python_generate_pyi_srcs bool`](#directive-python-generate-pyi-srcs) +: Controls whether to generate a `pyi_srcs` attribute if a sibling `.pyi` file + is found. When `false`, the `pyi_srcs` attribute is not added. + * Default: `true` + * Allowed Values: `true`, `false` + +[`# gazelle:python_generate_proto bool`](#directive-python-generate-proto) +: Controls whether to generate a {bzl:obj}`py_proto_library` for each + `proto_library` in the package. By default we load this rule from the + `@protobuf` repository; use `gazelle:map_kind` if you need to load this + from somewhere else. + * Default: `false` + * Allowed Values: `true`, `false` + +[`# gazelle:python_resolve_sibling_imports bool`](#directive-python-resolve-sibling-imports) +: Allows absolute imports to be resolved to sibling modules (Python 2's + behavior without `absolute_import`). + * Default: `false` + * Allowed Values: `true`, `false` + +[`# gazelle:python_include_ancestor_conftest bool`](#directive-python-include-ancestor-conftest) +: Controls whether ancestor conftest targets are added to {bzl:obj}`py_test` target + dependencies. + * Default: `true` + * Allowed Values: `true`, `false` + +(directive-python-extension)= +## `python_extension` + +:::{error} +Detailed docs are not yet written. +::: + + +(directive-python-root)= +## `python_root` + +Set this directive within the Bazel package that you want to use as the Python root. +For example, if using a `src` dir (as recommended by the [Python Packaging User +Guide][python-packaging-user-guide]), then set this directive in `src/BUILD.bazel`: + +```starlark +# ./src/BUILD.bazel +# Tell gazelle that are python root is the same dir as this Bazel package. +# gazelle:python_root +``` + +Note that the directive does not have any arguments. + +Gazelle will then add the necessary `imports` attribute to all targets that it +generates: + +```starlark +# in ./src/foo/BUILD.bazel +py_libary( + ... + imports = [".."], # Gazelle adds this + ... +) + +# in ./src/foo/bar/BUILD.bazel +py_libary( + ... + imports = ["../.."], # Gazelle adds this + ... +) +``` + +[python-packaging-user-guide]: https://github.com/pypa/packaging.python.org/blob/4c86169a/source/tutorials/packaging-projects.rst + + +(directive-python-manifest-file-name)= +## `python_manifest_file_name` + +:::{error} +Detailed docs are not yet written. +::: + + +(directive-python-ignore-files)= +## `python_ignore_files` + +:::{error} +Detailed docs are not yet written. +::: + + +(directive-python-ignore-dependencies)= +## `python_ignore_dependencies` + +:::{error} +Detailed docs are not yet written. +::: + + +(directive-python-validate-import-statements)= +## `python_validate_import_statements` + +:::{error} +Detailed docs are not yet written. +::: + + +(directive-python-generation-mode)= +## `python_generation_mode` + +:::{error} +Detailed docs are not yet written. +::: + + +(directive-python-generation-mode-per-file-include-init)= +## `python_generation_mode_per_file_include_init` + +:::{error} +Detailed docs are not yet written. +::: + + +(directive-python-generation-mode-per-package-require-test-entry-point)= +## `python_generation_mode_per_package_require_test_entry_point` + +When `# gazelle:python_generation_mode package`, whether a file called +`__test__.py` or a target called `__test__`, a.k.a., entry point, is required +to generate one test target per package. If this is set to true but no entry +point is found, Gazelle will fall back to file mode and generate one test target +per file. Setting this directive to false forces Gazelle to generate one test +target per package even without entry point. However, this means the `main` +attribute of the {bzl:obj}`py_test` will not be set and the target will not be runnable +unless either: + +1. there happen to be a file in the `srcs` with the same name as the {bzl:obj}`py_test` + target, or +2. a macro populating the `main` attribute of {bzl:obj}`py_test` is configured with + `gazelle:map_kind` to replace {bzl:obj}`py_test` when Gazelle is generating Python + test targets. For example, user can provide such a macro to Gazelle: + +```starlark +load("@rules_python//python:defs.bzl", _py_test="py_test") +load("@aspect_rules_py//py:defs.bzl", "py_pytest_main") + +def py_test(name, main=None, **kwargs): + deps = kwargs.pop("deps", []) + if not main: + py_pytest_main( + name = "__test__", + deps = ["@pip_pytest//:pkg"], # change this to the pytest target in your repo. + ) + + deps.append(":__test__") + main = ":__test__.py" + + _py_test( + name = name, + main = main, + deps = deps, + **kwargs, +) +``` + + +(directive-python-library-naming-convention)= +## `python_library_naming_convention` + +:::{error} +Detailed docs are not yet written. +::: + + +(directive-python-binary-naming-convention)= +## `python_binary_naming_convention` + +:::{error} +Detailed docs are not yet written. +::: + + +(directive-python-test-naming-convention)= +## `python_test_naming_convention` + +:::{error} +Detailed docs are not yet written. +::: + + +(directive-python-proto-naming-convention)= +## `python_proto_naming_convention` + +:::{versionadded} 1.6.0 +{gh-pr}`3093` +::: + +Set this directive to a string pattern to control how the generated +{bzl:obj}`py_proto_library` targets are named. When generating new +{bzl:obj}`py_proto_library` rules, Gazelle will replace `$proto_name$` in the +pattern with the name of the `proto_library` rule, stripping out a +trailing `_proto`. For example: + +```starlark +# gazelle:python_generate_proto true +# gazelle:python_proto_naming_convention my_custom_$proto_name$_pattern + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], +) +``` + +produces the following {bzl:obj}`py_proto_library` rule: + +```starlark +py_proto_library( + name = "my_custom_foo_pattern", + deps = [":foo_proto"], +) +``` + +The default naming convention is `$proto_name$_pb2_py` in accordance with +the [Bazel `py_proto_library` convention][bazel-py-proto-library], so by default +in the above example Gazelle would generate `foo_pb2_py`. Any pre-existing +rules are left in place and not renamed. + +[bazel-py-proto-library]: https://bazel.build/reference/be/protocol-buffer#py_proto_library + +Note that the Python library will always be imported as `foo_pb2` in Python +code, regardless of the naming convention. Also note that Gazelle is currently +not able to map said imports, e.g. `import foo_pb2`, to fill in +{bzl:obj}`py_proto_library` targets as dependencies of other rules. See +{gh-issue}`1703`. + + +(directive-resolve-py)= +## `resolve py` + +:::{error} +Detailed docs are not yet written. +::: + + +(directive-python-default-visibility)= +## `python_default_visibility` + +:::{versionadded} 0.32.0 +{gh-pr}`1787` +::: + +Instructs gazelle to use these visibility labels on all _python_ targets +(typically `py_*`, but can be modified via the `map_kind` directive). The arg +to this directive is a comma-separated list (without spaces) of labels. + +For example: + +```starlark +# gazelle:python_default_visibility //:__subpackages__,//tests:__subpackages__ +``` + +produces the following visibility attribute: + +```starlark +py_library( + ..., + visibility = [ + "//:__subpackages__", + "//tests:__subpackages__", + ], + ..., +) +``` + +You can also inject the `python_root` value by using the exact string +`$python_root$`. All instances of this string will be replaced by the `python_root` +value. + +```starlark +# gazelle:python_default_visibility //$python_root$:__pkg__,//foo/$python_root$/tests:__subpackages__ + +# Assuming the "# gazelle:python_root" directive is set in ./py/src/BUILD.bazel, +# the results will be: +py_library( + ..., + visibility = [ + "//foo/py/src/tests:__subpackages__", # sorted alphabetically + "//py/src:__pkg__", + ], + ..., +) +``` + +Two special values are also accepted as an argument to the directive: + +* `NONE`: This removes all default visibility. Labels added by the + `python_visibility` directive are still included. +* `DEFAULT`: This resets the default visibility. + +For example: + +```starlark +# gazelle:python_default_visibility NONE + +py_library( + name = "...", + srcs = [...], +) +``` + +```starlark +# gazelle:python_default_visibility //foo:bar +# gazelle:python_default_visibility DEFAULT + +py_library( + ..., + visibility = ["//:__subpackages__"], + ..., +) +``` + +These special values can be useful for sub-packages. + + +(directive-python-visibility)= +## `python_visibility` + +:::{versionadded} 0.32.0 +{gh-pr}`1784` +::: + +Appends additional `visibility` labels to each generated target. + +This directive can be set multiple times. The generated `visibility` attribute +will include the default visibility and all labels defined by this directive. +All labels will be ordered alphabetically. + +```starlark +# ./BUILD.bazel +# gazelle:python_visibility //tests:__pkg__ +# gazelle:python_visibility //bar:baz + +py_library( + ... + visibility = [ + "//:__subpackages__", # default visibility + "//bar:baz", + "//tests:__pkg__", + ], + ... +) +``` + +Child Bazel packages inherit values from parents: + +```starlark +# ./bar/BUILD.bazel +# gazelle:python_visibility //tests:__subpackages__ + +py_library( + ... + visibility = [ + "//:__subpackages__", # default visibility + "//bar:baz", # defined in ../BUILD.bazel + "//tests:__pkg__", # defined in ../BUILD.bazel + "//tests:__subpackages__", # defined in this ./BUILD.bazel + ], + ... +) + +``` + +This directive also supports the `$python_root$` placeholder that +`# gazelle:python_default_visibility` supports. + +```starlark +# gazlle:python_visibility //$python_root$/foo:bar + +py_library( + ... + visibility = ["//this_is_my_python_root/foo:bar"], + ... +) +``` + + +(directive-python-test-file-pattern)= +## `python_test_file_pattern` + +:::{versionadded} 0.32.0 +{gh-pr}`1819` +::: + +This directive adjusts which python files will be mapped to the {bzl:obj}`py_test` rule. + ++ The default is `*_test.py,test_*.py`: both `test_*.py` and `*_test.py` files + will generate {bzl:obj}`py_test` targets. ++ This directive must have a value. If no value is given, an error will be raised. ++ It is recommended, though not necessary, to include the `.py` extension in + the {command}`glob`: `foo*.py,?at.py`. ++ Like most directives, it applies to the current Bazel package and all subpackages + until the directive is set again. ++ This directive accepts multiple {command}`glob` patterns, separated by commas without spaces: + +```starlark +# gazelle:python_test_file_pattern foo*.py,?at + +py_library( + name = "mylib", + srcs = ["mylib.py"], +) + +py_test( + name = "foo_bar", + srcs = ["foo_bar.py"], +) + +py_test( + name = "cat", + srcs = ["cat.py"], +) + +py_test( + name = "hat", + srcs = ["hat.py"], +) +``` + + +### Notes + +Resetting to the default value (such as in a subpackage) is manual. Set: + +```starlark +# gazelle:python_test_file_pattern *_test.py,test_*.py +``` + +There currently is no way to tell gazelle that _no_ files in a package should +be mapped to {bzl:obj}`py_test` targets (see {gh-issue}`1826`). The workaround +is to set this directive to a pattern that will never match a `.py` file, such +as `foo.bar`: + +```starlark +# No files in this package should be mapped to py_test targets. +# gazelle:python_test_file_pattern foo.bar + +py_library( + name = "my_test", + srcs = ["my_test.py"], +) +``` + + +(directive-python-label-convention)= +## `python_label_convention` + +:::{versionadded} 0.34.0 +{gh-pr}`1976` +::: + +:::{error} +Detailed docs are not yet written. +::: + + +(directive-python-label-normalization)= +## `python_label_normalization` + +:::{versionadded} 0.34.0 +{gh-pr}`1976` +::: + +:::{error} +Detailed docs are not yet written. +::: + + +(directive-python-experimental-allow-relative-imports)= +## `python_experimental_allow_relative_imports` + +Enables experimental support for resolving relative imports in +`python_generation_mode package`. + +By default, when `# gazelle:python_generation_mode package` is enabled, +relative imports (e.g., `from .library import foo`) are not added to the +deps field of the generated target. This results in incomplete {bzl:obj}`py_library` +rules that lack required dependencies on sibling packages. + +Example: + +Given this Python file import: + +```python +from .library import add as _add +from .library import subtract as _subtract +``` + +Expected BUILD file output: + +```starlark +py_library( + name = "py_default_library", + srcs = ["__init__.py"], + deps = [ + "//example/library:py_default_library", + ], + visibility = ["//visibility:public"], +) +``` + +Actual output without this annotation: + +```starlark +py_library( + name = "py_default_library", + srcs = ["__init__.py"], + visibility = ["//visibility:public"], +) +``` + +If the directive is set to `true`, gazelle will resolve imports +that are relative to the current package. + + +(directive-python-generate-pyi-deps)= +## `python_generate_pyi_deps` + +:::{versionadded} 1.6.0 +{gh-pr}`3014` +::: + +:::{versionchanged} 2.1.0 +The default was changed from `false` to `true`. {gh-pr}`3753` +::: + +When `true`, Gazelle writes type-checking dependencies to the `pyi_deps` +attribute instead of merging them into `deps`. This is the default behavior. + +Gazelle treats imports inside `if TYPE_CHECKING:` and +`if typing.TYPE_CHECKING:` blocks as type-checking dependencies. It also adds +type stub packages, such as `boto3-stubs`, to `pyi_deps` when the corresponding +runtime package is imported normally. + +For example, assume you have the following file: + +```python +import boto3 +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + import requests +``` + +The generated target will be: + +```starlark +py_library( + name = "foo", + srcs = ["foo.py"], + pyi_deps = ["@pip//requests"], + deps = ["@pip//boto3"], +) +``` + +When `false`, Gazelle merges type-checking dependencies into `deps` and does +not write `pyi_deps`. + + +(directive-python-generate-pyi-srcs)= +## `python_generate_pyi_srcs` + +:::{versionadded} 1.6.0 +{gh-pr}`3356` +::: + +:::{versionchanged} 2.1.0 +The default was changed from `false` to `true`. {gh-pr}`3753` +::: + +When `true`, include any sibling `.pyi` files in the `pyi_srcs` target attribute. + +For example, assume you have the following files: + +``` +foo.py +foo.pyi +``` + +The generated target will be: + +```starlark +py_library( + name = "foo", + srcs = ["foo.py"], + pyi_srcs = ["foo.pyi"], +) +``` + + +(directive-python-generate-proto)= +## `python_generate_proto` + +:::{versionadded} 1.6.0 +{gh-pr}`3057` +::: + +When `# gazelle:python_generate_proto true`, Gazelle will generate one +{bzl:obj}`py_proto_library` for each `proto_library`, generating Python clients for +protobuf in each package. By default this is turned off. Gazelle will also +generate a load statement for the {bzl:obj}`py_proto_library` - attempting to detect +the configured name for the `@protobuf` / `@com_google_protobuf` repo in your +`MODULE.bazel`, and otherwise falling back to `@com_google_protobuf` for +compatibility with `WORKSPACE`. + +:::{note} +In order to use this, you must manually configure Gazelle to target multiple +languages. Place this in your root `BUILD.bazel` file: + +``` +load("@bazel_gazelle//:def.bzl", "gazelle", "gazelle_binary") + +gazelle_binary( + name = "gazelle_multilang", + languages = [ + "@bazel_gazelle//language/proto", + # The python gazelle plugin must be listed _after_ the proto language. + "@rules_python_gazelle_plugin//python", + ], +) + +gazelle( + name = "gazelle", + gazelle = "//:gazelle_multilang", +) +``` +::: + + +For example, in a package with `# gazelle:python_generate_proto true` and a +`foo.proto`, if you have both the proto extension and the Python extension +loaded into Gazelle, you'll get something like: + +```starlark +load("@protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_py_pb2", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) +``` + +When `false`, Gazelle will ignore any {bzl:obj}`py_proto_library`, including +previously-generated or hand-created rules. + + +(directive-python-resolve-sibling-imports)= +## `python_resolve_sibling_imports` + +:::{versionadded} 1.6.0 +{gh-pr}`3106` +::: + +:::{error} +Detailed docs are not yet written. +::: + +(directive-python-include-ancestor-conftest)= +## `python_include_ancestor_conftest` + +:::{versionadded} 1.9.0 +{gh-pr}`3596` +::: + +Version 1.9.0 includes a fix ({gh-pr}`3498`) for a long-standing issue +({gh-issue}`3497`) where ancestor `conftest.py` files were not automatically +added as dependencies of {bzl:obj}`py_test` targets. + +However, some people may not want this behavior (see ). +Thus the `python_include_ancestor_conftest` directive controls this behavior. +It defaults to `true`, which causes all ancestor `conftest.py` files to be +included as dependencies for {bzl:obj}`py_test` targets. + +Setting the directive to `false` reverts to the pre-1.9.0 behavior. + +For example, given this directory tree (not shown: intermediary `BUILD.bazel` +files) + +``` +./ +├── conftest.py +└── one/ + ├── conftest.py + └── two/ + ├── conftest.py + └── three/ + ├── BUILD.bazel + ├── conftest.py + └── my_test.py +``` + +Gazelle will generate this target for `foo_test.py` by default: + +```starlark +py_test( + name = "foo_test", + srcs = ["foo_test.py"], + deps = [ + ":conftest", # same as "//one:two/three:conftest" + "//:conftest", + "//one:conftest", + "//one/two:conftest", + ], +) +``` + +But when `python_include_ancestor_conftest` is `false`, only the sibling +`:conftest` target will be included as a dependency: + +:::{tip} +The [`include_pytest_conftest` annotation](annotation-include-pytest-conftest) +controls whether the sibling `:conftest` target is added to {bzl:obj}`py_test` +target dependency list. +::: + +```starlark +# gazelle:python_include_ancestor_conftest false +py_test( + name = "foo_test", + srcs = ["foo_test.py"], + deps = [ + ":conftest", + ], +) +``` diff --git a/gazelle/docs/index.md b/gazelle/docs/index.md new file mode 100644 index 0000000000..f276b0ca16 --- /dev/null +++ b/gazelle/docs/index.md @@ -0,0 +1,49 @@ +# Gazelle Plugin + +[Gazelle][gazelle] is a build file generator for Bazel projects. It can +create new `BUILD` or `BUILD.bazel` files for a project that +follows language conventions and update existing build files to include new +sources, dependencies, and options. + +[gazelle]: https://github.com/bazel-contrib/bazel-gazelle + +Bazel may run Gazelle using the Gazelle rule, or Gazelle may be installed and run +as a command line tool. + +The {gh-path}`gazelle` directory contains a plugin for Gazelle +that generates `BUILD` files content for Python code. When Gazelle is +run as a command line tool with this plugin, it embeds a Python interpreter +resolved during the plugin build. The behavior of the plugin is slightly +different with different version of the interpreter as the Python +`stdlib` changes with every minor version release. Distributors of Gazelle +binaries should, therefore, build a Gazelle binary for each OS+CPU +architecture+Minor Python version combination they are targeting. + +:::{note} +These instructions are for when you use [bzlmod][bzlmod]. Please refer to +older documentation that includes instructions on how to use Gazelle +without using bzlmod as your dependency manager. +::: + +[bzlmod]: https://bazel.build/external/module + +Gazelle is non-destructive. It will try to leave your edits to `BUILD` +files alone, only making updates to `py_*` targets. However it **will +remove** dependencies that appear to be unused, so it's a good idea to check +in your work before running Gazelle so you can easily revert any changes it made. + +The `rules_python` extension assumes some conventions about your Python code. +These are noted in the subsequent documents, and might require changes to your +existing code. + +Note that the `gazelle` program has multiple commands. At present, only +the `update` command (the default) does anything for Python code. + + +```{toctree} +:maxdepth: 1 +installation_and_usage +directives +annotations +development +``` diff --git a/gazelle/docs/installation_and_usage.md b/gazelle/docs/installation_and_usage.md new file mode 100644 index 0000000000..c3a004310c --- /dev/null +++ b/gazelle/docs/installation_and_usage.md @@ -0,0 +1,234 @@ +# Installation and Usage + +## Example + +Examples of using Gazelle with Python can be found in the `rules_python` +repo: + +* bzlmod: {gh-path}`gazelle/examples/bzlmod_build_file_generation` +* WORKSPACE: {gh-path}`examples/build_file_generation` + +:::{note} +The following documentation covers using bzlmod. +::: + + +## Adding Gazelle to your project + +First, you'll need to add Gazelle to your `MODULE.bazel` file. Get the current +version of [Gazelle][bcr-gazelle] from the [Bazel Central Registry][bcr]. Then +do the same for [`rules_python`][bcr-rules-python] and +[`rules_python_gazelle_plugin`][bcr-rules-python-gazelle-plugin]. + +[bcr-gazelle]: https://registry.bazel.build/modules/gazelle +[bcr]: https://registry.bazel.build/ +[bcr-rules-python]: https://registry.bazel.build/modules/rules_python +[bcr-rules-python-gazelle-plugin]: https://registry.bazel.build/modules/rules_python_gazelle_plugin + +Here is a snippet of a `MODULE.bazel` file. Note that most of it is just +general config for `rules_python` itself - the Gazelle plugin is only two lines +at the end. + +```starlark +################################################ +## START rules_python CONFIG ## +## See the main rules_python docs for details ## +################################################ +bazel_dep(name = "rules_python", version = "1.5.1") + +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.toolchain(python_version = "3.12.2") +use_repo(python, "python_3_12_2") + +pip = use_extension("@rules_python//python:extensions.bzl", "pip") +pip.parse( + hub_name = "pip", + requirements_lock = "//:requirements_lock.txt", + requirements_windows = "//:requirements_windows.txt", +) +use_repo(pip, "pip") + +############################################## +## START rules_python_gazelle_plugin CONFIG ## +############################################## + +# The Gazelle plugin depends on Gazelle. +bazel_dep(name = "gazelle", version = "0.33.0", repo_name = "bazel_gazelle") + +# Typically rules_python_gazelle_plugin is version matched to rules_python. +bazel_dep(name = "rules_python_gazelle_plugin", version = "1.5.1") +``` + +Next, we'll fetch metadata about your Python dependencies, so that gazelle can +determine which package a given import statement comes from. This is provided +by the `modules_mapping` rule. We'll make a target for consuming this +`modules_mapping`, and writing it as a manifest file for Gazelle to read. +This is checked into the repo for speed, as it takes some time to calculate +in a large monorepo. + +Gazelle will walk up the filesystem from a Python file to find this metadata, +looking for a file called `gazelle_python.yaml` in an ancestor folder +of the Python code. Create an empty file with this name. It might be next +to your `requirements.txt` file. (You can just use {command}`touch` at +this point, it just needs to exist.) + +To keep the metadata updated, put this in your `BUILD.bazel` file next +to `gazelle_python.yaml`: + +```starlark +# `@pip` is the hub_name from pip.parse in MODULE.bazel. +load("@pip//:requirements.bzl", "all_whl_requirements") +load("@rules_python_gazelle_plugin//manifest:defs.bzl", "gazelle_python_manifest") +load("@rules_python_gazelle_plugin//modules_mapping:def.bzl", "modules_mapping") + +# This rule fetches the metadata for python packages we depend on. That data is +# required for the gazelle_python_manifest rule to update our manifest file. +modules_mapping( + name = "modules_map", + wheels = all_whl_requirements, + + # include_stub_packages: bool (default: False) + # If set to True, this flag automatically includes any corresponding type stub packages + # for the third-party libraries that are present and used. For example, if you have + # `boto3` as a dependency, and this flag is enabled, the corresponding `boto3-stubs` + # package will be automatically included in the BUILD file. + # Enabling this feature helps ensure that type hints and stubs are readily available + # for tools like type checkers and IDEs, improving the development experience and + # reducing manual overhead in managing separate stub packages. + include_stub_packages = True, +) + +# Gazelle python extension needs a manifest file mapping from +# an import to the installed package that provides it. +# This macro produces two targets: +# - //:gazelle_python_manifest.update can be used with `bazel run` +# to recalculate the manifest +# - //:gazelle_python_manifest.test is a test target ensuring that +# the manifest doesn't need to be updated +gazelle_python_manifest( + name = "gazelle_python_manifest", + modules_mapping = ":modules_map", + + # This is what we called our `pip.parse` rule in MODULE.bazel, where third-party + # python libraries are loaded in BUILD files. + pip_repository_name = "pip", + + # This should point to wherever we declare our python dependencies + # (the same as what we passed to the modules_mapping rule in WORKSPACE) + # This argument is optional. If provided, the `.test` target is very + # fast because it just has to check an integrity field. If not provided, + # the integrity field is not added to the manifest which can help avoid + # merge conflicts in large repos. + requirements = "//:requirements_lock.txt", +) +``` + +Finally, you create a target that you'll invoke to run the Gazelle tool +with the `rules_python` extension included. This typically goes in your root +`/BUILD.bazel` file: + +```starlark +load("@bazel_gazelle//:def.bzl", "gazelle", "gazelle_binary") + +gazelle_binary( + name = "gazelle_multilang", + languages = [ + # List of language plugins. + # If you want to generate py_proto_library targets (PR #3057), then + # the proto language plugin _must_ come before the rules_python plugin. + #"@bazel_gazelle//language/proto", + "@rules_python_gazelle_plugin//python", + ], +) + +gazelle( + name = "gazelle", + gazelle = ":gazelle_multilang", +) +``` + +That's it, now you can finally run `bazel run //:gazelle` anytime +you edit Python code, and it should update your `BUILD` files correctly. + + +## Target Types and How They're Generated + +### Libraries + +Python source files are those ending in `.py` that are not matched as a test +file via the {term}`# gazelle:python_test_file_pattern value` directive. By default, +python source files are all `*.py` files except for `*_test.py` and +`test_*.py`. + +First, we look for the nearest ancestor `BUILD(.bazel)` file starting from +the folder containing the Python source file. + ++ In `package` generation mode, if there is no {bzl:obj}`py_library` in this + `BUILD(.bazel)` file, one is created using the package name as the target's + name. This makes it the default target in the package. Next, all source + files are collected into the `srcs` of the {bzl:obj}`py_library`. ++ In `project` generation mode, all source files in subdirectories (that don't + have `BUILD(.bazel)` files) are also collected. ++ In `file` generation mode, each python source file is given its own target. + +Finally, the `import` statements in the source files are parsed and +dependencies are added to the `deps` attribute of the target. + + +### Tests + +A {bzl:obj}`py_test` target is added to the `BUILD(.bazel)` file when gazelle +encounters a file named `__test__.py` or when files matching the +{term}`# gazelle:python_test_file_pattern value` directive are found. + +For example, if we had a folder that is a package named "foo" we could have a +Python file named `foo_test.py` and gazelle would create a {bzl:obj}`py_test` +block for the file. + +The following is an example of a {bzl:obj}`py_test` target that gazelle would +add when it encounters a file named `__test__.py`. + +```starlark +py_test( + name = "build_file_generation_test", + srcs = ["__test__.py"], + main = "__test__.py", + deps = [":build_file_generation"], +) +``` + +You can control the naming convention for test targets using the +{term}`# gazelle:python_test_naming_convention value` directive. + + +### Binaries + +When a `__main__.py` file is encountered, this indicates the entry point +of a Python program. A {bzl:obj}`py_binary` target will be created, named +`[package]_bin`. + +When no such entry point exists, Gazelle will look for a line like this in +the top level in every module: + +```python +if __name == "__main__": +``` + +Gazelle will create a {bzl:obj}`py_binary` target for every module with such +a line, with the target name the same as the module name. + +If the {term}`# gazelle:python_generation_mode value` directive is set to `file`, then +instead of one {bzl:obj}`py_binary` target per module, Gazelle will create +one {bzl:obj}`py_binary` target for each file with such a line, and the name +of the target will match the name of the script. + +:::{note} +It's possible for another script to depend on a {bzl:obj}`py_binary` target +and import from the {bzl:obj}`py_binary`'s scripts. This can have possible +negative effects on Bazel analysis time and runfiles size compared to +depending on a {bzl:obj}`py_library` target. The simplest way to avoid these +negative effects is to extract library code into a separate script without a +`main` line. Gazelle will then create a {bzl:obj}`py_library` target for +that library code, and other scripts can depend on that {bzl:obj}`py_library` +target. +::: diff --git a/gazelle/downloader_config.cfg b/gazelle/downloader_config.cfg new file mode 100644 index 0000000000..3fa6264eda --- /dev/null +++ b/gazelle/downloader_config.cfg @@ -0,0 +1,21 @@ +# Try GitHub first (primary) +rewrite ^github\.com/bazel-contrib/bazel_features/(.*) github.com/bazel-contrib/bazel_features/$1 +rewrite ^github\.com/bazel-contrib/rules_go/(.*) github.com/bazel-contrib/rules_go/$1 +rewrite ^github\.com/bazelbuild/bazel-skylib/(.*) github.com/bazelbuild/bazel-skylib/$1 +rewrite ^github\.com/bazelbuild/platforms/(.*) github.com/bazelbuild/platforms/$1 +rewrite ^github\.com/bazelbuild/rules_kotlin/(.*) github.com/bazelbuild/rules_kotlin/$1 +rewrite ^github\.com/bazelbuild/rules_shell/(.*) github.com/bazelbuild/rules_shell/$1 +rewrite ^github\.com/bazelbuild/rules_java/(.*) github.com/bazelbuild/rules_java/$1 +rewrite ^github\.com/bazelbuild/stardoc/(.*) github.com/bazelbuild/stardoc/$1 + + +# Fall back to mirror (secondary) +# Tracking upstream BCR mirror addition: https://github.com/bazelbuild/platforms/issues/139 +rewrite ^github\.com/bazel-contrib/bazel_features/(.*) mirror.bazel.build/github.com/bazel-contrib/bazel_features/$1 +rewrite ^github\.com/bazel-contrib/rules_go/(.*) mirror.bazel.build/github.com/bazel-contrib/rules_go/$1 +rewrite ^github\.com/bazelbuild/bazel-skylib/(.*) mirror.bazel.build/github.com/bazelbuild/bazel-skylib/$1 +rewrite ^github\.com/bazelbuild/platforms/(.*) mirror.bazel.build/github.com/bazelbuild/platforms/$1 +rewrite ^github\.com/bazelbuild/rules_kotlin/(.*) mirror.bazel.build/github.com/bazelbuild/rules_kotlin/$1 +rewrite ^github\.com/bazelbuild/rules_shell/(.*) mirror.bazel.build/github.com/bazelbuild/rules_shell/$1 +rewrite ^github\.com/bazelbuild/rules_java/(.*) mirror.bazel.build/github.com/bazelbuild/rules_java/$1 +rewrite ^github\.com/bazelbuild/stardoc/(.*) mirror.bazel.build/github.com/bazelbuild/stardoc/$1 diff --git a/examples/bzlmod_build_file_generation/.bazelignore b/gazelle/examples/bzlmod_build_file_generation/.bazelignore similarity index 100% rename from examples/bzlmod_build_file_generation/.bazelignore rename to gazelle/examples/bzlmod_build_file_generation/.bazelignore diff --git a/gazelle/examples/bzlmod_build_file_generation/.bazelrc b/gazelle/examples/bzlmod_build_file_generation/.bazelrc new file mode 100644 index 0000000000..31097b41de --- /dev/null +++ b/gazelle/examples/bzlmod_build_file_generation/.bazelrc @@ -0,0 +1,16 @@ +test --test_output=errors --enable_runfiles + +# Windows requires these for multi-python support: +build --enable_runfiles + +common --enable_bzlmod + +coverage --java_runtime_version=remotejdk_11 +common:bazel7.x --incompatible_python_disallow_native_rules + +# NOTE: This override is specific to the development of gazelle itself +# and the testing of it during its BCR release presubmits. +# In development of gazelle itself, we override it to the development +# rules_python code. In the BCR presubmits, this override is removed +# and the bazel_dep version of rules_python is used. +common --override_module=rules_python=../../../ diff --git a/examples/bzlmod_build_file_generation/.gitignore b/gazelle/examples/bzlmod_build_file_generation/.gitignore similarity index 100% rename from examples/bzlmod_build_file_generation/.gitignore rename to gazelle/examples/bzlmod_build_file_generation/.gitignore diff --git a/examples/bzlmod_build_file_generation/BUILD.bazel b/gazelle/examples/bzlmod_build_file_generation/BUILD.bazel similarity index 75% rename from examples/bzlmod_build_file_generation/BUILD.bazel rename to gazelle/examples/bzlmod_build_file_generation/BUILD.bazel index 33d01f4119..5ab2790e04 100644 --- a/examples/bzlmod_build_file_generation/BUILD.bazel +++ b/gazelle/examples/bzlmod_build_file_generation/BUILD.bazel @@ -7,8 +7,10 @@ # requirements. load("@bazel_gazelle//:def.bzl", "gazelle") load("@pip//:requirements.bzl", "all_whl_requirements") -load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test") load("@rules_python//python:pip.bzl", "compile_pip_requirements") +load("@rules_python//python:py_binary.bzl", "py_binary") +load("@rules_python//python:py_library.bzl", "py_library") +load("@rules_python//python:py_test.bzl", "py_test") load("@rules_python_gazelle_plugin//manifest:defs.bzl", "gazelle_python_manifest") load("@rules_python_gazelle_plugin//modules_mapping:def.bzl", "modules_mapping") @@ -30,11 +32,26 @@ modules_mapping( "^_|(\\._)+", # This is the default. "(\\.tests)+", # Add a custom one to get rid of the psutil tests. "^colorama", # Get rid of colorama on Windows. + "^tzdata", # Get rid of tzdata on Windows. "^lazy_object_proxy\\.cext$", # Get rid of this on Linux because it isn't included on Windows. ], wheels = all_whl_requirements, ) +modules_mapping( + name = "modules_map_with_types", + exclude_patterns = [ + "^_|(\\._)+", # This is the default. + "(\\.tests)+", # Add a custom one to get rid of the psutil tests. + "^colorama", # Get rid of colorama on Windows. + "^tzdata", # Get rid of tzdata on Windows. + "^lazy_object_proxy\\.cext$", # Get rid of this on Linux because it isn't included on Windows. + ], + include_stub_packages = True, + modules_mapping_name = "modules_mapping_with_types.json", + wheels = all_whl_requirements, +) + # Gazelle python extension needs a manifest file mapping from # an import to the installed package that provides it. # This macro produces two targets: @@ -52,11 +69,19 @@ gazelle_python_manifest( tags = ["exclusive"], ) +gazelle_python_manifest( + name = "gazelle_python_manifest_with_types", + manifest = "gazelle_python_with_types.yaml", + modules_mapping = ":modules_map_with_types", + pip_repository_name = "pip", + tags = ["exclusive"], +) + # Our gazelle target points to the python gazelle binary. # This is the simple case where we only need one language supported. # If you also had proto, go, or other gazelle-supported languages, # you would also need a gazelle_binary rule. -# See https://github.com/bazelbuild/bazel-gazelle/blob/master/extend.rst#example +# See https://github.com/bazel-contrib/bazel-gazelle/blob/master/extend.md#example # This is the primary gazelle target to run, so that you can update BUILD.bazel files. # You can execute: # - bazel run //:gazelle update diff --git a/examples/bzlmod_build_file_generation/MODULE.bazel b/gazelle/examples/bzlmod_build_file_generation/MODULE.bazel similarity index 82% rename from examples/bzlmod_build_file_generation/MODULE.bazel rename to gazelle/examples/bzlmod_build_file_generation/MODULE.bazel index 30ad567879..81719be01f 100644 --- a/examples/bzlmod_build_file_generation/MODULE.bazel +++ b/gazelle/examples/bzlmod_build_file_generation/MODULE.bazel @@ -12,33 +12,26 @@ module( # The following stanza defines the dependency rules_python. # For typical setups you set the version. # See the releases page for available versions. -# https://github.com/bazelbuild/rules_python/releases -bazel_dep(name = "rules_python", version = "0.0.0") - -# The following loads rules_python from the file system. -# For usual setups you should remove this local_path_override block. -local_path_override( - module_name = "rules_python", - path = "../..", -) +# https://github.com/bazel-contrib/rules_python/releases +bazel_dep(name = "rules_python", version = "1.5.0") # The following stanza defines the dependency rules_python_gazelle_plugin. # For typical setups you set the version. # See the releases page for available versions. -# https://github.com/bazelbuild/rules_python/releases -bazel_dep(name = "rules_python_gazelle_plugin", version = "0.0.0") +# https://github.com/bazel-contrib/rules_python/releases +bazel_dep(name = "rules_python_gazelle_plugin", version = "1.5.0") # The following starlark loads the gazelle plugin from the file system. # For usual setups you should remove this local_path_override block. local_path_override( module_name = "rules_python_gazelle_plugin", - path = "../../gazelle", + path = "../..", ) # The following stanza defines the dependency for gazelle # See here https://github.com/bazelbuild/bazel-gazelle/releases/ for the # latest version. -bazel_dep(name = "gazelle", version = "0.30.0", repo_name = "bazel_gazelle") +bazel_dep(name = "gazelle", version = "0.47.0", repo_name = "bazel_gazelle") # The following stanze returns a proxy object representing a module extension; # its methods can be invoked to create module extension tags. @@ -46,9 +39,13 @@ python = use_extension("@rules_python//python/extensions:python.bzl", "python") # We next initialize the python toolchain using the extension. # You can set different Python versions in this block. +python.defaults( + # The environment variable takes precedence if set. + python_version = "3.9", + python_version_env = "BAZEL_PYTHON_VERSION", +) python.toolchain( configure_coverage_tool = True, - is_default = True, python_version = "3.9", ) @@ -80,11 +77,11 @@ use_repo(pip, "pip") # This project includes a different module that is on the local file system. # Add the module to this parent project. -bazel_dep(name = "other_module", version = "", repo_name = "our_other_module") +bazel_dep(name = "other_module", version = "0.0.0", repo_name = "our_other_module") local_path_override( module_name = "other_module", path = "other_module", ) # Only needed to make rules_python's CI happy -bazel_dep(name = "rules_java", version = "8.3.1") +bazel_dep(name = "rules_java", version = "8.16.1") diff --git a/examples/bzlmod_build_file_generation/README.md b/gazelle/examples/bzlmod_build_file_generation/README.md similarity index 100% rename from examples/bzlmod_build_file_generation/README.md rename to gazelle/examples/bzlmod_build_file_generation/README.md diff --git a/examples/bzlmod_build_file_generation/WORKSPACE b/gazelle/examples/bzlmod_build_file_generation/WORKSPACE similarity index 100% rename from examples/bzlmod_build_file_generation/WORKSPACE rename to gazelle/examples/bzlmod_build_file_generation/WORKSPACE diff --git a/examples/bzlmod_build_file_generation/__main__.py b/gazelle/examples/bzlmod_build_file_generation/__main__.py similarity index 100% rename from examples/bzlmod_build_file_generation/__main__.py rename to gazelle/examples/bzlmod_build_file_generation/__main__.py diff --git a/examples/bzlmod_build_file_generation/__test__.py b/gazelle/examples/bzlmod_build_file_generation/__test__.py similarity index 100% rename from examples/bzlmod_build_file_generation/__test__.py rename to gazelle/examples/bzlmod_build_file_generation/__test__.py diff --git a/examples/bzlmod_build_file_generation/gazelle_python.yaml b/gazelle/examples/bzlmod_build_file_generation/gazelle_python.yaml similarity index 84% rename from examples/bzlmod_build_file_generation/gazelle_python.yaml rename to gazelle/examples/bzlmod_build_file_generation/gazelle_python.yaml index d0d322446e..019b051092 100644 --- a/examples/bzlmod_build_file_generation/gazelle_python.yaml +++ b/gazelle/examples/bzlmod_build_file_generation/gazelle_python.yaml @@ -3,19 +3,24 @@ # To update this file, run: # bazel run //:gazelle_python_manifest.update +--- manifest: modules_mapping: S3: s3cmd + asgiref: asgiref astroid: astroid certifi: certifi chardet: chardet dateutil: python_dateutil dill: dill + django: Django + django_stubs_ext: django_stubs_ext idna: idna isort: isort lazy_object_proxy: lazy_object_proxy magic: python_magic mccabe: mccabe + mypy_django_plugin: django_stubs pathspec: pathspec pkg_resources: setuptools platformdirs: platformdirs @@ -23,6 +28,7 @@ manifest: requests: requests setuptools: setuptools six: six + sqlparse: sqlparse tabulate: tabulate tomli: tomli tomlkit: tomlkit diff --git a/gazelle/examples/bzlmod_build_file_generation/gazelle_python_with_types.yaml b/gazelle/examples/bzlmod_build_file_generation/gazelle_python_with_types.yaml new file mode 100644 index 0000000000..7632235aa0 --- /dev/null +++ b/gazelle/examples/bzlmod_build_file_generation/gazelle_python_with_types.yaml @@ -0,0 +1,43 @@ +# GENERATED FILE - DO NOT EDIT! +# +# To update this file, run: +# bazel run //:gazelle_python_manifest_with_types.update + +--- +manifest: + modules_mapping: + S3: s3cmd + asgiref: asgiref + astroid: astroid + certifi: certifi + chardet: chardet + dateutil: python_dateutil + dill: dill + django: Django + django_stubs: django_stubs + django_stubs_ext: django_stubs_ext + idna: idna + isort: isort + lazy_object_proxy: lazy_object_proxy + magic: python_magic + mccabe: mccabe + pathspec: pathspec + pkg_resources: setuptools + platformdirs: platformdirs + pylint: pylint + requests: requests + setuptools: setuptools + six: six + sqlparse: sqlparse + tabulate: tabulate + tomli: tomli + tomlkit: tomlkit + types_pyyaml: types_pyyaml + types_tabulate: types_tabulate + typing_extensions: typing_extensions + urllib3: urllib3 + wrapt: wrapt + yaml: PyYAML + yamllint: yamllint + pip_repository: + name: pip diff --git a/examples/bzlmod_build_file_generation/lib.py b/gazelle/examples/bzlmod_build_file_generation/lib.py similarity index 100% rename from examples/bzlmod_build_file_generation/lib.py rename to gazelle/examples/bzlmod_build_file_generation/lib.py diff --git a/gazelle/examples/bzlmod_build_file_generation/other_module/MODULE.bazel b/gazelle/examples/bzlmod_build_file_generation/other_module/MODULE.bazel new file mode 100644 index 0000000000..3deeeb6ccc --- /dev/null +++ b/gazelle/examples/bzlmod_build_file_generation/other_module/MODULE.bazel @@ -0,0 +1,7 @@ +module( + name = "other_module", +) + +# Version doesn't matter because the root module overrides it, +# but Bazel requires it exist in the registry. +bazel_dep(name = "rules_python", version = "1.0.0") diff --git a/examples/bzlmod_build_file_generation/other_module/WORKSPACE b/gazelle/examples/bzlmod_build_file_generation/other_module/WORKSPACE similarity index 100% rename from examples/bzlmod_build_file_generation/other_module/WORKSPACE rename to gazelle/examples/bzlmod_build_file_generation/other_module/WORKSPACE diff --git a/examples/bzlmod_build_file_generation/other_module/other_module/pkg/BUILD.bazel b/gazelle/examples/bzlmod_build_file_generation/other_module/other_module/pkg/BUILD.bazel similarity index 77% rename from examples/bzlmod_build_file_generation/other_module/other_module/pkg/BUILD.bazel rename to gazelle/examples/bzlmod_build_file_generation/other_module/other_module/pkg/BUILD.bazel index 9a130e3554..90d41e752e 100644 --- a/examples/bzlmod_build_file_generation/other_module/other_module/pkg/BUILD.bazel +++ b/gazelle/examples/bzlmod_build_file_generation/other_module/other_module/pkg/BUILD.bazel @@ -1,4 +1,4 @@ -load("@rules_python//python:defs.bzl", "py_library") +load("@rules_python//python:py_library.bzl", "py_library") py_library( name = "lib", diff --git a/examples/bzlmod_build_file_generation/other_module/other_module/pkg/data/data.txt b/gazelle/examples/bzlmod_build_file_generation/other_module/other_module/pkg/data/data.txt similarity index 100% rename from examples/bzlmod_build_file_generation/other_module/other_module/pkg/data/data.txt rename to gazelle/examples/bzlmod_build_file_generation/other_module/other_module/pkg/data/data.txt diff --git a/examples/bzlmod_build_file_generation/other_module/other_module/pkg/lib.py b/gazelle/examples/bzlmod_build_file_generation/other_module/other_module/pkg/lib.py similarity index 100% rename from examples/bzlmod_build_file_generation/other_module/other_module/pkg/lib.py rename to gazelle/examples/bzlmod_build_file_generation/other_module/other_module/pkg/lib.py diff --git a/examples/bzlmod_build_file_generation/requirements.in b/gazelle/examples/bzlmod_build_file_generation/requirements.in similarity index 74% rename from examples/bzlmod_build_file_generation/requirements.in rename to gazelle/examples/bzlmod_build_file_generation/requirements.in index a709195442..fb3b45176c 100644 --- a/examples/bzlmod_build_file_generation/requirements.in +++ b/gazelle/examples/bzlmod_build_file_generation/requirements.in @@ -2,5 +2,8 @@ requests~=2.25.1 s3cmd~=2.1.0 yamllint>=1.28.0 tabulate~=0.9.0 +types-tabulate pylint~=2.15.5 python-dateutil>=2.8.2 +django +django-stubs diff --git a/examples/bzlmod_build_file_generation/requirements_lock.txt b/gazelle/examples/bzlmod_build_file_generation/requirements_lock.txt similarity index 90% rename from examples/bzlmod_build_file_generation/requirements_lock.txt rename to gazelle/examples/bzlmod_build_file_generation/requirements_lock.txt index 9d9ad9453e..5c1b7a86e8 100644 --- a/examples/bzlmod_build_file_generation/requirements_lock.txt +++ b/gazelle/examples/bzlmod_build_file_generation/requirements_lock.txt @@ -4,6 +4,12 @@ # # bazel run //:requirements.update # +asgiref==3.8.1 \ + --hash=sha256:3e1e3ecc849832fe52ccf2cb6686b7a55f82bb1d6aee72a58826471390335e47 \ + --hash=sha256:c343bd80a0bec947a9860adb4c432ffa7db769836c64238fc34bdc3fec84d590 + # via + # django + # django-stubs astroid==2.12.13 \ --hash=sha256:10e0ad5f7b79c435179d0d0f0df69998c4eef4597534aae44910db060baeb907 \ --hash=sha256:1493fe8bd3dfd73dc35bd53c9d5b6e49ead98497c47b2307662556a5692d29d7 @@ -20,6 +26,21 @@ dill==0.3.6 \ --hash=sha256:a07ffd2351b8c678dfc4a856a3005f8067aea51d6ba6c700796a4d9e280f39f0 \ --hash=sha256:e5db55f3687856d8fbdab002ed78544e1c4559a130302693d839dfe8f93f2373 # via pylint +django==4.2.20 \ + --hash=sha256:213381b6e4405f5c8703fffc29cd719efdf189dec60c67c04f76272b3dc845b9 \ + --hash=sha256:92bac5b4432a64532abb73b2ac27203f485e40225d2640a7fbef2b62b876e789 + # via + # -r requirements.in + # django-stubs + # django-stubs-ext +django-stubs==5.0.0 \ + --hash=sha256:084484cbe16a6d388e80ec687e46f529d67a232f3befaf55c936b3b476be289d \ + --hash=sha256:b8a792bee526d6cab31e197cb414ee7fa218abd931a50948c66a80b3a2548621 + # via -r requirements.in +django-stubs-ext==5.1.1 \ + --hash=sha256:3907f99e178c93323e2ce908aef8352adb8c047605161f8d9e5e7b4efb5a6a9c \ + --hash=sha256:db7364e4f50ae7e5360993dbd58a3a57ea4b2e7e5bab0fbd525ccdb3e7975d1c + # via django-stubs idna==2.10 \ --hash=sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6 \ --hash=sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0 @@ -129,6 +150,10 @@ six==1.16.0 \ --hash=sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926 \ --hash=sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254 # via python-dateutil +sqlparse==0.5.2 \ + --hash=sha256:9e37b35e16d1cc652a2545f0997c1deb23ea28fa1f3eefe609eee3063c3b105f \ + --hash=sha256:e99bc85c78160918c3e1d9230834ab8d80fc06c59d03f8db2618f65f65dda55e + # via django tabulate==0.9.0 \ --hash=sha256:0095b12bf5966de529c0feb1fa08671671b3368eec77d7ef7ab114be2c068b3c \ --hash=sha256:024ca478df22e9340661486f85298cff5f6dcdba14f3813e8830015b9ed1948f @@ -136,16 +161,29 @@ tabulate==0.9.0 \ tomli==2.0.1 \ --hash=sha256:939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc \ --hash=sha256:de526c12914f0c550d15924c62d72abc48d6fe7364aa87328337a31007fe8a4f - # via pylint + # via + # django-stubs + # pylint tomlkit==0.11.6 \ --hash=sha256:07de26b0d8cfc18f871aec595fda24d95b08fef89d147caa861939f37230bf4b \ --hash=sha256:71b952e5721688937fb02cf9d354dbcf0785066149d2855e44531ebdd2b65d73 # via pylint +types-pyyaml==6.0.12.20240917 \ + --hash=sha256:392b267f1c0fe6022952462bf5d6523f31e37f6cea49b14cee7ad634b6301570 \ + --hash=sha256:d1405a86f9576682234ef83bcb4e6fff7c9305c8b1fbad5e0bcd4f7dbdc9c587 + # via django-stubs +types-tabulate==0.9.0.20240106 \ + --hash=sha256:0378b7b6fe0ccb4986299496d027a6d4c218298ecad67199bbd0e2d7e9d335a1 \ + --hash=sha256:c9b6db10dd7fcf55bd1712dd3537f86ddce72a08fd62bb1af4338c7096ce947e + # via -r requirements.in typing-extensions==4.4.0 \ --hash=sha256:1511434bb92bf8dd198c12b1cc812e800d4181cfcb867674e0f8279cc93087aa \ --hash=sha256:16fa4864408f655d35ec496218b85f79b3437c829e93320c7c9215ccfd92489e # via + # asgiref # astroid + # django-stubs + # django-stubs-ext # pylint urllib3==1.26.13 \ --hash=sha256:47cc05d99aaa09c9e72ed5809b60e7ba354e64b59c9c173ac3018642d8bb41fc \ diff --git a/examples/bzlmod_build_file_generation/requirements_windows.txt b/gazelle/examples/bzlmod_build_file_generation/requirements_windows.txt similarity index 83% rename from examples/bzlmod_build_file_generation/requirements_windows.txt rename to gazelle/examples/bzlmod_build_file_generation/requirements_windows.txt index 5b31ff5541..309dfbcf40 100644 --- a/examples/bzlmod_build_file_generation/requirements_windows.txt +++ b/gazelle/examples/bzlmod_build_file_generation/requirements_windows.txt @@ -4,6 +4,12 @@ # # bazel run //:requirements.update # +asgiref==3.8.1 \ + --hash=sha256:3e1e3ecc849832fe52ccf2cb6686b7a55f82bb1d6aee72a58826471390335e47 \ + --hash=sha256:c343bd80a0bec947a9860adb4c432ffa7db769836c64238fc34bdc3fec84d590 + # via + # django + # django-stubs astroid==2.12.13 \ --hash=sha256:10e0ad5f7b79c435179d0d0f0df69998c4eef4597534aae44910db060baeb907 \ --hash=sha256:1493fe8bd3dfd73dc35bd53c9d5b6e49ead98497c47b2307662556a5692d29d7 @@ -24,6 +30,21 @@ dill==0.3.6 \ --hash=sha256:a07ffd2351b8c678dfc4a856a3005f8067aea51d6ba6c700796a4d9e280f39f0 \ --hash=sha256:e5db55f3687856d8fbdab002ed78544e1c4559a130302693d839dfe8f93f2373 # via pylint +django==4.2.20 \ + --hash=sha256:213381b6e4405f5c8703fffc29cd719efdf189dec60c67c04f76272b3dc845b9 \ + --hash=sha256:92bac5b4432a64532abb73b2ac27203f485e40225d2640a7fbef2b62b876e789 + # via + # -r requirements.in + # django-stubs + # django-stubs-ext +django-stubs==5.1.1 \ + --hash=sha256:126d354bbdff4906c4e93e6361197f6fbfb6231c3df6def85a291dae6f9f577b \ + --hash=sha256:c4dc64260bd72e6d32b9e536e8dd0d9247922f0271f82d1d5132a18f24b388ac + # via -r requirements.in +django-stubs-ext==5.1.1 \ + --hash=sha256:3907f99e178c93323e2ce908aef8352adb8c047605161f8d9e5e7b4efb5a6a9c \ + --hash=sha256:db7364e4f50ae7e5360993dbd58a3a57ea4b2e7e5bab0fbd525ccdb3e7975d1c + # via django-stubs idna==2.10 \ --hash=sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6 \ --hash=sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0 @@ -133,6 +154,10 @@ six==1.16.0 \ --hash=sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926 \ --hash=sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254 # via python-dateutil +sqlparse==0.5.2 \ + --hash=sha256:9e37b35e16d1cc652a2545f0997c1deb23ea28fa1f3eefe609eee3063c3b105f \ + --hash=sha256:e99bc85c78160918c3e1d9230834ab8d80fc06c59d03f8db2618f65f65dda55e + # via django tabulate==0.9.0 \ --hash=sha256:0095b12bf5966de529c0feb1fa08671671b3368eec77d7ef7ab114be2c068b3c \ --hash=sha256:024ca478df22e9340661486f85298cff5f6dcdba14f3813e8830015b9ed1948f @@ -140,17 +165,34 @@ tabulate==0.9.0 \ tomli==2.0.1 \ --hash=sha256:939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc \ --hash=sha256:de526c12914f0c550d15924c62d72abc48d6fe7364aa87328337a31007fe8a4f - # via pylint + # via + # django-stubs + # pylint tomlkit==0.11.6 \ --hash=sha256:07de26b0d8cfc18f871aec595fda24d95b08fef89d147caa861939f37230bf4b \ --hash=sha256:71b952e5721688937fb02cf9d354dbcf0785066149d2855e44531ebdd2b65d73 # via pylint -typing-extensions==4.4.0 \ - --hash=sha256:1511434bb92bf8dd198c12b1cc812e800d4181cfcb867674e0f8279cc93087aa \ - --hash=sha256:16fa4864408f655d35ec496218b85f79b3437c829e93320c7c9215ccfd92489e +types-pyyaml==6.0.12.20240917 \ + --hash=sha256:392b267f1c0fe6022952462bf5d6523f31e37f6cea49b14cee7ad634b6301570 \ + --hash=sha256:d1405a86f9576682234ef83bcb4e6fff7c9305c8b1fbad5e0bcd4f7dbdc9c587 + # via django-stubs +types-tabulate==0.9.0.20240106 \ + --hash=sha256:0378b7b6fe0ccb4986299496d027a6d4c218298ecad67199bbd0e2d7e9d335a1 \ + --hash=sha256:c9b6db10dd7fcf55bd1712dd3537f86ddce72a08fd62bb1af4338c7096ce947e + # via -r requirements.in +typing-extensions==4.12.2 \ + --hash=sha256:04e5ca0351e0f3f85c6853954072df659d0d13fac324d0072316b67d7794700d \ + --hash=sha256:1a7ead55c7e559dd4dee8856e3a88b41225abfe1ce8df57b7c13915fe121ffb8 # via + # asgiref # astroid + # django-stubs + # django-stubs-ext # pylint +tzdata==2024.2 \ + --hash=sha256:7d85cc416e9382e69095b7bdf4afd9e3880418a2413feec7069d533d6b4e31cc \ + --hash=sha256:a48093786cdcde33cad18c2555e8532f34422074448fbc874186f0abd79565cd + # via django urllib3==1.26.13 \ --hash=sha256:47cc05d99aaa09c9e72ed5809b60e7ba354e64b59c9c173ac3018642d8bb41fc \ --hash=sha256:c083dd0dce68dbfbe1129d5271cb90f9447dea7d52097c6e0126120c521ddea8 @@ -162,23 +204,30 @@ wrapt==1.14.1 \ --hash=sha256:07f7a7d0f388028b2df1d916e94bbb40624c59b48ecc6cbc232546706fac74c2 \ --hash=sha256:11871514607b15cfeb87c547a49bca19fde402f32e2b1c24a632506c0a756656 \ --hash=sha256:1b376b3f4896e7930f1f772ac4b064ac12598d1c38d04907e696cc4d794b43d3 \ + --hash=sha256:2020f391008ef874c6d9e208b24f28e31bcb85ccff4f335f15a3251d222b92d9 \ --hash=sha256:21ac0156c4b089b330b7666db40feee30a5d52634cc4560e1905d6529a3897ff \ + --hash=sha256:240b1686f38ae665d1b15475966fe0472f78e71b1b4903c143a842659c8e4cb9 \ --hash=sha256:257fd78c513e0fb5cdbe058c27a0624c9884e735bbd131935fd49e9fe719d310 \ + --hash=sha256:26046cd03936ae745a502abf44dac702a5e6880b2b01c29aea8ddf3353b68224 \ --hash=sha256:2b39d38039a1fdad98c87279b48bc5dce2c0ca0d73483b12cb72aa9609278e8a \ --hash=sha256:2cf71233a0ed05ccdabe209c606fe0bac7379fdcf687f39b944420d2a09fdb57 \ --hash=sha256:2fe803deacd09a233e4762a1adcea5db5d31e6be577a43352936179d14d90069 \ + --hash=sha256:2feecf86e1f7a86517cab34ae6c2f081fd2d0dac860cb0c0ded96d799d20b335 \ --hash=sha256:3232822c7d98d23895ccc443bbdf57c7412c5a65996c30442ebe6ed3df335383 \ --hash=sha256:34aa51c45f28ba7f12accd624225e2b1e5a3a45206aa191f6f9aac931d9d56fe \ + --hash=sha256:358fe87cc899c6bb0ddc185bf3dbfa4ba646f05b1b0b9b5a27c2cb92c2cea204 \ --hash=sha256:36f582d0c6bc99d5f39cd3ac2a9062e57f3cf606ade29a0a0d6b323462f4dd87 \ --hash=sha256:380a85cf89e0e69b7cfbe2ea9f765f004ff419f34194018a6827ac0e3edfed4d \ --hash=sha256:40e7bc81c9e2b2734ea4bc1aceb8a8f0ceaac7c5299bc5d69e37c44d9081d43b \ --hash=sha256:43ca3bbbe97af00f49efb06e352eae40434ca9d915906f77def219b88e85d907 \ + --hash=sha256:49ef582b7a1152ae2766557f0550a9fcbf7bbd76f43fbdc94dd3bf07cc7168be \ --hash=sha256:4fcc4649dc762cddacd193e6b55bc02edca674067f5f98166d7713b193932b7f \ --hash=sha256:5a0f54ce2c092aaf439813735584b9537cad479575a09892b8352fea5e988dc0 \ --hash=sha256:5a9a0d155deafd9448baff28c08e150d9b24ff010e899311ddd63c45c2445e28 \ --hash=sha256:5b02d65b9ccf0ef6c34cba6cf5bf2aab1bb2f49c6090bafeecc9cd81ad4ea1c1 \ --hash=sha256:60db23fa423575eeb65ea430cee741acb7c26a1365d103f7b0f6ec412b893853 \ --hash=sha256:642c2e7a804fcf18c222e1060df25fc210b9c58db7c91416fb055897fc27e8cc \ + --hash=sha256:6447e9f3ba72f8e2b985a1da758767698efa72723d5b59accefd716e9e8272bf \ --hash=sha256:6a9a25751acb379b466ff6be78a315e2b439d4c94c1e99cb7266d40a537995d3 \ --hash=sha256:6b1a564e6cb69922c7fe3a678b9f9a3c54e72b469875aa8018f18b4d1dd1adf3 \ --hash=sha256:6d323e1554b3d22cfc03cd3243b5bb815a51f5249fdcbb86fda4bf62bab9e164 \ @@ -201,8 +250,10 @@ wrapt==1.14.1 \ --hash=sha256:9e0fd32e0148dd5dea6af5fee42beb949098564cc23211a88d799e434255a1f4 \ --hash=sha256:9f3e6f9e05148ff90002b884fbc2a86bd303ae847e472f44ecc06c2cd2fcdb2d \ --hash=sha256:a85d2b46be66a71bedde836d9e41859879cc54a2a04fad1191eb50c2066f6e9d \ + --hash=sha256:a9008dad07d71f68487c91e96579c8567c98ca4c3881b9b113bc7b33e9fd78b8 \ --hash=sha256:a9a52172be0b5aae932bef82a79ec0a0ce87288c7d132946d645eba03f0ad8a8 \ --hash=sha256:aa31fdcc33fef9eb2552cbcbfee7773d5a6792c137b359e82879c101e98584c5 \ + --hash=sha256:acae32e13a4153809db37405f5eba5bac5fbe2e2ba61ab227926a22901051c0a \ --hash=sha256:b014c23646a467558be7da3d6b9fa409b2c567d2110599b7cf9a0c5992b3b471 \ --hash=sha256:b21bb4c09ffabfa0e85e3a6b623e19b80e7acd709b9f91452b8297ace2a8ab00 \ --hash=sha256:b5901a312f4d14c59918c221323068fad0540e34324925c8475263841dbdfe68 \ @@ -217,6 +268,7 @@ wrapt==1.14.1 \ --hash=sha256:dee60e1de1898bde3b238f18340eec6148986da0455d8ba7848d50470a7a32fb \ --hash=sha256:e2f83e18fe2f4c9e7db597e988f72712c0c3676d337d8b101f6758107c42425b \ --hash=sha256:e3fb1677c720409d5f671e39bac6c9e0e422584e5f518bfd50aa4cbbea02433f \ + --hash=sha256:ecee4132c6cd2ce5308e21672015ddfed1ff975ad0ac8d27168ea82e71413f55 \ --hash=sha256:ee2b1b1769f6707a8a445162ea16dddf74285c3964f605877a20e38545c3c462 \ --hash=sha256:ee6acae74a2b91865910eef5e7de37dc6895ad96fa23603d1d27ea69df545015 \ --hash=sha256:ef3f72c9666bba2bab70d2a8b79f2c6d2c1a42a7f7e2b0ec83bb2f9e383950af diff --git a/examples/bzlmod_build_file_generation/runfiles/BUILD.bazel b/gazelle/examples/bzlmod_build_file_generation/runfiles/BUILD.bazel similarity index 90% rename from examples/bzlmod_build_file_generation/runfiles/BUILD.bazel rename to gazelle/examples/bzlmod_build_file_generation/runfiles/BUILD.bazel index 3503ac3017..8806668a3f 100644 --- a/examples/bzlmod_build_file_generation/runfiles/BUILD.bazel +++ b/gazelle/examples/bzlmod_build_file_generation/runfiles/BUILD.bazel @@ -1,4 +1,4 @@ -load("@rules_python//python:defs.bzl", "py_test") +load("@rules_python//python:py_test.bzl", "py_test") # gazelle:ignore py_test( diff --git a/examples/bzlmod_build_file_generation/runfiles/data/data.txt b/gazelle/examples/bzlmod_build_file_generation/runfiles/data/data.txt similarity index 100% rename from examples/bzlmod_build_file_generation/runfiles/data/data.txt rename to gazelle/examples/bzlmod_build_file_generation/runfiles/data/data.txt diff --git a/examples/bzlmod_build_file_generation/runfiles/runfiles_test.py b/gazelle/examples/bzlmod_build_file_generation/runfiles/runfiles_test.py similarity index 84% rename from examples/bzlmod_build_file_generation/runfiles/runfiles_test.py rename to gazelle/examples/bzlmod_build_file_generation/runfiles/runfiles_test.py index 5bfa5302ef..6ce4c2db37 100644 --- a/examples/bzlmod_build_file_generation/runfiles/runfiles_test.py +++ b/gazelle/examples/bzlmod_build_file_generation/runfiles/runfiles_test.py @@ -29,36 +29,36 @@ def testRunfilesWithRepoMapping(self): data_path = runfiles.Create().Rlocation( "example_bzlmod_build_file_generation/runfiles/data/data.txt" ) - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, example_bzlmod!") def testRunfileWithRlocationpath(self): data_rlocationpath = os.getenv("DATA_RLOCATIONPATH") data_path = runfiles.Create().Rlocation(data_rlocationpath) - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, example_bzlmod!") def testRunfileInOtherModuleWithOurRepoMapping(self): data_path = runfiles.Create().Rlocation( "our_other_module/other_module/pkg/data/data.txt" ) - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, other_module!") def testRunfileInOtherModuleWithItsRepoMapping(self): data_path = lib.GetRunfilePathWithRepoMapping() - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, other_module!") def testRunfileInOtherModuleWithCurrentRepository(self): data_path = lib.GetRunfilePathWithCurrentRepository() - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, other_module!") def testRunfileInOtherModuleWithRlocationpath(self): data_rlocationpath = os.getenv("OTHER_MODULE_DATA_RLOCATIONPATH") data_path = runfiles.Create().Rlocation(data_rlocationpath) - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, other_module!") diff --git a/gazelle/go.mod b/gazelle/go.mod index 4b65e71d67..9ad4951536 100644 --- a/gazelle/go.mod +++ b/gazelle/go.mod @@ -1,26 +1,28 @@ -module github.com/bazelbuild/rules_python/gazelle +module github.com/bazel-contrib/rules_python/gazelle -go 1.19 +go 1.22.9 + +toolchain go1.24.13 require ( - github.com/bazelbuild/bazel-gazelle v0.31.1 - github.com/bazelbuild/buildtools v0.0.0-20231103205921-433ea8554e82 - github.com/bazelbuild/rules_go v0.41.0 - github.com/bmatcuk/doublestar/v4 v4.6.1 + github.com/bazelbuild/bazel-gazelle v0.47.0 + github.com/bazelbuild/buildtools v0.0.0-20250930140053-2eb4fccefb52 + github.com/bazelbuild/rules_go v0.55.1 + github.com/bmatcuk/doublestar/v4 v4.9.1 github.com/emirpasic/gods v1.18.1 github.com/ghodss/yaml v1.0.0 - github.com/smacker/go-tree-sitter v0.0.0-20240422154435-0628b34cbf9c + github.com/smacker/go-tree-sitter v0.0.0-20240827094217-dd81d9e9be82 github.com/stretchr/testify v1.9.0 - golang.org/x/sync v0.2.0 + golang.org/x/sync v0.11.0 gopkg.in/yaml.v2 v2.4.0 ) require ( github.com/davecgh/go-spew v1.1.1 // indirect - github.com/google/go-cmp v0.5.9 // indirect + github.com/google/go-cmp v0.6.0 // indirect github.com/pmezard/go-difflib v1.0.0 // indirect - golang.org/x/mod v0.10.0 // indirect - golang.org/x/sys v0.8.0 // indirect - golang.org/x/tools v0.9.1 // indirect + golang.org/x/mod v0.23.0 // indirect + golang.org/x/sys v0.30.0 // indirect + golang.org/x/tools/go/vcs v0.1.0-deprecated // indirect gopkg.in/yaml.v3 v3.0.1 // indirect ) diff --git a/gazelle/go.sum b/gazelle/go.sum index 46e0127e8f..d5fc2b1af9 100644 --- a/gazelle/go.sum +++ b/gazelle/go.sum @@ -1,112 +1,36 @@ -cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= -github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= -github.com/bazelbuild/bazel-gazelle v0.31.1 h1:ROyUyUHzoEdvoOs1e0haxJx1l5EjZX6AOqiKdVlaBbg= -github.com/bazelbuild/bazel-gazelle v0.31.1/go.mod h1:Ul0pqz50f5wxz0QNzsZ+mrEu4AVAVJZEB5xLnHgIG9c= -github.com/bazelbuild/buildtools v0.0.0-20231103205921-433ea8554e82 h1:HTepWP/jhtWTC1gvK0RnvKCgjh4gLqiwaOwGozAXcbw= -github.com/bazelbuild/buildtools v0.0.0-20231103205921-433ea8554e82/go.mod h1:689QdV3hBP7Vo9dJMmzhoYIyo/9iMhEmHkJcnaPRCbo= -github.com/bazelbuild/rules_go v0.41.0 h1:JzlRxsFNhlX+g4drDRPhIaU5H5LnI978wdMJ0vK4I+k= -github.com/bazelbuild/rules_go v0.41.0/go.mod h1:TMHmtfpvyfsxaqfL9WnahCsXMWDMICTw7XeK9yVb+YU= -github.com/bmatcuk/doublestar/v4 v4.6.1 h1:FH9SifrbvJhnlQpztAx++wlkk70QBf0iBWDwNy7PA4I= -github.com/bmatcuk/doublestar/v4 v4.6.1/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc= -github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= -github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI= -github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI= -github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= -github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= -github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/bazelbuild/bazel-gazelle v0.47.0 h1:g3Rr1ZbkC1Pk20aOgBITxSD/efS1WbaSty5jC786Z3Q= +github.com/bazelbuild/bazel-gazelle v0.47.0/go.mod h1:8Ozf20jhv+in87nCUHdmUPPcVGTfKg/gotZ/hce3T+w= +github.com/bazelbuild/buildtools v0.0.0-20250930140053-2eb4fccefb52 h1:njQAmjTv/YHRm/0Lfv9DXHFZ4MdT2IA/RKHTnqZkgDw= +github.com/bazelbuild/buildtools v0.0.0-20250930140053-2eb4fccefb52/go.mod h1:PLNUetjLa77TCCziPsz0EI8a6CUxgC+1jgmWv0H25tg= +github.com/bazelbuild/rules_go v0.55.1 h1:cQYGcunY8myOB+0Ym6PGQRhc/milkRcNv0my3XgxaDU= +github.com/bazelbuild/rules_go v0.55.1/go.mod h1:T90Gpyq4HDFlsrvtQa2CBdHNJ2P4rAu/uUTmQbanzf0= +github.com/bmatcuk/doublestar/v4 v4.9.1 h1:X8jg9rRZmJd4yRy7ZeNDRnM+T3ZfHv15JiBJ/avrEXE= +github.com/bmatcuk/doublestar/v4 v4.9.1/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/emirpasic/gods v1.18.1 h1:FXtiHYKDGKCW2KzwZKx0iC0PQmdlorYgdFG9jPXJ1Bc= github.com/emirpasic/gods v1.18.1/go.mod h1:8tpGGwCnJ5H4r6BWwaV6OrWmMoPhUl5jm/FMNAnJvWQ= -github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= -github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= github.com/ghodss/yaml v1.0.0 h1:wQHKEahhL6wmXdzwWG11gIVCkOv05bNOh+Rxn0yngAk= github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04= -github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= -github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= -github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= -github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= -github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8= -github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA= -github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs= -github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w= -github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0= -github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8= -github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= -github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= -github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= -github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= -github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38= -github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= +github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= +github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= -github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= -github.com/smacker/go-tree-sitter v0.0.0-20240422154435-0628b34cbf9c h1:7QZKUmQfnxncZIJGyvX8M8YeMfn8kM10j3J/2KwVTN4= -github.com/smacker/go-tree-sitter v0.0.0-20240422154435-0628b34cbf9c/go.mod h1:q99oHDsbP0xRwmn7Vmob8gbSMNyvJ83OauXPSuHQuKE= -github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= -github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= -github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= -github.com/stretchr/testify v1.7.4/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= +github.com/smacker/go-tree-sitter v0.0.0-20240827094217-dd81d9e9be82 h1:6C8qej6f1bStuePVkLSFxoU22XBS165D3klxlzRg8F4= +github.com/smacker/go-tree-sitter v0.0.0-20240827094217-dd81d9e9be82/go.mod h1:xe4pgH49k4SsmkQq5OT8abwhWmnzkhpgnXeekbx2efw= github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg= github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= -go.starlark.net v0.0.0-20210223155950-e043a3d3c984/go.mod h1:t3mmBBPzAVvK0L0n1drDmrQsJ8FoIx4INCqVMTr/Zo0= -golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= -golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= -golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= -golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU= -golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= -golang.org/x/mod v0.10.0 h1:lFO9qtOdlre5W1jxS3r/4szv2/6iXxScdzjoBMXNhYk= -golang.org/x/mod v0.10.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs= -golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= -golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= -golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.2.0 h1:PUR+T4wwASmuSTYdKjYHI5TD22Wy5ogLU5qZCOLxBrI= -golang.org/x/sync v0.2.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.8.0 h1:EBmGv8NaZBZTWvrbjNoL6HVt+IVy3QDQpJs7VRIw3tU= -golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= -golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= -golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY= -golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= -golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= -golang.org/x/tools v0.9.1 h1:8WMNJAz3zrtPmnYC7ISf5dEn3MT0gY7jBJfw27yrrLo= -golang.org/x/tools v0.9.1/go.mod h1:owI94Op576fPu3cIGQeHs3joujW/2Oc6MtlxbF5dfNc= -golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= -golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= -google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= -google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= -google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc= -google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= -google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo= -google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= -google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg= -google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= -google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= -google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0= -google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM= -google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE= -google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo= -google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= -google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= -google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= -google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c= +golang.org/x/mod v0.23.0 h1:Zb7khfcRGKk+kqfxFaP5tZqCnDZMjC5VtUBs87Hr6QM= +golang.org/x/mod v0.23.0/go.mod h1:6SkKJ3Xj0I0BrPOZoBy3bdMptDDU9oJrpohJ3eWZ1fY= +golang.org/x/sync v0.11.0 h1:GGz8+XQP4FvTTrjZPzNKTMFtSXH80RAzG+5ghFPgK9w= +golang.org/x/sync v0.11.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= +golang.org/x/sys v0.30.0 h1:QjkSwP/36a20jFYWkSue1YwXzLmsV5Gfq7Eiy72C1uc= +golang.org/x/sys v0.30.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/tools/go/vcs v0.1.0-deprecated h1:cOIJqWBl99H1dH5LWizPa+0ImeeJq3t3cJjaeOWUAL4= +golang.org/x/tools/go/vcs v0.1.0-deprecated/go.mod h1:zUrvATBAvEI9535oC0yWYsLsHIV4Z7g63sNPVMtuBy8= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= -gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= -honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= -honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= diff --git a/gazelle/internal_dev_deps.bzl b/gazelle/internal_dev_deps.bzl new file mode 100644 index 0000000000..f05f5fbb88 --- /dev/null +++ b/gazelle/internal_dev_deps.bzl @@ -0,0 +1,47 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Module extension for internal dev_dependency=True setup.""" + +load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_file") + +def internal_dev_deps(): + """This extension creates internal rules_python_gazelle dev dependencies.""" + http_file( + name = "pytest", + downloaded_file_path = "pytest-8.3.3-py3-none-any.whl", + sha256 = "a6853c7375b2663155079443d2e45de913a911a11d669df02a50814944db57b2", + urls = [ + "https://files.pythonhosted.org/packages/6b/77/7440a06a8ead44c7757a64362dd22df5760f9b12dc5f11b6188cd2fc27a0/pytest-8.3.3-py3-none-any.whl", + ], + ) + http_file( + name = "django-types", + downloaded_file_path = "django_types-0.19.1-py3-none-any.whl", + sha256 = "b3f529de17f6374d41ca67232aa01330c531bbbaa3ac4097896f31ac33c96c30", + urls = [ + "https://files.pythonhosted.org/packages/25/cb/d088c67245a9d5759a08dbafb47e040ee436e06ee433a3cdc7f3233b3313/django_types-0.19.1-py3-none-any.whl", + ], + ) + +def _internal_dev_deps_impl(mctx): + _ = mctx # @unused + + # This wheel is purely here to validate the wheel extraction code. It's not + # intended for anything else. + internal_dev_deps() + +internal_dev_deps_extension = module_extension( + implementation = _internal_dev_deps_impl, + doc = "This extension creates internal rules_python_gazelle dev dependencies.", +) diff --git a/gazelle/manifest/BUILD.bazel b/gazelle/manifest/BUILD.bazel index 33b5a46947..68f4e1be30 100644 --- a/gazelle/manifest/BUILD.bazel +++ b/gazelle/manifest/BUILD.bazel @@ -1,3 +1,4 @@ +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_test") exports_files([ @@ -8,11 +9,11 @@ exports_files([ go_library( name = "manifest", srcs = ["manifest.go"], - importpath = "github.com/bazelbuild/rules_python/gazelle/manifest", + importpath = "github.com/bazel-contrib/rules_python/gazelle/manifest", visibility = ["//visibility:public"], deps = [ - "@com_github_emirpasic_gods//sets/treeset", - "@in_gopkg_yaml_v2//:yaml_v2", + "@com_github_emirpasic_gods//sets/treeset:go_default_library", + "@in_gopkg_yaml_v2//:go_default_library", ], ) @@ -32,3 +33,14 @@ filegroup( ], visibility = ["//:__pkg__"], ) + +bzl_library( + name = "defs", + srcs = ["defs.bzl"], + visibility = ["//visibility:public"], + deps = [ + "@bazel_skylib//rules:diff_test", + "@io_bazel_rules_go//go:def", + "@rules_python//python:defs_bzl", + ], +) diff --git a/gazelle/manifest/copy_to_source.py b/gazelle/manifest/copy_to_source.py index 4ebb958c3d..4342dee843 100644 --- a/gazelle/manifest/copy_to_source.py +++ b/gazelle/manifest/copy_to_source.py @@ -6,7 +6,6 @@ import os import shutil -import stat import sys from pathlib import Path @@ -20,7 +19,9 @@ def copy_to_source(generated_relative_path: Path, target_relative_path: Path) -> generated_absolute_path = Path.cwd() / generated_relative_path # Similarly, the target is relative to the source directory. - target_absolute_path = os.getenv("BUILD_WORKSPACE_DIRECTORY") / target_relative_path + target_absolute_path = ( + os.environ["BUILD_WORKSPACE_DIRECTORY"] / target_relative_path + ) print(f"Copying {generated_absolute_path} to {target_absolute_path}") target_absolute_path.parent.mkdir(parents=True, exist_ok=True) diff --git a/gazelle/manifest/defs.bzl b/gazelle/manifest/defs.bzl index 3a65bffec4..71a4d57156 100644 --- a/gazelle/manifest/defs.bzl +++ b/gazelle/manifest/defs.bzl @@ -79,7 +79,7 @@ def gazelle_python_manifest( update_args = [ "--manifest-generator-hash=$(execpath {})".format(manifest_generator_hash), - "--requirements=$(rootpath {})".format(requirements) if requirements else "--requirements=", + "--requirements=$(execpath {})".format(requirements) if requirements else "--requirements=", "--pip-repository-name={}".format(pip_repository_name), "--modules-mapping=$(execpath {})".format(modules_mapping), "--output=$(execpath {})".format(generated_manifest), @@ -117,9 +117,9 @@ def gazelle_python_manifest( if requirements: attrs = { "env": { - "_TEST_MANIFEST": "$(rootpath {})".format(manifest), + "_TEST_MANIFEST": "$(rlocationpath {})".format(manifest), "_TEST_MANIFEST_GENERATOR_HASH": "$(rlocationpath {})".format(manifest_generator_hash), - "_TEST_REQUIREMENTS": "$(rootpath {})".format(requirements), + "_TEST_REQUIREMENTS": "$(rlocationpath {})".format(requirements), }, "size": "small", } @@ -161,7 +161,7 @@ AllSourcesInfo = provider(fields = {"all_srcs": "All sources collected from the _rules_python_workspace = Label("@rules_python//:WORKSPACE") def _get_all_sources_impl(target, ctx): - is_rules_python = target.label.workspace_name == _rules_python_workspace.workspace_name + is_rules_python = target.label.repo_name == _rules_python_workspace.repo_name if not is_rules_python: # Avoid adding third-party dependency files to the checksum of the srcs. return AllSourcesInfo(all_srcs = depset()) @@ -184,6 +184,7 @@ def _sources_hash_impl(ctx): args.add_all(all_srcs) ctx.actions.run( outputs = [hash_file], + mnemonic = "PyGazelleManifestHash", inputs = all_srcs, arguments = [args], executable = ctx.executable._hasher, diff --git a/gazelle/manifest/generate/BUILD.bazel b/gazelle/manifest/generate/BUILD.bazel index 96248f4e08..77d2467cef 100644 --- a/gazelle/manifest/generate/BUILD.bazel +++ b/gazelle/manifest/generate/BUILD.bazel @@ -4,7 +4,7 @@ load("//manifest:defs.bzl", "sources_hash") go_library( name = "generate_lib", srcs = ["generate.go"], - importpath = "github.com/bazelbuild/rules_python/gazelle/manifest/generate", + importpath = "github.com/bazel-contrib/rules_python/gazelle/manifest/generate", visibility = ["//visibility:public"], deps = ["//manifest"], ) diff --git a/gazelle/manifest/generate/generate.go b/gazelle/manifest/generate/generate.go index 27cf2a21d8..52100713e3 100644 --- a/gazelle/manifest/generate/generate.go +++ b/gazelle/manifest/generate/generate.go @@ -28,7 +28,7 @@ import ( "os" "strings" - "github.com/bazelbuild/rules_python/gazelle/manifest" + "github.com/bazel-contrib/rules_python/gazelle/manifest" ) func main() { @@ -151,7 +151,7 @@ func writeOutput( } defer outputFile.Close() - if _, err := fmt.Fprintf(outputFile, "%s\n", header); err != nil { + if _, err := fmt.Fprintf(outputFile, "%s\n---\n", header); err != nil { return fmt.Errorf("failed to write output: %w", err) } diff --git a/gazelle/manifest/hasher/BUILD.bazel b/gazelle/manifest/hasher/BUILD.bazel index 2e7b125cc0..c6e3c4c29b 100644 --- a/gazelle/manifest/hasher/BUILD.bazel +++ b/gazelle/manifest/hasher/BUILD.bazel @@ -3,7 +3,7 @@ load("@io_bazel_rules_go//go:def.bzl", "go_binary", "go_library") go_library( name = "hasher_lib", srcs = ["main.go"], - importpath = "github.com/bazelbuild/rules_python/gazelle/manifest/hasher", + importpath = "github.com/bazel-contrib/rules_python/gazelle/manifest/hasher", visibility = ["//visibility:private"], ) diff --git a/gazelle/manifest/manifest.go b/gazelle/manifest/manifest.go index 26b0dfb394..c5cd8a7d69 100644 --- a/gazelle/manifest/manifest.go +++ b/gazelle/manifest/manifest.go @@ -70,6 +70,9 @@ func (f *File) VerifyIntegrity(manifestGeneratorHashFile, requirements io.Reader return false, fmt.Errorf("failed to verify integrity: %w", err) } valid := (f.Integrity == fmt.Sprintf("%x", integrityBytes)) + if (!valid) { + fmt.Printf("WARN: Integrity hash was %v but expected %x\n", f.Integrity, integrityBytes) + } return valid, nil } diff --git a/gazelle/manifest/manifest_test.go b/gazelle/manifest/manifest_test.go index e80c7fcccc..320361a8e1 100644 --- a/gazelle/manifest/manifest_test.go +++ b/gazelle/manifest/manifest_test.go @@ -22,7 +22,7 @@ import ( "strings" "testing" - "github.com/bazelbuild/rules_python/gazelle/manifest" + "github.com/bazel-contrib/rules_python/gazelle/manifest" ) var modulesMapping = manifest.ModulesMapping{ diff --git a/gazelle/manifest/test/test.go b/gazelle/manifest/test/test.go index a7647f3f7c..77b354b495 100644 --- a/gazelle/manifest/test/test.go +++ b/gazelle/manifest/test/test.go @@ -26,23 +26,32 @@ import ( "path/filepath" "testing" + "github.com/bazel-contrib/rules_python/gazelle/manifest" "github.com/bazelbuild/rules_go/go/runfiles" - "github.com/bazelbuild/rules_python/gazelle/manifest" ) -func TestGazelleManifestIsUpdated(t *testing.T) { - requirementsPath := os.Getenv("_TEST_REQUIREMENTS") - if requirementsPath == "" { - t.Fatal("_TEST_REQUIREMENTS must be set") +// getResolvedRunfile resolves an environment variable to a runfiles path. +// It handles getting the env var, checking it's set, and resolving it through +// the runfiles mechanism, providing detailed error messages if anything fails. +func getResolvedRunfile(t *testing.T, envVar string) string { + t.Helper() + path := os.Getenv(envVar) + if path == "" { + t.Fatalf("%s must be set", envVar) } - - manifestPath := os.Getenv("_TEST_MANIFEST") - if manifestPath == "" { - t.Fatal("_TEST_MANIFEST must be set") + resolvedPath, err := runfiles.Rlocation(path) + if err != nil { + t.Fatalf("failed to resolve runfiles path for %s (%q): %v", envVar, path, err) } + return resolvedPath +} + +func TestGazelleManifestIsUpdated(t *testing.T) { + requirementsPathResolved := getResolvedRunfile(t, "_TEST_REQUIREMENTS") + manifestPathResolved := getResolvedRunfile(t, "_TEST_MANIFEST") manifestFile := new(manifest.File) - if err := manifestFile.Decode(manifestPath); err != nil { + if err := manifestFile.Decode(manifestPathResolved); err != nil { t.Fatalf("decoding manifest file: %v", err) } @@ -50,11 +59,7 @@ func TestGazelleManifestIsUpdated(t *testing.T) { t.Fatal("failed to find the Gazelle manifest file integrity") } - manifestGeneratorHashPath, err := runfiles.Rlocation( - os.Getenv("_TEST_MANIFEST_GENERATOR_HASH")) - if err != nil { - t.Fatalf("failed to resolve runfiles path of manifest: %v", err) - } + manifestGeneratorHashPath := getResolvedRunfile(t, "_TEST_MANIFEST_GENERATOR_HASH") manifestGeneratorHash, err := os.Open(manifestGeneratorHashPath) if err != nil { @@ -62,9 +67,9 @@ func TestGazelleManifestIsUpdated(t *testing.T) { } defer manifestGeneratorHash.Close() - requirements, err := os.Open(requirementsPath) + requirements, err := os.Open(requirementsPathResolved) if err != nil { - t.Fatalf("opening %q: %v", requirementsPath, err) + t.Fatalf("opening %q: %v", requirementsPathResolved, err) } defer requirements.Close() @@ -73,9 +78,9 @@ func TestGazelleManifestIsUpdated(t *testing.T) { t.Fatalf("verifying integrity: %v", err) } if !valid { - manifestRealpath, err := filepath.EvalSymlinks(manifestPath) + manifestRealpath, err := filepath.EvalSymlinks(manifestPathResolved) if err != nil { - t.Fatalf("evaluating symlink %q: %v", manifestPath, err) + t.Fatalf("evaluating symlink %q: %v", manifestPathResolved, err) } t.Errorf( "%q is out-of-date. Follow the update instructions in that file to resolve this", diff --git a/gazelle/modules_mapping/BUILD.bazel b/gazelle/modules_mapping/BUILD.bazel index d78b1fb51f..0e740f2636 100644 --- a/gazelle/modules_mapping/BUILD.bazel +++ b/gazelle/modules_mapping/BUILD.bazel @@ -1,4 +1,6 @@ -load("@rules_python//python:defs.bzl", "py_binary") +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") +load("@bazel_skylib//rules:copy_file.bzl", "copy_file") +load("@rules_python//python:defs.bzl", "py_binary", "py_test") # gazelle:exclude *.py @@ -8,8 +10,56 @@ py_binary( visibility = ["//visibility:public"], ) +py_binary( + name = "merger", + srcs = ["merger.py"], + visibility = ["//visibility:public"], +) + +copy_file( + name = "pytest_wheel", + src = "@pytest//file", + out = "pytest-8.3.3-py3-none-any.whl", +) + +copy_file( + name = "django_types_wheel", + src = "@django-types//file", + out = "django_types-0.19.1-py3-none-any.whl", +) + +py_test( + name = "test_generator", + srcs = ["test_generator.py"], + data = [ + "django_types_wheel", + "pytest_wheel", + ], + imports = ["."], + main = "test_generator.py", + deps = [":generator"], +) + +py_test( + name = "test_merger", + srcs = ["test_merger.py"], + data = [ + "django_types_wheel", + "pytest_wheel", + ], + imports = ["."], + main = "test_merger.py", + deps = [":merger"], +) + filegroup( name = "distribution", srcs = glob(["**"]), visibility = ["//:__pkg__"], ) + +bzl_library( + name = "def", + srcs = ["def.bzl"], + visibility = ["//visibility:public"], +) diff --git a/gazelle/modules_mapping/def.bzl b/gazelle/modules_mapping/def.bzl index 4da6267493..8466edd39a 100644 --- a/gazelle/modules_mapping/def.bzl +++ b/gazelle/modules_mapping/def.bzl @@ -25,21 +25,46 @@ module name doesn't match the wheel distribution name. def _modules_mapping_impl(ctx): modules_mapping = ctx.actions.declare_file(ctx.attr.modules_mapping_name) - args = ctx.actions.args() all_wheels = depset( [whl for whl in ctx.files.wheels], transitive = [dep[DefaultInfo].files for dep in ctx.attr.wheels] + [dep[DefaultInfo].data_runfiles.files for dep in ctx.attr.wheels], ) - args.add("--output_file", modules_mapping.path) - args.add_all("--exclude_patterns", ctx.attr.exclude_patterns) - args.add_all("--wheels", [whl.path for whl in all_wheels.to_list()]) + + # Run the generator once per-wheel (to leverage caching) + per_wheel_outputs = [] + for idx, whl in enumerate(all_wheels.to_list()): + wheel_modules_mapping = ctx.actions.declare_file("{}.{}".format(modules_mapping.short_path, idx)) + args = ctx.actions.args() + args.add("--output_file", wheel_modules_mapping.path) + if ctx.attr.include_stub_packages: + args.add("--include_stub_packages") + args.add_all("--exclude_patterns", ctx.attr.exclude_patterns) + args.add("--wheel", whl.path) + + ctx.actions.run( + inputs = [whl], + mnemonic = "PyGazelleModMapGen", + outputs = [wheel_modules_mapping], + executable = ctx.executable._generator, + arguments = [args], + use_default_shell_env = False, + ) + per_wheel_outputs.append(wheel_modules_mapping) + + # Then merge the individual JSONs together + merge_args = ctx.actions.args() + merge_args.add("--output", modules_mapping.path) + merge_args.add_all("--inputs", [f.path for f in per_wheel_outputs]) + ctx.actions.run( - inputs = all_wheels.to_list(), + inputs = per_wheel_outputs, + mnemonic = "PyGazelleModMapMerge", outputs = [modules_mapping], - executable = ctx.executable._generator, - arguments = [args], + executable = ctx.executable._merger, + arguments = [merge_args], use_default_shell_env = False, ) + return [DefaultInfo(files = depset([modules_mapping]))] modules_mapping = rule( @@ -50,6 +75,11 @@ modules_mapping = rule( doc = "A set of regex patterns to match against each calculated module path. By default, exclude the modules starting with underscores.", mandatory = False, ), + "include_stub_packages": attr.bool( + default = False, + doc = "Whether to include stub packages in the mapping.", + mandatory = False, + ), "modules_mapping_name": attr.string( default = "modules_mapping.json", doc = "The name for the output JSON file.", @@ -65,6 +95,11 @@ modules_mapping = rule( default = "//modules_mapping:generator", executable = True, ), + "_merger": attr.label( + cfg = "exec", + default = "//modules_mapping:merger", + executable = True, + ), }, doc = "Creates a modules_mapping.json file for mapping module names to wheel distribution names.", ) diff --git a/gazelle/modules_mapping/generator.py b/gazelle/modules_mapping/generator.py index bbd579d416..611910c669 100644 --- a/gazelle/modules_mapping/generator.py +++ b/gazelle/modules_mapping/generator.py @@ -25,16 +25,25 @@ class Generator: stderr = None output_file = None excluded_patterns = None - mapping = {} - def __init__(self, stderr, output_file, excluded_patterns): + def __init__(self, stderr, output_file, excluded_patterns, include_stub_packages): self.stderr = stderr self.output_file = output_file self.excluded_patterns = [re.compile(pattern) for pattern in excluded_patterns] + self.include_stub_packages = include_stub_packages + self.mapping = {} # dig_wheel analyses the wheel .whl file determining the modules it provides # by looking at the directory structure. def dig_wheel(self, whl): + # Skip stubs and types wheels. + wheel_name = get_wheel_name(whl) + if self.include_stub_packages and ( + wheel_name.endswith(("_stubs", "_types")) + or wheel_name.startswith(("types_", "stubs_")) + ): + self.mapping[wheel_name.lower()] = wheel_name.lower() + return with zipfile.ZipFile(whl, "r") as zip_file: for path in zip_file.namelist(): if is_metadata(path): @@ -87,8 +96,7 @@ def module_for_path(self, path, whl): ext = "".join(pathlib.Path(root).suffixes) module = root[: -len(ext)].replace("/", ".") if not self.is_excluded(module): - if not self.is_excluded(module): - self.mapping[module] = wheel_name + self.mapping[module] = wheel_name def is_excluded(self, module): for pattern in self.excluded_patterns: @@ -96,14 +104,20 @@ def is_excluded(self, module): return True return False - # run is the entrypoint for the generator. - def run(self, wheels): - for whl in wheels: - try: - self.dig_wheel(whl) - except AssertionError as error: - print(error, file=self.stderr) - return 1 + def run(self, wheel: pathlib.Path) -> int: + """ + Entrypoint for the generator. + + Args: + wheel: The path to the wheel file (`.whl`) + Returns: + Exit code (for `sys.exit`) + """ + try: + self.dig_wheel(wheel) + except AssertionError as error: + print(error, file=self.stderr) + return 1 self.simplify() mapping_json = json.dumps(self.mapping) with open(self.output_file, "w") as f: @@ -145,8 +159,11 @@ def data_has_purelib_or_platlib(path): description="Generates the modules mapping used by the Gazelle manifest.", ) parser.add_argument("--output_file", type=str) + parser.add_argument("--include_stub_packages", action="store_true") parser.add_argument("--exclude_patterns", nargs="+", default=[]) - parser.add_argument("--wheels", nargs="+", default=[]) + parser.add_argument("--wheel", type=pathlib.Path) args = parser.parse_args() - generator = Generator(sys.stderr, args.output_file, args.exclude_patterns) - exit(generator.run(args.wheels)) + generator = Generator( + sys.stderr, args.output_file, args.exclude_patterns, args.include_stub_packages + ) + sys.exit(generator.run(args.wheel)) diff --git a/gazelle/modules_mapping/merger.py b/gazelle/modules_mapping/merger.py new file mode 100644 index 0000000000..deb0cb2666 --- /dev/null +++ b/gazelle/modules_mapping/merger.py @@ -0,0 +1,45 @@ +#!/usr/bin/env python3 +"""Merges multiple modules_mapping.json files into a single file.""" + +import argparse +import json +from pathlib import Path + + +def merge_modules_mappings(input_files: list[Path], output_file: Path) -> None: + """Merge multiple modules_mapping.json files into one. + + Args: + input_files: List of paths to input JSON files to merge + output_file: Path where the merged output should be written + """ + merged_mapping = {} + for input_file in input_files: + mapping = json.loads(input_file.read_text()) + # Merge the mappings, with later files overwriting earlier ones + # if there are conflicts + merged_mapping.update(mapping) + + output_file.write_text(json.dumps(merged_mapping)) + + +if __name__ == "__main__": + parser = argparse.ArgumentParser( + description="Merge multiple modules_mapping.json files" + ) + parser.add_argument( + "--output", + required=True, + type=Path, + help="Output file path for merged mapping", + ) + parser.add_argument( + "--inputs", + required=True, + nargs="+", + type=Path, + help="Input JSON files to merge", + ) + + args = parser.parse_args() + merge_modules_mappings(args.inputs, args.output) diff --git a/gazelle/modules_mapping/test_generator.py b/gazelle/modules_mapping/test_generator.py new file mode 100644 index 0000000000..d6d2f19039 --- /dev/null +++ b/gazelle/modules_mapping/test_generator.py @@ -0,0 +1,44 @@ +import pathlib +import unittest + +from generator import Generator + + +class GeneratorTest(unittest.TestCase): + def test_generator(self): + whl = pathlib.Path(__file__).parent / "pytest-8.3.3-py3-none-any.whl" + gen = Generator(None, None, {}, False) + gen.dig_wheel(whl) + self.assertLessEqual( + { + "_pytest": "pytest", + "_pytest.__init__": "pytest", + "_pytest._argcomplete": "pytest", + "_pytest.config.argparsing": "pytest", + }.items(), + gen.mapping.items(), + ) + + def test_stub_generator(self): + whl = pathlib.Path(__file__).parent / "django_types-0.19.1-py3-none-any.whl" + gen = Generator(None, None, {}, True) + gen.dig_wheel(whl) + self.assertLessEqual( + { + "django_types": "django_types", + }.items(), + gen.mapping.items(), + ) + + def test_stub_excluded(self): + whl = pathlib.Path(__file__).parent / "django_types-0.19.1-py3-none-any.whl" + gen = Generator(None, None, {}, False) + gen.dig_wheel(whl) + self.assertEqual( + {}.items(), + gen.mapping.items(), + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/gazelle/modules_mapping/test_merger.py b/gazelle/modules_mapping/test_merger.py new file mode 100644 index 0000000000..87c35f4404 --- /dev/null +++ b/gazelle/modules_mapping/test_merger.py @@ -0,0 +1,61 @@ +import json +import pathlib +import tempfile +import unittest + +from merger import merge_modules_mappings + + +class MergerTest(unittest.TestCase): + _tmpdir: tempfile.TemporaryDirectory + + def setUp(self) -> None: + super().setUp() + self._tmpdir = tempfile.TemporaryDirectory() + + def tearDown(self) -> None: + super().tearDown() + self._tmpdir.cleanup() + del self._tmpdir + + @property + def tmppath(self) -> pathlib.Path: + return pathlib.Path(self._tmpdir.name) + + def make_input(self, mapping: dict[str, str]) -> pathlib.Path: + _fd, file = tempfile.mkstemp(suffix=".json", dir=self._tmpdir.name) + path = pathlib.Path(file) + path.write_text(json.dumps(mapping)) + return path + + def test_merger(self): + output_path = self.tmppath / "output.json" + merge_modules_mappings( + [ + self.make_input( + { + "_pytest": "pytest", + "_pytest.__init__": "pytest", + "_pytest._argcomplete": "pytest", + "_pytest.config.argparsing": "pytest", + } + ), + self.make_input({"django_types": "django_types"}), + ], + output_path, + ) + + self.assertEqual( + { + "_pytest": "pytest", + "_pytest.__init__": "pytest", + "_pytest._argcomplete": "pytest", + "_pytest.config.argparsing": "pytest", + "django_types": "django_types", + }, + json.loads(output_path.read_text()), + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/gazelle/python/BUILD.bazel b/gazelle/python/BUILD.bazel index 627a867c68..1ffa2890e1 100644 --- a/gazelle/python/BUILD.bazel +++ b/gazelle/python/BUILD.bazel @@ -26,24 +26,23 @@ go_library( # See following for more info: # https://github.com/bazelbuild/bazel-gazelle/issues/1513 embedsrcs = ["stdlib_list.txt"], # keep # TODO: use user-defined version? - importpath = "github.com/bazelbuild/rules_python/gazelle/python", + importpath = "github.com/bazel-contrib/rules_python/gazelle/python", visibility = ["//visibility:public"], deps = [ - "//manifest", "//pythonconfig", - "@bazel_gazelle//config:go_default_library", - "@bazel_gazelle//label:go_default_library", - "@bazel_gazelle//language:go_default_library", - "@bazel_gazelle//repo:go_default_library", - "@bazel_gazelle//resolve:go_default_library", - "@bazel_gazelle//rule:go_default_library", - "@com_github_bazelbuild_buildtools//build:go_default_library", + "@bazel_gazelle//config", + "@bazel_gazelle//label", + "@bazel_gazelle//language", + "@bazel_gazelle//repo", + "@bazel_gazelle//resolve", + "@bazel_gazelle//rule", + "@com_github_bazelbuild_buildtools//build", "@com_github_bmatcuk_doublestar_v4//:doublestar", - "@com_github_emirpasic_gods//lists/singlylinkedlist", - "@com_github_emirpasic_gods//sets/treeset", - "@com_github_emirpasic_gods//utils", - "@com_github_smacker_go_tree_sitter//:go-tree-sitter", - "@com_github_smacker_go_tree_sitter//python", + "@com_github_emirpasic_gods//lists/singlylinkedlist:go_default_library", + "@com_github_emirpasic_gods//sets/treeset:go_default_library", + "@com_github_emirpasic_gods//utils:go_default_library", + "@com_github_smacker_go_tree_sitter//:go_default_library", + "@com_github_smacker_go_tree_sitter//python:go_default_library", "@org_golang_x_sync//errgroup", ], ) @@ -55,10 +54,12 @@ copy_file( "@rules_python//python/config_settings:is_python_3.10": "@python_stdlib_list//:stdlib_list/lists/3.10.txt", "@rules_python//python/config_settings:is_python_3.11": "@python_stdlib_list//:stdlib_list/lists/3.11.txt", "@rules_python//python/config_settings:is_python_3.12": "@python_stdlib_list//:stdlib_list/lists/3.12.txt", + "@rules_python//python/config_settings:is_python_3.13": "@python_stdlib_list//:stdlib_list/lists/3.13.txt", + "@rules_python//python/config_settings:is_python_3.14": "@python_stdlib_list//:stdlib_list/lists/3.14.txt", "@rules_python//python/config_settings:is_python_3.8": "@python_stdlib_list//:stdlib_list/lists/3.8.txt", "@rules_python//python/config_settings:is_python_3.9": "@python_stdlib_list//:stdlib_list/lists/3.9.txt", - # This is the same behaviour as previously - "//conditions:default": "@python_stdlib_list//:stdlib_list/lists/3.11.txt", + # Fall back to the latest stdlib list + "//conditions:default": "@python_stdlib_list//:stdlib_list/lists/3.14.txt", }, ), out = "stdlib_list.txt", @@ -66,11 +67,15 @@ copy_file( ) # gazelle:exclude testdata/ +# gazelle:exclude extensions.bzl +# Exclude test-only Starlark helper from Gazelle to avoid generating an unnecessary bzl_library +# gazelle:exclude gazelle_test.bzl gazelle_test( name = "python_test", srcs = ["python_test.go"], data = [ + ":_gazelle_binary_with_proto", ":gazelle_binary", ], test_dirs = glob( @@ -95,6 +100,16 @@ gazelle_binary( visibility = ["//visibility:public"], ) +# Only used by testing +gazelle_binary( + name = "_gazelle_binary_with_proto", + languages = [ + "@bazel_gazelle//language/proto", + ":python", + ], + visibility = ["//visibility:private"], +) + filegroup( name = "distribution", srcs = glob(["**"]), diff --git a/gazelle/python/configure.go b/gazelle/python/configure.go index a369a64b8e..1fe95a1683 100644 --- a/gazelle/python/configure.go +++ b/gazelle/python/configure.go @@ -18,7 +18,6 @@ import ( "flag" "fmt" "log" - "os" "path/filepath" "strconv" "strings" @@ -27,8 +26,7 @@ import ( "github.com/bazelbuild/bazel-gazelle/rule" "github.com/bmatcuk/doublestar/v4" - "github.com/bazelbuild/rules_python/gazelle/manifest" - "github.com/bazelbuild/rules_python/gazelle/pythonconfig" + "github.com/bazel-contrib/rules_python/gazelle/pythonconfig" ) // Configurer satisfies the config.Configurer interface. It's the @@ -65,11 +63,18 @@ func (py *Configurer) KnownDirectives() []string { pythonconfig.LibraryNamingConvention, pythonconfig.BinaryNamingConvention, pythonconfig.TestNamingConvention, + pythonconfig.ProtoNamingConvention, pythonconfig.DefaultVisibilty, pythonconfig.Visibility, pythonconfig.TestFilePattern, pythonconfig.LabelConvention, pythonconfig.LabelNormalization, + pythonconfig.GeneratePyiDeps, + pythonconfig.GeneratePyiSrcs, + pythonconfig.ExperimentalAllowRelativeImports, + pythonconfig.GenerateProto, + pythonconfig.PythonResolveSiblingImports, + pythonconfig.PythonIncludeAncestorConftest, } } @@ -178,6 +183,8 @@ func (py *Configurer) Configure(c *config.Config, rel string, f *rule.File) { config.SetBinaryNamingConvention(strings.TrimSpace(d.Value)) case pythonconfig.TestNamingConvention: config.SetTestNamingConvention(strings.TrimSpace(d.Value)) + case pythonconfig.ProtoNamingConvention: + config.SetProtoNamingConvention(strings.TrimSpace(d.Value)) case pythonconfig.DefaultVisibilty: switch directiveArg := strings.TrimSpace(d.Value); directiveArg { case "NONE": @@ -224,29 +231,46 @@ func (py *Configurer) Configure(c *config.Config, rel string, f *rule.File) { default: config.SetLabelNormalization(pythonconfig.DefaultLabelNormalizationType) } + case pythonconfig.ExperimentalAllowRelativeImports: + v, err := strconv.ParseBool(strings.TrimSpace(d.Value)) + if err != nil { + log.Printf("invalid value for gazelle:%s in %q: %q", + pythonconfig.ExperimentalAllowRelativeImports, rel, d.Value) + } + config.SetExperimentalAllowRelativeImports(v) + case pythonconfig.GeneratePyiDeps: + v, err := strconv.ParseBool(strings.TrimSpace(d.Value)) + if err != nil { + log.Fatal(err) + } + config.SetGeneratePyiDeps(v) + case pythonconfig.GeneratePyiSrcs: + v, err := strconv.ParseBool(strings.TrimSpace(d.Value)) + if err != nil { + log.Fatal(err) + } + config.SetGeneratePyiSrcs(v) + case pythonconfig.GenerateProto: + v, err := strconv.ParseBool(strings.TrimSpace(d.Value)) + if err != nil { + log.Fatal(err) + } + config.SetGenerateProto(v) + case pythonconfig.PythonResolveSiblingImports: + v, err := strconv.ParseBool(strings.TrimSpace(d.Value)) + if err != nil { + log.Fatal(err) + } + config.SetResolveSiblingImports(v) + case pythonconfig.PythonIncludeAncestorConftest: + v, err := strconv.ParseBool(strings.TrimSpace(d.Value)) + if err != nil { + log.Fatal(err) + } + config.SetIncludeAncestorConftest(v) } } gazelleManifestPath := filepath.Join(c.RepoRoot, rel, gazelleManifestFilename) - gazelleManifest, err := py.loadGazelleManifest(gazelleManifestPath) - if err != nil { - log.Fatal(err) - } - if gazelleManifest != nil { - config.SetGazelleManifest(gazelleManifest) - } -} - -func (py *Configurer) loadGazelleManifest(gazelleManifestPath string) (*manifest.Manifest, error) { - if _, err := os.Stat(gazelleManifestPath); err != nil { - if os.IsNotExist(err) { - return nil, nil - } - return nil, fmt.Errorf("failed to load Gazelle manifest at %q: %w", gazelleManifestPath, err) - } - manifestFile := new(manifest.File) - if err := manifestFile.Decode(gazelleManifestPath); err != nil { - return nil, fmt.Errorf("failed to load Gazelle manifest at %q: %w", gazelleManifestPath, err) - } - return manifestFile.Manifest, nil + config.SetGazelleManifestPath(gazelleManifestPath) } diff --git a/gazelle/python/file_parser.go b/gazelle/python/file_parser.go index a1f47f400c..e129337e11 100644 --- a/gazelle/python/file_parser.go +++ b/gazelle/python/file_parser.go @@ -41,15 +41,16 @@ const ( type ParserOutput struct { FileName string - Modules []module - Comments []comment + Modules []Module + Comments []Comment HasMain bool } type FileParser struct { - code []byte - relFilepath string - output ParserOutput + code []byte + relFilepath string + output ParserOutput + inTypeCheckingBlock bool } func NewFileParser() *FileParser { @@ -115,10 +116,6 @@ func (p *FileParser) parseMain(ctx context.Context, node *sitter.Node) bool { a, b = b, a } if a.Type() == sitterNodeTypeIdentifier && a.Content(p.code) == "__name__" && - // at github.com/smacker/go-tree-sitter@latest (after v0.0.0-20240422154435-0628b34cbf9c we used) - // "__main__" is the second child of b. But now, it isn't. - // we cannot use the latest go-tree-sitter because of the top level reference in scanner.c. - // https://github.com/smacker/go-tree-sitter/blob/04d6b33fe138a98075210f5b770482ded024dc0f/python/scanner.c#L1 b.Type() == sitterNodeTypeString && string(p.code[b.StartByte()+1:b.EndByte()-1]) == "__main__" { return true } @@ -127,24 +124,34 @@ func (p *FileParser) parseMain(ctx context.Context, node *sitter.Node) bool { return false } -// parseImportStatement parses a node for an import statement, returning a `module` and a boolean +// parseImportStatement parses a node for an import statement, returning a `Module` and a boolean // representing if the parse was OK or not. -func parseImportStatement(node *sitter.Node, code []byte) (module, bool) { +func parseImportStatement(node *sitter.Node, code []byte) (Module, bool) { switch node.Type() { case sitterNodeTypeDottedName: - return module{ + return Module{ Name: node.Content(code), LineNumber: node.StartPoint().Row + 1, }, true case sitterNodeTypeAliasedImport: return parseImportStatement(node.Child(0), code) case sitterNodeTypeWildcardImport: - return module{ + return Module{ Name: "*", LineNumber: node.StartPoint().Row + 1, }, true } - return module{}, false + return Module{}, false +} + +// cleanImportString removes backslashes and all whitespace from the string. +func cleanImportString(s string) string { + s = strings.ReplaceAll(s, "\r\n", "") + s = strings.ReplaceAll(s, "\\", "") + s = strings.ReplaceAll(s, " ", "") + s = strings.ReplaceAll(s, "\n", "") + s = strings.ReplaceAll(s, "\t", "") + return s } // parseImportStatements parses a node for import statements, returning true if the node is @@ -157,7 +164,10 @@ func (p *FileParser) parseImportStatements(node *sitter.Node) bool { if !ok { continue } + m.From = cleanImportString(m.From) + m.Name = cleanImportString(m.Name) m.Filepath = p.relFilepath + m.TypeCheckingOnly = p.inTypeCheckingBlock if strings.HasPrefix(m.Name, ".") { continue } @@ -165,7 +175,10 @@ func (p *FileParser) parseImportStatements(node *sitter.Node) bool { } } else if node.Type() == sitterNodeTypeImportFromStatement { from := node.Child(1).Content(p.code) - if strings.HasPrefix(from, ".") { + from = cleanImportString(from) + // If the import is from the current package, we don't need to add it to the modules i.e. from . import Class1. + // If the import is from a different relative package i.e. from .package1 import foo, we need to add it to the modules. + if from == "." { return true } for j := 3; j < int(node.ChildCount()); j++ { @@ -175,7 +188,9 @@ func (p *FileParser) parseImportStatements(node *sitter.Node) bool { } m.Filepath = p.relFilepath m.From = from + m.Name = cleanImportString(m.Name) m.Name = fmt.Sprintf("%s.%s", from, m.Name) + m.TypeCheckingOnly = p.inTypeCheckingBlock p.output.Modules = append(p.output.Modules, m) } } else { @@ -188,7 +203,7 @@ func (p *FileParser) parseImportStatements(node *sitter.Node) bool { // It updates FileParser.output.Comments with the parsed comment. func (p *FileParser) parseComments(node *sitter.Node) bool { if node.Type() == sitterNodeTypeComment { - p.output.Comments = append(p.output.Comments, comment(node.Content(p.code))) + p.output.Comments = append(p.output.Comments, Comment(node.Content(p.code))) return true } return false @@ -200,10 +215,43 @@ func (p *FileParser) SetCodeAndFile(code []byte, relPackagePath, filename string p.output.FileName = filename } +// isTypeCheckingBlock returns true if the given node is an `if TYPE_CHECKING:` block. +func (p *FileParser) isTypeCheckingBlock(node *sitter.Node) bool { + if node.Type() != sitterNodeTypeIfStatement || node.ChildCount() < 2 { + return false + } + + condition := node.Child(1) + + // Handle `if TYPE_CHECKING:` + if condition.Type() == sitterNodeTypeIdentifier && condition.Content(p.code) == "TYPE_CHECKING" { + return true + } + + // Handle `if typing.TYPE_CHECKING:` + if condition.Type() == "attribute" && condition.ChildCount() >= 3 { + object := condition.Child(0) + attr := condition.Child(2) + if object.Type() == sitterNodeTypeIdentifier && object.Content(p.code) == "typing" && + attr.Type() == sitterNodeTypeIdentifier && attr.Content(p.code) == "TYPE_CHECKING" { + return true + } + } + + return false +} + func (p *FileParser) parse(ctx context.Context, node *sitter.Node) { if node == nil { return } + + // Check if this is a TYPE_CHECKING block + wasInTypeCheckingBlock := p.inTypeCheckingBlock + if p.isTypeCheckingBlock(node) { + p.inTypeCheckingBlock = true + } + for i := 0; i < int(node.ChildCount()); i++ { if err := ctx.Err(); err != nil { return @@ -217,6 +265,9 @@ func (p *FileParser) parse(ctx context.Context, node *sitter.Node) { } p.parse(ctx, child) } + + // Restore the previous state + p.inTypeCheckingBlock = wasInTypeCheckingBlock } func (p *FileParser) Parse(ctx context.Context) (*ParserOutput, error) { diff --git a/gazelle/python/file_parser_test.go b/gazelle/python/file_parser_test.go index 3682cff753..0a6fd1b4ab 100644 --- a/gazelle/python/file_parser_test.go +++ b/gazelle/python/file_parser_test.go @@ -27,7 +27,7 @@ func TestParseImportStatements(t *testing.T) { name string code string filepath string - result []module + result []Module }{ { name: "not has import", @@ -39,7 +39,7 @@ func TestParseImportStatements(t *testing.T) { name: "has import", code: "import unittest\nimport os.path\nfrom foo.bar import abc.xyz", filepath: "abc.py", - result: []module{ + result: []Module{ { Name: "unittest", LineNumber: 1, @@ -66,7 +66,7 @@ func TestParseImportStatements(t *testing.T) { import unittest `, filepath: "abc.py", - result: []module{ + result: []Module{ { Name: "unittest", LineNumber: 2, @@ -79,7 +79,7 @@ func TestParseImportStatements(t *testing.T) { name: "invalid syntax", code: "import os\nimport", filepath: "abc.py", - result: []module{ + result: []Module{ { Name: "os", LineNumber: 1, @@ -92,7 +92,7 @@ func TestParseImportStatements(t *testing.T) { name: "import as", code: "import os as b\nfrom foo import bar as c# 123", filepath: "abc.py", - result: []module{ + result: []Module{ { Name: "os", LineNumber: 1, @@ -111,7 +111,7 @@ func TestParseImportStatements(t *testing.T) { { name: "complex import", code: "from unittest import *\nfrom foo import (bar as c, baz, qux as d)\nfrom . import abc", - result: []module{ + result: []Module{ { Name: "unittest.*", LineNumber: 1, @@ -152,7 +152,7 @@ func TestParseComments(t *testing.T) { units := []struct { name string code string - result []comment + result []Comment }{ { name: "not has comment", @@ -162,17 +162,17 @@ func TestParseComments(t *testing.T) { { name: "has comment", code: "# a = 1\n# b = 2", - result: []comment{"# a = 1", "# b = 2"}, + result: []Comment{"# a = 1", "# b = 2"}, }, { name: "has comment in if", code: "if True:\n # a = 1\n # b = 2", - result: []comment{"# a = 1", "# b = 2"}, + result: []Comment{"# a = 1", "# b = 2"}, }, { name: "has comment inline", code: "import os# 123\nfrom pathlib import Path as b#456", - result: []comment{"# 123", "#456"}, + result: []Comment{"# 123", "#456"}, }, } for _, u := range units { @@ -248,9 +248,138 @@ func TestParseFull(t *testing.T) { output, err := p.Parse(context.Background()) assert.NoError(t, err) assert.Equal(t, ParserOutput{ - Modules: []module{{Name: "bar.abc", LineNumber: 1, Filepath: "foo/a.py", From: "bar"}}, + Modules: []Module{{Name: "bar.abc", LineNumber: 1, Filepath: "foo/a.py", From: "bar"}}, Comments: nil, HasMain: false, FileName: "a.py", }, *output) } + +func TestTypeCheckingImports(t *testing.T) { + code := ` +import sys +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + import boto3 + from rest_framework import serializers + +def example_function(): + _ = sys.version_info +` + p := NewFileParser() + p.SetCodeAndFile([]byte(code), "", "test.py") + + result, err := p.Parse(context.Background()) + if err != nil { + t.Fatalf("Failed to parse: %v", err) + } + + // Check that we found the expected modules + expectedModules := map[string]bool{ + "sys": false, + "typing.TYPE_CHECKING": false, + "boto3": true, + "rest_framework.serializers": true, + } + + for _, mod := range result.Modules { + if expected, exists := expectedModules[mod.Name]; exists { + if mod.TypeCheckingOnly != expected { + t.Errorf("Module %s: expected TypeCheckingOnly=%v, got %v", mod.Name, expected, mod.TypeCheckingOnly) + } + } + } +} + +func TestParseImportStatements_MultilineWithBackslashAndWhitespace(t *testing.T) { + t.Parallel() + t.Run("multiline from import", func(t *testing.T) { + p := NewFileParser() + code := []byte(`from foo.bar.\ + baz import ( + Something, + AnotherThing +) + +from foo\ + .test import ( + Foo, + Bar +) +`) + p.SetCodeAndFile(code, "", "test.py") + output, err := p.Parse(context.Background()) + assert.NoError(t, err) + // Updated expected to match parser output + expected := []Module{ + { + Name: "foo.bar.baz.Something", + LineNumber: 3, + Filepath: "test.py", + From: "foo.bar.baz", + }, + { + Name: "foo.bar.baz.AnotherThing", + LineNumber: 4, + Filepath: "test.py", + From: "foo.bar.baz", + }, + { + Name: "foo.test.Foo", + LineNumber: 9, + Filepath: "test.py", + From: "foo.test", + }, + { + Name: "foo.test.Bar", + LineNumber: 10, + Filepath: "test.py", + From: "foo.test", + }, + } + assert.ElementsMatch(t, expected, output.Modules) + }) + t.Run("multiline import", func(t *testing.T) { + p := NewFileParser() + code := []byte(`import foo.bar.\ + baz +`) + p.SetCodeAndFile(code, "", "test.py") + output, err := p.Parse(context.Background()) + assert.NoError(t, err) + // Updated expected to match parser output + expected := []Module{ + { + Name: "foo.bar.baz", + LineNumber: 1, + Filepath: "test.py", + From: "", + }, + } + assert.ElementsMatch(t, expected, output.Modules) + }) + t.Run("windows line endings", func(t *testing.T) { + p := NewFileParser() + code := []byte("from foo.bar.\r\n baz import (\r\n Something,\r\n AnotherThing\r\n)\r\n") + p.SetCodeAndFile(code, "", "test.py") + output, err := p.Parse(context.Background()) + assert.NoError(t, err) + // Updated expected to match parser output + expected := []Module{ + { + Name: "foo.bar.baz.Something", + LineNumber: 3, + Filepath: "test.py", + From: "foo.bar.baz", + }, + { + Name: "foo.bar.baz.AnotherThing", + LineNumber: 4, + Filepath: "test.py", + From: "foo.bar.baz", + }, + } + assert.ElementsMatch(t, expected, output.Modules) + }) +} diff --git a/gazelle/python/generate.go b/gazelle/python/generate.go index c563b47bf3..90e06a1546 100644 --- a/gazelle/python/generate.go +++ b/gazelle/python/generate.go @@ -32,7 +32,7 @@ import ( "github.com/emirpasic/gods/sets/treeset" godsutils "github.com/emirpasic/gods/utils" - "github.com/bazelbuild/rules_python/gazelle/pythonconfig" + "github.com/bazel-contrib/rules_python/gazelle/pythonconfig" ) const ( @@ -48,13 +48,21 @@ var ( buildFilenames = []string{"BUILD", "BUILD.bazel"} ) -func GetActualKindName(kind string, args language.GenerateArgs) string { - if kindOverride, ok := args.Config.KindMap[kind]; ok { - return kindOverride.KindName +// Returns the mapped kind, or kind if no mapping is configured with the map_kind directive. +func getMappedKind(c *config.Config, kind string) string { + if mapped, ok := c.KindMap[kind]; ok { + return mapped.KindName } return kind } +// kindMatches returns whether r matches the canonical Python rule kind `expected`, respecting `# gazelle:map_kind` and +// `# gazelle:alias_kind` directives in the config.Config c. +func kindMatches(c *config.Config, r *rule.Rule, expected string) bool { + kind := r.Kind() + return kind == getMappedKind(c, expected) || c.AliasMap[kind] == expected +} + func matchesAnyGlob(s string, globs []string) bool { // This function assumes that the globs have already been validated. If a glob is // invalid, it's considered a non-match and we move on to the next pattern. @@ -66,6 +74,30 @@ func matchesAnyGlob(s string, globs []string) bool { return false } +// findConftestPaths returns package paths containing conftest.py, from currentPkg +// up through ancestors, stopping at module root. +func findConftestPaths(repoRoot, currentPkg, pythonProjectRoot string, includeAncestorConftest bool) []string { + var result []string + for pkg := currentPkg; ; pkg = filepath.Dir(pkg) { + if pkg == "." { + pkg = "" + } + if _, err := os.Stat(filepath.Join(repoRoot, pkg, conftestFilename)); err == nil { + result = append(result, pkg) + } + // We traverse up the tree to find conftest files and we start in + // the current package. Thus if we find one in the current package + // and do not want ancestors, we break early. + if !includeAncestorConftest { + break + } + if pkg == "" { + break + } + } + return result +} + // GenerateRules extracts build metadata from source files in a directory. // GenerateRules is called in each directory where an update is requested // in depth-first post-order. @@ -85,15 +117,9 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes if parent != nil && parent.CoarseGrainedGeneration() { return language.GenerateResult{} } - } else if !hasEntrypointFile(args.Dir) { - return language.GenerateResult{} } } - actualPyBinaryKind := GetActualKindName(pyBinaryKind, args) - actualPyLibraryKind := GetActualKindName(pyLibraryKind, args) - actualPyTestKind := GetActualKindName(pyTestKind, args) - pythonProjectRoot := cfg.PythonProjectRoot() packageName := filepath.Base(args.Dir) @@ -172,9 +198,6 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes // 2. The directory has a BUILD or BUILD.bazel files. Then // it doesn't matter at all what it has since it's a // separate Bazel package. - // 3. (only for package generation) The directory has an - // __init__.py, __main__.py or __test__.py, meaning a - // BUILD file will be generated. if cfg.PerFileGeneration() { return fs.SkipDir } @@ -184,7 +207,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes return nil } - if !cfg.CoarseGrainedGeneration() && hasEntrypointFile(path) { + if !cfg.CoarseGrainedGeneration() { return fs.SkipDir } @@ -231,10 +254,28 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes var result language.GenerateResult result.Gen = make([]*rule.Rule, 0) + if cfg.GenerateProto() { + generateProtoLibraries(args, cfg, pythonProjectRoot, visibility, &result) + } + collisionErrors := singlylinkedlist.New() + // Create a validFilesMap of mainModules to validate if python macros have valid srcs. + validFilesMap := make(map[string]struct{}) + + // Determine whether we have an __init__.py file in this package and whether we'll implicitly include it in srcs. + var hasPopulatedInit bool + var autoIncludeInit bool + if cfg.PerFileGeneration() { + var hasInit bool + hasInit, hasPopulatedInit = hasLibraryEntrypointFile(args.Dir) + autoIncludeInit = cfg.PerFileGenerationIncludeInit() && hasInit && hasPopulatedInit + } appendPyLibrary := func(srcs *treeset.Set, pyLibraryTargetName string) { allDeps, mainModules, annotations, err := parser.parse(srcs) + for name := range mainModules { + validFilesMap[name] = struct{}{} + } if err != nil { log.Fatalf("ERROR: %v\n", err) } @@ -249,23 +290,41 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes // that we don't also generate a py_library target for it. if cfg.PerFileGeneration() { srcs.Remove(name) + // Also remove the __init__.py that was added earlier. + if autoIncludeInit { + srcs.Remove(pyLibraryEntrypointFilename) + } } } + sort.Strings(mainFileNames) for _, filename := range mainFileNames { pyBinaryTargetName := strings.TrimSuffix(filepath.Base(filename), ".py") - if err := ensureNoCollision(args.File, pyBinaryTargetName, actualPyBinaryKind); err != nil { + if err := ensureNoCollision(args.Config, args.File, pyBinaryTargetName, pyBinaryKind); err != nil { fqTarget := label.New("", args.Rel, pyBinaryTargetName) log.Printf("failed to generate target %q of kind %q: %v", - fqTarget.String(), actualPyBinaryKind, err) + fqTarget.String(), getMappedKind(args.Config, pyBinaryKind), err) continue } - pyBinary := newTargetBuilder(pyBinaryKind, pyBinaryTargetName, pythonProjectRoot, args.Rel, pyFileNames). + + // Add any sibling .pyi files to pyi_srcs + filenames := treeset.NewWith(godsutils.StringComparator, filename) + pyiSrcs, _ := getPyiFilenames(filenames, cfg.GeneratePyiSrcs(), args.Dir) + + pyBinaryBuilder := newTargetBuilder(pyBinaryKind, pyBinaryTargetName, pythonProjectRoot, args.Rel, pyFileNames, cfg.ResolveSiblingImports()). addVisibility(visibility). addSrc(filename). + addPyiSrcs(pyiSrcs). addModuleDependencies(mainModules[filename]). addResolvedDependencies(annotations.includeDeps). - generateImportsAttribute().build() + generateImportsAttribute(). + setAnnotations(*annotations) + + if autoIncludeInit { + pyBinaryBuilder.addSrc(pyLibraryEntrypointFilename) + } + + pyBinary := pyBinaryBuilder.build() result.Gen = append(result.Gen, pyBinary) result.Imports = append(result.Imports, pyBinary.PrivateAttr(config.GazelleImportsKey)) } @@ -279,7 +338,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes } generateEmptyLibrary := false for _, r := range args.File.Rules { - if r.Kind() == actualPyLibraryKind && r.Name() == pyLibraryTargetName { + if r.Name() == pyLibraryTargetName && kindMatches(args.Config, r, pyLibraryKind) { generateEmptyLibrary = true } } @@ -288,42 +347,47 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes } } + // Add any sibling .pyi files to pyi_srcs + pyiSrcs, _ := getPyiFilenames(srcs, cfg.GeneratePyiSrcs(), args.Dir) + // Check if a target with the same name we are generating already // exists, and if it is of a different kind from the one we are // generating. If so, we have to throw an error since Gazelle won't // generate it correctly. - if err := ensureNoCollision(args.File, pyLibraryTargetName, actualPyLibraryKind); err != nil { + if err := ensureNoCollision(args.Config, args.File, pyLibraryTargetName, pyLibraryKind); err != nil { fqTarget := label.New("", args.Rel, pyLibraryTargetName) err := fmt.Errorf("failed to generate target %q of kind %q: %w. "+ "Use the '# gazelle:%s' directive to change the naming convention.", - fqTarget.String(), actualPyLibraryKind, err, pythonconfig.LibraryNamingConvention) + fqTarget.String(), getMappedKind(args.Config, pyLibraryKind), err, pythonconfig.LibraryNamingConvention) collisionErrors.Add(err) } - pyLibrary := newTargetBuilder(pyLibraryKind, pyLibraryTargetName, pythonProjectRoot, args.Rel, pyFileNames). + pyLibrary := newTargetBuilder(pyLibraryKind, pyLibraryTargetName, pythonProjectRoot, args.Rel, pyFileNames, cfg.ResolveSiblingImports()). addVisibility(visibility). addSrcs(srcs). + addPyiSrcs(pyiSrcs). addModuleDependencies(allDeps). addResolvedDependencies(annotations.includeDeps). generateImportsAttribute(). + setAnnotations(*annotations). build() if pyLibrary.IsEmpty(py.Kinds()[pyLibrary.Kind()]) { - result.Empty = append(result.Gen, pyLibrary) + result.Empty = append(result.Empty, pyLibrary) } else { result.Gen = append(result.Gen, pyLibrary) result.Imports = append(result.Imports, pyLibrary.PrivateAttr(config.GazelleImportsKey)) } } + if cfg.PerFileGeneration() { - hasInit, nonEmptyInit := hasLibraryEntrypointFile(args.Dir) pyLibraryFilenames.Each(func(index int, filename interface{}) { pyLibraryTargetName := strings.TrimSuffix(filepath.Base(filename.(string)), ".py") - if filename == pyLibraryEntrypointFilename && !nonEmptyInit { + if filename == pyLibraryEntrypointFilename && !hasPopulatedInit { return // ignore empty __init__.py. } srcs := treeset.NewWith(godsutils.StringComparator, filename) - if cfg.PerFileGenerationIncludeInit() && hasInit && nonEmptyInit { + if autoIncludeInit { srcs.Add(pyLibraryEntrypointFilename) } appendPyLibrary(srcs, pyLibraryTargetName) @@ -344,20 +408,26 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes // exists, and if it is of a different kind from the one we are // generating. If so, we have to throw an error since Gazelle won't // generate it correctly. - if err := ensureNoCollision(args.File, pyBinaryTargetName, actualPyBinaryKind); err != nil { + if err := ensureNoCollision(args.Config, args.File, pyBinaryTargetName, pyBinaryKind); err != nil { fqTarget := label.New("", args.Rel, pyBinaryTargetName) err := fmt.Errorf("failed to generate target %q of kind %q: %w. "+ "Use the '# gazelle:%s' directive to change the naming convention.", - fqTarget.String(), actualPyBinaryKind, err, pythonconfig.BinaryNamingConvention) + fqTarget.String(), getMappedKind(args.Config, pyBinaryKind), err, pythonconfig.BinaryNamingConvention) collisionErrors.Add(err) } - pyBinaryTarget := newTargetBuilder(pyBinaryKind, pyBinaryTargetName, pythonProjectRoot, args.Rel, pyFileNames). + // Add any sibling .pyi files to pyi_srcs + filenames := treeset.NewWith(godsutils.StringComparator, pyBinaryEntrypointFilename) + pyiSrcs, _ := getPyiFilenames(filenames, cfg.GeneratePyiSrcs(), args.Dir) + + pyBinaryTarget := newTargetBuilder(pyBinaryKind, pyBinaryTargetName, pythonProjectRoot, args.Rel, pyFileNames, cfg.ResolveSiblingImports()). setMain(pyBinaryEntrypointFilename). addVisibility(visibility). addSrc(pyBinaryEntrypointFilename). + addPyiSrcs(pyiSrcs). addModuleDependencies(deps). addResolvedDependencies(annotations.includeDeps). + setAnnotations(*annotations). generateImportsAttribute() pyBinary := pyBinaryTarget.build() @@ -377,17 +447,23 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes // exists, and if it is of a different kind from the one we are // generating. If so, we have to throw an error since Gazelle won't // generate it correctly. - if err := ensureNoCollision(args.File, conftestTargetname, actualPyLibraryKind); err != nil { + if err := ensureNoCollision(args.Config, args.File, conftestTargetname, pyLibraryKind); err != nil { fqTarget := label.New("", args.Rel, conftestTargetname) err := fmt.Errorf("failed to generate target %q of kind %q: %w. ", - fqTarget.String(), actualPyLibraryKind, err) + fqTarget.String(), getMappedKind(args.Config, pyLibraryKind), err) collisionErrors.Add(err) } - conftestTarget := newTargetBuilder(pyLibraryKind, conftestTargetname, pythonProjectRoot, args.Rel, pyFileNames). + // Add any sibling .pyi files to pyi_srcs + filenames := treeset.NewWith(godsutils.StringComparator, conftestFilename) + pyiSrcs, _ := getPyiFilenames(filenames, cfg.GeneratePyiSrcs(), args.Dir) + + conftestTarget := newTargetBuilder(pyLibraryKind, conftestTargetname, pythonProjectRoot, args.Rel, pyFileNames, cfg.ResolveSiblingImports()). addSrc(conftestFilename). + addPyiSrcs(pyiSrcs). addModuleDependencies(deps). addResolvedDependencies(annotations.includeDeps). + setAnnotations(*annotations). addVisibility(visibility). setTestonly(). generateImportsAttribute() @@ -408,17 +484,23 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes // exists, and if it is of a different kind from the one we are // generating. If so, we have to throw an error since Gazelle won't // generate it correctly. - if err := ensureNoCollision(args.File, pyTestTargetName, actualPyTestKind); err != nil { + if err := ensureNoCollision(args.Config, args.File, pyTestTargetName, pyTestKind); err != nil { fqTarget := label.New("", args.Rel, pyTestTargetName) err := fmt.Errorf("failed to generate target %q of kind %q: %w. "+ "Use the '# gazelle:%s' directive to change the naming convention.", - fqTarget.String(), actualPyTestKind, err, pythonconfig.TestNamingConvention) + fqTarget.String(), getMappedKind(args.Config, pyTestKind), err, pythonconfig.TestNamingConvention) collisionErrors.Add(err) } - return newTargetBuilder(pyTestKind, pyTestTargetName, pythonProjectRoot, args.Rel, pyFileNames). + + // Add any sibling .pyi files to pyi_srcs + pyiSrcs, _ := getPyiFilenames(srcs, cfg.GeneratePyiSrcs(), args.Dir) + + return newTargetBuilder(pyTestKind, pyTestTargetName, pythonProjectRoot, args.Rel, pyFileNames, cfg.ResolveSiblingImports()). addSrcs(srcs). + addPyiSrcs(pyiSrcs). addModuleDependencies(deps). addResolvedDependencies(annotations.includeDeps). + setAnnotations(*annotations). generateImportsAttribute() } if (!cfg.PerPackageGenerationRequireTestEntryPoint() || hasPyTestEntryPointFile || hasPyTestEntryPointTarget || cfg.CoarseGrainedGeneration()) && !cfg.PerFileGeneration() { @@ -470,15 +552,26 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes } for _, pyTestTarget := range pyTestTargets { - if conftest != nil { - pyTestTarget.addModuleDependency(module{Name: strings.TrimSuffix(conftestFilename, ".py")}) + shouldAddConftest := pyTestTarget.annotations.includePytestConftest == nil || + *pyTestTarget.annotations.includePytestConftest + + if shouldAddConftest { + for _, conftestPkg := range findConftestPaths(args.Config.RepoRoot, args.Rel, pythonProjectRoot, cfg.IncludeAncestorConftest()) { + pyTestTarget.addModuleDependency( + Module{ + Name: importSpecFromSrc(pythonProjectRoot, conftestPkg, conftestFilename).Imp, + Filepath: filepath.Join(conftestPkg, conftestFilename), + }, + ) + } } pyTest := pyTestTarget.build() result.Gen = append(result.Gen, pyTest) result.Imports = append(result.Imports, pyTest.PrivateAttr(config.GazelleImportsKey)) } - + emptyRules := py.getRulesWithInvalidSrcs(args, validFilesMap) + result.Empty = append(result.Empty, emptyRules...) if !collisionErrors.Empty() { it := collisionErrors.Iterator() for it.Next() { @@ -490,6 +583,68 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes return result } +// getRulesWithInvalidSrcs checks existing Python rules in the BUILD file and return the rules with invalid source files. +// Invalid source files are files that do not exist or not a target. +func (py *Python) getRulesWithInvalidSrcs(args language.GenerateArgs, validFilesMap map[string]struct{}) (invalidRules []*rule.Rule) { + if args.File == nil { + return + } + for _, file := range args.GenFiles { + validFilesMap[file] = struct{}{} + } + + // allFilesMap extends validFilesMap with all regular files on disk. + // py_binary uses validFilesMap (main modules + generated files), while py_library + // and py_test use allFilesMap since any file is a valid src for them. + allFilesMap := make(map[string]struct{}, len(validFilesMap)+len(args.RegularFiles)) + for file := range validFilesMap { + allFilesMap[file] = struct{}{} + } + for _, file := range args.RegularFiles { + allFilesMap[file] = struct{}{} + } + + isTarget := func(src string) bool { + return strings.HasPrefix(src, "@") || strings.HasPrefix(src, "//") || strings.HasPrefix(src, ":") + } + for _, existingRule := range args.File.Rules { + var matchedKind string + var filesMap map[string]struct{} + if kindMatches(args.Config, existingRule, pyBinaryKind) { + matchedKind = pyBinaryKind + filesMap = validFilesMap + } else if kindMatches(args.Config, existingRule, pyLibraryKind) { + matchedKind = pyLibraryKind + filesMap = allFilesMap + } else if kindMatches(args.Config, existingRule, pyTestKind) { + matchedKind = pyTestKind + filesMap = allFilesMap + } else { + continue + } + + srcs := existingRule.AttrStrings("srcs") + if len(srcs) == 0 { + continue + } + var hasValidSrcs bool + for _, src := range srcs { + if isTarget(src) { + hasValidSrcs = true + break + } + if _, ok := filesMap[src]; ok { + hasValidSrcs = true + break + } + } + if !hasValidSrcs { + invalidRules = append(invalidRules, newTargetBuilder(matchedKind, existingRule.Name(), "", "", nil, false).build()) + } + } + return invalidRules +} + // isBazelPackage determines if the directory is a Bazel package by probing for // the existence of a known BUILD file name. func isBazelPackage(dir string) bool { @@ -545,14 +700,95 @@ func isEntrypointFile(path string) bool { } } -func ensureNoCollision(file *rule.File, targetName, kind string) error { +func ensureNoCollision(c *config.Config, file *rule.File, targetName, kind string) error { if file == nil { return nil } for _, t := range file.Rules { - if t.Name() == targetName && t.Kind() != kind { + if t.Name() == targetName && !kindMatches(c, t, kind) { return fmt.Errorf("a target of kind %q with the same name already exists", t.Kind()) } } return nil } + +func generateProtoLibraries(args language.GenerateArgs, cfg *pythonconfig.Config, pythonProjectRoot string, visibility []string, res *language.GenerateResult) { + // First, enumerate all the proto_library in this package. + var protoRuleNames []string + for _, r := range args.OtherGen { + if r.Kind() != "proto_library" { + continue + } + protoRuleNames = append(protoRuleNames, r.Name()) + } + sort.Strings(protoRuleNames) + + // Next, enumerate all the pre-existing py_proto_library in this package, so we can delete unnecessary rules later. + pyProtoRules := map[string]bool{} + pyProtoRulesForProto := map[string]string{} + if args.File != nil { + for _, r := range args.File.Rules { + if kindMatches(args.Config, r, pyProtoLibraryKind) { + pyProtoRules[r.Name()] = false + + protos := r.AttrStrings("deps") + for _, proto := range protos { + pyProtoRulesForProto[strings.TrimPrefix(proto, ":")] = r.Name() + } + } + } + } + + emptySiblings := treeset.Set{} + // Generate a py_proto_library for each proto_library. + for _, protoRuleName := range protoRuleNames { + pyProtoLibraryName := cfg.RenderProtoName(protoRuleName) + if ruleName, ok := pyProtoRulesForProto[protoRuleName]; ok { + // There exists a pre-existing py_proto_library for this proto. Keep this name. + pyProtoLibraryName = ruleName + } + + pyProtoLibrary := newTargetBuilder(pyProtoLibraryKind, pyProtoLibraryName, pythonProjectRoot, args.Rel, &emptySiblings, false). + addVisibility(visibility). + addResolvedDependency(":" + protoRuleName). + generateImportsAttribute().build() + + res.Gen = append(res.Gen, pyProtoLibrary) + res.Imports = append(res.Imports, pyProtoLibrary.PrivateAttr(config.GazelleImportsKey)) + pyProtoRules[pyProtoLibrary.Name()] = true + + } + + // Finally, emit an empty rule for each pre-existing py_proto_library that we didn't already generate. + for ruleName, generated := range pyProtoRules { + if generated { + continue + } + + emptyRule := newTargetBuilder(pyProtoLibraryKind, ruleName, pythonProjectRoot, args.Rel, &emptySiblings, false).build() + res.Empty = append(res.Empty, emptyRule) + } + +} + +// getPyiFilenames returns a set of existing .pyi source file names for a given set of source +// file names if GeneratePyiSrcs is set. Otherwise, returns an empty set. +func getPyiFilenames(filenames *treeset.Set, generatePyiSrcs bool, basePath string) (*treeset.Set, error) { + pyiSrcs := treeset.NewWith(godsutils.StringComparator) + if !generatePyiSrcs { + return pyiSrcs, nil + } + + it := filenames.Iterator() + for it.Next() { + pyiFilename := it.Value().(string) + "i" // foo.py --> foo.pyi + + _, err := os.Stat(filepath.Join(basePath, pyiFilename)) + // If the file DNE or there's some other error, there's nothing to do. + if err == nil { + // pyi file exists, add it + pyiSrcs.Add(pyiFilename) + } + } + return pyiSrcs, nil +} diff --git a/gazelle/python/kinds.go b/gazelle/python/kinds.go index a9483372e2..dac271bec6 100644 --- a/gazelle/python/kinds.go +++ b/gazelle/python/kinds.go @@ -15,13 +15,16 @@ package python import ( + "fmt" + "github.com/bazelbuild/bazel-gazelle/rule" ) const ( - pyBinaryKind = "py_binary" - pyLibraryKind = "py_library" - pyTestKind = "py_test" + pyBinaryKind = "py_binary" + pyLibraryKind = "py_library" + pyProtoLibraryKind = "py_proto_library" + pyTestKind = "py_test" ) // Kinds returns a map that maps rule names (kinds) and information on how to @@ -32,7 +35,8 @@ func (*Python) Kinds() map[string]rule.KindInfo { var pyKinds = map[string]rule.KindInfo{ pyBinaryKind: { - MatchAny: true, + MatchAny: false, + MatchAttrs: []string{"srcs"}, NonEmptyAttrs: map[string]bool{ "deps": true, "main": true, @@ -42,9 +46,12 @@ var pyKinds = map[string]rule.KindInfo{ SubstituteAttrs: map[string]bool{}, MergeableAttrs: map[string]bool{ "srcs": true, + "imports": true, }, ResolveAttrs: map[string]bool{ - "deps": true, + "deps": true, + "pyi_deps": true, + "pyi_srcs": true, }, }, pyLibraryKind: { @@ -60,8 +67,16 @@ var pyKinds = map[string]rule.KindInfo{ "srcs": true, }, ResolveAttrs: map[string]bool{ + "deps": true, + "pyi_deps": true, + "pyi_srcs": true, + }, + }, + pyProtoLibraryKind: { + NonEmptyAttrs: map[string]bool{ "deps": true, }, + ResolveAttrs: map[string]bool{"deps": true}, }, pyTestKind: { MatchAny: false, @@ -76,25 +91,44 @@ var pyKinds = map[string]rule.KindInfo{ "srcs": true, }, ResolveAttrs: map[string]bool{ - "deps": true, + "deps": true, + "pyi_deps": true, + "pyi_srcs": true, }, }, } +func (py *Python) Loads() []rule.LoadInfo { + panic("ApparentLoads should be called instead") +} + // Loads returns .bzl files and symbols they define. Every rule generated by // GenerateRules, now or in the past, should be loadable from one of these // files. -func (py *Python) Loads() []rule.LoadInfo { - return pyLoads +func (py *Python) ApparentLoads(moduleToApparentName func(string) string) []rule.LoadInfo { + return apparentLoads(moduleToApparentName) } -var pyLoads = []rule.LoadInfo{ - { - Name: "@rules_python//python:defs.bzl", - Symbols: []string{ - pyBinaryKind, - pyLibraryKind, - pyTestKind, +func apparentLoads(moduleToApparentName func(string) string) []rule.LoadInfo { + protobuf := moduleToApparentName("protobuf") + if protobuf == "" { + protobuf = "com_google_protobuf" + } + + return []rule.LoadInfo{ + { + Name: "@rules_python//python:defs.bzl", + Symbols: []string{ + pyBinaryKind, + pyLibraryKind, + pyTestKind, + }, }, - }, + { + Name: fmt.Sprintf("@%s//bazel:py_proto_library.bzl", protobuf), + Symbols: []string{ + pyProtoLibraryKind, + }, + }, + } } diff --git a/gazelle/python/parser.go b/gazelle/python/parser.go index 1b2a90dddf..3d0dbe7a5f 100644 --- a/gazelle/python/parser.go +++ b/gazelle/python/parser.go @@ -18,6 +18,8 @@ import ( "context" _ "embed" "fmt" + "log" + "strconv" "strings" "github.com/emirpasic/gods/sets/treeset" @@ -112,9 +114,9 @@ func (p *python3Parser) parse(pyFilenames *treeset.Set) (*treeset.Set, map[strin continue } - modules.Add(m) + addModuleToTreeSet(modules, m) if res.HasMain { - mainModules[res.FileName].Add(m) + addModuleToTreeSet(mainModules[res.FileName], m) } } @@ -123,6 +125,7 @@ func (p *python3Parser) parse(pyFilenames *treeset.Set) (*treeset.Set, map[strin allAnnotations.ignore[k] = v } allAnnotations.includeDeps = append(allAnnotations.includeDeps, annotations.includeDeps...) + allAnnotations.includePytestConftest = annotations.includePytestConftest } allAnnotations.includeDeps = removeDupesFromStringTreeSetSlice(allAnnotations.includeDeps) @@ -145,9 +148,9 @@ func removeDupesFromStringTreeSetSlice(array []string) []string { return dedupe } -// module represents a fully-qualified, dot-separated, Python module as seen on +// Module represents a fully-qualified, dot-separated, Python module as seen on // the import statement, alongside the line number where it happened. -type module struct { +type Module struct { // The fully-qualified, dot-separated, Python module name as seen on import // statements. Name string `json:"name"` @@ -158,11 +161,22 @@ type module struct { // If this was a from import, e.g. from foo import bar, From indicates the module // from which it is imported. From string `json:"from"` + // Whether this import is type-checking only (inside if TYPE_CHECKING block). + TypeCheckingOnly bool `json:"type_checking_only"` } // moduleComparator compares modules by name. func moduleComparator(a, b interface{}) int { - return godsutils.StringComparator(a.(module).Name, b.(module).Name) + return godsutils.StringComparator(a.(Module).Name, b.(Module).Name) +} + +// addModuleToTreeSet adds a module to a treeset.Set, ensuring that a TypeCheckingOnly=false module is +// prefered over a TypeCheckingOnly=true module. +func addModuleToTreeSet(set *treeset.Set, mod Module) { + if mod.TypeCheckingOnly && set.Contains(mod) { + return + } + set.Add(mod) } // annotationKind represents Gazelle annotation kinds. @@ -172,16 +186,20 @@ const ( // The Gazelle annotation prefix. annotationPrefix string = "gazelle:" // The ignore annotation kind. E.g. '# gazelle:ignore '. - annotationKindIgnore annotationKind = "ignore" - annotationKindIncludeDep annotationKind = "include_dep" + annotationKindIgnore annotationKind = "ignore" + // Force a particular target to be added to `deps`. Multiple invocations are + // accumulated and the value can be comma separated. + // Eg: '# gazelle:include_dep //foo/bar:baz,@repo//:target + annotationKindIncludeDep annotationKind = "include_dep" + annotationKindIncludePytestConftest annotationKind = "include_pytest_conftest" ) -// comment represents a Python comment. -type comment string +// Comment represents a Python comment. +type Comment string // asAnnotation returns an annotation object if the comment has the // annotationPrefix. -func (c *comment) asAnnotation() (*annotation, error) { +func (c *Comment) asAnnotation() (*annotation, error) { uncomment := strings.TrimLeft(string(*c), "# ") if !strings.HasPrefix(uncomment, annotationPrefix) { return nil, nil @@ -211,13 +229,18 @@ type annotations struct { ignore map[string]struct{} // Labels that Gazelle should include as deps of the generated target. includeDeps []string + // Whether the conftest.py file, found in the same directory as the current + // python test file, should be added to the py_test target's `deps` attribute. + // A *bool is used so that we can handle the "not set" state. + includePytestConftest *bool } // annotationsFromComments returns all the annotations parsed out of the // comments of a Python module. -func annotationsFromComments(comments []comment) (*annotations, error) { +func annotationsFromComments(comments []Comment) (*annotations, error) { ignore := make(map[string]struct{}) includeDeps := []string{} + var includePytestConftest *bool for _, comment := range comments { annotation, err := comment.asAnnotation() if err != nil { @@ -244,11 +267,21 @@ func annotationsFromComments(comments []comment) (*annotations, error) { includeDeps = append(includeDeps, t) } } + if annotation.kind == annotationKindIncludePytestConftest { + val := annotation.value + parsedVal, err := strconv.ParseBool(val) + if err != nil { + log.Printf("WARNING: unable to cast %q to bool in %q. Ignoring annotation", val, comment) + continue + } + includePytestConftest = &parsedVal + } } } return &annotations{ - ignore: ignore, - includeDeps: includeDeps, + ignore: ignore, + includeDeps: includeDeps, + includePytestConftest: includePytestConftest, }, nil } diff --git a/gazelle/python/private/BUILD.bazel b/gazelle/python/private/BUILD.bazel index e69de29bb2..ccdc080664 100644 --- a/gazelle/python/private/BUILD.bazel +++ b/gazelle/python/private/BUILD.bazel @@ -0,0 +1 @@ +# gazelle:exclude extensions.bzl diff --git a/gazelle/python/python_test.go b/gazelle/python/python_test.go index dd8c2411f1..e7b95cc1e6 100644 --- a/gazelle/python/python_test.go +++ b/gazelle/python/python_test.go @@ -38,7 +38,7 @@ import ( const ( extensionDir = "python" + string(os.PathSeparator) testDataPath = extensionDir + "testdata" + string(os.PathSeparator) - gazelleBinaryName = "gazelle_binary" + gazelleBinaryName = "_gazelle_binary_with_proto" ) func TestGazelleBinary(t *testing.T) { diff --git a/gazelle/python/resolve.go b/gazelle/python/resolve.go index a7b716a829..cc57180a49 100644 --- a/gazelle/python/resolve.go +++ b/gazelle/python/resolve.go @@ -30,7 +30,7 @@ import ( "github.com/emirpasic/gods/sets/treeset" godsutils "github.com/emirpasic/gods/utils" - "github.com/bazelbuild/rules_python/gazelle/pythonconfig" + "github.com/bazel-contrib/rules_python/gazelle/pythonconfig" ) const languageName = "py" @@ -123,6 +123,20 @@ func (py *Resolver) Embeds(r *rule.Rule, from label.Label) []label.Label { return make([]label.Label, 0) } +// addDependency adds a dependency to either the regular deps or pyiDeps set based on +// whether the module is type-checking only. If a module is added as both +// non-type-checking and type-checking, it should end up in deps and not pyiDeps. +func addDependency(dep string, typeCheckingOnly bool, deps, pyiDeps *treeset.Set) { + if typeCheckingOnly { + if !deps.Contains(dep) { + pyiDeps.Add(dep) + } + } else { + deps.Add(dep) + pyiDeps.Remove(dep) + } +} + // Resolve translates imported libraries for a given rule into Bazel // dependencies. Information about imported libraries is returned for each // rule generated by language.GenerateRules in @@ -141,9 +155,11 @@ func (py *Resolver) Resolve( // join with the main Gazelle binary with other rules. It may conflict with // other generators that generate py_* targets. deps := treeset.NewWith(godsutils.StringComparator) + pyiDeps := treeset.NewWith(godsutils.StringComparator) + cfgs := c.Exts[languageName].(pythonconfig.Configs) + cfg := cfgs[from.Pkg] + if modulesRaw != nil { - cfgs := c.Exts[languageName].(pythonconfig.Configs) - cfg := cfgs[from.Pkg] pythonProjectRoot := cfg.PythonProjectRoot() modules := modulesRaw.(*treeset.Set) it := modules.Iterator() @@ -151,9 +167,56 @@ func (py *Resolver) Resolve( hasFatalError := false MODULES_LOOP: for it.Next() { - mod := it.Value().(module) - moduleParts := strings.Split(mod.Name, ".") - possibleModules := []string{mod.Name} + mod := it.Value().(Module) + moduleName := mod.Name + // Transform relative imports `.` or `..foo.bar` into the package path from root. + if strings.HasPrefix(mod.From, ".") { + if !cfg.ExperimentalAllowRelativeImports() { + continue MODULES_LOOP + } + + // Count number of leading dots in mod.From (e.g., ".." = 2, "...foo.bar" = 3) + relativeDepth := strings.IndexFunc(mod.From, func(r rune) bool { return r != '.' }) + if relativeDepth == -1 { + relativeDepth = len(mod.From) + } + + // Extract final symbol (e.g., "some_function") from mod.Name + imported := mod.Name + if idx := strings.LastIndex(mod.Name, "."); idx >= 0 { + imported = mod.Name[idx+1:] + } + + // Optional subpath in 'from' clause, e.g. "from ...my_library.foo import x" + fromPath := strings.TrimLeft(mod.From, ".") + var fromParts []string + if fromPath != "" { + fromParts = strings.Split(fromPath, ".") + } + + // Current Bazel package as path segments + pkgParts := strings.Split(from.Pkg, "/") + + if relativeDepth-1 > len(pkgParts) { + log.Printf("ERROR: Invalid relative import %q in %q: exceeds package root.", mod.Name, mod.Filepath) + continue MODULES_LOOP + } + + // Go up relativeDepth - 1 levels + baseParts := pkgParts + if relativeDepth > 1 { + baseParts = pkgParts[:len(pkgParts)-(relativeDepth-1)] + } + // Build absolute module path + absParts := append([]string{}, baseParts...) // base path + absParts = append(absParts, fromParts...) // subpath from 'from' + absParts = append(absParts, imported) // actual imported symbol + + moduleName = strings.Join(absParts, ".") + } + + moduleParts := strings.Split(moduleName, ".") + possibleModules := []string{moduleName} for len(moduleParts) > 1 { // Iterate back through the possible imports until // a match is found. @@ -179,7 +242,7 @@ func (py *Resolver) Resolve( override.Repo = "" } dep := override.Rel(from.Repo, from.Pkg).String() - deps.Add(dep) + addDependency(dep, mod.TypeCheckingOnly, deps, pyiDeps) if explainDependency == dep { log.Printf("Explaining dependency (%s): "+ "in the target %q, the file %q imports %q at line %d, "+ @@ -189,8 +252,21 @@ func (py *Resolver) Resolve( continue MODULES_LOOP } } else { - if dep, ok := cfg.FindThirdPartyDependency(moduleName); ok { - deps.Add(dep) + if dep, distributionName, ok := cfg.FindThirdPartyDependency(moduleName); ok { + addDependency(dep, mod.TypeCheckingOnly, deps, pyiDeps) + // Add the type and stub dependencies if they exist. + modules := []string{ + fmt.Sprintf("%s_stubs", strings.ToLower(distributionName)), + fmt.Sprintf("%s_types", strings.ToLower(distributionName)), + fmt.Sprintf("types_%s", strings.ToLower(distributionName)), + fmt.Sprintf("stubs_%s", strings.ToLower(distributionName)), + } + for _, module := range modules { + if dep, _, ok := cfg.FindThirdPartyDependency(module); ok { + // Type stub packages are added as type-checking only. + addDependency(dep, true, deps, pyiDeps) + } + } if explainDependency == dep { log.Printf("Explaining dependency (%s): "+ "in the target %q, the file %q imports %q at line %d, "+ @@ -202,7 +278,7 @@ func (py *Resolver) Resolve( matches := ix.FindRulesByImportWithConfig(c, imp, languageName) if len(matches) == 0 { // Check if the imported module is part of the standard library. - if isStdModule(module{Name: moduleName}) { + if isStdModule(Module{Name: moduleName}) { continue MODULES_LOOP } else if cfg.ValidateImportStatements() { err := fmt.Errorf( @@ -247,7 +323,7 @@ func (py *Resolver) Resolve( } matchLabel := filteredMatches[0].Label.Rel(from.Repo, from.Pkg) dep := matchLabel.String() - deps.Add(dep) + addDependency(dep, mod.TypeCheckingOnly, deps, pyiDeps) if explainDependency == dep { log.Printf("Explaining dependency (%s): "+ "in the target %q, the file %q imports %q at line %d, "+ @@ -272,6 +348,34 @@ func (py *Resolver) Resolve( os.Exit(1) } } + + addResolvedDeps(r, deps) + + if cfg.GeneratePyiDeps() { + if !deps.Empty() { + r.SetAttr("deps", convertDependencySetToExpr(deps)) + } + if !pyiDeps.Empty() { + r.SetAttr("pyi_deps", convertDependencySetToExpr(pyiDeps)) + } + } else { + // When generate_pyi_deps is false, merge both deps and pyiDeps into deps + combinedDeps := treeset.NewWith(godsutils.StringComparator) + combinedDeps.Add(deps.Values()...) + combinedDeps.Add(pyiDeps.Values()...) + + if !combinedDeps.Empty() { + r.SetAttr("deps", convertDependencySetToExpr(combinedDeps)) + } + } +} + +// addResolvedDeps adds the pre-resolved dependencies from the rule's private attributes +// to the provided deps set. +func addResolvedDeps( + r *rule.Rule, + deps *treeset.Set, +) { resolvedDeps := r.PrivateAttr(resolvedDepsKey).(*treeset.Set) if !resolvedDeps.Empty() { it := resolvedDeps.Iterator() @@ -279,9 +383,6 @@ func (py *Resolver) Resolve( deps.Add(it.Value()) } } - if !deps.Empty() { - r.SetAttr("deps", convertDependencySetToExpr(deps)) - } } // targetListFromResults returns a string with the human-readable list of diff --git a/gazelle/python/std_modules.go b/gazelle/python/std_modules.go index e10f87b6ea..ecb4f4c454 100644 --- a/gazelle/python/std_modules.go +++ b/gazelle/python/std_modules.go @@ -34,7 +34,7 @@ func init() { } } -func isStdModule(m module) bool { +func isStdModule(m Module) bool { _, ok := stdModules[m.Name] return ok } diff --git a/gazelle/python/std_modules_test.go b/gazelle/python/std_modules_test.go index bc22638e69..dbcd18c9d6 100644 --- a/gazelle/python/std_modules_test.go +++ b/gazelle/python/std_modules_test.go @@ -21,7 +21,7 @@ import ( ) func TestIsStdModule(t *testing.T) { - assert.True(t, isStdModule(module{Name: "unittest"})) - assert.True(t, isStdModule(module{Name: "os.path"})) - assert.False(t, isStdModule(module{Name: "foo"})) + assert.True(t, isStdModule(Module{Name: "unittest"})) + assert.True(t, isStdModule(Module{Name: "os.path"})) + assert.False(t, isStdModule(Module{Name: "foo"})) } diff --git a/gazelle/python/target.go b/gazelle/python/target.go index c40d6fb3b7..c7009a6a84 100644 --- a/gazelle/python/target.go +++ b/gazelle/python/target.go @@ -15,41 +15,48 @@ package python import ( + "path/filepath" + "github.com/bazelbuild/bazel-gazelle/config" "github.com/bazelbuild/bazel-gazelle/rule" "github.com/emirpasic/gods/sets/treeset" godsutils "github.com/emirpasic/gods/utils" - "path/filepath" ) // targetBuilder builds targets to be generated by Gazelle. type targetBuilder struct { - kind string - name string - pythonProjectRoot string - bzlPackage string - srcs *treeset.Set - siblingSrcs *treeset.Set - deps *treeset.Set - resolvedDeps *treeset.Set - visibility *treeset.Set - main *string - imports []string - testonly bool + kind string + name string + pythonProjectRoot string + bzlPackage string + srcs *treeset.Set + pyiSrcs *treeset.Set + siblingSrcs *treeset.Set + deps *treeset.Set + resolvedDeps *treeset.Set + visibility *treeset.Set + main *string + imports []string + testonly bool + annotations *annotations + resolveSiblingImports bool } // newTargetBuilder constructs a new targetBuilder. -func newTargetBuilder(kind, name, pythonProjectRoot, bzlPackage string, siblingSrcs *treeset.Set) *targetBuilder { +func newTargetBuilder(kind, name, pythonProjectRoot, bzlPackage string, siblingSrcs *treeset.Set, resolveSiblingImports bool) *targetBuilder { return &targetBuilder{ - kind: kind, - name: name, - pythonProjectRoot: pythonProjectRoot, - bzlPackage: bzlPackage, - srcs: treeset.NewWith(godsutils.StringComparator), - siblingSrcs: siblingSrcs, - deps: treeset.NewWith(moduleComparator), - resolvedDeps: treeset.NewWith(godsutils.StringComparator), - visibility: treeset.NewWith(godsutils.StringComparator), + kind: kind, + name: name, + pythonProjectRoot: pythonProjectRoot, + bzlPackage: bzlPackage, + srcs: treeset.NewWith(godsutils.StringComparator), + pyiSrcs: treeset.NewWith(godsutils.StringComparator), + siblingSrcs: siblingSrcs, + deps: treeset.NewWith(moduleComparator), + resolvedDeps: treeset.NewWith(godsutils.StringComparator), + visibility: treeset.NewWith(godsutils.StringComparator), + annotations: new(annotations), + resolveSiblingImports: resolveSiblingImports, } } @@ -68,18 +75,34 @@ func (t *targetBuilder) addSrcs(srcs *treeset.Set) *targetBuilder { return t } +// addPyiSrc adds a single pyi_src to the target. +func (t *targetBuilder) addPyiSrc(pyiSrc string) *targetBuilder { + t.pyiSrcs.Add(pyiSrc) + return t +} + +// addPyiSrcs adds multiple pyi_srcs to the target. +func (t *targetBuilder) addPyiSrcs(pyiSrcs *treeset.Set) *targetBuilder { + it := pyiSrcs.Iterator() + for it.Next() { + t.pyiSrcs.Add(it.Value().(string)) + } + return t +} + // addModuleDependency adds a single module dep to the target. -func (t *targetBuilder) addModuleDependency(dep module) *targetBuilder { +func (t *targetBuilder) addModuleDependency(dep Module) *targetBuilder { fileName := dep.Name + ".py" if dep.From != "" { fileName = dep.From + ".py" } - if t.siblingSrcs.Contains(fileName) && fileName != filepath.Base(dep.Filepath) { + if t.resolveSiblingImports && t.siblingSrcs.Contains(fileName) && fileName != filepath.Base(dep.Filepath) { // importing another module from the same package, converting to absolute imports to make // dependency resolution easier dep.Name = importSpecFromSrc(t.pythonProjectRoot, t.bzlPackage, fileName).Imp } - t.deps.Add(dep) + + addModuleToTreeSet(t.deps, dep) return t } @@ -87,7 +110,7 @@ func (t *targetBuilder) addModuleDependency(dep module) *targetBuilder { func (t *targetBuilder) addModuleDependencies(deps *treeset.Set) *targetBuilder { it := deps.Iterator() for it.Next() { - t.addModuleDependency(it.Value().(module)) + t.addModuleDependency(it.Value().(Module)) } return t } @@ -128,6 +151,12 @@ func (t *targetBuilder) setTestonly() *targetBuilder { return t } +// setAnnotations sets the annotations attribute on the target. +func (t *targetBuilder) setAnnotations(val annotations) *targetBuilder { + t.annotations = &val + return t +} + // generateImportsAttribute generates the imports attribute. // These are a list of import directories to be added to the PYTHONPATH. In our // case, the value we add is on Bazel sub-packages to be able to perform imports @@ -153,6 +182,9 @@ func (t *targetBuilder) build() *rule.Rule { if !t.srcs.Empty() { r.SetAttr("srcs", t.srcs.Values()) } + if !t.pyiSrcs.Empty() { + r.SetAttr("pyi_srcs", t.pyiSrcs.Values()) + } if !t.visibility.Empty() { r.SetAttr("visibility", t.visibility.Values()) } diff --git a/gazelle/python/testdata/add_type_stub_packages/BUILD.in b/gazelle/python/testdata/add_type_stub_packages/BUILD.in new file mode 100644 index 0000000000..99d122ad12 --- /dev/null +++ b/gazelle/python/testdata/add_type_stub_packages/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_generate_pyi_deps true diff --git a/gazelle/python/testdata/add_type_stub_packages/BUILD.out b/gazelle/python/testdata/add_type_stub_packages/BUILD.out new file mode 100644 index 0000000000..1a5b640ac8 --- /dev/null +++ b/gazelle/python/testdata/add_type_stub_packages/BUILD.out @@ -0,0 +1,18 @@ +load("@rules_python//python:defs.bzl", "py_binary") + +# gazelle:python_generate_pyi_deps true + +py_binary( + name = "add_type_stub_packages_bin", + srcs = ["__main__.py"], + main = "__main__.py", + pyi_deps = [ + "@gazelle_python_test//boto3_stubs", + "@gazelle_python_test//django_types", + ], + visibility = ["//:__subpackages__"], + deps = [ + "@gazelle_python_test//boto3", + "@gazelle_python_test//django", + ], +) diff --git a/gazelle/python/testdata/add_type_stub_packages/README.md b/gazelle/python/testdata/add_type_stub_packages/README.md new file mode 100644 index 0000000000..e3a2afee81 --- /dev/null +++ b/gazelle/python/testdata/add_type_stub_packages/README.md @@ -0,0 +1,4 @@ +# Add stubs to `deps` of `py_library` target + +This test case asserts that +* if a package has the corresponding stub available, it is added to the `pyi_deps` of the `py_library` target. diff --git a/gazelle/python/testdata/remove_invalid_library/WORKSPACE b/gazelle/python/testdata/add_type_stub_packages/WORKSPACE similarity index 100% rename from gazelle/python/testdata/remove_invalid_library/WORKSPACE rename to gazelle/python/testdata/add_type_stub_packages/WORKSPACE diff --git a/third_party/rules_pycross/pycross/private/BUILD.bazel b/gazelle/python/testdata/add_type_stub_packages/__main__.py similarity index 91% rename from third_party/rules_pycross/pycross/private/BUILD.bazel rename to gazelle/python/testdata/add_type_stub_packages/__main__.py index f59b087027..96384cfb13 100644 --- a/third_party/rules_pycross/pycross/private/BUILD.bazel +++ b/gazelle/python/testdata/add_type_stub_packages/__main__.py @@ -1,4 +1,3 @@ -# Copyright 2023 Jeremy Volkman. All rights reserved. # Copyright 2023 The Bazel Authors. All rights reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -12,3 +11,6 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. + +import boto3 +import django diff --git a/python/private/register_extension_info.bzl b/gazelle/python/testdata/add_type_stub_packages/gazelle_python.yaml similarity index 70% rename from python/private/register_extension_info.bzl rename to gazelle/python/testdata/add_type_stub_packages/gazelle_python.yaml index 408df6261e..f498d07f2f 100644 --- a/python/private/register_extension_info.bzl +++ b/gazelle/python/testdata/add_type_stub_packages/gazelle_python.yaml @@ -4,15 +4,19 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. -"""Stub implementation to make patching easier.""" -# buildifier: disable=unused-variable -def register_extension_info(**kwargs): - """A no-op stub to make Google patching easier.""" +manifest: + modules_mapping: + boto3: boto3 + boto3_stubs: boto3_stubs + django_types: django_types + django: Django + + pip_deps_repository_name: gazelle_python_test diff --git a/gazelle/python/testdata/relative_imports/test.yaml b/gazelle/python/testdata/add_type_stub_packages/test.yaml similarity index 100% rename from gazelle/python/testdata/relative_imports/test.yaml rename to gazelle/python/testdata/add_type_stub_packages/test.yaml diff --git a/gazelle/python/testdata/annotation_include_dep/BUILD.in b/gazelle/python/testdata/annotation_include_dep/BUILD.in index af2c2cea4b..5131712aca 100644 --- a/gazelle/python/testdata/annotation_include_dep/BUILD.in +++ b/gazelle/python/testdata/annotation_include_dep/BUILD.in @@ -1 +1,2 @@ # gazelle:python_generation_mode file +# gazelle:python_resolve_sibling_imports true diff --git a/gazelle/python/testdata/annotation_include_dep/BUILD.out b/gazelle/python/testdata/annotation_include_dep/BUILD.out index 1cff8f4676..412bf456f5 100644 --- a/gazelle/python/testdata/annotation_include_dep/BUILD.out +++ b/gazelle/python/testdata/annotation_include_dep/BUILD.out @@ -1,6 +1,7 @@ load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test") # gazelle:python_generation_mode file +# gazelle:python_resolve_sibling_imports true py_library( name = "__init__", diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/README.md b/gazelle/python/testdata/annotation_include_pytest_conftest/README.md new file mode 100644 index 0000000000..6a347d154e --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/README.md @@ -0,0 +1,25 @@ +# Annotation: Include Pytest Conftest + +Validate that the `# gazelle:include_pytest_conftest` annotation follows +this logic: + ++ When a `conftest.py` file does not exist: + + all values have no affect ++ When a `conftest.py` file does exist: + + Truthy values add `:conftest` to `deps`. + + Falsey values do not add `:conftest` to `deps`. + + Unset (no annotation) performs the default action. + +Additionally, we test that: + ++ invalid values (eg `foo`) print a warning and then act as if + the annotation was not present. ++ last annotation (highest line number) wins. ++ the annotation has no effect on non-test files/targets. ++ the `include_dep` can still inject `:conftest` even when `include_pytest_conftest` + is false. ++ `import conftest` will still add the dep even when `include_pytest_conftest` is + false. + +An annotation without a value is not tested, as that's part of the core +annotation framework and not specific to this annotation. diff --git a/tests/integration/ignore_root_user_error/submodule/WORKSPACE b/gazelle/python/testdata/annotation_include_pytest_conftest/WORKSPACE similarity index 100% rename from tests/integration/ignore_root_user_error/submodule/WORKSPACE rename to gazelle/python/testdata/annotation_include_pytest_conftest/WORKSPACE diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/test.yaml b/gazelle/python/testdata/annotation_include_pytest_conftest/test.yaml new file mode 100644 index 0000000000..e643d0e90c --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/test.yaml @@ -0,0 +1,5 @@ +--- +expect: + stderr: | + gazelle: WARNING: unable to cast "foo" to bool in "# gazelle:include_pytest_conftest foo". Ignoring annotation + exit_code: 0 diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/BUILD.in b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/BUILD.in new file mode 100644 index 0000000000..5c25b0d5a6 --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_resolve_sibling_imports true diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/BUILD.out b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/BUILD.out new file mode 100644 index 0000000000..52b915208e --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/BUILD.out @@ -0,0 +1,70 @@ +load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test") + +# gazelle:python_resolve_sibling_imports true + +py_binary( + name = "binary", + srcs = ["binary.py"], + visibility = ["//:__subpackages__"], +) + +py_library( + name = "with_conftest", + srcs = [ + "binary.py", + "library.py", + ], + visibility = ["//:__subpackages__"], +) + +py_library( + name = "conftest", + testonly = True, + srcs = ["conftest.py"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "bad_value_test", + srcs = ["bad_value_test.py"], + deps = [":conftest"], +) + +py_test( + name = "conftest_imported_test", + srcs = ["conftest_imported_test.py"], + deps = [":conftest"], +) + +py_test( + name = "conftest_included_test", + srcs = ["conftest_included_test.py"], + deps = [":conftest"], +) + +py_test( + name = "false_test", + srcs = ["false_test.py"], +) + +py_test( + name = "falsey_test", + srcs = ["falsey_test.py"], +) + +py_test( + name = "last_value_wins_test", + srcs = ["last_value_wins_test.py"], +) + +py_test( + name = "true_test", + srcs = ["true_test.py"], + deps = [":conftest"], +) + +py_test( + name = "unset_test", + srcs = ["unset_test.py"], + deps = [":conftest"], +) diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/bad_value_test.py b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/bad_value_test.py new file mode 100644 index 0000000000..af2e8c54e0 --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/bad_value_test.py @@ -0,0 +1 @@ +# gazelle:include_pytest_conftest foo diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/binary.py b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/binary.py new file mode 100644 index 0000000000..d6dc8413d4 --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/binary.py @@ -0,0 +1,3 @@ +# gazelle:include_pytest_conftest true +if __name__ == "__main__": + pass diff --git a/gazelle/python/testdata/relative_imports/package2/BUILD.in b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/conftest.py similarity index 100% rename from gazelle/python/testdata/relative_imports/package2/BUILD.in rename to gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/conftest.py diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/conftest_imported_test.py b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/conftest_imported_test.py new file mode 100644 index 0000000000..2c72ca4df1 --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/conftest_imported_test.py @@ -0,0 +1,3 @@ +import conftest + +# gazelle:include_pytest_conftest false diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/conftest_included_test.py b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/conftest_included_test.py new file mode 100644 index 0000000000..c942bfb1ab --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/conftest_included_test.py @@ -0,0 +1,2 @@ +# gazelle:include_dep :conftest +# gazelle:include_pytest_conftest false diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/false_test.py b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/false_test.py new file mode 100644 index 0000000000..ba71a2818b --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/false_test.py @@ -0,0 +1 @@ +# gazelle:include_pytest_conftest false diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/falsey_test.py b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/falsey_test.py new file mode 100644 index 0000000000..c4387b3a8c --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/falsey_test.py @@ -0,0 +1 @@ +# gazelle:include_pytest_conftest 0 diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/last_value_wins_test.py b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/last_value_wins_test.py new file mode 100644 index 0000000000..6ffc06f9c0 --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/last_value_wins_test.py @@ -0,0 +1,6 @@ +# gazelle:include_pytest_conftest true +# gazelle:include_pytest_conftest TRUE +# gazelle:include_pytest_conftest False +# gazelle:include_pytest_conftest 0 +# gazelle:include_pytest_conftest 1 +# gazelle:include_pytest_conftest F diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/library.py b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/library.py new file mode 100644 index 0000000000..b2d10359da --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/library.py @@ -0,0 +1 @@ +# gazelle:include_pytest_conftest true diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/true_test.py b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/true_test.py new file mode 100644 index 0000000000..b2d10359da --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/true_test.py @@ -0,0 +1 @@ +# gazelle:include_pytest_conftest true diff --git a/sphinxdocs/src/sphinx_bzl/__init__.py b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/unset_test.py similarity index 100% rename from sphinxdocs/src/sphinx_bzl/__init__.py rename to gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/unset_test.py diff --git a/tests/integration/ignore_root_user_error/submodule/BUILD.bazel b/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/BUILD.in similarity index 100% rename from tests/integration/ignore_root_user_error/submodule/BUILD.bazel rename to gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/BUILD.in diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/BUILD.out b/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/BUILD.out new file mode 100644 index 0000000000..01383344c5 --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/BUILD.out @@ -0,0 +1,16 @@ +load("@rules_python//python:defs.bzl", "py_test") + +py_test( + name = "false_test", + srcs = ["false_test.py"], +) + +py_test( + name = "true_test", + srcs = ["true_test.py"], +) + +py_test( + name = "unset_test", + srcs = ["unset_test.py"], +) diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/false_test.py b/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/false_test.py new file mode 100644 index 0000000000..ba71a2818b --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/false_test.py @@ -0,0 +1 @@ +# gazelle:include_pytest_conftest false diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/true_test.py b/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/true_test.py new file mode 100644 index 0000000000..b2d10359da --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/true_test.py @@ -0,0 +1 @@ +# gazelle:include_pytest_conftest true diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/unset_test.py b/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/unset_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/BUILD.in b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/BUILD.in new file mode 100644 index 0000000000..63b547f0b3 --- /dev/null +++ b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/BUILD.in @@ -0,0 +1,9 @@ +load("@rules_python//python:defs.bzl", "py_binary") + +# gazelle:python_generation_mode file + +py_binary( + name = "a", + srcs = ["a.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/BUILD.out b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/BUILD.out new file mode 100644 index 0000000000..8f49cccd9f --- /dev/null +++ b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/BUILD.out @@ -0,0 +1,15 @@ +load("@rules_python//python:defs.bzl", "py_binary") + +# gazelle:python_generation_mode file + +py_binary( + name = "a", + srcs = ["a.py"], + visibility = ["//:__subpackages__"], +) + +py_binary( + name = "b", + srcs = ["b.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/README.md b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/README.md new file mode 100644 index 0000000000..5aa499f4ad --- /dev/null +++ b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/README.md @@ -0,0 +1,3 @@ +# Partial update with multiple per-file binaries + +This test case asserts that when there are multiple binaries in a package, and no __main__.py, and the BUILD file already includes a py_binary for one of the files, a py_binary is generated for the other file. diff --git a/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/WORKSPACE b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/a.py b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/a.py new file mode 100644 index 0000000000..9c97da4809 --- /dev/null +++ b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/a.py @@ -0,0 +1,2 @@ +if __name__ == "__main__": + print("Hello, world!") diff --git a/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/b.py b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/b.py new file mode 100644 index 0000000000..9c97da4809 --- /dev/null +++ b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/b.py @@ -0,0 +1,2 @@ +if __name__ == "__main__": + print("Hello, world!") diff --git a/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/test.yaml b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/test.yaml new file mode 100644 index 0000000000..346ecd7ae8 --- /dev/null +++ b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/test.yaml @@ -0,0 +1,17 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +--- +expect: + exit_code: 0 diff --git a/gazelle/python/testdata/clear_out_deps/BUILD.in b/gazelle/python/testdata/clear_out_deps/BUILD.in new file mode 100644 index 0000000000..99d122ad12 --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_generate_pyi_deps true diff --git a/gazelle/python/testdata/clear_out_deps/BUILD.out b/gazelle/python/testdata/clear_out_deps/BUILD.out new file mode 100644 index 0000000000..99d122ad12 --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/BUILD.out @@ -0,0 +1 @@ +# gazelle:python_generate_pyi_deps true diff --git a/gazelle/python/testdata/clear_out_deps/README.md b/gazelle/python/testdata/clear_out_deps/README.md new file mode 100644 index 0000000000..53b62a46d5 --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/README.md @@ -0,0 +1,9 @@ +# Clearing deps / pyi_deps + +This test case asserts that an existing `py_library` specifying `deps` and +`pyi_deps` have these attributes removed if the corresponding imports are +removed. + +`a/BUILD.in` declares `deps`/`pyi_deps` on non-existing libraries, `b/BUILD.in` declares dependency on `//a` +without a matching import, and `c/BUILD.in` declares both `deps` and `pyi_deps` as `["//a", "//b"]`, but +it should have only `//a` as `deps` and only `//b` as `pyi_deps`. diff --git a/gazelle/python/testdata/clear_out_deps/WORKSPACE b/gazelle/python/testdata/clear_out_deps/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/clear_out_deps/a/BUILD.in b/gazelle/python/testdata/clear_out_deps/a/BUILD.in new file mode 100644 index 0000000000..832683b22a --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/a/BUILD.in @@ -0,0 +1,9 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "a", + srcs = ["__init__.py"], + pyi_deps = ["//:nonexistent_pyi_dep"], + visibility = ["//:__subpackages__"], + deps = ["//nonexistent_dep"], +) diff --git a/gazelle/python/testdata/clear_out_deps/a/BUILD.out b/gazelle/python/testdata/clear_out_deps/a/BUILD.out new file mode 100644 index 0000000000..2668e97c42 --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/a/BUILD.out @@ -0,0 +1,7 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "a", + srcs = ["__init__.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/clear_out_deps/a/__init__.py b/gazelle/python/testdata/clear_out_deps/a/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/clear_out_deps/b/BUILD.in b/gazelle/python/testdata/clear_out_deps/b/BUILD.in new file mode 100644 index 0000000000..14cce87498 --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/b/BUILD.in @@ -0,0 +1,9 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "b", + srcs = ["__init__.py"], + pyi_deps = ["//a"], + visibility = ["//:__subpackages__"], + deps = ["//a"], +) diff --git a/gazelle/python/testdata/clear_out_deps/b/BUILD.out b/gazelle/python/testdata/clear_out_deps/b/BUILD.out new file mode 100644 index 0000000000..7305850a2e --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/b/BUILD.out @@ -0,0 +1,7 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "b", + srcs = ["__init__.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/clear_out_deps/b/__init__.py b/gazelle/python/testdata/clear_out_deps/b/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/clear_out_deps/c/BUILD.in b/gazelle/python/testdata/clear_out_deps/c/BUILD.in new file mode 100644 index 0000000000..10ace67dd2 --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/c/BUILD.in @@ -0,0 +1,9 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "c", + srcs = ["__init__.py"], + pyi_deps = ["//a", "//b"], + visibility = ["//:__subpackages__"], + deps = ["//a", "//b"], +) diff --git a/gazelle/python/testdata/clear_out_deps/c/BUILD.out b/gazelle/python/testdata/clear_out_deps/c/BUILD.out new file mode 100644 index 0000000000..d1aa97e5aa --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/c/BUILD.out @@ -0,0 +1,9 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "c", + srcs = ["__init__.py"], + pyi_deps = ["//b"], + visibility = ["//:__subpackages__"], + deps = ["//a"], +) diff --git a/gazelle/python/testdata/clear_out_deps/c/__init__.py b/gazelle/python/testdata/clear_out_deps/c/__init__.py new file mode 100644 index 0000000000..32d017f28a --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/c/__init__.py @@ -0,0 +1,6 @@ +from typing import TYPE_CHECKING + +import a + +if TYPE_CHECKING: + import b diff --git a/gazelle/python/testdata/clear_out_deps/test.yaml b/gazelle/python/testdata/clear_out_deps/test.yaml new file mode 100644 index 0000000000..88a0cbf018 --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/test.yaml @@ -0,0 +1,2 @@ + +--- diff --git a/gazelle/python/testdata/dependency_resolution_order/__init__.py b/gazelle/python/testdata/dependency_resolution_order/__init__.py index e2d0a8a979..4b40aa9f54 100644 --- a/gazelle/python/testdata/dependency_resolution_order/__init__.py +++ b/gazelle/python/testdata/dependency_resolution_order/__init__.py @@ -22,9 +22,8 @@ # we can still override "third_party.foo.bar" import third_party.foo.bar -from third_party import baz - import third_party +from third_party import baz _ = sys _ = bar diff --git a/gazelle/python/testdata/directive_python_default_visibility/README.md b/gazelle/python/testdata/directive_python_default_visibility/README.md index be42792375..60582d6407 100644 --- a/gazelle/python/testdata/directive_python_default_visibility/README.md +++ b/gazelle/python/testdata/directive_python_default_visibility/README.md @@ -18,4 +18,4 @@ correctly: they interact with sub-packages. -[gh-1682]: https://github.com/bazelbuild/rules_python/issues/1682 +[gh-1682]: https://github.com/bazel-contrib/rules_python/issues/1682 diff --git a/gazelle/python/testdata/directive_python_generate_proto/README.md b/gazelle/python/testdata/directive_python_generate_proto/README.md new file mode 100644 index 0000000000..54261f47ca --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/README.md @@ -0,0 +1,9 @@ +# Directive: `python_generate_proto` + +This test case asserts that the `# gazelle:python_generate_proto` directive +correctly: + +1. Uses the default value when `python_generate_proto` is not set. +2. Generates (or not) `py_proto_library` when `python_generate_proto` is set, based on whether a proto is present. + +[gh-2994]: https://github.com/bazel-contrib/rules_python/issues/2994 diff --git a/gazelle/python/testdata/directive_python_generate_proto/WORKSPACE b/gazelle/python/testdata/directive_python_generate_proto/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/directive_python_generate_proto/test.yaml b/gazelle/python/testdata/directive_python_generate_proto/test.yaml new file mode 100644 index 0000000000..36dd656b39 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test.yaml @@ -0,0 +1,3 @@ +--- +expect: + exit_code: 0 diff --git a/gazelle/python/testdata/directive_python_generate_proto/test1_default_with_proto/BUILD.in b/gazelle/python/testdata/directive_python_generate_proto/test1_default_with_proto/BUILD.in new file mode 100644 index 0000000000..9784aafc17 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test1_default_with_proto/BUILD.in @@ -0,0 +1,9 @@ +load("@rules_proto//proto:defs.bzl", "proto_library") + +# python_generate_proto is not set, so py_proto_library is not generated. + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto/test1_default_with_proto/BUILD.out b/gazelle/python/testdata/directive_python_generate_proto/test1_default_with_proto/BUILD.out new file mode 100644 index 0000000000..9784aafc17 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test1_default_with_proto/BUILD.out @@ -0,0 +1,9 @@ +load("@rules_proto//proto:defs.bzl", "proto_library") + +# python_generate_proto is not set, so py_proto_library is not generated. + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto/test1_default_with_proto/foo.proto b/gazelle/python/testdata/directive_python_generate_proto/test1_default_with_proto/foo.proto new file mode 100644 index 0000000000..fe2af27aa6 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test1_default_with_proto/foo.proto @@ -0,0 +1,7 @@ +syntax = "proto3"; + +package foo; + +message Foo { + string bar = 1; +} diff --git a/gazelle/python/testdata/directive_python_generate_proto/test2_default_without_proto/BUILD.in b/gazelle/python/testdata/directive_python_generate_proto/test2_default_without_proto/BUILD.in new file mode 100644 index 0000000000..0a869d0fd5 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test2_default_without_proto/BUILD.in @@ -0,0 +1 @@ +# python_generate_proto is not set, so py_proto_library is not generated. diff --git a/gazelle/python/testdata/directive_python_generate_proto/test2_default_without_proto/BUILD.out b/gazelle/python/testdata/directive_python_generate_proto/test2_default_without_proto/BUILD.out new file mode 100644 index 0000000000..0a869d0fd5 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test2_default_without_proto/BUILD.out @@ -0,0 +1 @@ +# python_generate_proto is not set, so py_proto_library is not generated. diff --git a/gazelle/python/testdata/directive_python_generate_proto/test3_disabled_with_proto/BUILD.in b/gazelle/python/testdata/directive_python_generate_proto/test3_disabled_with_proto/BUILD.in new file mode 100644 index 0000000000..62fd4be661 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test3_disabled_with_proto/BUILD.in @@ -0,0 +1,9 @@ +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto false + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto/test3_disabled_with_proto/BUILD.out b/gazelle/python/testdata/directive_python_generate_proto/test3_disabled_with_proto/BUILD.out new file mode 100644 index 0000000000..62fd4be661 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test3_disabled_with_proto/BUILD.out @@ -0,0 +1,9 @@ +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto false + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto/test3_disabled_with_proto/foo.proto b/gazelle/python/testdata/directive_python_generate_proto/test3_disabled_with_proto/foo.proto new file mode 100644 index 0000000000..022e29ae69 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test3_disabled_with_proto/foo.proto @@ -0,0 +1,7 @@ +syntax = "proto3"; + +package foo.bar; + +message Foo { + string bar = 1; +} diff --git a/gazelle/python/testdata/directive_python_generate_proto/test4_disabled_without_proto/BUILD.in b/gazelle/python/testdata/directive_python_generate_proto/test4_disabled_without_proto/BUILD.in new file mode 100644 index 0000000000..b283b5fb51 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test4_disabled_without_proto/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_generate_proto false diff --git a/gazelle/python/testdata/directive_python_generate_proto/test4_disabled_without_proto/BUILD.out b/gazelle/python/testdata/directive_python_generate_proto/test4_disabled_without_proto/BUILD.out new file mode 100644 index 0000000000..b283b5fb51 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test4_disabled_without_proto/BUILD.out @@ -0,0 +1 @@ +# gazelle:python_generate_proto false diff --git a/gazelle/python/testdata/directive_python_generate_proto/test5_enabled_with_proto/BUILD.in b/gazelle/python/testdata/directive_python_generate_proto/test5_enabled_with_proto/BUILD.in new file mode 100644 index 0000000000..4713404b19 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test5_enabled_with_proto/BUILD.in @@ -0,0 +1,9 @@ +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto/test5_enabled_with_proto/BUILD.out b/gazelle/python/testdata/directive_python_generate_proto/test5_enabled_with_proto/BUILD.out new file mode 100644 index 0000000000..686252f27c --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test5_enabled_with_proto/BUILD.out @@ -0,0 +1,16 @@ +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_py_pb2", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto/test5_enabled_with_proto/foo.proto b/gazelle/python/testdata/directive_python_generate_proto/test5_enabled_with_proto/foo.proto new file mode 100644 index 0000000000..fe2af27aa6 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test5_enabled_with_proto/foo.proto @@ -0,0 +1,7 @@ +syntax = "proto3"; + +package foo; + +message Foo { + string bar = 1; +} diff --git a/gazelle/python/testdata/directive_python_generate_proto/test6_enabled_without_proto/BUILD.in b/gazelle/python/testdata/directive_python_generate_proto/test6_enabled_without_proto/BUILD.in new file mode 100644 index 0000000000..ce3eec6001 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test6_enabled_without_proto/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_generate_proto true diff --git a/gazelle/python/testdata/directive_python_generate_proto/test6_enabled_without_proto/BUILD.out b/gazelle/python/testdata/directive_python_generate_proto/test6_enabled_without_proto/BUILD.out new file mode 100644 index 0000000000..ce3eec6001 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test6_enabled_without_proto/BUILD.out @@ -0,0 +1 @@ +# gazelle:python_generate_proto true diff --git a/gazelle/python/testdata/directive_python_generate_proto/test7_removes_when_unnecessary/BUILD.in b/gazelle/python/testdata/directive_python_generate_proto/test7_removes_when_unnecessary/BUILD.in new file mode 100644 index 0000000000..686252f27c --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test7_removes_when_unnecessary/BUILD.in @@ -0,0 +1,16 @@ +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_py_pb2", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto/test7_removes_when_unnecessary/BUILD.out b/gazelle/python/testdata/directive_python_generate_proto/test7_removes_when_unnecessary/BUILD.out new file mode 100644 index 0000000000..ce3eec6001 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test7_removes_when_unnecessary/BUILD.out @@ -0,0 +1 @@ +# gazelle:python_generate_proto true diff --git a/gazelle/python/testdata/directive_python_generate_proto/test8_disabled_ignores_py_proto_library/BUILD.in b/gazelle/python/testdata/directive_python_generate_proto/test8_disabled_ignores_py_proto_library/BUILD.in new file mode 100644 index 0000000000..f14ed4fc2d --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test8_disabled_ignores_py_proto_library/BUILD.in @@ -0,0 +1,16 @@ +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto false + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_py_pb2", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto/test8_disabled_ignores_py_proto_library/BUILD.out b/gazelle/python/testdata/directive_python_generate_proto/test8_disabled_ignores_py_proto_library/BUILD.out new file mode 100644 index 0000000000..f14ed4fc2d --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test8_disabled_ignores_py_proto_library/BUILD.out @@ -0,0 +1,16 @@ +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto false + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_py_pb2", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto/test8_disabled_ignores_py_proto_library/foo.proto b/gazelle/python/testdata/directive_python_generate_proto/test8_disabled_ignores_py_proto_library/foo.proto new file mode 100644 index 0000000000..022e29ae69 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test8_disabled_ignores_py_proto_library/foo.proto @@ -0,0 +1,7 @@ +syntax = "proto3"; + +package foo.bar; + +message Foo { + string bar = 1; +} diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/BUILD.in b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/BUILD.in new file mode 100644 index 0000000000..4713404b19 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/BUILD.in @@ -0,0 +1,9 @@ +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/BUILD.out b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/BUILD.out new file mode 100644 index 0000000000..dab84a6777 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/BUILD.out @@ -0,0 +1,16 @@ +load("@protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_py_pb2", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/MODULE.bazel b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/MODULE.bazel new file mode 100644 index 0000000000..66d64afe03 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/MODULE.bazel @@ -0,0 +1 @@ +bazel_dep(name = "protobuf", version = "29.3") diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/README.md b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/README.md new file mode 100644 index 0000000000..2d91ccff56 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/README.md @@ -0,0 +1,6 @@ +# Directive: `python_generate_proto` + +This test case asserts that the `# gazelle:python_generate_proto` directive +correctly reads the name of the protobuf repository when bzlmod is being used. + +[gh-2994]: https://github.com/bazel-contrib/rules_python/issues/2994 diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/WORKSPACE b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/WORKSPACE new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/foo.proto b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/foo.proto new file mode 100644 index 0000000000..fe2af27aa6 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/foo.proto @@ -0,0 +1,7 @@ +syntax = "proto3"; + +package foo; + +message Foo { + string bar = 1; +} diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/test.yaml b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/test.yaml new file mode 100644 index 0000000000..36dd656b39 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/test.yaml @@ -0,0 +1,3 @@ +--- +expect: + exit_code: 0 diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/BUILD.in b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/BUILD.in new file mode 100644 index 0000000000..4713404b19 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/BUILD.in @@ -0,0 +1,9 @@ +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/BUILD.out b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/BUILD.out new file mode 100644 index 0000000000..686252f27c --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/BUILD.out @@ -0,0 +1,16 @@ +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_py_pb2", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/MODULE.bazel b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/MODULE.bazel new file mode 100644 index 0000000000..9ab4c175aa --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/MODULE.bazel @@ -0,0 +1 @@ +bazel_dep(name = "protobuf", version = "29.3", repo_name = "com_google_protobuf") diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/README.md b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/README.md new file mode 100644 index 0000000000..7900d49084 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/README.md @@ -0,0 +1,7 @@ +# Directive: `python_generate_proto` + +This test case asserts that the `# gazelle:python_generate_proto` directive +correctly reads the name of the protobuf repository when bzlmod is being used, +but the repository is renamed. + +[gh-2994]: https://github.com/bazel-contrib/rules_python/issues/2994 diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/WORKSPACE b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/WORKSPACE new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/foo.proto b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/foo.proto new file mode 100644 index 0000000000..fe2af27aa6 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/foo.proto @@ -0,0 +1,7 @@ +syntax = "proto3"; + +package foo; + +message Foo { + string bar = 1; +} diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/test.yaml b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/test.yaml new file mode 100644 index 0000000000..36dd656b39 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/test.yaml @@ -0,0 +1,3 @@ +--- +expect: + exit_code: 0 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/BUILD.in b/gazelle/python/testdata/directive_python_generate_pyi_srcs/BUILD.in new file mode 100644 index 0000000000..a8459ad349 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_pyi_srcs/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_generate_pyi_srcs true diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/BUILD.out b/gazelle/python/testdata/directive_python_generate_pyi_srcs/BUILD.out new file mode 100644 index 0000000000..5beb703423 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_pyi_srcs/BUILD.out @@ -0,0 +1,36 @@ +load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test") + +# gazelle:python_generate_pyi_srcs true + +py_library( + name = "directive_python_generate_pyi_srcs", + srcs = [ + "__init__.py", + "bar.py", + "baz.py", + "foo.py", + ], + pyi_srcs = [ + "baz.pyi", + "foo.pyi", + ], + visibility = ["//:__subpackages__"], +) + +py_binary( + name = "directive_python_generate_pyi_srcs_bin", + srcs = ["__main__.py"], + main = "__main__.py", + pyi_srcs = ["__main__.pyi"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "directive_python_generate_pyi_srcs_test", + srcs = [ + "__test__.py", + "foo_test.py", + ], + main = "__test__.py", + pyi_srcs = ["foo_test.pyi"], +) diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/README.md b/gazelle/python/testdata/directive_python_generate_pyi_srcs/README.md new file mode 100644 index 0000000000..3ef10bab06 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_pyi_srcs/README.md @@ -0,0 +1,13 @@ +# Directive: python_generate_pyi_srcs + +Test that the `python_generate_pyi_srcs` directive will add `pyi_srcs` to +generated targets and that it can be toggled on/off on a per-package basis. + +The root of the test case asserts that the default generation mode (package) +will compile multiple .pyi files into a single py_* target. + +The `per_file` directory asserts that the `file` generation mode will attach +a single .pyi file to a given target. + +Lastly, the `per_file/turn_off` directory asserts that we can turn off the +directive for subpackages. It continues with per-file generation mode. diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/WORKSPACE b/gazelle/python/testdata/directive_python_generate_pyi_srcs/WORKSPACE new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/__init__.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/__main__.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/__main__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/__main__.pyi b/gazelle/python/testdata/directive_python_generate_pyi_srcs/__main__.pyi new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/__test__.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/__test__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/bar.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/bar.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/baz.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/baz.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/baz.pyi b/gazelle/python/testdata/directive_python_generate_pyi_srcs/baz.pyi new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/foo.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/foo.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/foo.pyi b/gazelle/python/testdata/directive_python_generate_pyi_srcs/foo.pyi new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/foo_test.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/foo_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/foo_test.pyi b/gazelle/python/testdata/directive_python_generate_pyi_srcs/foo_test.pyi new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/BUILD.in b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/BUILD.in new file mode 100644 index 0000000000..a7dbb7e4c1 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/BUILD.in @@ -0,0 +1,2 @@ +# gazelle:python_generate_pyi_srcs true +# gazelle:python_generation_mode file diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/BUILD.out b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/BUILD.out new file mode 100644 index 0000000000..d0889fc46f --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/BUILD.out @@ -0,0 +1,30 @@ +load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test") + +# gazelle:python_generate_pyi_srcs true +# gazelle:python_generation_mode file + +py_library( + name = "bar", + srcs = ["bar.py"], + visibility = ["//:__subpackages__"], +) + +py_library( + name = "foo", + srcs = ["foo.py"], + pyi_srcs = ["foo.pyi"], + visibility = ["//:__subpackages__"], +) + +py_binary( + name = "my_binary", + srcs = ["my_binary.py"], + pyi_srcs = ["my_binary.pyi"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "bar_test", + srcs = ["bar_test.py"], + pyi_srcs = ["bar_test.pyi"], +) diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/__init__.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/bar.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/bar.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/bar_test.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/bar_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/bar_test.pyi b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/bar_test.pyi new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/foo.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/foo.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/foo.pyi b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/foo.pyi new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/my_binary.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/my_binary.py new file mode 100644 index 0000000000..6a81382943 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/my_binary.py @@ -0,0 +1,2 @@ +if __name__ == "__main__": + print("hey") diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/my_binary.pyi b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/my_binary.pyi new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/BUILD.in b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/BUILD.in new file mode 100644 index 0000000000..7a24055cd0 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_generate_pyi_srcs false diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/BUILD.out b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/BUILD.out new file mode 100644 index 0000000000..25b508a7f0 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/BUILD.out @@ -0,0 +1,9 @@ +load("@rules_python//python:defs.bzl", "py_library") + +# gazelle:python_generate_pyi_srcs false + +py_library( + name = "foo", + srcs = ["foo.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/foo.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/foo.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/foo.pyi b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/foo.pyi new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/test.yaml b/gazelle/python/testdata/directive_python_generate_pyi_srcs/test.yaml new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/BUILD.in b/gazelle/python/testdata/directive_python_include_ancestor_conftest/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/BUILD.out b/gazelle/python/testdata/directive_python_include_ancestor_conftest/BUILD.out new file mode 100644 index 0000000000..c7adad8336 --- /dev/null +++ b/gazelle/python/testdata/directive_python_include_ancestor_conftest/BUILD.out @@ -0,0 +1,8 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "conftest", + testonly = True, + srcs = ["conftest.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/MODULE.bazel b/gazelle/python/testdata/directive_python_include_ancestor_conftest/MODULE.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/README.md b/gazelle/python/testdata/directive_python_include_ancestor_conftest/README.md new file mode 100644 index 0000000000..956e65155b --- /dev/null +++ b/gazelle/python/testdata/directive_python_include_ancestor_conftest/README.md @@ -0,0 +1,23 @@ +# Directive: `python_include_ancestor_conftest` + +This test case asserts that the `# gazelle:python_include_ancestor_conftest` +directive correctly includes or excludes ancestor `conftest` targets in +`py_test` target dependencies. + +The test also asserts that the directive can be applied at any level and that +child levels will inherit the value: + ++ The root level does not set the directive (it defaults to True). ++ The next level, `one/`, inherits that value. ++ The next level, `one/two/`, sets the directive to False; consequently the + `py_test` target only includes the sibling `:conftest` target. + + The `one/two/no_conftest/` directory does not contain a `conftest.py` file + thereby asserting that we correctly do not include any `conftest` targets + whatsoever. ++ The final level, `one/two/three/`, sets the directive back to True, meaning + the `py_test` target includes a total of 4 `conftest` targets. + + The `one/two/three/no_conftest/` directory does not contain a `conftest.py` + file and thus asserts that the code includes _only_ ancestor `conftest` + targets. + +See [Issue #3595](https://github.com/bazel-contrib/rules_python/issues/3595). diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/WORKSPACE b/gazelle/python/testdata/directive_python_include_ancestor_conftest/WORKSPACE new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/conftest.py b/gazelle/python/testdata/directive_python_include_ancestor_conftest/conftest.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/BUILD.in b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/BUILD.out b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/BUILD.out new file mode 100644 index 0000000000..9d0405e92f --- /dev/null +++ b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/BUILD.out @@ -0,0 +1,17 @@ +load("@rules_python//python:defs.bzl", "py_library", "py_test") + +py_library( + name = "conftest", + testonly = True, + srcs = ["conftest.py"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "my_test", + srcs = ["my_test.py"], + deps = [ + ":conftest", + "//:conftest", + ], +) diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/conftest.py b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/conftest.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/my_test.py b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/my_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/BUILD.in b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/BUILD.in new file mode 100644 index 0000000000..3805e248e2 --- /dev/null +++ b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_include_ancestor_conftest false diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/BUILD.out b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/BUILD.out new file mode 100644 index 0000000000..edb91a38f5 --- /dev/null +++ b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/BUILD.out @@ -0,0 +1,16 @@ +load("@rules_python//python:defs.bzl", "py_library", "py_test") + +# gazelle:python_include_ancestor_conftest false + +py_library( + name = "conftest", + testonly = True, + srcs = ["conftest.py"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "my_test", + srcs = ["my_test.py"], + deps = [":conftest"], +) diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/conftest.py b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/conftest.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/my_test.py b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/my_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/no_conftest/BUILD.in b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/no_conftest/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/no_conftest/BUILD.out b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/no_conftest/BUILD.out new file mode 100644 index 0000000000..764e2b4172 --- /dev/null +++ b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/no_conftest/BUILD.out @@ -0,0 +1,6 @@ +load("@rules_python//python:defs.bzl", "py_test") + +py_test( + name = "my_test", + srcs = ["my_test.py"], +) diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/no_conftest/my_test.py b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/no_conftest/my_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/BUILD.in b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/BUILD.in new file mode 100644 index 0000000000..987cd7dc20 --- /dev/null +++ b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_include_ancestor_conftest true diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/BUILD.out b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/BUILD.out new file mode 100644 index 0000000000..605aa00f88 --- /dev/null +++ b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/BUILD.out @@ -0,0 +1,21 @@ +load("@rules_python//python:defs.bzl", "py_library", "py_test") + +# gazelle:python_include_ancestor_conftest true + +py_library( + name = "conftest", + testonly = True, + srcs = ["conftest.py"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "my_test", + srcs = ["my_test.py"], + deps = [ + ":conftest", + "//:conftest", + "//one:conftest", + "//one/two:conftest", + ], +) diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/conftest.py b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/conftest.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/my_test.py b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/my_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/no_conftest/BUILD.in b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/no_conftest/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/no_conftest/BUILD.out b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/no_conftest/BUILD.out new file mode 100644 index 0000000000..c1bddccc30 --- /dev/null +++ b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/no_conftest/BUILD.out @@ -0,0 +1,12 @@ +load("@rules_python//python:defs.bzl", "py_test") + +py_test( + name = "my_test", + srcs = ["my_test.py"], + deps = [ + "//:conftest", + "//one:conftest", + "//one/two:conftest", + "//one/two/three:conftest", + ], +) diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/no_conftest/my_test.py b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/no_conftest/my_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/test.yaml b/gazelle/python/testdata/directive_python_include_ancestor_conftest/test.yaml new file mode 100644 index 0000000000..36dd656b39 --- /dev/null +++ b/gazelle/python/testdata/directive_python_include_ancestor_conftest/test.yaml @@ -0,0 +1,3 @@ +--- +expect: + exit_code: 0 diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/README.md b/gazelle/python/testdata/directive_python_proto_naming_convention/README.md new file mode 100644 index 0000000000..594379cdfc --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/README.md @@ -0,0 +1,9 @@ +# Directive: `python_proto_naming_convention` + +This test case asserts that the `# gazelle:python_proto_naming_convention` directive +correctly: + +1. Has no effect on pre-existing `py_proto_library` when `gazelle:python_generate_proto` is disabled. +2. Uses the default value when proto generation is on and `python_proto_naming_convention` is not set. +3. Uses the provided naming convention when proto generation is on and `python_proto_naming_convention` is set. +4. With a pre-existing `py_proto_library` not following a given naming convention, keeps it intact and does not rename it. \ No newline at end of file diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/WORKSPACE b/gazelle/python/testdata/directive_python_proto_naming_convention/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test.yaml b/gazelle/python/testdata/directive_python_proto_naming_convention/test.yaml new file mode 100644 index 0000000000..36dd656b39 --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test.yaml @@ -0,0 +1,3 @@ +--- +expect: + exit_code: 0 diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/BUILD.in b/gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/BUILD.in new file mode 100644 index 0000000000..2171d877f4 --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/BUILD.in @@ -0,0 +1,17 @@ +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto false +# gazelle:python_proto_naming_convention some_$proto_name$_value + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_proto_custom_name", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/BUILD.out b/gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/BUILD.out new file mode 100644 index 0000000000..2171d877f4 --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/BUILD.out @@ -0,0 +1,17 @@ +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto false +# gazelle:python_proto_naming_convention some_$proto_name$_value + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_proto_custom_name", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/foo.proto b/gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/foo.proto new file mode 100644 index 0000000000..022e29ae69 --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/foo.proto @@ -0,0 +1,7 @@ +syntax = "proto3"; + +package foo.bar; + +message Foo { + string bar = 1; +} diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test2_python_generation_enabled_uses_default/BUILD.in b/gazelle/python/testdata/directive_python_proto_naming_convention/test2_python_generation_enabled_uses_default/BUILD.in new file mode 100644 index 0000000000..4713404b19 --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test2_python_generation_enabled_uses_default/BUILD.in @@ -0,0 +1,9 @@ +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test2_python_generation_enabled_uses_default/BUILD.out b/gazelle/python/testdata/directive_python_proto_naming_convention/test2_python_generation_enabled_uses_default/BUILD.out new file mode 100644 index 0000000000..686252f27c --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test2_python_generation_enabled_uses_default/BUILD.out @@ -0,0 +1,16 @@ +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_py_pb2", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test2_python_generation_enabled_uses_default/foo.proto b/gazelle/python/testdata/directive_python_proto_naming_convention/test2_python_generation_enabled_uses_default/foo.proto new file mode 100644 index 0000000000..fe2af27aa6 --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test2_python_generation_enabled_uses_default/foo.proto @@ -0,0 +1,7 @@ +syntax = "proto3"; + +package foo; + +message Foo { + string bar = 1; +} diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test3_python_generation_enabled_uses_value/BUILD.in b/gazelle/python/testdata/directive_python_proto_naming_convention/test3_python_generation_enabled_uses_value/BUILD.in new file mode 100644 index 0000000000..b68a9937dc --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test3_python_generation_enabled_uses_value/BUILD.in @@ -0,0 +1,10 @@ +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true +# gazelle:python_proto_naming_convention some_$proto_name$_value + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test3_python_generation_enabled_uses_value/BUILD.out b/gazelle/python/testdata/directive_python_proto_naming_convention/test3_python_generation_enabled_uses_value/BUILD.out new file mode 100644 index 0000000000..f432e9a0c3 --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test3_python_generation_enabled_uses_value/BUILD.out @@ -0,0 +1,17 @@ +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true +# gazelle:python_proto_naming_convention some_$proto_name$_value + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "some_foo_value", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test3_python_generation_enabled_uses_value/foo.proto b/gazelle/python/testdata/directive_python_proto_naming_convention/test3_python_generation_enabled_uses_value/foo.proto new file mode 100644 index 0000000000..fe2af27aa6 --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test3_python_generation_enabled_uses_value/foo.proto @@ -0,0 +1,7 @@ +syntax = "proto3"; + +package foo; + +message Foo { + string bar = 1; +} diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test4_python_generation_enabled_with_preexisting_keeps_intact/BUILD.in b/gazelle/python/testdata/directive_python_proto_naming_convention/test4_python_generation_enabled_with_preexisting_keeps_intact/BUILD.in new file mode 100644 index 0000000000..cc7d120a7e --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test4_python_generation_enabled_with_preexisting_keeps_intact/BUILD.in @@ -0,0 +1,16 @@ +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true +# gazelle:python_proto_naming_convention $proto_name$_bar + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_py_proto", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test4_python_generation_enabled_with_preexisting_keeps_intact/BUILD.out b/gazelle/python/testdata/directive_python_proto_naming_convention/test4_python_generation_enabled_with_preexisting_keeps_intact/BUILD.out new file mode 100644 index 0000000000..080b83f1fb --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test4_python_generation_enabled_with_preexisting_keeps_intact/BUILD.out @@ -0,0 +1,17 @@ +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true +# gazelle:python_proto_naming_convention $proto_name$_bar + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_py_proto", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test4_python_generation_enabled_with_preexisting_keeps_intact/foo.proto b/gazelle/python/testdata/directive_python_proto_naming_convention/test4_python_generation_enabled_with_preexisting_keeps_intact/foo.proto new file mode 100644 index 0000000000..fe2af27aa6 --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test4_python_generation_enabled_with_preexisting_keeps_intact/foo.proto @@ -0,0 +1,7 @@ +syntax = "proto3"; + +package foo; + +message Foo { + string bar = 1; +} diff --git a/gazelle/python/testdata/directive_python_test_file_pattern_no_value/README.md b/gazelle/python/testdata/directive_python_test_file_pattern_no_value/README.md index 2c38eb78d2..d6fb0b6a72 100644 --- a/gazelle/python/testdata/directive_python_test_file_pattern_no_value/README.md +++ b/gazelle/python/testdata/directive_python_test_file_pattern_no_value/README.md @@ -5,4 +5,4 @@ fails with a nice message if the directive has no value. See discussion in [PR #1819 (comment)][comment]. -[comment]: https://github.com/bazelbuild/rules_python/pull/1819#discussion_r1536906287 +[comment]: https://github.com/bazel-contrib/rules_python/pull/1819#discussion_r1536906287 diff --git a/gazelle/python/testdata/dont_ignore_setup/BUILD.in b/gazelle/python/testdata/dont_ignore_setup/BUILD.in new file mode 100644 index 0000000000..af2c2cea4b --- /dev/null +++ b/gazelle/python/testdata/dont_ignore_setup/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_generation_mode file diff --git a/gazelle/python/testdata/dont_ignore_setup/BUILD.out b/gazelle/python/testdata/dont_ignore_setup/BUILD.out new file mode 100644 index 0000000000..acf9324d3d --- /dev/null +++ b/gazelle/python/testdata/dont_ignore_setup/BUILD.out @@ -0,0 +1,9 @@ +load("@rules_python//python:defs.bzl", "py_library") + +# gazelle:python_generation_mode file + +py_library( + name = "setup", + srcs = ["setup.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/dont_ignore_setup/README.md b/gazelle/python/testdata/dont_ignore_setup/README.md new file mode 100644 index 0000000000..d170364cb2 --- /dev/null +++ b/gazelle/python/testdata/dont_ignore_setup/README.md @@ -0,0 +1,8 @@ +# Don't ignore setup.py files + +Make sure that files named `setup.py` are processed by Gazelle. + +It's believed that `setup.py` was originally ignored because it, when found +in the repository root directory, is part of the `setuptools` build system +and could cause some issues for Gazelle. However, files within source code can +also be called `setup.py` and thus should be processed by Gazelle. diff --git a/gazelle/python/testdata/dont_ignore_setup/WORKSPACE b/gazelle/python/testdata/dont_ignore_setup/WORKSPACE new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/dont_ignore_setup/setup.py b/gazelle/python/testdata/dont_ignore_setup/setup.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/dont_ignore_setup/test.yaml b/gazelle/python/testdata/dont_ignore_setup/test.yaml new file mode 100644 index 0000000000..c27e6c854b --- /dev/null +++ b/gazelle/python/testdata/dont_ignore_setup/test.yaml @@ -0,0 +1,15 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +--- diff --git a/gazelle/python/testdata/from_imports/import_nested_var/__init__.py b/gazelle/python/testdata/from_imports/import_nested_var/__init__.py index d0f51c443c..20eda530e5 100644 --- a/gazelle/python/testdata/from_imports/import_nested_var/__init__.py +++ b/gazelle/python/testdata/from_imports/import_nested_var/__init__.py @@ -13,4 +13,8 @@ # limitations under the License. # baz is a variable in foo/bar/baz.py -from foo.bar.baz import baz +from foo\ + .bar.\ + baz import ( + baz + ) diff --git a/gazelle/python/testdata/naming_convention/BUILD.in b/gazelle/python/testdata/naming_convention/BUILD.in index 7517848a92..fee53ba7ff 100644 --- a/gazelle/python/testdata/naming_convention/BUILD.in +++ b/gazelle/python/testdata/naming_convention/BUILD.in @@ -1,3 +1,4 @@ # gazelle:python_library_naming_convention my_$package_name$_library # gazelle:python_binary_naming_convention my_$package_name$_binary # gazelle:python_test_naming_convention my_$package_name$_test +# gazelle:python_resolve_sibling_imports true diff --git a/gazelle/python/testdata/naming_convention/BUILD.out b/gazelle/python/testdata/naming_convention/BUILD.out index e2f067489c..7392cfeb35 100644 --- a/gazelle/python/testdata/naming_convention/BUILD.out +++ b/gazelle/python/testdata/naming_convention/BUILD.out @@ -3,6 +3,7 @@ load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test") # gazelle:python_library_naming_convention my_$package_name$_library # gazelle:python_binary_naming_convention my_$package_name$_binary # gazelle:python_test_naming_convention my_$package_name$_test +# gazelle:python_resolve_sibling_imports true py_library( name = "my_naming_convention_library", diff --git a/gazelle/python/testdata/naming_convention_mapped_fail/BUILD.in b/gazelle/python/testdata/naming_convention_mapped_fail/BUILD.in new file mode 100644 index 0000000000..63ca6b805f --- /dev/null +++ b/gazelle/python/testdata/naming_convention_mapped_fail/BUILD.in @@ -0,0 +1,9 @@ +# gazelle:map_kind py_library my_lib :mytest.bzl +# gazelle:map_kind py_binary my_bin :mytest.bzl +# gazelle:map_kind py_test my_test :mytest.bzl + +py_library(name = "naming_convention_mapped_fail") + +py_binary(name = "naming_convention_mapped_fail_bin") + +py_test(name = "naming_convention_mapped_fail_test") diff --git a/gazelle/python/testdata/naming_convention_mapped_fail/BUILD.out b/gazelle/python/testdata/naming_convention_mapped_fail/BUILD.out new file mode 100644 index 0000000000..63ca6b805f --- /dev/null +++ b/gazelle/python/testdata/naming_convention_mapped_fail/BUILD.out @@ -0,0 +1,9 @@ +# gazelle:map_kind py_library my_lib :mytest.bzl +# gazelle:map_kind py_binary my_bin :mytest.bzl +# gazelle:map_kind py_test my_test :mytest.bzl + +py_library(name = "naming_convention_mapped_fail") + +py_binary(name = "naming_convention_mapped_fail_bin") + +py_test(name = "naming_convention_mapped_fail_test") diff --git a/gazelle/python/testdata/naming_convention_mapped_fail/README.md b/gazelle/python/testdata/naming_convention_mapped_fail/README.md new file mode 100644 index 0000000000..85a561c575 --- /dev/null +++ b/gazelle/python/testdata/naming_convention_mapped_fail/README.md @@ -0,0 +1,4 @@ +# Naming convention fail with `map_kind` + +This test case asserts that collision error messages reference the mapped rule +kind over than the canonical Python kind when using `# gazelle:map_kind`. diff --git a/gazelle/python/testdata/naming_convention_mapped_fail/WORKSPACE b/gazelle/python/testdata/naming_convention_mapped_fail/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/naming_convention_mapped_fail/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/naming_convention_mapped_fail/__init__.py b/gazelle/python/testdata/naming_convention_mapped_fail/__init__.py new file mode 100644 index 0000000000..a75c47faaf --- /dev/null +++ b/gazelle/python/testdata/naming_convention_mapped_fail/__init__.py @@ -0,0 +1 @@ +# Empty test file. \ No newline at end of file diff --git a/gazelle/python/testdata/naming_convention_mapped_fail/__main__.py b/gazelle/python/testdata/naming_convention_mapped_fail/__main__.py new file mode 100644 index 0000000000..a75c47faaf --- /dev/null +++ b/gazelle/python/testdata/naming_convention_mapped_fail/__main__.py @@ -0,0 +1 @@ +# Empty test file. \ No newline at end of file diff --git a/gazelle/python/testdata/naming_convention_mapped_fail/__test__.py b/gazelle/python/testdata/naming_convention_mapped_fail/__test__.py new file mode 100644 index 0000000000..a75c47faaf --- /dev/null +++ b/gazelle/python/testdata/naming_convention_mapped_fail/__test__.py @@ -0,0 +1 @@ +# Empty test file. \ No newline at end of file diff --git a/gazelle/python/testdata/naming_convention_mapped_fail/test.yaml b/gazelle/python/testdata/naming_convention_mapped_fail/test.yaml new file mode 100644 index 0000000000..1bdf1acbc5 --- /dev/null +++ b/gazelle/python/testdata/naming_convention_mapped_fail/test.yaml @@ -0,0 +1,7 @@ +--- +expect: + exit_code: 1 + stderr: | + gazelle: ERROR: failed to generate target "//:naming_convention_mapped_fail" of kind "my_lib": a target of kind "py_library" with the same name already exists. Use the '# gazelle:python_library_naming_convention' directive to change the naming convention. + gazelle: ERROR: failed to generate target "//:naming_convention_mapped_fail_bin" of kind "my_bin": a target of kind "py_binary" with the same name already exists. Use the '# gazelle:python_binary_naming_convention' directive to change the naming convention. + gazelle: ERROR: failed to generate target "//:naming_convention_mapped_fail_test" of kind "my_test": a target of kind "py_test" with the same name already exists. Use the '# gazelle:python_test_naming_convention' directive to change the naming convention. diff --git a/gazelle/python/testdata/per_file_non_empty_init/BUILD.out b/gazelle/python/testdata/per_file_non_empty_init/BUILD.out index ee4a417966..38dcaa5ee2 100644 --- a/gazelle/python/testdata/per_file_non_empty_init/BUILD.out +++ b/gazelle/python/testdata/per_file_non_empty_init/BUILD.out @@ -1,4 +1,4 @@ -load("@rules_python//python:defs.bzl", "py_library") +load("@rules_python//python:defs.bzl", "py_binary", "py_library") # gazelle:python_generation_mode file # gazelle:python_generation_mode_per_file_include_init true @@ -18,3 +18,13 @@ py_library( ], visibility = ["//:__subpackages__"], ) + +py_binary( + name = "foobin", + srcs = [ + "__init__.py", + "foobin.py", + ], + visibility = ["//:__subpackages__"], + deps = [":foo"], +) diff --git a/gazelle/python/testdata/per_file_non_empty_init/README.md b/gazelle/python/testdata/per_file_non_empty_init/README.md index 6e6e9e245d..0fad0f3d43 100644 --- a/gazelle/python/testdata/per_file_non_empty_init/README.md +++ b/gazelle/python/testdata/per_file_non_empty_init/README.md @@ -1,3 +1,3 @@ # Per-file generation -This test case generates one `py_library` per file, including `__init__.py`. +This test case generates one `py_library` or `py_binary` per file, including `__init__.py`. diff --git a/gazelle/python/testdata/per_file_non_empty_init/foo.py b/gazelle/python/testdata/per_file_non_empty_init/foo.py index 730755995d..b70cc96b7e 100644 --- a/gazelle/python/testdata/per_file_non_empty_init/foo.py +++ b/gazelle/python/testdata/per_file_non_empty_init/foo.py @@ -13,3 +13,4 @@ # limitations under the License. # For test purposes only. +BAR = "baz" diff --git a/gazelle/python/testdata/per_file_non_empty_init/foobin.py b/gazelle/python/testdata/per_file_non_empty_init/foobin.py new file mode 100644 index 0000000000..6b493b531e --- /dev/null +++ b/gazelle/python/testdata/per_file_non_empty_init/foobin.py @@ -0,0 +1,4 @@ +from foo import BAR + +if __name__ == "__main__": + print(BAR) diff --git a/gazelle/python/testdata/py312_syntax/BUILD.in b/gazelle/python/testdata/py312_syntax/BUILD.in new file mode 100644 index 0000000000..af2c2cea4b --- /dev/null +++ b/gazelle/python/testdata/py312_syntax/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_generation_mode file diff --git a/gazelle/python/testdata/py312_syntax/BUILD.out b/gazelle/python/testdata/py312_syntax/BUILD.out new file mode 100644 index 0000000000..7457f335a7 --- /dev/null +++ b/gazelle/python/testdata/py312_syntax/BUILD.out @@ -0,0 +1,16 @@ +load("@rules_python//python:defs.bzl", "py_binary", "py_library") + +# gazelle:python_generation_mode file + +py_library( + name = "_other_module", + srcs = ["_other_module.py"], + visibility = ["//:__subpackages__"], +) + +py_binary( + name = "pep_695_type_parameter", + srcs = ["pep_695_type_parameter.py"], + visibility = ["//:__subpackages__"], + deps = [":_other_module"], +) diff --git a/gazelle/python/testdata/py312_syntax/README.md b/gazelle/python/testdata/py312_syntax/README.md new file mode 100644 index 0000000000..854a0a3aa6 --- /dev/null +++ b/gazelle/python/testdata/py312_syntax/README.md @@ -0,0 +1,4 @@ +# py312 syntax + +This test case checks that we properly parse certain python 3.12 syntax, such +as pep 695 type parameters, with go-tree-sitter. diff --git a/gazelle/python/testdata/py312_syntax/WORKSPACE b/gazelle/python/testdata/py312_syntax/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/py312_syntax/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/py312_syntax/__init__.py b/gazelle/python/testdata/py312_syntax/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/py312_syntax/_other_module.py b/gazelle/python/testdata/py312_syntax/_other_module.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/py312_syntax/pep_695_type_parameter.py b/gazelle/python/testdata/py312_syntax/pep_695_type_parameter.py new file mode 100644 index 0000000000..eb6263b334 --- /dev/null +++ b/gazelle/python/testdata/py312_syntax/pep_695_type_parameter.py @@ -0,0 +1,21 @@ +def search_one_more_level[T]( + graph: dict[T, set[T]], seen: set[T], routes: list[list[T]], target: T +) -> list[T] | None: + """This function fails to parse with older versions of go-tree-sitter. + + Args: + graph: The graph to search as input. + seen: The nodes that have been visited as input/output. + routes: The current routes in the breadth-first search as input/output. + target: The target to search in this extra search level. + + Returns: + a route if it ends on the target, or None if no route reaches the + target. + """ + + +import _other_module + +if __name__ == "__main__": + pass diff --git a/gazelle/python/testdata/py312_syntax/test.yaml b/gazelle/python/testdata/py312_syntax/test.yaml new file mode 100644 index 0000000000..ed97d539c0 --- /dev/null +++ b/gazelle/python/testdata/py312_syntax/test.yaml @@ -0,0 +1 @@ +--- diff --git a/gazelle/python/testdata/python_ignore_files_directive/BUILD.out b/gazelle/python/testdata/python_ignore_files_directive/BUILD.out index 1fe6030053..234ff71b13 100644 --- a/gazelle/python/testdata/python_ignore_files_directive/BUILD.out +++ b/gazelle/python/testdata/python_ignore_files_directive/BUILD.out @@ -4,6 +4,9 @@ load("@rules_python//python:defs.bzl", "py_library") py_library( name = "python_ignore_files_directive", - srcs = ["__init__.py"], + srcs = [ + "__init__.py", + "setup.py", + ], visibility = ["//:__subpackages__"], ) diff --git a/gazelle/python/testdata/relative_imports/README.md b/gazelle/python/testdata/relative_imports/README.md deleted file mode 100644 index 1937cbcf4a..0000000000 --- a/gazelle/python/testdata/relative_imports/README.md +++ /dev/null @@ -1,4 +0,0 @@ -# Relative imports - -This test case asserts that the generated targets handle relative imports in -Python correctly. diff --git a/gazelle/python/testdata/relative_imports_package_mode/BUILD.in b/gazelle/python/testdata/relative_imports_package_mode/BUILD.in new file mode 100644 index 0000000000..52bcb68600 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/BUILD.in @@ -0,0 +1,2 @@ +# gazelle:python_generation_mode package +# gazelle:python_experimental_allow_relative_imports true diff --git a/gazelle/python/testdata/relative_imports_package_mode/BUILD.out b/gazelle/python/testdata/relative_imports_package_mode/BUILD.out new file mode 100644 index 0000000000..8775c114ef --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/BUILD.out @@ -0,0 +1,15 @@ +load("@rules_python//python:defs.bzl", "py_binary") + +# gazelle:python_generation_mode package +# gazelle:python_experimental_allow_relative_imports true + +py_binary( + name = "relative_imports_package_mode_bin", + srcs = ["__main__.py"], + main = "__main__.py", + visibility = ["//:__subpackages__"], + deps = [ + "//package1", + "//package2", + ], +) diff --git a/gazelle/python/testdata/relative_imports_package_mode/README.md b/gazelle/python/testdata/relative_imports_package_mode/README.md new file mode 100644 index 0000000000..eb9f8c096c --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/README.md @@ -0,0 +1,6 @@ +# Resolve deps for relative imports + +This test case verifies that the generated targets correctly handle relative imports in +Python. Specifically, when the Python generation mode is set to "package," it ensures +that relative import statements such as from .foo import X are properly resolved to +their corresponding modules. diff --git a/gazelle/python/testdata/relative_imports/WORKSPACE b/gazelle/python/testdata/relative_imports_package_mode/WORKSPACE similarity index 100% rename from gazelle/python/testdata/relative_imports/WORKSPACE rename to gazelle/python/testdata/relative_imports_package_mode/WORKSPACE diff --git a/gazelle/python/testdata/relative_imports_package_mode/__main__.py b/gazelle/python/testdata/relative_imports_package_mode/__main__.py new file mode 100644 index 0000000000..4fb887a803 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/__main__.py @@ -0,0 +1,5 @@ +from package1.module1 import function1 +from package2.module3 import function3 + +print(function1()) +print(function3()) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/BUILD.in b/gazelle/python/testdata/relative_imports_package_mode/package1/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/BUILD.out b/gazelle/python/testdata/relative_imports_package_mode/package1/BUILD.out new file mode 100644 index 0000000000..c562ff07de --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/BUILD.out @@ -0,0 +1,11 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "package1", + srcs = [ + "__init__.py", + "module1.py", + "module2.py", + ], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/__init__.py b/gazelle/python/testdata/relative_imports_package_mode/package1/__init__.py new file mode 100644 index 0000000000..11ffb98647 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/__init__.py @@ -0,0 +1,2 @@ +def some_function(): + pass diff --git a/gazelle/python/testdata/relative_imports/package1/module1.py b/gazelle/python/testdata/relative_imports_package_mode/package1/module1.py similarity index 100% rename from gazelle/python/testdata/relative_imports/package1/module1.py rename to gazelle/python/testdata/relative_imports_package_mode/package1/module1.py diff --git a/gazelle/python/testdata/relative_imports/package1/module2.py b/gazelle/python/testdata/relative_imports_package_mode/package1/module2.py similarity index 100% rename from gazelle/python/testdata/relative_imports/package1/module2.py rename to gazelle/python/testdata/relative_imports_package_mode/package1/module2.py diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/BUILD.in b/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/BUILD.in new file mode 100644 index 0000000000..80a4a22348 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/BUILD.in @@ -0,0 +1,7 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "my_library", + srcs = ["__init__.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/BUILD.out b/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/BUILD.out new file mode 100644 index 0000000000..80a4a22348 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/BUILD.out @@ -0,0 +1,7 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "my_library", + srcs = ["__init__.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/__init__.py b/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/__init__.py new file mode 100644 index 0000000000..aaa161cd59 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/__init__.py @@ -0,0 +1,2 @@ +def some_function(): + return "some_function" diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/foo/BUILD.in b/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/foo/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/foo/BUILD.out b/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/foo/BUILD.out new file mode 100644 index 0000000000..58498ee3b3 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/foo/BUILD.out @@ -0,0 +1,7 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "foo", + srcs = ["__init__.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/foo/__init__.py b/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/foo/__init__.py new file mode 100644 index 0000000000..aaa161cd59 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/foo/__init__.py @@ -0,0 +1,2 @@ +def some_function(): + return "some_function" diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/BUILD.in b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/BUILD.in new file mode 100644 index 0000000000..0a5b665c8d --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/BUILD.in @@ -0,0 +1,10 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "subpackage1", + srcs = [ + "__init__.py", + "some_module.py", + ], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/BUILD.out b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/BUILD.out new file mode 100644 index 0000000000..0a5b665c8d --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/BUILD.out @@ -0,0 +1,10 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "subpackage1", + srcs = [ + "__init__.py", + "some_module.py", + ], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/__init__.py b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/__init__.py new file mode 100644 index 0000000000..02feaeb848 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/__init__.py @@ -0,0 +1,3 @@ + +def some_init(): + return "some_init" diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/some_module.py b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/some_module.py new file mode 100644 index 0000000000..3cae706242 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/some_module.py @@ -0,0 +1,3 @@ + +def some_function(): + return "some_function" diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/BUILD.in b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/BUILD.out b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/BUILD.out new file mode 100644 index 0000000000..8c34081210 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/BUILD.out @@ -0,0 +1,16 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "subpackage2", + srcs = [ + "__init__.py", + "script.py", + ], + visibility = ["//:__subpackages__"], + deps = [ + "//package1/my_library", + "//package1/my_library/foo", + "//package1/subpackage1", + "//package1/subpackage1/subpackage2/library", + ], +) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/__init__.py b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/library/BUILD.in b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/library/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/library/BUILD.out b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/library/BUILD.out new file mode 100644 index 0000000000..9fe2e3d1d7 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/library/BUILD.out @@ -0,0 +1,7 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "library", + srcs = ["other_module.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/library/other_module.py b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/library/other_module.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/script.py b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/script.py new file mode 100644 index 0000000000..e93f07719a --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/script.py @@ -0,0 +1,11 @@ +from ...my_library import ( + some_function, +) # Import path should be package1.my_library.some_function +from ...my_library.foo import ( + some_function, +) # Import path should be package1.my_library.foo.some_function +from .library import ( + other_module, +) # Import path should be package1.subpackage1.subpackage2.library.other_module +from .. import some_module # Import path should be package1.subpackage1.some_module +from .. import some_function # Import path should be package1.subpackage1.some_function diff --git a/gazelle/python/testdata/relative_imports_package_mode/package2/BUILD.in b/gazelle/python/testdata/relative_imports_package_mode/package2/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/relative_imports_package_mode/package2/BUILD.out b/gazelle/python/testdata/relative_imports_package_mode/package2/BUILD.out new file mode 100644 index 0000000000..bd78108159 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package2/BUILD.out @@ -0,0 +1,12 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "package2", + srcs = [ + "__init__.py", + "module3.py", + "module4.py", + ], + visibility = ["//:__subpackages__"], + deps = ["//package2/library"], +) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package2/__init__.py b/gazelle/python/testdata/relative_imports_package_mode/package2/__init__.py new file mode 100644 index 0000000000..3d19d80e21 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package2/__init__.py @@ -0,0 +1,20 @@ +from .library import add as _add +from .library import divide as _divide +from .library import multiply as _multiply +from .library import subtract as _subtract + + +def add(a, b): + return _add(a, b) + + +def divide(a, b): + return _divide(a, b) + + +def multiply(a, b): + return _multiply(a, b) + + +def subtract(a, b): + return _subtract(a, b) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package2/library/BUILD.in b/gazelle/python/testdata/relative_imports_package_mode/package2/library/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/relative_imports_package_mode/package2/library/BUILD.out b/gazelle/python/testdata/relative_imports_package_mode/package2/library/BUILD.out new file mode 100644 index 0000000000..d704b7fe93 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package2/library/BUILD.out @@ -0,0 +1,7 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "library", + srcs = ["__init__.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package2/library/__init__.py b/gazelle/python/testdata/relative_imports_package_mode/package2/library/__init__.py new file mode 100644 index 0000000000..5f8fc62492 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package2/library/__init__.py @@ -0,0 +1,14 @@ +def add(a, b): + return a + b + + +def divide(a, b): + return a / b + + +def multiply(a, b): + return a * b + + +def subtract(a, b): + return a - b diff --git a/gazelle/python/testdata/relative_imports_package_mode/package2/module3.py b/gazelle/python/testdata/relative_imports_package_mode/package2/module3.py new file mode 100644 index 0000000000..6b955cfda6 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package2/module3.py @@ -0,0 +1,5 @@ +from .library import function5 + + +def function3(): + return "function3 " + function5() diff --git a/gazelle/python/testdata/relative_imports_package_mode/package2/module4.py b/gazelle/python/testdata/relative_imports_package_mode/package2/module4.py new file mode 100644 index 0000000000..6e69699985 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package2/module4.py @@ -0,0 +1,2 @@ +def function4(): + return "function4" diff --git a/gazelle/python/testdata/remove_invalid_library/test.yaml b/gazelle/python/testdata/relative_imports_package_mode/test.yaml similarity index 100% rename from gazelle/python/testdata/remove_invalid_library/test.yaml rename to gazelle/python/testdata/relative_imports_package_mode/test.yaml diff --git a/gazelle/python/testdata/relative_imports/BUILD.in b/gazelle/python/testdata/relative_imports_project_mode/BUILD.in similarity index 61% rename from gazelle/python/testdata/relative_imports/BUILD.in rename to gazelle/python/testdata/relative_imports_project_mode/BUILD.in index c04b5e5434..1059942bfb 100644 --- a/gazelle/python/testdata/relative_imports/BUILD.in +++ b/gazelle/python/testdata/relative_imports_project_mode/BUILD.in @@ -1 +1,2 @@ # gazelle:resolve py resolved_package //package2:resolved_package +# gazelle:python_generation_mode project diff --git a/gazelle/python/testdata/relative_imports/BUILD.out b/gazelle/python/testdata/relative_imports_project_mode/BUILD.out similarity index 70% rename from gazelle/python/testdata/relative_imports/BUILD.out rename to gazelle/python/testdata/relative_imports_project_mode/BUILD.out index bf9524480a..acdc914541 100644 --- a/gazelle/python/testdata/relative_imports/BUILD.out +++ b/gazelle/python/testdata/relative_imports_project_mode/BUILD.out @@ -1,9 +1,10 @@ load("@rules_python//python:defs.bzl", "py_binary", "py_library") # gazelle:resolve py resolved_package //package2:resolved_package +# gazelle:python_generation_mode project py_library( - name = "relative_imports", + name = "relative_imports_project_mode", srcs = [ "package1/module1.py", "package1/module2.py", @@ -12,12 +13,12 @@ py_library( ) py_binary( - name = "relative_imports_bin", + name = "relative_imports_project_mode_bin", srcs = ["__main__.py"], main = "__main__.py", visibility = ["//:__subpackages__"], deps = [ - ":relative_imports", + ":relative_imports_project_mode", "//package2", ], ) diff --git a/gazelle/python/testdata/relative_imports_project_mode/README.md b/gazelle/python/testdata/relative_imports_project_mode/README.md new file mode 100644 index 0000000000..3c95a36e62 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_project_mode/README.md @@ -0,0 +1,5 @@ +# Relative imports + +This test case asserts that the generated targets handle relative imports in +Python correctly. This tests that if python generation mode is project, +the relative paths are included in the subdirectories. diff --git a/gazelle/python/testdata/relative_imports_project_mode/WORKSPACE b/gazelle/python/testdata/relative_imports_project_mode/WORKSPACE new file mode 100644 index 0000000000..4959898cdd --- /dev/null +++ b/gazelle/python/testdata/relative_imports_project_mode/WORKSPACE @@ -0,0 +1 @@ +# This is a test data Bazel workspace. diff --git a/gazelle/python/testdata/relative_imports/__main__.py b/gazelle/python/testdata/relative_imports_project_mode/__main__.py similarity index 100% rename from gazelle/python/testdata/relative_imports/__main__.py rename to gazelle/python/testdata/relative_imports_project_mode/__main__.py diff --git a/examples/pip_repository_annotations/data/copy_executable.py b/gazelle/python/testdata/relative_imports_project_mode/package1/module1.py old mode 100755 new mode 100644 similarity index 86% rename from examples/pip_repository_annotations/data/copy_executable.py rename to gazelle/python/testdata/relative_imports_project_mode/package1/module1.py index 5cb1af7fdb..28502f1f84 --- a/examples/pip_repository_annotations/data/copy_executable.py +++ b/gazelle/python/testdata/relative_imports_project_mode/package1/module1.py @@ -1,4 +1,3 @@ -#!/usr/bin/env python # Copyright 2023 The Bazel Authors. All rights reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -13,6 +12,8 @@ # See the License for the specific language governing permissions and # limitations under the License. +from .module2 import function2 -if __name__ == "__main__": - print("Hello world from copied executable") + +def function1(): + return "function1 " + function2() diff --git a/gazelle/python/testdata/relative_imports_project_mode/package1/module2.py b/gazelle/python/testdata/relative_imports_project_mode/package1/module2.py new file mode 100644 index 0000000000..0cbc5f0be0 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_project_mode/package1/module2.py @@ -0,0 +1,17 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +def function2(): + return "function2" diff --git a/gazelle/python/testdata/relative_imports_project_mode/package2/BUILD.in b/gazelle/python/testdata/relative_imports_project_mode/package2/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/relative_imports/package2/BUILD.out b/gazelle/python/testdata/relative_imports_project_mode/package2/BUILD.out similarity index 100% rename from gazelle/python/testdata/relative_imports/package2/BUILD.out rename to gazelle/python/testdata/relative_imports_project_mode/package2/BUILD.out diff --git a/gazelle/python/testdata/relative_imports/package2/__init__.py b/gazelle/python/testdata/relative_imports_project_mode/package2/__init__.py similarity index 100% rename from gazelle/python/testdata/relative_imports/package2/__init__.py rename to gazelle/python/testdata/relative_imports_project_mode/package2/__init__.py diff --git a/gazelle/python/testdata/relative_imports/package2/module3.py b/gazelle/python/testdata/relative_imports_project_mode/package2/module3.py similarity index 100% rename from gazelle/python/testdata/relative_imports/package2/module3.py rename to gazelle/python/testdata/relative_imports_project_mode/package2/module3.py diff --git a/gazelle/python/testdata/relative_imports/package2/module4.py b/gazelle/python/testdata/relative_imports_project_mode/package2/module4.py similarity index 100% rename from gazelle/python/testdata/relative_imports/package2/module4.py rename to gazelle/python/testdata/relative_imports_project_mode/package2/module4.py diff --git a/gazelle/python/testdata/relative_imports/package2/subpackage1/module5.py b/gazelle/python/testdata/relative_imports_project_mode/package2/subpackage1/module5.py similarity index 100% rename from gazelle/python/testdata/relative_imports/package2/subpackage1/module5.py rename to gazelle/python/testdata/relative_imports_project_mode/package2/subpackage1/module5.py diff --git a/gazelle/python/testdata/relative_imports_project_mode/test.yaml b/gazelle/python/testdata/relative_imports_project_mode/test.yaml new file mode 100644 index 0000000000..fcea77710f --- /dev/null +++ b/gazelle/python/testdata/relative_imports_project_mode/test.yaml @@ -0,0 +1,15 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +--- diff --git a/gazelle/python/testdata/remove_invalid_binary/BUILD.in b/gazelle/python/testdata/remove_invalid_binary/BUILD.in new file mode 100644 index 0000000000..f4bfd65c1b --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_binary/BUILD.in @@ -0,0 +1,20 @@ +load("@rules_python//python:defs.bzl", "py_binary", "py_library") + +py_library( + name = "remove_invalid_binary", + srcs = ["__init__.py"], + deps = ["//keep_binary:foo"], +) + +py_binary( + name = "remove_invalid_binary_bin", + srcs = ["__main__.py"], + data = ["testdata/test.txt"], + visibility = ["//:__subpackages__"], +) + +py_binary( + name = "another_removed_binary", + srcs = ["foo.py"], # eg a now-deleted file that used to have `if __name__` block + imports = ["."], +) diff --git a/gazelle/python/testdata/remove_invalid_binary/BUILD.out b/gazelle/python/testdata/remove_invalid_binary/BUILD.out new file mode 100644 index 0000000000..a217b4bdaf --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_binary/BUILD.out @@ -0,0 +1,7 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "remove_invalid_binary", + srcs = ["__init__.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/remove_invalid_binary/README.md b/gazelle/python/testdata/remove_invalid_binary/README.md new file mode 100644 index 0000000000..be8a894420 --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_binary/README.md @@ -0,0 +1,3 @@ +# Remove invalid binary + +This test case asserts that `py_binary` should be deleted if invalid (no source files). diff --git a/gazelle/python/testdata/remove_invalid_binary/WORKSPACE b/gazelle/python/testdata/remove_invalid_binary/WORKSPACE new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/remove_invalid_binary/__init__.py b/gazelle/python/testdata/remove_invalid_binary/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/remove_invalid_binary/keep_binary/BUILD.in b/gazelle/python/testdata/remove_invalid_binary/keep_binary/BUILD.in new file mode 100644 index 0000000000..0036c6797a --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_binary/keep_binary/BUILD.in @@ -0,0 +1,13 @@ +load("@rules_python//python:defs.bzl", "py_binary", "py_library") + +py_binary( + name = "foo", + srcs = ["foo.py"], + visibility = ["//:__subpackages__"], +) + +py_library( + name = "keep_binary", + srcs = ["foo.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/remove_invalid_binary/keep_binary/BUILD.out b/gazelle/python/testdata/remove_invalid_binary/keep_binary/BUILD.out new file mode 100644 index 0000000000..0036c6797a --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_binary/keep_binary/BUILD.out @@ -0,0 +1,13 @@ +load("@rules_python//python:defs.bzl", "py_binary", "py_library") + +py_binary( + name = "foo", + srcs = ["foo.py"], + visibility = ["//:__subpackages__"], +) + +py_library( + name = "keep_binary", + srcs = ["foo.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/remove_invalid_binary/keep_binary/foo.py b/gazelle/python/testdata/remove_invalid_binary/keep_binary/foo.py new file mode 100644 index 0000000000..d3b51eea8e --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_binary/keep_binary/foo.py @@ -0,0 +1,2 @@ +if __name__ == "__main__": + print("foo") diff --git a/gazelle/python/testdata/remove_invalid_binary/test.yaml b/gazelle/python/testdata/remove_invalid_binary/test.yaml new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/remove_invalid_library/BUILD.in b/gazelle/python/testdata/remove_invalid_library_package_mode/BUILD.in similarity index 68% rename from gazelle/python/testdata/remove_invalid_library/BUILD.in rename to gazelle/python/testdata/remove_invalid_library_package_mode/BUILD.in index 3f24c8df35..815e673ee1 100644 --- a/gazelle/python/testdata/remove_invalid_library/BUILD.in +++ b/gazelle/python/testdata/remove_invalid_library_package_mode/BUILD.in @@ -1,4 +1,4 @@ -load("@rules_python//python:defs.bzl", "py_library") +load("@rules_python//python:defs.bzl", "py_library", "py_test") py_library( name = "remove_invalid_library", @@ -14,3 +14,8 @@ py_library( "@pypi//foo", ], ) + +py_test( + name = "my_test", + srcs = ["my_test.py"], +) diff --git a/gazelle/python/testdata/remove_invalid_library/BUILD.out b/gazelle/python/testdata/remove_invalid_library_package_mode/BUILD.out similarity index 55% rename from gazelle/python/testdata/remove_invalid_library/BUILD.out rename to gazelle/python/testdata/remove_invalid_library_package_mode/BUILD.out index 4a6fffa183..80d47076c3 100644 --- a/gazelle/python/testdata/remove_invalid_library/BUILD.out +++ b/gazelle/python/testdata/remove_invalid_library_package_mode/BUILD.out @@ -1,4 +1,4 @@ -load("@rules_python//python:defs.bzl", "py_library") +load("@rules_python//python:defs.bzl", "py_library", "py_test") py_library( name = "deps_with_no_srcs_library", @@ -8,3 +8,8 @@ py_library( "@pypi//foo", ], ) + +py_test( + name = "my_test", + srcs = ["my_test.py"], +) diff --git a/gazelle/python/testdata/remove_invalid_library/README.md b/gazelle/python/testdata/remove_invalid_library_package_mode/README.md similarity index 100% rename from gazelle/python/testdata/remove_invalid_library/README.md rename to gazelle/python/testdata/remove_invalid_library_package_mode/README.md diff --git a/gazelle/python/testdata/remove_invalid_library_package_mode/WORKSPACE b/gazelle/python/testdata/remove_invalid_library_package_mode/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_library_package_mode/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/remove_invalid_library_package_mode/my_test.py b/gazelle/python/testdata/remove_invalid_library_package_mode/my_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/remove_invalid_library/others/BUILD.in b/gazelle/python/testdata/remove_invalid_library_package_mode/others/BUILD.in similarity index 100% rename from gazelle/python/testdata/remove_invalid_library/others/BUILD.in rename to gazelle/python/testdata/remove_invalid_library_package_mode/others/BUILD.in diff --git a/gazelle/python/testdata/remove_invalid_library/others/BUILD.out b/gazelle/python/testdata/remove_invalid_library_package_mode/others/BUILD.out similarity index 100% rename from gazelle/python/testdata/remove_invalid_library/others/BUILD.out rename to gazelle/python/testdata/remove_invalid_library_package_mode/others/BUILD.out diff --git a/gazelle/python/testdata/remove_invalid_library_package_mode/test.yaml b/gazelle/python/testdata/remove_invalid_library_package_mode/test.yaml new file mode 100644 index 0000000000..fcea77710f --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_library_package_mode/test.yaml @@ -0,0 +1,15 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +--- diff --git a/gazelle/python/testdata/remove_invalid_per_file/BUILD.in b/gazelle/python/testdata/remove_invalid_per_file/BUILD.in new file mode 100644 index 0000000000..d3b15a7866 --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_per_file/BUILD.in @@ -0,0 +1,45 @@ +load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test") + +# gazelle:python_generation_mode file + +# Valid py_library — bar.py exists on disk, should be kept. +py_library( + name = "bar", + srcs = ["bar.py"], + visibility = ["//:__subpackages__"], +) + +# Stale py_library — deleted.py does not exist, should be removed. +py_library( + name = "deleted_lib", + srcs = ["deleted.py"], + visibility = ["//:__subpackages__"], +) + +# Valid py_binary — __main__.py exists on disk, should be kept. +py_binary( + name = "remove_invalid_per_file_bin", + srcs = ["__main__.py"], + visibility = ["//:__subpackages__"], +) + +# Stale py_binary — deleted_bin.py does not exist, should be removed. +py_binary( + name = "deleted_bin", + srcs = ["deleted_bin.py"], + visibility = ["//:__subpackages__"], +) + +# Valid py_test — bar_test.py exists on disk, should be kept. +py_test( + name = "bar_test", + srcs = ["bar_test.py"], + visibility = ["//:__subpackages__"], +) + +# Stale py_test — deleted_test.py does not exist, should be removed. +py_test( + name = "deleted_test", + srcs = ["deleted_test.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/remove_invalid_per_file/BUILD.out b/gazelle/python/testdata/remove_invalid_per_file/BUILD.out new file mode 100644 index 0000000000..376c14c0e1 --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_per_file/BUILD.out @@ -0,0 +1,24 @@ +load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test") + +# gazelle:python_generation_mode file + +# Valid py_library — bar.py exists on disk, should be kept. +py_library( + name = "bar", + srcs = ["bar.py"], + visibility = ["//:__subpackages__"], +) + +# Valid py_test — bar_test.py exists on disk, should be kept. +py_test( + name = "bar_test", + srcs = ["bar_test.py"], + visibility = ["//:__subpackages__"], +) + +py_binary( + name = "remove_invalid_per_file_bin", + srcs = ["__main__.py"], + main = "__main__.py", + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/remove_invalid_per_file/WORKSPACE b/gazelle/python/testdata/remove_invalid_per_file/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_per_file/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/remove_invalid_per_file/__main__.py b/gazelle/python/testdata/remove_invalid_per_file/__main__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/remove_invalid_per_file/alias_kind/BUILD.in b/gazelle/python/testdata/remove_invalid_per_file/alias_kind/BUILD.in new file mode 100644 index 0000000000..e56ea1617a --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_per_file/alias_kind/BUILD.in @@ -0,0 +1,37 @@ +load(":mylib.bzl", "my_py_library", "my_py_test") + +# gazelle:python_generation_mode file +# gazelle:alias_kind my_py_library py_library +# gazelle:alias_kind my_py_test py_test + +# Valid aliased py_library — bar.py exists on disk, should be kept. +my_py_library( + name = "bar", + srcs = ["bar.py"], + visibility = ["//:__subpackages__"], +) + +# Stale aliased py_library — deleted.py does not exist, should be removed. +# TODO: Known limitation: not fully deleted until gazelle is bumped to include +# https://github.com/bazel-contrib/bazel-gazelle/pull/2362 +my_py_library( + name = "deleted_lib", + srcs = ["deleted.py"], + visibility = ["//:__subpackages__"], +) + +# Valid aliased py_test — bar_test.py exists on disk, should be kept. +my_py_test( + name = "bar_test", + srcs = ["bar_test.py"], + visibility = ["//:__subpackages__"], +) + +# Stale aliased py_test — deleted_test.py does not exist, should be removed. +# TODO: Known limitation: not fully deleted until gazelle is bumped to include +# https://github.com/bazel-contrib/bazel-gazelle/pull/2362 +my_py_test( + name = "deleted_test", + srcs = ["deleted_test.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/remove_invalid_per_file/alias_kind/BUILD.out b/gazelle/python/testdata/remove_invalid_per_file/alias_kind/BUILD.out new file mode 100644 index 0000000000..19d6c01789 --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_per_file/alias_kind/BUILD.out @@ -0,0 +1,35 @@ +load(":mylib.bzl", "my_py_library", "my_py_test") + +# gazelle:python_generation_mode file +# gazelle:alias_kind my_py_library py_library +# gazelle:alias_kind my_py_test py_test + +# Valid aliased py_library — bar.py exists on disk, should be kept. +my_py_library( + name = "bar", + srcs = ["bar.py"], + visibility = ["//:__subpackages__"], +) + +# Stale aliased py_library — deleted.py does not exist, should be removed. +# TODO: Known limitation: not fully deleted until gazelle is bumped to include +# https://github.com/bazel-contrib/bazel-gazelle/pull/2362 +my_py_library( + name = "deleted_lib", + visibility = ["//:__subpackages__"], +) + +# Valid aliased py_test — bar_test.py exists on disk, should be kept. +my_py_test( + name = "bar_test", + srcs = ["bar_test.py"], + visibility = ["//:__subpackages__"], +) + +# Stale aliased py_test — deleted_test.py does not exist, should be removed. +# TODO: Known limitation: not fully deleted until gazelle is bumped to include +# https://github.com/bazel-contrib/bazel-gazelle/pull/2362 +my_py_test( + name = "deleted_test", + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/remove_invalid_per_file/alias_kind/bar.py b/gazelle/python/testdata/remove_invalid_per_file/alias_kind/bar.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/remove_invalid_per_file/alias_kind/bar_test.py b/gazelle/python/testdata/remove_invalid_per_file/alias_kind/bar_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/remove_invalid_per_file/bar.py b/gazelle/python/testdata/remove_invalid_per_file/bar.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/remove_invalid_per_file/bar_test.py b/gazelle/python/testdata/remove_invalid_per_file/bar_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/remove_invalid_per_file/map_kind/BUILD.in b/gazelle/python/testdata/remove_invalid_per_file/map_kind/BUILD.in new file mode 100644 index 0000000000..00b9a60d6f --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_per_file/map_kind/BUILD.in @@ -0,0 +1,37 @@ +load(":mylib.bzl", "my_py_binary", "my_py_test") + +# gazelle:python_generation_mode file +# gazelle:map_kind py_binary my_py_binary :mylib.bzl +# gazelle:map_kind py_test my_py_test :mylib.bzl + +# Valid py_library — bar.py exists on disk, should be kept. +py_library( + name = "bar", + srcs = ["bar.py"], + visibility = ["//:__subpackages__"], +) + +# Stale py_library — deleted.py does not exist, should be removed. +py_library( + name = "deleted_lib", + srcs = ["deleted.py"], + visibility = ["//:__subpackages__"], +) + +# Stale mapped py_binary — deleted_bin.py does not exist, should be removed. +my_py_binary( + name = "deleted_bin", + srcs = ["deleted_bin.py"], +) + +# Valid mapped py_test — bar_test.py exists on disk, should be kept. +my_py_test( + name = "bar_test", + srcs = ["bar_test.py"], +) + +# Stale mapped py_test — deleted_test.py does not exist, should be removed. +my_py_test( + name = "deleted_test", + srcs = ["deleted_test.py"], +) diff --git a/gazelle/python/testdata/remove_invalid_per_file/map_kind/BUILD.out b/gazelle/python/testdata/remove_invalid_per_file/map_kind/BUILD.out new file mode 100644 index 0000000000..84ca905031 --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_per_file/map_kind/BUILD.out @@ -0,0 +1,19 @@ +load("@rules_python//python:defs.bzl", "py_library") +load(":mylib.bzl", "my_py_test") + +# gazelle:python_generation_mode file +# gazelle:map_kind py_binary my_py_binary :mylib.bzl +# gazelle:map_kind py_test my_py_test :mylib.bzl + +# Valid py_library — bar.py exists on disk, should be kept. +py_library( + name = "bar", + srcs = ["bar.py"], + visibility = ["//:__subpackages__"], +) + +# Valid mapped py_test — bar_test.py exists on disk, should be kept. +my_py_test( + name = "bar_test", + srcs = ["bar_test.py"], +) diff --git a/gazelle/python/testdata/remove_invalid_per_file/map_kind/bar.py b/gazelle/python/testdata/remove_invalid_per_file/map_kind/bar.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/remove_invalid_per_file/map_kind/bar_test.py b/gazelle/python/testdata/remove_invalid_per_file/map_kind/bar_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/remove_invalid_per_file/test.yaml b/gazelle/python/testdata/remove_invalid_per_file/test.yaml new file mode 100644 index 0000000000..ed97d539c0 --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_per_file/test.yaml @@ -0,0 +1 @@ +--- diff --git a/gazelle/python/testdata/respect_alias_and_map_kind/BUILD.in b/gazelle/python/testdata/respect_alias_and_map_kind/BUILD.in new file mode 100644 index 0000000000..06ebfbdfa3 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_and_map_kind/BUILD.in @@ -0,0 +1,16 @@ +load(":mylib.bzl", "my_py_library") + +# gazelle:alias_kind my_py_library py_library +# gazelle:map_kind py_test my_test :mytest.bzl + +my_py_library( + name = "respect_alias_and_map_kind", + srcs = ["__init__.py"], +) + +my_test( + name = "respect_alias_and_map_kind_test", + srcs = ["__test__.py"], + main = "__test__.py", + deps = [], +) diff --git a/gazelle/python/testdata/respect_alias_and_map_kind/BUILD.out b/gazelle/python/testdata/respect_alias_and_map_kind/BUILD.out new file mode 100644 index 0000000000..6c7eee06e3 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_and_map_kind/BUILD.out @@ -0,0 +1,21 @@ +load(":mylib.bzl", "my_py_library") +load(":mytest.bzl", "my_test") + +# gazelle:alias_kind my_py_library py_library +# gazelle:map_kind py_test my_test :mytest.bzl + +my_py_library( + name = "respect_alias_and_map_kind", + srcs = [ + "__init__.py", + "foo.py", + ], + visibility = ["//:__subpackages__"], +) + +my_test( + name = "respect_alias_and_map_kind_test", + srcs = ["__test__.py"], + main = "__test__.py", + deps = [":respect_alias_and_map_kind"], +) diff --git a/gazelle/python/testdata/respect_alias_and_map_kind/README.md b/gazelle/python/testdata/respect_alias_and_map_kind/README.md new file mode 100644 index 0000000000..5e5ca20d27 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_and_map_kind/README.md @@ -0,0 +1,4 @@ +# Respect `alias_kind` and `map_kind` + +This test case asserts that `# gazelle:alias_kind` and `# gazelle:map_kind` are +both respected as the expected kind at once and do not collide. diff --git a/gazelle/python/testdata/respect_alias_and_map_kind/WORKSPACE b/gazelle/python/testdata/respect_alias_and_map_kind/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/respect_alias_and_map_kind/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/respect_alias_and_map_kind/__init__.py b/gazelle/python/testdata/respect_alias_and_map_kind/__init__.py new file mode 100644 index 0000000000..6a49193fe4 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_and_map_kind/__init__.py @@ -0,0 +1,3 @@ +from foo import foo + +_ = foo diff --git a/gazelle/python/testdata/respect_alias_and_map_kind/__test__.py b/gazelle/python/testdata/respect_alias_and_map_kind/__test__.py new file mode 100644 index 0000000000..d6085a41b4 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_and_map_kind/__test__.py @@ -0,0 +1,12 @@ +import unittest + +from __init__ import foo + + +class FooTest(unittest.TestCase): + def test_foo(self): + self.assertEqual("foo", foo()) + + +if __name__ == "__main__": + unittest.main() diff --git a/gazelle/python/testdata/respect_alias_and_map_kind/foo.py b/gazelle/python/testdata/respect_alias_and_map_kind/foo.py new file mode 100644 index 0000000000..3f049df738 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_and_map_kind/foo.py @@ -0,0 +1,17 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +def foo(): + return "foo" diff --git a/gazelle/python/testdata/respect_alias_and_map_kind/test.yaml b/gazelle/python/testdata/respect_alias_and_map_kind/test.yaml new file mode 100644 index 0000000000..36dd656b39 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_and_map_kind/test.yaml @@ -0,0 +1,3 @@ +--- +expect: + exit_code: 0 diff --git a/gazelle/python/testdata/respect_alias_kind/BUILD.in b/gazelle/python/testdata/respect_alias_kind/BUILD.in new file mode 100644 index 0000000000..3dc5ca7151 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_kind/BUILD.in @@ -0,0 +1,8 @@ +load(":mylib.bzl", "my_py_library") + +# gazelle:alias_kind my_py_library py_library + +my_py_library( + name = "respect_alias_kind", + srcs = ["__init__.py"], +) diff --git a/gazelle/python/testdata/respect_alias_kind/BUILD.out b/gazelle/python/testdata/respect_alias_kind/BUILD.out new file mode 100644 index 0000000000..9e65a95923 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_kind/BUILD.out @@ -0,0 +1,12 @@ +load(":mylib.bzl", "my_py_library") + +# gazelle:alias_kind my_py_library py_library + +my_py_library( + name = "respect_alias_kind", + srcs = [ + "__init__.py", + "foo.py", + ], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/respect_alias_kind/README.md b/gazelle/python/testdata/respect_alias_kind/README.md new file mode 100644 index 0000000000..d0d8702106 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_kind/README.md @@ -0,0 +1,4 @@ +# Respect `alias_kind` + +This test case asserts that Gazelle updates the existing wrapper-macro rule +declared with `# gazelle:alias_kind` in place instead of reporting a collision. diff --git a/gazelle/python/testdata/respect_alias_kind/WORKSPACE b/gazelle/python/testdata/respect_alias_kind/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/respect_alias_kind/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/respect_alias_kind/__init__.py b/gazelle/python/testdata/respect_alias_kind/__init__.py new file mode 100644 index 0000000000..6a49193fe4 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_kind/__init__.py @@ -0,0 +1,3 @@ +from foo import foo + +_ = foo diff --git a/gazelle/python/testdata/respect_alias_kind/foo.py b/gazelle/python/testdata/respect_alias_kind/foo.py new file mode 100644 index 0000000000..cf68624419 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_kind/foo.py @@ -0,0 +1,2 @@ +def foo(): + return "foo" diff --git a/gazelle/python/testdata/respect_alias_kind/test.yaml b/gazelle/python/testdata/respect_alias_kind/test.yaml new file mode 100644 index 0000000000..36dd656b39 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_kind/test.yaml @@ -0,0 +1,3 @@ +--- +expect: + exit_code: 0 diff --git a/gazelle/python/testdata/respect_kind_mapping/BUILD.in b/gazelle/python/testdata/respect_kind_mapping/BUILD.in index 6a06737623..3c6ec07014 100644 --- a/gazelle/python/testdata/respect_kind_mapping/BUILD.in +++ b/gazelle/python/testdata/respect_kind_mapping/BUILD.in @@ -1,6 +1,7 @@ load("@rules_python//python:defs.bzl", "py_library") # gazelle:map_kind py_test my_test :mytest.bzl +# gazelle:map_kind py_binary my_bin :mytest.bzl py_library( name = "respect_kind_mapping", @@ -13,3 +14,8 @@ my_test( main = "__test__.py", deps = [":respect_kind_mapping"], ) + +my_bin( + name = "my_bin_gets_removed", + srcs = ["__main__.py"], +) diff --git a/gazelle/python/testdata/respect_kind_mapping/BUILD.out b/gazelle/python/testdata/respect_kind_mapping/BUILD.out index 7c5fb0bd20..eb6894f1c8 100644 --- a/gazelle/python/testdata/respect_kind_mapping/BUILD.out +++ b/gazelle/python/testdata/respect_kind_mapping/BUILD.out @@ -1,7 +1,8 @@ -load(":mytest.bzl", "my_test") load("@rules_python//python:defs.bzl", "py_library") +load(":mytest.bzl", "my_test") # gazelle:map_kind py_test my_test :mytest.bzl +# gazelle:map_kind py_binary my_bin :mytest.bzl py_library( name = "respect_kind_mapping", diff --git a/gazelle/python/testdata/sibling_imports/README.md b/gazelle/python/testdata/sibling_imports/README.md index e59be07634..d21a671b1c 100644 --- a/gazelle/python/testdata/sibling_imports/README.md +++ b/gazelle/python/testdata/sibling_imports/README.md @@ -1,3 +1,13 @@ # Sibling imports -This test case asserts that imports from sibling modules are resolved correctly. It covers 3 different types of imports in `pkg/unit_test.py` \ No newline at end of file +This test case asserts that imports from sibling modules are resolved correctly +when the `python_resolve_sibling_imports` directive is enabled (default +behavior). It covers 3 different types of imports in `pkg/unit_test.py`: + +- `import a` - resolves to the sibling `a.py` in the same package +- `import test_util` - resolves to the sibling `test_util.py` in the same + package +- `from b import run` - resolves to the sibling `b.py` in the same package + +When sibling imports are enabled, we allow them to be satisfied by sibling +modules (ie. modules in the same package). diff --git a/gazelle/python/testdata/sibling_imports/pkg/BUILD.in b/gazelle/python/testdata/sibling_imports/pkg/BUILD.in index e69de29bb2..5c25b0d5a6 100644 --- a/gazelle/python/testdata/sibling_imports/pkg/BUILD.in +++ b/gazelle/python/testdata/sibling_imports/pkg/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_resolve_sibling_imports true diff --git a/gazelle/python/testdata/sibling_imports/pkg/BUILD.out b/gazelle/python/testdata/sibling_imports/pkg/BUILD.out index cae6c3f17a..e8c13098c2 100644 --- a/gazelle/python/testdata/sibling_imports/pkg/BUILD.out +++ b/gazelle/python/testdata/sibling_imports/pkg/BUILD.out @@ -1,5 +1,7 @@ load("@rules_python//python:defs.bzl", "py_library", "py_test") +# gazelle:python_resolve_sibling_imports true + py_library( name = "pkg", srcs = [ @@ -23,4 +25,3 @@ py_test( ":test_util", ], ) - diff --git a/gazelle/python/testdata/sibling_imports_disabled/BUILD.in b/gazelle/python/testdata/sibling_imports_disabled/BUILD.in new file mode 100644 index 0000000000..44f7406e58 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/BUILD.in @@ -0,0 +1,2 @@ +# gazelle:python_resolve_sibling_imports false +# gazelle:python_experimental_allow_relative_imports true diff --git a/gazelle/python/testdata/sibling_imports_disabled/BUILD.out b/gazelle/python/testdata/sibling_imports_disabled/BUILD.out new file mode 100644 index 0000000000..d3d5c6bfab --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/BUILD.out @@ -0,0 +1,18 @@ +load("@rules_python//python:defs.bzl", "py_library", "py_test") + +# gazelle:python_resolve_sibling_imports false +# gazelle:python_experimental_allow_relative_imports true + +py_library( + name = "sibling_imports_disabled", + srcs = [ + "a.py", + "b.py", + ], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "test_util", + srcs = ["test_util.py"], +) diff --git a/gazelle/python/testdata/sibling_imports_disabled/README.md b/gazelle/python/testdata/sibling_imports_disabled/README.md new file mode 100644 index 0000000000..a39023e8a3 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/README.md @@ -0,0 +1,22 @@ +# Sibling imports disabled + +This test case asserts that imports from sibling modules are NOT resolved as +absolute imports when the `python_resolve_sibling_imports` directive is +disabled. It covers different types of imports in `pkg/unit_test.py`: + +- `import a` - resolves to the root-level `a.py` instead of the sibling + `pkg/a.py` +- `from typing import Iterable` - resolves to the stdlib `typing` module + (not the sibling `typing.py`). +- `from .b import run` / `from .typing import A` - resolves to the sibling + `pkg/b.py` / `pkg/typing.py` (with + `gazelle:python_experimental_allow_relative_imports` enabled) +- `import test_util` - resolves to the root-level `test_util.py` instead of + the sibling `pkg/test_util.py` +- `from b import run` - resolves to the root-level `b.py` instead of the + sibling `pkg/b.py` + +When sibling imports are disabled with +`# gazelle:python_resolve_sibling_imports false`, the imports remain as-is +and follow standard Python resolution rules where absolute imports can't refer +to sibling modules. diff --git a/gazelle/python/testdata/sibling_imports_disabled/WORKSPACE b/gazelle/python/testdata/sibling_imports_disabled/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/sibling_imports_disabled/a.py b/gazelle/python/testdata/sibling_imports_disabled/a.py new file mode 100644 index 0000000000..fad4fb1ff9 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/a.py @@ -0,0 +1 @@ +# Root level a.py file for testing disabled sibling imports diff --git a/gazelle/python/testdata/sibling_imports_disabled/b.py b/gazelle/python/testdata/sibling_imports_disabled/b.py new file mode 100644 index 0000000000..a5eafc436f --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/b.py @@ -0,0 +1,3 @@ +# Root level b.py file for testing disabled sibling imports +def run(): + pass diff --git a/gazelle/python/testdata/sibling_imports_disabled/pkg/BUILD.in b/gazelle/python/testdata/sibling_imports_disabled/pkg/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/sibling_imports_disabled/pkg/BUILD.out b/gazelle/python/testdata/sibling_imports_disabled/pkg/BUILD.out new file mode 100644 index 0000000000..e778ce1076 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/pkg/BUILD.out @@ -0,0 +1,27 @@ +load("@rules_python//python:defs.bzl", "py_library", "py_test") + +py_library( + name = "pkg", + srcs = [ + "__init__.py", + "a.py", + "b.py", + "typing.py", + ], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "test_util", + srcs = ["test_util.py"], + deps = [":pkg"], +) + +py_test( + name = "unit_test", + srcs = ["unit_test.py"], + deps = [ + "//:sibling_imports_disabled", + "//:test_util", + ], +) diff --git a/gazelle/python/testdata/sibling_imports_disabled/pkg/__init__.py b/gazelle/python/testdata/sibling_imports_disabled/pkg/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/sibling_imports_disabled/pkg/a.py b/gazelle/python/testdata/sibling_imports_disabled/pkg/a.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/sibling_imports_disabled/pkg/b.py b/gazelle/python/testdata/sibling_imports_disabled/pkg/b.py new file mode 100644 index 0000000000..d04d423678 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/pkg/b.py @@ -0,0 +1,2 @@ +def run(): + pass diff --git a/gazelle/python/testdata/sibling_imports_disabled/pkg/test_util.py b/gazelle/python/testdata/sibling_imports_disabled/pkg/test_util.py new file mode 100644 index 0000000000..01cc15da86 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/pkg/test_util.py @@ -0,0 +1,2 @@ +from .b import run +from .typing import A diff --git a/gazelle/python/testdata/sibling_imports_disabled/pkg/typing.py b/gazelle/python/testdata/sibling_imports_disabled/pkg/typing.py new file mode 100644 index 0000000000..76f516f79f --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/pkg/typing.py @@ -0,0 +1 @@ +A = 1 diff --git a/gazelle/python/testdata/sibling_imports_disabled/pkg/unit_test.py b/gazelle/python/testdata/sibling_imports_disabled/pkg/unit_test.py new file mode 100644 index 0000000000..3c551a1cb1 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/pkg/unit_test.py @@ -0,0 +1,5 @@ +from typing import Iterable + +import a +import test_util +from b import run diff --git a/gazelle/python/testdata/sibling_imports_disabled/test.yaml b/gazelle/python/testdata/sibling_imports_disabled/test.yaml new file mode 100644 index 0000000000..ed97d539c0 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/test.yaml @@ -0,0 +1 @@ +--- diff --git a/gazelle/python/testdata/sibling_imports_disabled/test_util.py b/gazelle/python/testdata/sibling_imports_disabled/test_util.py new file mode 100644 index 0000000000..f5fa1b34ea --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/test_util.py @@ -0,0 +1 @@ +# Root level test_util.py file for testing disabled sibling imports diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/BUILD.in b/gazelle/python/testdata/sibling_imports_disabled_file_mode/BUILD.in new file mode 100644 index 0000000000..32b0bec20f --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/BUILD.in @@ -0,0 +1,3 @@ +# gazelle:python_generation_mode file +# gazelle:python_resolve_sibling_imports false +# gazelle:python_experimental_allow_relative_imports true diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/BUILD.out b/gazelle/python/testdata/sibling_imports_disabled_file_mode/BUILD.out new file mode 100644 index 0000000000..d7a829e8ea --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/BUILD.out @@ -0,0 +1,22 @@ +load("@rules_python//python:defs.bzl", "py_library", "py_test") + +# gazelle:python_generation_mode file +# gazelle:python_resolve_sibling_imports false +# gazelle:python_experimental_allow_relative_imports true + +py_library( + name = "a", + srcs = ["a.py"], + visibility = ["//:__subpackages__"], +) + +py_library( + name = "b", + srcs = ["b.py"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "test_util", + srcs = ["test_util.py"], +) diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/README.md b/gazelle/python/testdata/sibling_imports_disabled_file_mode/README.md new file mode 100644 index 0000000000..124e751b10 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/README.md @@ -0,0 +1,22 @@ +# Sibling imports disabled (file generation mode) + +This test case asserts that imports from sibling modules are NOT resolved as +absolute imports when the `python_resolve_sibling_imports` directive is +disabled. It covers different types of imports in `pkg/unit_test.py`: + +- `import a` - resolves to the root-level `a.py` instead of the sibling + `pkg/a.py` +- `from typing import Iterable` - resolves to the stdlib `typing` module + (not the sibling `typing.py`). +- `from .b import run` / `from .typing import A` - resolves to the sibling + `pkg/b.py` / `pkg/typing.py` (with + `gazelle:python_experimental_allow_relative_imports` enabled) +- `import test_util` - resolves to the root-level `test_util.py` instead of + the sibling `pkg/test_util.py` +- `from b import run` - resolves to the root-level `b.py` instead of the + sibling `pkg/b.py` + +When sibling imports are disabled with +`# gazelle:python_resolve_sibling_imports false`, the imports remain as-is +and follow standard Python resolution rules where absolute imports can't refer +to sibling modules. diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/WORKSPACE b/gazelle/python/testdata/sibling_imports_disabled_file_mode/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/a.py b/gazelle/python/testdata/sibling_imports_disabled_file_mode/a.py new file mode 100644 index 0000000000..fad4fb1ff9 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/a.py @@ -0,0 +1 @@ +# Root level a.py file for testing disabled sibling imports diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/b.py b/gazelle/python/testdata/sibling_imports_disabled_file_mode/b.py new file mode 100644 index 0000000000..a5eafc436f --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/b.py @@ -0,0 +1,3 @@ +# Root level b.py file for testing disabled sibling imports +def run(): + pass diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/BUILD.in b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/BUILD.out b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/BUILD.out new file mode 100644 index 0000000000..ab161e135f --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/BUILD.out @@ -0,0 +1,38 @@ +load("@rules_python//python:defs.bzl", "py_library", "py_test") + +py_library( + name = "a", + srcs = ["a.py"], + visibility = ["//:__subpackages__"], +) + +py_library( + name = "b", + srcs = ["b.py"], + visibility = ["//:__subpackages__"], +) + +py_library( + name = "typing", + srcs = ["typing.py"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "test_util", + srcs = ["test_util.py"], + deps = [ + ":b", + ":typing", + ], +) + +py_test( + name = "unit_test", + srcs = ["unit_test.py"], + deps = [ + "//:a", + "//:b", + "//:test_util", + ], +) diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/__init__.py b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/a.py b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/a.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/b.py b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/b.py new file mode 100644 index 0000000000..d04d423678 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/b.py @@ -0,0 +1,2 @@ +def run(): + pass diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/test_util.py b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/test_util.py new file mode 100644 index 0000000000..01cc15da86 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/test_util.py @@ -0,0 +1,2 @@ +from .b import run +from .typing import A diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/typing.py b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/typing.py new file mode 100644 index 0000000000..76f516f79f --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/typing.py @@ -0,0 +1 @@ +A = 1 diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/unit_test.py b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/unit_test.py new file mode 100644 index 0000000000..3c551a1cb1 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/unit_test.py @@ -0,0 +1,5 @@ +from typing import Iterable + +import a +import test_util +from b import run diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/test.yaml b/gazelle/python/testdata/sibling_imports_disabled_file_mode/test.yaml new file mode 100644 index 0000000000..ed97d539c0 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/test.yaml @@ -0,0 +1 @@ +--- diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/test_util.py b/gazelle/python/testdata/sibling_imports_disabled_file_mode/test_util.py new file mode 100644 index 0000000000..f5fa1b34ea --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/test_util.py @@ -0,0 +1 @@ +# Root level test_util.py file for testing disabled sibling imports diff --git a/gazelle/python/testdata/simple_test_with_conftest/BUILD.in b/gazelle/python/testdata/simple_test_with_conftest/BUILD.in index 3f2beb3147..6dfab75442 100644 --- a/gazelle/python/testdata/simple_test_with_conftest/BUILD.in +++ b/gazelle/python/testdata/simple_test_with_conftest/BUILD.in @@ -1 +1,3 @@ load("@rules_python//python:defs.bzl", "py_library") + +# gazelle:python_resolve_sibling_imports true diff --git a/gazelle/python/testdata/simple_test_with_conftest/BUILD.out b/gazelle/python/testdata/simple_test_with_conftest/BUILD.out index 18079bf2f4..62e1c550e6 100644 --- a/gazelle/python/testdata/simple_test_with_conftest/BUILD.out +++ b/gazelle/python/testdata/simple_test_with_conftest/BUILD.out @@ -1,5 +1,7 @@ load("@rules_python//python:defs.bzl", "py_library", "py_test") +# gazelle:python_resolve_sibling_imports true + py_library( name = "simple_test_with_conftest", srcs = [ diff --git a/gazelle/python/testdata/simple_test_with_conftest/bar/BUILD.out b/gazelle/python/testdata/simple_test_with_conftest/bar/BUILD.out index 4a1204e989..9b500d4733 100644 --- a/gazelle/python/testdata/simple_test_with_conftest/bar/BUILD.out +++ b/gazelle/python/testdata/simple_test_with_conftest/bar/BUILD.out @@ -23,5 +23,6 @@ py_test( deps = [ ":bar", ":conftest", + "//:conftest", ], ) diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/BUILD.in b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/BUILD.in new file mode 100644 index 0000000000..f8a40fe26c --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/BUILD.in @@ -0,0 +1,3 @@ +load("@rules_python//python:defs.bzl", "py_library") + +# gazelle:python_resolve_sibling_imports false diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/BUILD.out b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/BUILD.out new file mode 100644 index 0000000000..b5a7066aff --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/BUILD.out @@ -0,0 +1,29 @@ +load("@rules_python//python:defs.bzl", "py_library", "py_test") + +# gazelle:python_resolve_sibling_imports false + +py_library( + name = "simple_test_with_conftest_sibling_imports_disabled", + srcs = [ + "__init__.py", + "foo.py", + ], + visibility = ["//:__subpackages__"], +) + +py_library( + name = "conftest", + testonly = True, + srcs = ["conftest.py"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "simple_test_with_conftest_sibling_imports_disabled_test", + srcs = ["__test__.py"], + main = "__test__.py", + deps = [ + ":conftest", + ":simple_test_with_conftest_sibling_imports_disabled", + ], +) diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/README.md b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/README.md new file mode 100644 index 0000000000..98793c23de --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/README.md @@ -0,0 +1,4 @@ +# Simple test with conftest.py (sibling imports disable) + +This test case asserts that a simple `py_test` is generated as expected when a +`conftest.py` is present with sibling imports disabled. diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/WORKSPACE b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/__init__.py b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/__init__.py new file mode 100644 index 0000000000..6a49193fe4 --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/__init__.py @@ -0,0 +1,3 @@ +from foo import foo + +_ = foo diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/__test__.py b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/__test__.py new file mode 100644 index 0000000000..d6085a41b4 --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/__test__.py @@ -0,0 +1,12 @@ +import unittest + +from __init__ import foo + + +class FooTest(unittest.TestCase): + def test_foo(self): + self.assertEqual("foo", foo()) + + +if __name__ == "__main__": + unittest.main() diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/BUILD.in b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/BUILD.in new file mode 100644 index 0000000000..3f2beb3147 --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/BUILD.in @@ -0,0 +1 @@ +load("@rules_python//python:defs.bzl", "py_library") diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/BUILD.out b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/BUILD.out new file mode 100644 index 0000000000..25a2d39672 --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/BUILD.out @@ -0,0 +1,28 @@ +load("@rules_python//python:defs.bzl", "py_library", "py_test") + +py_library( + name = "bar", + srcs = [ + "__init__.py", + "bar.py", + ], + visibility = ["//:__subpackages__"], +) + +py_library( + name = "conftest", + testonly = True, + srcs = ["conftest.py"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "bar_test", + srcs = ["__test__.py"], + main = "__test__.py", + deps = [ + ":conftest", + "//:conftest", + "//:simple_test_with_conftest_sibling_imports_disabled", + ], +) diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/__init__.py b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/__init__.py new file mode 100644 index 0000000000..0c59205559 --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/__init__.py @@ -0,0 +1,3 @@ +from bar import bar + +_ = bar diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/__test__.py b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/__test__.py new file mode 100644 index 0000000000..c3d4734eed --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/__test__.py @@ -0,0 +1,12 @@ +import unittest + +from __init__ import bar + + +class BarTest(unittest.TestCase): + def test_bar(self): + self.assertEqual("bar", bar()) + + +if __name__ == "__main__": + unittest.main() diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/bar.py b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/bar.py new file mode 100644 index 0000000000..ee70a51f03 --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/bar.py @@ -0,0 +1,2 @@ +def bar(): + return "bar" diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/conftest.py b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/conftest.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/conftest.py b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/conftest.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/foo.py b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/foo.py new file mode 100644 index 0000000000..cf68624419 --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/foo.py @@ -0,0 +1,2 @@ +def foo(): + return "foo" diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/test.yaml b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/test.yaml new file mode 100644 index 0000000000..8071ef4094 --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/test.yaml @@ -0,0 +1,4 @@ + +--- +expect: + exit_code: 0 diff --git a/gazelle/python/testdata/subdir_sources/BUILD.in b/gazelle/python/testdata/subdir_sources/BUILD.in index e69de29bb2..e8f3827bd2 100644 --- a/gazelle/python/testdata/subdir_sources/BUILD.in +++ b/gazelle/python/testdata/subdir_sources/BUILD.in @@ -0,0 +1,2 @@ +# gazelle:python_generation_mode project +# gazelle:python_resolve_sibling_imports true diff --git a/gazelle/python/testdata/subdir_sources/BUILD.out b/gazelle/python/testdata/subdir_sources/BUILD.out index d03a8f05ac..5b96ad7576 100644 --- a/gazelle/python/testdata/subdir_sources/BUILD.out +++ b/gazelle/python/testdata/subdir_sources/BUILD.out @@ -1,5 +1,9 @@ + load("@rules_python//python:defs.bzl", "py_binary") +# gazelle:python_generation_mode project +# gazelle:python_resolve_sibling_imports true + py_binary( name = "subdir_sources_bin", srcs = ["__main__.py"], diff --git a/gazelle/python/testdata/type_checking_imports/BUILD.in b/gazelle/python/testdata/type_checking_imports/BUILD.in new file mode 100644 index 0000000000..d4dce063ef --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports/BUILD.in @@ -0,0 +1,2 @@ +# gazelle:python_generation_mode file +# gazelle:python_generate_pyi_deps true diff --git a/gazelle/python/testdata/type_checking_imports/BUILD.out b/gazelle/python/testdata/type_checking_imports/BUILD.out new file mode 100644 index 0000000000..690210682c --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports/BUILD.out @@ -0,0 +1,33 @@ +load("@rules_python//python:defs.bzl", "py_library") + +# gazelle:python_generation_mode file +# gazelle:python_generate_pyi_deps true + +py_library( + name = "bar", + srcs = ["bar.py"], + pyi_deps = [":foo"], + visibility = ["//:__subpackages__"], + deps = [":baz"], +) + +py_library( + name = "baz", + srcs = ["baz.py"], + pyi_deps = [ + "@gazelle_python_test//boto3", + "@gazelle_python_test//boto3_stubs", + ], + visibility = ["//:__subpackages__"], +) + +py_library( + name = "foo", + srcs = ["foo.py"], + pyi_deps = [ + "@gazelle_python_test//boto3_stubs", + "@gazelle_python_test//djangorestframework", + ], + visibility = ["//:__subpackages__"], + deps = ["@gazelle_python_test//boto3"], +) diff --git a/gazelle/python/testdata/type_checking_imports/README.md b/gazelle/python/testdata/type_checking_imports/README.md new file mode 100644 index 0000000000..b09f442be3 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports/README.md @@ -0,0 +1,5 @@ +# Type Checking Imports + +Test that the Python gazelle correctly handles type-only imports inside `if TYPE_CHECKING:` blocks. + +Type-only imports should be added to the `pyi_deps` attribute instead of the regular `deps` attribute. diff --git a/gazelle/python/testdata/type_checking_imports/WORKSPACE b/gazelle/python/testdata/type_checking_imports/WORKSPACE new file mode 100644 index 0000000000..3e6e74e7f4 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports/WORKSPACE @@ -0,0 +1 @@ +workspace(name = "gazelle_python_test") diff --git a/gazelle/python/testdata/type_checking_imports/bar.py b/gazelle/python/testdata/type_checking_imports/bar.py new file mode 100644 index 0000000000..47c7d93d08 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports/bar.py @@ -0,0 +1,9 @@ +from typing import TYPE_CHECKING + +# foo should be added as a pyi_deps, since it is only imported in a type-checking context, but baz should be +# added as a deps. +from baz import X + +if TYPE_CHECKING: + import baz + import foo diff --git a/third_party/rules_pycross/pycross/private/tools/BUILD.bazel b/gazelle/python/testdata/type_checking_imports/baz.py similarity index 65% rename from third_party/rules_pycross/pycross/private/tools/BUILD.bazel rename to gazelle/python/testdata/type_checking_imports/baz.py index 41485c18a3..1c69e25da4 100644 --- a/third_party/rules_pycross/pycross/private/tools/BUILD.bazel +++ b/gazelle/python/testdata/type_checking_imports/baz.py @@ -1,4 +1,3 @@ -# Copyright 2023 Jeremy Volkman. All rights reserved. # Copyright 2023 The Bazel Authors. All rights reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -13,14 +12,12 @@ # See the License for the specific language governing permissions and # limitations under the License. -load("//python:defs.bzl", "py_binary") -py_binary( - name = "wheel_installer", - srcs = ["wheel_installer.py"], - visibility = ["//visibility:public"], - deps = [ - "//python/private/pypi/whl_installer:lib", - "@pypi__installer//:lib", - ], -) +# While this format is not official, it is supported by most type checkers and +# is used in the wild to avoid importing the typing module. +TYPE_CHECKING = False +if TYPE_CHECKING: + # Both boto3 and boto3_stubs should be added to pyi_deps. + import boto3 + +X = 1 diff --git a/gazelle/python/testdata/type_checking_imports/foo.py b/gazelle/python/testdata/type_checking_imports/foo.py new file mode 100644 index 0000000000..655cb54675 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports/foo.py @@ -0,0 +1,21 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import typing + +# boto3 should be added to deps. boto3_stubs and djangorestframework should be added to pyi_deps. +import boto3 + +if typing.TYPE_CHECKING: + from rest_framework import serializers diff --git a/gazelle/python/testdata/type_checking_imports/gazelle_python.yaml b/gazelle/python/testdata/type_checking_imports/gazelle_python.yaml new file mode 100644 index 0000000000..a782354215 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports/gazelle_python.yaml @@ -0,0 +1,20 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +manifest: + modules_mapping: + boto3: boto3 + boto3_stubs: boto3_stubs + rest_framework: djangorestframework + pip_deps_repository_name: gazelle_python_test diff --git a/gazelle/python/testdata/type_checking_imports/test.yaml b/gazelle/python/testdata/type_checking_imports/test.yaml new file mode 100644 index 0000000000..fcea77710f --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports/test.yaml @@ -0,0 +1,15 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +--- diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/BUILD.in b/gazelle/python/testdata/type_checking_imports_across_packages/BUILD.in new file mode 100644 index 0000000000..8e6c1cbabb --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_across_packages/BUILD.in @@ -0,0 +1,2 @@ +# gazelle:python_generation_mode package +# gazelle:python_generate_pyi_deps true diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/BUILD.out b/gazelle/python/testdata/type_checking_imports_across_packages/BUILD.out new file mode 100644 index 0000000000..8e6c1cbabb --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_across_packages/BUILD.out @@ -0,0 +1,2 @@ +# gazelle:python_generation_mode package +# gazelle:python_generate_pyi_deps true diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/README.md b/gazelle/python/testdata/type_checking_imports_across_packages/README.md new file mode 100644 index 0000000000..75fb3aae56 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_across_packages/README.md @@ -0,0 +1,6 @@ +# Overlapping deps and pyi_deps across packages + +This test reproduces a case where a dependency may be added to both `deps` and +`pyi_deps`. Package `b` imports `a.foo` normally and imports `a.bar` as a +type-checking only import. The dependency on package `a` should appear only in +`deps` (and not `pyi_deps`) of package `b`. diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/WORKSPACE b/gazelle/python/testdata/type_checking_imports_across_packages/WORKSPACE new file mode 100644 index 0000000000..3e6e74e7f4 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_across_packages/WORKSPACE @@ -0,0 +1 @@ +workspace(name = "gazelle_python_test") diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/a/BUILD.in b/gazelle/python/testdata/type_checking_imports_across_packages/a/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/a/BUILD.out b/gazelle/python/testdata/type_checking_imports_across_packages/a/BUILD.out new file mode 100644 index 0000000000..cf9be008b1 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_across_packages/a/BUILD.out @@ -0,0 +1,10 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "a", + srcs = [ + "bar.py", + "foo.py", + ], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/a/bar.py b/gazelle/python/testdata/type_checking_imports_across_packages/a/bar.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/a/foo.py b/gazelle/python/testdata/type_checking_imports_across_packages/a/foo.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/b/BUILD.in b/gazelle/python/testdata/type_checking_imports_across_packages/b/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/b/BUILD.out b/gazelle/python/testdata/type_checking_imports_across_packages/b/BUILD.out new file mode 100644 index 0000000000..15f4d343e1 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_across_packages/b/BUILD.out @@ -0,0 +1,8 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "b", + srcs = ["b.py"], + visibility = ["//:__subpackages__"], + deps = ["//a"], +) diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/b/b.py b/gazelle/python/testdata/type_checking_imports_across_packages/b/b.py new file mode 100644 index 0000000000..93d09c0baa --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_across_packages/b/b.py @@ -0,0 +1,6 @@ +from typing import TYPE_CHECKING + +from a import foo + +if TYPE_CHECKING: + from a import bar diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/test.yaml b/gazelle/python/testdata/type_checking_imports_across_packages/test.yaml new file mode 100644 index 0000000000..ed97d539c0 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_across_packages/test.yaml @@ -0,0 +1 @@ +--- diff --git a/gazelle/python/testdata/type_checking_imports_disabled/BUILD.in b/gazelle/python/testdata/type_checking_imports_disabled/BUILD.in new file mode 100644 index 0000000000..ab6d30f5a7 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_disabled/BUILD.in @@ -0,0 +1,2 @@ +# gazelle:python_generation_mode file +# gazelle:python_generate_pyi_deps false diff --git a/gazelle/python/testdata/type_checking_imports_disabled/BUILD.out b/gazelle/python/testdata/type_checking_imports_disabled/BUILD.out new file mode 100644 index 0000000000..bf23d28da9 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_disabled/BUILD.out @@ -0,0 +1,35 @@ +load("@rules_python//python:defs.bzl", "py_library") + +# gazelle:python_generation_mode file +# gazelle:python_generate_pyi_deps false + +py_library( + name = "bar", + srcs = ["bar.py"], + visibility = ["//:__subpackages__"], + deps = [ + ":baz", + ":foo", + ], +) + +py_library( + name = "baz", + srcs = ["baz.py"], + visibility = ["//:__subpackages__"], + deps = [ + "@gazelle_python_test//boto3", + "@gazelle_python_test//boto3_stubs", + ], +) + +py_library( + name = "foo", + srcs = ["foo.py"], + visibility = ["//:__subpackages__"], + deps = [ + "@gazelle_python_test//boto3", + "@gazelle_python_test//boto3_stubs", + "@gazelle_python_test//djangorestframework", + ], +) diff --git a/gazelle/python/testdata/type_checking_imports_disabled/README.md b/gazelle/python/testdata/type_checking_imports_disabled/README.md new file mode 100644 index 0000000000..0e3b623614 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_disabled/README.md @@ -0,0 +1,3 @@ +# Type Checking Imports (disabled) + +See `type_checking_imports`; this is the same test case, but with the directive disabled. diff --git a/gazelle/python/testdata/type_checking_imports_disabled/WORKSPACE b/gazelle/python/testdata/type_checking_imports_disabled/WORKSPACE new file mode 100644 index 0000000000..3e6e74e7f4 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_disabled/WORKSPACE @@ -0,0 +1 @@ +workspace(name = "gazelle_python_test") diff --git a/gazelle/python/testdata/type_checking_imports_disabled/bar.py b/gazelle/python/testdata/type_checking_imports_disabled/bar.py new file mode 100644 index 0000000000..47c7d93d08 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_disabled/bar.py @@ -0,0 +1,9 @@ +from typing import TYPE_CHECKING + +# foo should be added as a pyi_deps, since it is only imported in a type-checking context, but baz should be +# added as a deps. +from baz import X + +if TYPE_CHECKING: + import baz + import foo diff --git a/gazelle/python/testdata/type_checking_imports_disabled/baz.py b/gazelle/python/testdata/type_checking_imports_disabled/baz.py new file mode 100644 index 0000000000..1c69e25da4 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_disabled/baz.py @@ -0,0 +1,23 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# While this format is not official, it is supported by most type checkers and +# is used in the wild to avoid importing the typing module. +TYPE_CHECKING = False +if TYPE_CHECKING: + # Both boto3 and boto3_stubs should be added to pyi_deps. + import boto3 + +X = 1 diff --git a/gazelle/python/testdata/type_checking_imports_disabled/foo.py b/gazelle/python/testdata/type_checking_imports_disabled/foo.py new file mode 100644 index 0000000000..655cb54675 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_disabled/foo.py @@ -0,0 +1,21 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import typing + +# boto3 should be added to deps. boto3_stubs and djangorestframework should be added to pyi_deps. +import boto3 + +if typing.TYPE_CHECKING: + from rest_framework import serializers diff --git a/gazelle/python/testdata/type_checking_imports_disabled/gazelle_python.yaml b/gazelle/python/testdata/type_checking_imports_disabled/gazelle_python.yaml new file mode 100644 index 0000000000..a782354215 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_disabled/gazelle_python.yaml @@ -0,0 +1,20 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +manifest: + modules_mapping: + boto3: boto3 + boto3_stubs: boto3_stubs + rest_framework: djangorestframework + pip_deps_repository_name: gazelle_python_test diff --git a/gazelle/python/testdata/type_checking_imports_disabled/test.yaml b/gazelle/python/testdata/type_checking_imports_disabled/test.yaml new file mode 100644 index 0000000000..fcea77710f --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_disabled/test.yaml @@ -0,0 +1,15 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +--- diff --git a/gazelle/python/testdata/type_checking_imports_package/BUILD.in b/gazelle/python/testdata/type_checking_imports_package/BUILD.in new file mode 100644 index 0000000000..8e6c1cbabb --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_package/BUILD.in @@ -0,0 +1,2 @@ +# gazelle:python_generation_mode package +# gazelle:python_generate_pyi_deps true diff --git a/gazelle/python/testdata/type_checking_imports_package/BUILD.out b/gazelle/python/testdata/type_checking_imports_package/BUILD.out new file mode 100644 index 0000000000..0091e9c5c9 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_package/BUILD.out @@ -0,0 +1,19 @@ +load("@rules_python//python:defs.bzl", "py_library") + +# gazelle:python_generation_mode package +# gazelle:python_generate_pyi_deps true + +py_library( + name = "type_checking_imports_package", + srcs = [ + "bar.py", + "baz.py", + "foo.py", + ], + pyi_deps = [ + "@gazelle_python_test//boto3_stubs", + "@gazelle_python_test//djangorestframework", + ], + visibility = ["//:__subpackages__"], + deps = ["@gazelle_python_test//boto3"], +) diff --git a/gazelle/python/testdata/type_checking_imports_package/README.md b/gazelle/python/testdata/type_checking_imports_package/README.md new file mode 100644 index 0000000000..3e2cafe992 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_package/README.md @@ -0,0 +1,3 @@ +# Type Checking Imports (package mode) + +See `type_checking_imports`; this is the same test case, but using the package generation mode. diff --git a/gazelle/python/testdata/type_checking_imports_package/WORKSPACE b/gazelle/python/testdata/type_checking_imports_package/WORKSPACE new file mode 100644 index 0000000000..3e6e74e7f4 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_package/WORKSPACE @@ -0,0 +1 @@ +workspace(name = "gazelle_python_test") diff --git a/gazelle/python/testdata/type_checking_imports_package/bar.py b/gazelle/python/testdata/type_checking_imports_package/bar.py new file mode 100644 index 0000000000..47c7d93d08 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_package/bar.py @@ -0,0 +1,9 @@ +from typing import TYPE_CHECKING + +# foo should be added as a pyi_deps, since it is only imported in a type-checking context, but baz should be +# added as a deps. +from baz import X + +if TYPE_CHECKING: + import baz + import foo diff --git a/gazelle/python/testdata/type_checking_imports_package/baz.py b/gazelle/python/testdata/type_checking_imports_package/baz.py new file mode 100644 index 0000000000..1c69e25da4 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_package/baz.py @@ -0,0 +1,23 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# While this format is not official, it is supported by most type checkers and +# is used in the wild to avoid importing the typing module. +TYPE_CHECKING = False +if TYPE_CHECKING: + # Both boto3 and boto3_stubs should be added to pyi_deps. + import boto3 + +X = 1 diff --git a/gazelle/python/testdata/type_checking_imports_package/foo.py b/gazelle/python/testdata/type_checking_imports_package/foo.py new file mode 100644 index 0000000000..655cb54675 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_package/foo.py @@ -0,0 +1,21 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import typing + +# boto3 should be added to deps. boto3_stubs and djangorestframework should be added to pyi_deps. +import boto3 + +if typing.TYPE_CHECKING: + from rest_framework import serializers diff --git a/gazelle/python/testdata/type_checking_imports_package/gazelle_python.yaml b/gazelle/python/testdata/type_checking_imports_package/gazelle_python.yaml new file mode 100644 index 0000000000..a782354215 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_package/gazelle_python.yaml @@ -0,0 +1,20 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +manifest: + modules_mapping: + boto3: boto3 + boto3_stubs: boto3_stubs + rest_framework: djangorestframework + pip_deps_repository_name: gazelle_python_test diff --git a/gazelle/python/testdata/type_checking_imports_package/test.yaml b/gazelle/python/testdata/type_checking_imports_package/test.yaml new file mode 100644 index 0000000000..fcea77710f --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_package/test.yaml @@ -0,0 +1,15 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +--- diff --git a/gazelle/python/testdata/type_checking_imports_project/BUILD.in b/gazelle/python/testdata/type_checking_imports_project/BUILD.in new file mode 100644 index 0000000000..808e3e044e --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_project/BUILD.in @@ -0,0 +1,2 @@ +# gazelle:python_generation_mode project +# gazelle:python_generate_pyi_deps true diff --git a/gazelle/python/testdata/type_checking_imports_project/BUILD.out b/gazelle/python/testdata/type_checking_imports_project/BUILD.out new file mode 100644 index 0000000000..6d6ac3cef9 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_project/BUILD.out @@ -0,0 +1,19 @@ +load("@rules_python//python:defs.bzl", "py_library") + +# gazelle:python_generation_mode project +# gazelle:python_generate_pyi_deps true + +py_library( + name = "type_checking_imports_project", + srcs = [ + "bar.py", + "baz.py", + "foo.py", + ], + pyi_deps = [ + "@gazelle_python_test//boto3_stubs", + "@gazelle_python_test//djangorestframework", + ], + visibility = ["//:__subpackages__"], + deps = ["@gazelle_python_test//boto3"], +) diff --git a/gazelle/python/testdata/type_checking_imports_project/README.md b/gazelle/python/testdata/type_checking_imports_project/README.md new file mode 100644 index 0000000000..ead09e1994 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_project/README.md @@ -0,0 +1,3 @@ +# Type Checking Imports (project mode) + +See `type_checking_imports`; this is the same test case, but using the project generation mode. diff --git a/gazelle/python/testdata/type_checking_imports_project/WORKSPACE b/gazelle/python/testdata/type_checking_imports_project/WORKSPACE new file mode 100644 index 0000000000..3e6e74e7f4 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_project/WORKSPACE @@ -0,0 +1 @@ +workspace(name = "gazelle_python_test") diff --git a/gazelle/python/testdata/type_checking_imports_project/bar.py b/gazelle/python/testdata/type_checking_imports_project/bar.py new file mode 100644 index 0000000000..47c7d93d08 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_project/bar.py @@ -0,0 +1,9 @@ +from typing import TYPE_CHECKING + +# foo should be added as a pyi_deps, since it is only imported in a type-checking context, but baz should be +# added as a deps. +from baz import X + +if TYPE_CHECKING: + import baz + import foo diff --git a/gazelle/python/testdata/type_checking_imports_project/baz.py b/gazelle/python/testdata/type_checking_imports_project/baz.py new file mode 100644 index 0000000000..1c69e25da4 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_project/baz.py @@ -0,0 +1,23 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# While this format is not official, it is supported by most type checkers and +# is used in the wild to avoid importing the typing module. +TYPE_CHECKING = False +if TYPE_CHECKING: + # Both boto3 and boto3_stubs should be added to pyi_deps. + import boto3 + +X = 1 diff --git a/gazelle/python/testdata/type_checking_imports_project/foo.py b/gazelle/python/testdata/type_checking_imports_project/foo.py new file mode 100644 index 0000000000..655cb54675 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_project/foo.py @@ -0,0 +1,21 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import typing + +# boto3 should be added to deps. boto3_stubs and djangorestframework should be added to pyi_deps. +import boto3 + +if typing.TYPE_CHECKING: + from rest_framework import serializers diff --git a/gazelle/python/testdata/type_checking_imports_project/gazelle_python.yaml b/gazelle/python/testdata/type_checking_imports_project/gazelle_python.yaml new file mode 100644 index 0000000000..a782354215 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_project/gazelle_python.yaml @@ -0,0 +1,20 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +manifest: + modules_mapping: + boto3: boto3 + boto3_stubs: boto3_stubs + rest_framework: djangorestframework + pip_deps_repository_name: gazelle_python_test diff --git a/gazelle/python/testdata/type_checking_imports_project/test.yaml b/gazelle/python/testdata/type_checking_imports_project/test.yaml new file mode 100644 index 0000000000..fcea77710f --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_project/test.yaml @@ -0,0 +1,15 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +--- diff --git a/gazelle/python/testdata/with_third_party_requirements_from_imports/README.md b/gazelle/python/testdata/with_third_party_requirements_from_imports/README.md index c50a1ca100..8713d3d7e1 100644 --- a/gazelle/python/testdata/with_third_party_requirements_from_imports/README.md +++ b/gazelle/python/testdata/with_third_party_requirements_from_imports/README.md @@ -12,4 +12,4 @@ for example from google.cloud import aiplatform, storage ``` -See https://github.com/bazelbuild/rules_python/issues/709 and https://github.com/sramirezmartin/gazelle-toy-example. +See https://github.com/bazel-contrib/rules_python/issues/709 and https://github.com/sramirezmartin/gazelle-toy-example. diff --git a/gazelle/pythonconfig/BUILD.bazel b/gazelle/pythonconfig/BUILD.bazel index d80902e7ce..a82a7f19ca 100644 --- a/gazelle/pythonconfig/BUILD.bazel +++ b/gazelle/pythonconfig/BUILD.bazel @@ -6,12 +6,12 @@ go_library( "pythonconfig.go", "types.go", ], - importpath = "github.com/bazelbuild/rules_python/gazelle/pythonconfig", + importpath = "github.com/bazel-contrib/rules_python/gazelle/pythonconfig", visibility = ["//visibility:public"], deps = [ "//manifest", - "@bazel_gazelle//label:go_default_library", - "@com_github_emirpasic_gods//lists/singlylinkedlist", + "@bazel_gazelle//label", + "@com_github_emirpasic_gods//lists/singlylinkedlist:go_default_library", ], ) diff --git a/gazelle/pythonconfig/pythonconfig.go b/gazelle/pythonconfig/pythonconfig.go index a24a90efeb..c88d59abcf 100644 --- a/gazelle/pythonconfig/pythonconfig.go +++ b/gazelle/pythonconfig/pythonconfig.go @@ -16,14 +16,16 @@ package pythonconfig import ( "fmt" + "log" + "os" "path" "regexp" "strings" "github.com/emirpasic/gods/lists/singlylinkedlist" + "github.com/bazel-contrib/rules_python/gazelle/manifest" "github.com/bazelbuild/bazel-gazelle/label" - "github.com/bazelbuild/rules_python/gazelle/manifest" ) // Directives @@ -72,6 +74,12 @@ const ( // naming convention. See python_library_naming_convention for more info on // the package name interpolation. TestNamingConvention = "python_test_naming_convention" + // ProtoNamingConvention represents the directive that controls the + // py_proto_library naming convention. It interpolates $proto_name$ with + // the proto_library rule name, minus any trailing _proto. E.g. if the + // proto_library name is `foo_proto`, setting this to `$proto_name$_my_lib` + // would render to `foo_my_lib`. + ProtoNamingConvention = "python_proto_naming_convention" // DefaultVisibilty represents the directive that controls what visibility // labels are added to generated python targets. DefaultVisibilty = "python_default_visibility" @@ -89,6 +97,34 @@ const ( // names of labels to third-party dependencies are normalized. Supported values // are 'none', 'pep503' and 'snake_case' (default). See LabelNormalizationType. LabelNormalization = "python_label_normalization" + // ExperimentalAllowRelativeImports represents the directive that controls + // whether relative imports are allowed. + ExperimentalAllowRelativeImports = "python_experimental_allow_relative_imports" + // GeneratePyiDeps represents the directive that controls whether to generate + // separate pyi_deps attribute or merge type-checking dependencies into deps. + // Defaults to true. + GeneratePyiDeps = "python_generate_pyi_deps" + // GeneratePyiSrcs represents the directive that controls whether to include + // a pyi_srcs attribute if a sibling .pyi file is found. + // Defaults to true. + GeneratePyiSrcs = "python_generate_pyi_srcs" + // GenerateProto represents the directive that controls whether to generate + // python_generate_proto targets. + GenerateProto = "python_generate_proto" + // PythonResolveSiblingImports represents the directive that controls whether + // absolute imports can be solved to sibling modules. When enabled, imports + // like "import a" can be resolved to sibling modules. When disabled, they + // can only be resolved as an absolute import. + PythonResolveSiblingImports = "python_resolve_sibling_imports" + // PythonIncludeAncestorConftest represents the directive that controls + // whether ancestor conftest.py files are added as dependencies to py_test + // targets. When enabled (the default), ancestor conftest.py files are + // included as deps. + // See also https://github.com/bazel-contrib/rules_python/pull/3498, which + // fixed previous behavior that was incorrectly _not_ adding the files and + // https://github.com/bazel-contrib/rules_python/issues/3595 which requested + // that the behavior be configurable. + PythonIncludeAncestorConftest = "python_include_ancestor_conftest" ) // GenerationModeType represents one of the generation modes for the Python @@ -109,6 +145,7 @@ const ( const ( packageNameNamingConventionSubstitution = "$package_name$" + protoNameNamingConventionSubstitution = "$proto_name$" distributionNameLabelConventionSubstitution = "$distribution_name$" ) @@ -125,32 +162,39 @@ const ( // defaultIgnoreFiles is the list of default values used in the // python_ignore_files option. -var defaultIgnoreFiles = map[string]struct{}{ - "setup.py": {}, -} +var defaultIgnoreFiles = map[string]struct{}{} // Configs is an extension of map[string]*Config. It provides finding methods // on top of the mapping. type Configs map[string]*Config // ParentForPackage returns the parent Config for the given Bazel package. -func (c *Configs) ParentForPackage(pkg string) *Config { - dir := path.Dir(pkg) - if dir == "." { - dir = "" +func (c Configs) ParentForPackage(pkg string) *Config { + for { + dir := path.Dir(pkg) + if dir == "." { + dir = "" + } + parent := (map[string]*Config)(c)[dir] + if parent != nil { + return parent + } + if dir == "" { + return nil + } + pkg = dir } - parent := (map[string]*Config)(*c)[dir] - return parent } // Config represents a config extension for a specific Bazel package. type Config struct { parent *Config - extensionEnabled bool - repoRoot string - pythonProjectRoot string - gazelleManifest *manifest.Manifest + extensionEnabled bool + repoRoot string + pythonProjectRoot string + gazelleManifestPath string + gazelleManifest *manifest.Manifest excludedPatterns *singlylinkedlist.List ignoreFiles map[string]struct{} @@ -163,11 +207,18 @@ type Config struct { libraryNamingConvention string binaryNamingConvention string testNamingConvention string + protoNamingConvention string defaultVisibility []string visibility []string testFilePattern []string labelConvention string labelNormalization LabelNormalizationType + experimentalAllowRelativeImports bool + generatePyiDeps bool + generatePyiSrcs bool + generateProto bool + resolveSiblingImports bool + includeAncestorConftest bool } type LabelNormalizationType int @@ -198,11 +249,18 @@ func New( libraryNamingConvention: packageNameNamingConventionSubstitution, binaryNamingConvention: fmt.Sprintf("%s_bin", packageNameNamingConventionSubstitution), testNamingConvention: fmt.Sprintf("%s_test", packageNameNamingConventionSubstitution), + protoNamingConvention: fmt.Sprintf("%s_py_pb2", protoNameNamingConventionSubstitution), defaultVisibility: []string{fmt.Sprintf(DefaultVisibilityFmtString, "")}, visibility: []string{}, testFilePattern: strings.Split(DefaultTestFilePatternString, ","), labelConvention: DefaultLabelConvention, labelNormalization: DefaultLabelNormalizationType, + experimentalAllowRelativeImports: false, + generatePyiDeps: true, + generatePyiSrcs: true, + generateProto: false, + resolveSiblingImports: false, + includeAncestorConftest: true, } } @@ -230,11 +288,18 @@ func (c *Config) NewChild() *Config { libraryNamingConvention: c.libraryNamingConvention, binaryNamingConvention: c.binaryNamingConvention, testNamingConvention: c.testNamingConvention, + protoNamingConvention: c.protoNamingConvention, defaultVisibility: c.defaultVisibility, visibility: c.visibility, testFilePattern: c.testFilePattern, labelConvention: c.labelConvention, labelNormalization: c.labelNormalization, + experimentalAllowRelativeImports: c.experimentalAllowRelativeImports, + generatePyiDeps: c.generatePyiDeps, + generatePyiSrcs: c.generatePyiSrcs, + generateProto: c.generateProto, + resolveSiblingImports: c.resolveSiblingImports, + includeAncestorConftest: c.includeAncestorConftest, } } @@ -275,11 +340,26 @@ func (c *Config) SetGazelleManifest(gazelleManifest *manifest.Manifest) { c.gazelleManifest = gazelleManifest } +// SetGazelleManifestPath sets the path to the gazelle_python.yaml file +// for the current configuration. +func (c *Config) SetGazelleManifestPath(gazelleManifestPath string) { + c.gazelleManifestPath = gazelleManifestPath +} + // FindThirdPartyDependency scans the gazelle manifests for the current config // and the parent configs up to the root finding if it can resolve the module // name. -func (c *Config) FindThirdPartyDependency(modName string) (string, bool) { +func (c *Config) FindThirdPartyDependency(modName string) (string, string, bool) { for currentCfg := c; currentCfg != nil; currentCfg = currentCfg.parent { + // Attempt to load the manifest if needed. + if currentCfg.gazelleManifestPath != "" && currentCfg.gazelleManifest == nil { + currentCfgManifest, err := loadGazelleManifest(currentCfg.gazelleManifestPath) + if err != nil { + log.Fatal(err) + } + currentCfg.SetGazelleManifest(currentCfgManifest) + } + if currentCfg.gazelleManifest != nil { gazelleManifest := currentCfg.gazelleManifest if distributionName, ok := gazelleManifest.ModulesMapping[modName]; ok { @@ -291,11 +371,11 @@ func (c *Config) FindThirdPartyDependency(modName string) (string, bool) { } lbl := currentCfg.FormatThirdPartyDependency(distributionRepositoryName, distributionName) - return lbl.String(), true + return lbl.String(), distributionName, true } } } - return "", false + return "", "", false } // AddIgnoreFile adds a file to the list of ignored files for a given package. @@ -446,6 +526,17 @@ func (c *Config) RenderTestName(packageName string) string { return strings.ReplaceAll(c.testNamingConvention, packageNameNamingConventionSubstitution, packageName) } +// SetProtoNamingConvention sets the py_proto_library target naming convention. +func (c *Config) SetProtoNamingConvention(protoNamingConvention string) { + c.protoNamingConvention = protoNamingConvention +} + +// RenderProtoName returns the py_proto_library target name by performing all +// substitutions. +func (c *Config) RenderProtoName(protoName string) string { + return strings.ReplaceAll(c.protoNamingConvention, protoNameNamingConventionSubstitution, strings.TrimSuffix(protoName, "_proto")) +} + // AppendVisibility adds additional items to the target's visibility. func (c *Config) AppendVisibility(visibility string) { c.visibility = append(c.visibility, visibility) @@ -496,6 +587,70 @@ func (c *Config) LabelNormalization() LabelNormalizationType { return c.labelNormalization } +// SetExperimentalAllowRelativeImports sets whether relative imports are allowed. +func (c *Config) SetExperimentalAllowRelativeImports(allowRelativeImports bool) { + c.experimentalAllowRelativeImports = allowRelativeImports +} + +// ExperimentalAllowRelativeImports returns whether relative imports are allowed. +func (c *Config) ExperimentalAllowRelativeImports() bool { + return c.experimentalAllowRelativeImports +} + +// SetGeneratePyiDeps sets whether pyi_deps attribute should be generated separately +// or type-checking dependencies should be merged into the regular deps attribute. +func (c *Config) SetGeneratePyiDeps(generatePyiDeps bool) { + c.generatePyiDeps = generatePyiDeps +} + +// GeneratePyiDeps returns whether pyi_deps attribute should be generated separately +// or type-checking dependencies should be merged into the regular deps attribute. +func (c *Config) GeneratePyiDeps() bool { + return c.generatePyiDeps +} + +// SetGeneratePyiSrcs sets whether pyi_srcs attribute should be generated if a sibling +// .pyi file is found. +func (c *Config) SetGeneratePyiSrcs(generatePyiSrcs bool) { + c.generatePyiSrcs = generatePyiSrcs +} + +// GeneratePyiSrcs returns whether pyi_srcs attribute should be generated if a sibling +// .pyi file is found. +func (c *Config) GeneratePyiSrcs() bool { + return c.generatePyiSrcs +} + +// SetGenerateProto sets whether py_proto_library should be generated for proto_library. +func (c *Config) SetGenerateProto(generateProto bool) { + c.generateProto = generateProto +} + +// GenerateProto returns whether py_proto_library should be generated for proto_library. +func (c *Config) GenerateProto() bool { + return c.generateProto +} + +// SetResolveSiblingImports sets whether absolute imports can be resolved to sibling modules. +func (c *Config) SetResolveSiblingImports(resolveSiblingImports bool) { + c.resolveSiblingImports = resolveSiblingImports +} + +// ResolveSiblingImports returns whether absolute imports can be resolved to sibling modules. +func (c *Config) ResolveSiblingImports() bool { + return c.resolveSiblingImports +} + +// SetIncludeAncestorConftest sets whether ancestor conftest files are added to py_test targets. +func (c *Config) SetIncludeAncestorConftest(includeAncestorConftest bool) { + c.includeAncestorConftest = includeAncestorConftest +} + +// IncludeAncestorConftest returns whether ancestor conftest files are added to py_test targets. +func (c *Config) IncludeAncestorConftest() bool { + return c.includeAncestorConftest +} + // FormatThirdPartyDependency returns a label to a third-party dependency performing all formating and normalization. func (c *Config) FormatThirdPartyDependency(repositoryName string, distributionName string) label.Label { conventionalDistributionName := strings.ReplaceAll(c.labelConvention, distributionNameLabelConventionSubstitution, distributionName) @@ -520,3 +675,17 @@ func (c *Config) FormatThirdPartyDependency(repositoryName string, distributionN return label.New(repositoryName, normConventionalDistributionName, normConventionalDistributionName) } + +func loadGazelleManifest(gazelleManifestPath string) (*manifest.Manifest, error) { + if _, err := os.Stat(gazelleManifestPath); err != nil { + if os.IsNotExist(err) { + return nil, nil + } + return nil, fmt.Errorf("failed to load Gazelle manifest at %q: %w", gazelleManifestPath, err) + } + manifestFile := new(manifest.File) + if err := manifestFile.Decode(gazelleManifestPath); err != nil { + return nil, fmt.Errorf("failed to load Gazelle manifest at %q: %w", gazelleManifestPath, err) + } + return manifestFile.Manifest, nil +} diff --git a/gazelle/pythonconfig/pythonconfig_test.go b/gazelle/pythonconfig/pythonconfig_test.go index 7cdb9af1d1..fe21ce236e 100644 --- a/gazelle/pythonconfig/pythonconfig_test.go +++ b/gazelle/pythonconfig/pythonconfig_test.go @@ -248,3 +248,35 @@ func TestFormatThirdPartyDependency(t *testing.T) { }) } } + +func TestConfigsMap(t *testing.T) { + t.Run("only root", func(t *testing.T) { + configs := Configs{"": New("root/dir", "")} + + if configs.ParentForPackage("") == nil { + t.Fatal("expected non-nil for root config") + } + + if configs.ParentForPackage("a/b/c") != configs[""] { + t.Fatal("expected root for subpackage") + } + }) + + t.Run("sparse child configs", func(t *testing.T) { + configs := Configs{"": New("root/dir", "")} + configs["a"] = configs[""].NewChild() + configs["a/b/c"] = configs["a"].NewChild() + + if configs.ParentForPackage("a/b/c/d") != configs["a/b/c"] { + t.Fatal("child should match direct parent") + } + + if configs.ParentForPackage("a/b/c/d/e") != configs["a/b/c"] { + t.Fatal("grandchild should match first parant") + } + + if configs.ParentForPackage("other/root/path") != configs[""] { + t.Fatal("non-configured subpackage should match root") + } + }) +} diff --git a/internal_dev_deps.bzl b/internal_dev_deps.bzl index 80196815a0..6d4d7656b7 100644 --- a/internal_dev_deps.bzl +++ b/internal_dev_deps.bzl @@ -12,10 +12,12 @@ # See the License for the specific language governing permissions and # limitations under the License. -"""Dependencies that are needed for development and testing of rules_python itself.""" +"""Dependencies that are needed for development and testing of rules_python itself in WORKSPACE mode.""" load("@bazel_tools//tools/build_defs/repo:http.bzl", _http_archive = "http_archive", _http_file = "http_file") +load("@bazel_tools//tools/build_defs/repo:local.bzl", "local_repository") load("@bazel_tools//tools/build_defs/repo:utils.bzl", "maybe") +load("//python/private:internal_config_repo.bzl", "internal_config_repo") # buildifier: disable=bzl-visibility def http_archive(name, **kwargs): maybe( @@ -32,20 +34,43 @@ def http_file(name, **kwargs): ) def rules_python_internal_deps(): - """Fetches all required dependencies for developing/testing rules_python itself. + """Fetches all required dependencies for developing/testing rules_python itself in WORKSPACE mode. Setup of these dependencies is done by `internal_dev_setup.bzl` For dependencies needed by *users* of rules_python, see python/private/py_repositories.bzl. """ + internal_config_repo( + name = "rules_python_internal", + transition_settings = [ + str(Label("//tests/multi_pypi:external_deps_name")), + ], + ) + + # Sphinxdocs doesn't support workspace mode, but we have to define it + # so that load() passes. + local_repository( + name = "sphinxdocs", + path = "sphinxdocs", + ) + + local_repository( + name = "other", + path = "tests/modules/other", + ) + + local_repository( + name = "another_module", + path = "tests/modules/another_module", + ) http_archive( name = "bazel_skylib", - sha256 = "bc283cdfcd526a52c3201279cda4bc298652efa898b10b4db0837dc51652756f", + sha256 = "6e78f0e57de26801f6f564fa7c4a48dc8b36873e416257a92bbb0937eeac8446", urls = [ - "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.7.1/bazel-skylib-1.7.1.tar.gz", - "https://github.com/bazelbuild/bazel-skylib/releases/download/1.7.1/bazel-skylib-1.7.1.tar.gz", + "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.8.2/bazel-skylib-1.8.2.tar.gz", + "https://github.com/bazelbuild/bazel-skylib/releases/download/1.8.2/bazel-skylib-1.8.2.tar.gz", ], ) @@ -60,10 +85,10 @@ def rules_python_internal_deps(): http_archive( name = "rules_pkg", urls = [ - "https://mirror.bazel.build/github.com/bazelbuild/rules_pkg/releases/download/0.7.0/rules_pkg-0.7.0.tar.gz", - "https://github.com/bazelbuild/rules_pkg/releases/download/0.7.0/rules_pkg-0.7.0.tar.gz", + "https://mirror.bazel.build/github.com/bazelbuild/rules_pkg/releases/download/1.0.1/rules_pkg-1.0.1.tar.gz", + "https://github.com/bazelbuild/rules_pkg/releases/download/1.0.1/rules_pkg-1.0.1.tar.gz", ], - sha256 = "8a298e832762eda1830597d64fe7db58178aa84cd5926d76d5b744d6558941c2", + sha256 = "d20c951960ed77cb7b341c2a59488534e494d5ad1d30c4818c736d57772a9fef", ) http_archive( @@ -82,6 +107,13 @@ def rules_python_internal_deps(): ], ) + # Stub repository for rules_pyrefly in WORKSPACE mode so that load() + # statements for @rules_pyrefly resolve without requiring full Pyrefly. + local_repository( + name = "rules_pyrefly", + path = "tests/modules/rules_pyrefly_stub", + ) + # The below two deps are required for the integration test with bazel # gazelle. Maybe the test should be moved to the `gazelle` workspace? http_archive( @@ -175,13 +207,6 @@ def rules_python_internal_deps(): ], ) - http_archive( - name = "rules_proto", - sha256 = "904a8097fae42a690c8e08d805210e40cccb069f5f9a0f6727cf4faa7bed2c9c", - strip_prefix = "rules_proto-6.0.0-rc1", - url = "https://github.com/bazelbuild/rules_proto/releases/download/6.0.0-rc1/rules_proto-6.0.0-rc1.tar.gz", - ) - http_archive( name = "com_google_protobuf", sha256 = "23082dca1ca73a1e9c6cbe40097b41e81f71f3b4d6201e36c134acc30a1b3660", @@ -193,10 +218,9 @@ def rules_python_internal_deps(): http_archive( name = "rules_java", urls = [ - "https://mirror.bazel.build/github.com/bazelbuild/rules_java/releases/download/8.3.1/rules_java-8.3.1.tar.gz", - "https://github.com/bazelbuild/rules_java/releases/download/8.3.1/rules_java-8.3.1.tar.gz", + "https://github.com/bazelbuild/rules_java/releases/download/8.6.2/rules_java-8.6.2.tar.gz", ], - sha256 = "ee786b943e00da4fea7c233e70e5f5b8a01cc69b9341b3f49169f174fe0df1c5", + sha256 = "a64ab04616e76a448c2c2d8165d836f0d2fb0906200d0b7c7376f46dd62e59cc", ) RULES_JVM_EXTERNAL_TAG = "5.2" @@ -228,9 +252,9 @@ def rules_python_internal_deps(): http_archive( name = "rules_cc", - urls = ["https://github.com/bazelbuild/rules_cc/releases/download/0.0.16/rules_cc-0.0.16.tar.gz"], - sha256 = "bbf1ae2f83305b7053b11e4467d317a7ba3517a12cef608543c1b1c5bf48a4df", - strip_prefix = "rules_cc-0.0.16", + urls = ["https://github.com/bazelbuild/rules_cc/releases/download/0.1.5/rules_cc-0.1.5.tar.gz"], + sha256 = "b8b918a85f9144c01f6cfe0f45e4f2838c7413961a8ff23bc0c6cdf8bb07a3b6", + strip_prefix = "rules_cc-0.1.5", ) http_archive( diff --git a/internal_dev_setup.bzl b/internal_dev_setup.bzl index 554ff926f2..0bbcd97748 100644 --- a/internal_dev_setup.bzl +++ b/internal_dev_setup.bzl @@ -20,45 +20,66 @@ load("@cgrindel_bazel_starlib//:deps.bzl", "bazel_starlib_dependencies") load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") load("@rules_bazel_integration_test//bazel_integration_test:deps.bzl", "bazel_integration_test_rules_dependencies") load("@rules_bazel_integration_test//bazel_integration_test:repo_defs.bzl", "bazel_binaries") -load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") -load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains") load("@rules_shell//shell:repositories.bzl", "rules_shell_dependencies", "rules_shell_toolchains") load("//:version.bzl", "SUPPORTED_BAZEL_VERSIONS") load("//python:versions.bzl", "MINOR_MAPPING", "TOOL_VERSIONS") -load("//python/private:internal_config_repo.bzl", "internal_config_repo") # buildifier: disable=bzl-visibility load("//python/private:pythons_hub.bzl", "hub_repo") # buildifier: disable=bzl-visibility +load("//python/private:runtime_env_repo.bzl", "runtime_env_repo") # buildifier: disable=bzl-visibility load("//python/private/pypi:deps.bzl", "pypi_deps") # buildifier: disable=bzl-visibility +load("//python/private/pypi:whl_library.bzl", "whl_library") # buildifier: disable=bzl-visibility +load("//tests/support/whl_from_dir:whl_from_dir_repo.bzl", "whl_from_dir_repo") # buildifier: disable=bzl-visibility def rules_python_internal_setup(): """Setup for development and testing of rules_python itself.""" - internal_config_repo(name = "rules_python_internal") hub_repo( name = "pythons_hub", minor_mapping = MINOR_MAPPING, default_python_version = "", - toolchain_prefixes = [], - toolchain_python_versions = [], - toolchain_set_python_version_constraints = [], - toolchain_user_repository_names = [], python_versions = sorted(TOOL_VERSIONS.keys()), + toolchain_names = [], + toolchain_repo_names = {}, + toolchain_target_compatible_with_map = {}, + toolchain_target_settings_map = {}, + toolchain_platform_keys = {}, + toolchain_python_versions = {}, + toolchain_set_python_version_constraints = {}, + host_compatible_repo_names = [], ) + runtime_env_repo(name = "rules_python_runtime_env_tc_info") + pypi_deps() bazel_skylib_workspace() - rules_proto_dependencies() - rules_proto_toolchains() - protobuf_deps() - rules_java_dependencies() - rules_java_toolchains() - bazel_integration_test_rules_dependencies() bazel_starlib_dependencies() bazel_binaries(versions = SUPPORTED_BAZEL_VERSIONS) bazel_features_deps() rules_shell_dependencies() rules_shell_toolchains() + + whl_from_dir_repo( + name = "whl_with_data1_whl", + root = "//tests/repos/whl_with_data1:BUILD.bazel", + output = "whl_with_data1-1.0-any-none-any.whl", + ) + whl_library( + name = "whl_with_data1", + whl_file = "@whl_with_data1_whl//:whl_with_data1-1.0-any-none-any.whl", + requirement = "whl-with-data1", + ) + + whl_from_dir_repo( + name = "whl_with_data2_whl", + root = "//tests/repos/whl_with_data2:BUILD.bazel", + output = "whl_with_data2-1.0-any-none-any.whl", + ) + whl_library( + name = "whl_with_data2", + whl_file = "@whl_with_data2_whl//:whl_with_data2-1.0-any-none-any.whl", + requirement = "whl-with-data2", + ) diff --git a/news/.gitkeep b/news/.gitkeep new file mode 100644 index 0000000000..35e8e662aa --- /dev/null +++ b/news/.gitkeep @@ -0,0 +1 @@ +# Keep this directory even if empty so git tracks it. diff --git a/news/BUILD.bazel b/news/BUILD.bazel new file mode 100644 index 0000000000..79b48675ca --- /dev/null +++ b/news/BUILD.bazel @@ -0,0 +1,6 @@ +package(default_visibility = ["//:__subpackages__"]) + +filegroup( + name = "news_files", + srcs = glob(["**"]), +) diff --git a/news/add_runfiles_raise_api.added.md b/news/add_runfiles_raise_api.added.md new file mode 100644 index 0000000000..5b2ff574e6 --- /dev/null +++ b/news/add_runfiles_raise_api.added.md @@ -0,0 +1,2 @@ +(runfiles) Added {obj}`Runfiles.CreateOrRaise` to return a `Runfiles` instance +or raise an error if runfiles cannot be found. diff --git a/private/BUILD.bazel b/private/BUILD.bazel index 68fefe910f..c6bfe6bd9f 100644 --- a/private/BUILD.bazel +++ b/private/BUILD.bazel @@ -13,8 +13,9 @@ multirun( "requirements_linux", ] ] + [ - "//docs:requirements.update", + "//dev:requirements.update", ], + tags = ["manual"], ) # NOTE: The requirements for the pip dependencies may sometimes break the build @@ -24,4 +25,5 @@ multirun( alias( name = "whl_library_requirements.update", actual = "//tools/private/update_deps:update_pip_deps", + tags = ["manual"], ) diff --git a/python/BUILD.bazel b/python/BUILD.bazel index f3b5b136a7..a4dd98c572 100644 --- a/python/BUILD.bazel +++ b/python/BUILD.bazel @@ -24,6 +24,10 @@ In an ideal renaming, we'd move the packaging rules to a different package so that @rules_python//python is only concerned with the core rules. """ +# TODO: make current_py_toolchain.bzl private +# gazelle:resolve starlark @rules_python//python:current_py_toolchain.bzl //python:current_py_toolchain_bzl +# gazelle:resolve starlark //python:current_py_toolchain.bzl //python:current_py_toolchain_bzl + load("@bazel_skylib//:bzl_library.bzl", "bzl_library") load(":current_py_toolchain.bzl", "current_py_toolchain") @@ -35,215 +39,25 @@ filegroup( name = "distribution", srcs = glob(["**"]) + [ "//python/api:distribution", + "//python/bin:distribution", "//python/cc:distribution", "//python/config_settings:distribution", "//python/constraints:distribution", "//python/entry_points:distribution", "//python/extensions:distribution", + "//python/local_toolchains:distribution", "//python/pip_install:distribution", "//python/private:distribution", "//python/runfiles:distribution", + "//python/runtime_env_toolchains:distribution", "//python/uv:distribution", + "//python/zipapp:distribution", ], visibility = ["//:__pkg__"], ) # ========= bzl_library targets end ========= -bzl_library( - name = "current_py_toolchain_bzl", - srcs = ["current_py_toolchain.bzl"], -) - -bzl_library( - name = "defs_bzl", - srcs = [ - "defs.bzl", - ], - visibility = ["//visibility:public"], - deps = [ - ":current_py_toolchain_bzl", - ":py_binary_bzl", - ":py_import_bzl", - ":py_info_bzl", - ":py_library_bzl", - ":py_runtime_bzl", - ":py_runtime_info_bzl", - ":py_runtime_pair_bzl", - ":py_test_bzl", - "//python/private:bazel_tools_bzl", - ], -) - -bzl_library( - name = "features_bzl", - srcs = ["features.bzl"], -) - -bzl_library( - name = "packaging_bzl", - srcs = ["packaging.bzl"], - deps = [ - ":py_binary_bzl", - "//python/private:bzlmod_enabled_bzl", - "//python/private:py_package.bzl", - "//python/private:py_wheel_bzl", - "//python/private:py_wheel_normalize_pep440.bzl", - "//python/private:stamp_bzl", - "//python/private:util_bzl", - "@bazel_skylib//rules:native_binary", - ], -) - -bzl_library( - name = "pip_bzl", - srcs = ["pip.bzl"], - deps = [ - "//python/private:normalize_name_bzl", - "//python/private/pypi:multi_pip_parse_bzl", - "//python/private/pypi:package_annotation_bzl", - "//python/private/pypi:pip_compile_bzl", - "//python/private/pypi:pip_repository_bzl", - "//python/private/pypi:whl_library_alias_bzl", - "//python/private/whl_filegroup:whl_filegroup_bzl", - ], -) - -bzl_library( - name = "proto_bzl", - srcs = [ - "proto.bzl", - ], - visibility = ["//visibility:public"], - deps = [ - "//python/private/proto:py_proto_library_bzl", - ], -) - -bzl_library( - name = "py_binary_bzl", - srcs = ["py_binary.bzl"], - deps = [ - "//python/private:py_binary_macro_bzl", - "//python/private:register_extension_info_bzl", - "//python/private:util_bzl", - "@rules_python_internal//:rules_python_config_bzl", - ], -) - -bzl_library( - name = "py_cc_link_params_info_bzl", - srcs = ["py_cc_link_params_info.bzl"], - deps = [ - "//python/private:py_cc_link_params_info_bzl", - "@rules_python_internal//:rules_python_config_bzl", - ], -) - -bzl_library( - name = "py_exec_tools_info_bzl", - srcs = ["py_exec_tools_info.bzl"], - deps = ["//python/private:py_exec_tools_info_bzl"], -) - -bzl_library( - name = "py_exec_tools_toolchain_bzl", - srcs = ["py_exec_tools_toolchain.bzl"], - deps = ["//python/private:py_exec_tools_toolchain_bzl"], -) - -bzl_library( - name = "py_executable_info_bzl", - srcs = ["py_executable_info.bzl"], - deps = ["//python/private:py_executable_info_bzl"], -) - -bzl_library( - name = "py_import_bzl", - srcs = ["py_import.bzl"], - deps = [":py_info_bzl"], -) - -bzl_library( - name = "py_info_bzl", - srcs = ["py_info.bzl"], - deps = [ - "//python/private:py_info_bzl", - "//python/private:reexports_bzl", - "@rules_python_internal//:rules_python_config_bzl", - ], -) - -bzl_library( - name = "py_library_bzl", - srcs = ["py_library.bzl"], - deps = [ - "//python/private:py_library_macro_bzl", - "//python/private:register_extension_info_bzl", - "//python/private:util_bzl", - "@rules_python_internal//:rules_python_config_bzl", - ], -) - -bzl_library( - name = "py_runtime_bzl", - srcs = ["py_runtime.bzl"], - deps = [ - "//python/private:py_runtime_macro_bzl", - "//python/private:util_bzl", - ], -) - -bzl_library( - name = "py_runtime_pair_bzl", - srcs = ["py_runtime_pair.bzl"], - deps = [ - "//python/private:bazel_tools_bzl", - "//python/private:py_runtime_pair_macro_bzl", - "//python/private:util_bzl", - ], -) - -bzl_library( - name = "py_runtime_info_bzl", - srcs = ["py_runtime_info.bzl"], - deps = [ - "//python/private:py_runtime_info_bzl", - "//python/private:reexports_bzl", - "//python/private:util_bzl", - "@rules_python_internal//:rules_python_config_bzl", - ], -) - -bzl_library( - name = "py_test_bzl", - srcs = ["py_test.bzl"], - deps = [ - "//python/private:py_test_macro_bzl", - "//python/private:register_extension_info_bzl", - "//python/private:util_bzl", - "@rules_python_internal//:rules_python_config_bzl", - ], -) - -bzl_library( - name = "repositories_bzl", - srcs = ["repositories.bzl"], - deps = [ - "//python/private:is_standalone_interpreter_bzl", - "//python/private:py_repositories_bzl", - "//python/private:python_register_multi_toolchains_bzl", - "//python/private:python_register_toolchains_bzl", - "//python/private:python_repository_bzl", - ], -) - -bzl_library( - name = "versions_bzl", - srcs = ["versions.bzl"], - visibility = ["//:__subpackages__"], -) - # NOTE: Remember to add bzl_library targets to //tests:bzl_libraries # ========= bzl_library targets end ========= @@ -363,3 +177,286 @@ exports_files([ current_py_toolchain( name = "current_py_toolchain", ) + +# Keep this target because it is a public API and Gazelle might otherwise +# remove it if it is not loaded by other bzl_library targets. +# keep +bzl_library( + name = "current_py_toolchain_bzl", + srcs = ["current_py_toolchain.bzl"], + visibility = ["//visibility:public"], +) + +bzl_library( + name = "defs", + srcs = ["defs.bzl"], + deps = [ + ":py_binary", + ":py_import", + ":py_info", + ":py_library", + ":py_runtime", + ":py_runtime_info", + ":py_runtime_pair", + ":py_test", + "//python:current_py_toolchain_bzl", + ], +) + +alias( + name = "defs_bzl", + actual = ":defs", + deprecation = "Use //python:defs instead", +) + +alias( + name = "features_bzl", + actual = ":features", + deprecation = "Use //python:features instead", +) + +bzl_library( + name = "packaging", + srcs = ["packaging.bzl"], + deps = [ + ":py_binary", + "//python/private:bzlmod_enabled", + "//python/private:py_package", + "//python/private:py_wheel", + "//python/private:util", + "@bazel_skylib//rules:native_binary", + ], +) + +alias( + name = "packaging_bzl", + actual = ":packaging", + deprecation = "Use //python:packaging instead", +) + +bzl_library( + name = "pip", + srcs = ["pip.bzl"], + deps = [ + "//python/private:normalize_name", + "//python/private/pypi:multi_pip_parse", + "//python/private/pypi:package_annotation", + "//python/private/pypi:pip_compile", + "//python/private/pypi:pip_repository", + "//python/private/pypi:whl_library_alias", + "//python/private/whl_filegroup", + ], +) + +alias( + name = "pip_bzl", + actual = ":pip", + deprecation = "Use //python:pip instead", +) + +bzl_library( + name = "proto", + srcs = ["proto.bzl"], + deps = ["@com_google_protobuf//bazel:py_proto_library_bzl"], +) + +alias( + name = "proto_bzl", + actual = ":proto", + deprecation = "Use //python:proto instead", +) + +bzl_library( + name = "py_binary", + srcs = ["py_binary.bzl"], + deps = ["//python/private:py_binary_macro"], +) + +alias( + name = "py_binary_bzl", + actual = ":py_binary", + deprecation = "Use //python:py_binary instead", +) + +bzl_library( + name = "py_cc_link_params_info", + srcs = ["py_cc_link_params_info.bzl"], + deps = ["//python/private:py_cc_link_params_info"], +) + +alias( + name = "py_cc_link_params_info_bzl", + actual = ":py_cc_link_params_info", + deprecation = "Use //python:py_cc_link_params_info instead", +) + +bzl_library( + name = "py_exec_tools_info", + srcs = ["py_exec_tools_info.bzl"], + deps = ["//python/private:py_exec_tools_info"], +) + +alias( + name = "py_exec_tools_info_bzl", + actual = ":py_exec_tools_info", + deprecation = "Use //python:py_exec_tools_info instead", +) + +bzl_library( + name = "py_exec_tools_toolchain", + srcs = ["py_exec_tools_toolchain.bzl"], + deps = ["//python/private:py_exec_tools_toolchain"], +) + +alias( + name = "py_exec_tools_toolchain_bzl", + actual = ":py_exec_tools_toolchain", + deprecation = "Use //python:py_exec_tools_toolchain instead", +) + +bzl_library( + name = "py_executable_info", + srcs = ["py_executable_info.bzl"], + deps = ["//python/private:py_executable_info"], +) + +alias( + name = "py_executable_info_bzl", + actual = ":py_executable_info", + deprecation = "Use //python:py_executable_info instead", +) + +bzl_library( + name = "py_import", + srcs = ["py_import.bzl"], + deps = [":py_info"], +) + +alias( + name = "py_import_bzl", + actual = ":py_import", + deprecation = "Use //python:py_import instead", +) + +bzl_library( + name = "py_info", + srcs = ["py_info.bzl"], + deps = ["//python/private:py_info"], +) + +alias( + name = "py_info_bzl", + actual = ":py_info", + deprecation = "Use //python:py_info instead", +) + +bzl_library( + name = "py_library", + srcs = ["py_library.bzl"], + deps = ["//python/private:py_library_macro"], +) + +alias( + name = "py_library_bzl", + actual = ":py_library", + deprecation = "Use //python:py_library instead", +) + +bzl_library( + name = "py_runtime", + srcs = ["py_runtime.bzl"], + deps = ["//python/private:py_runtime_macro"], +) + +alias( + name = "py_runtime_bzl", + actual = ":py_runtime", + deprecation = "Use //python:py_runtime instead", +) + +bzl_library( + name = "py_runtime_info", + srcs = ["py_runtime_info.bzl"], + deps = ["//python/private:py_runtime_info"], +) + +alias( + name = "py_runtime_info_bzl", + actual = ":py_runtime_info", + deprecation = "Use //python:py_runtime_info instead", +) + +bzl_library( + name = "py_runtime_pair", + srcs = ["py_runtime_pair.bzl"], + deps = ["//python/private:py_runtime_pair_macro"], +) + +alias( + name = "py_runtime_pair_bzl", + actual = ":py_runtime_pair", + deprecation = "Use //python:py_runtime_pair instead", +) + +# keep +bzl_library( + name = "py_test", + srcs = ["py_test.bzl"], + deps = ["//python/private:py_test_macro"], +) + +alias( + name = "py_test_bzl", + actual = ":py_test", + deprecation = "Use //python:py_test instead", +) + +alias( + name = "python_bzl", + actual = ":python", + deprecation = "Use //python:python instead", +) + +bzl_library( + name = "repositories", + srcs = ["repositories.bzl"], + deps = [ + "//python/private:is_standalone_interpreter", + "//python/private:py_repositories", + "//python/private:python_register_multi_toolchains", + "//python/private:python_register_toolchains", + "//python/private:python_repository", + ], +) + +alias( + name = "repositories_bzl", + actual = ":repositories", + deprecation = "Use //python:repositories instead", +) + +bzl_library( + name = "versions", + srcs = ["versions.bzl"], + deps = [ + "//python/private:pbs_manifest", + "//python/private:platform_info", + "//python/private:runtimes_manifest_workspace", + ], +) + +alias( + name = "versions_bzl", + actual = ":versions", + deprecation = "Use //python:versions instead", +) + +bzl_library( + name = "features", + srcs = ["features.bzl"], +) + +bzl_library( + name = "python", + srcs = ["python.bzl"], +) diff --git a/python/api/BUILD.bazel b/python/api/BUILD.bazel index 1df6877ef8..93c70f4c8b 100644 --- a/python/api/BUILD.bazel +++ b/python/api/BUILD.bazel @@ -18,14 +18,41 @@ package( default_visibility = ["//:__subpackages__"], ) +filegroup( + name = "distribution", + srcs = glob(["**"]), +) + bzl_library( - name = "api_bzl", + name = "api", srcs = ["api.bzl"], - visibility = ["//visibility:public"], - deps = ["//python/private/api:api_bzl"], + deps = ["//python/private/api"], ) -filegroup( - name = "distribution", - srcs = glob(["**"]), +bzl_library( + name = "attr_builders", + srcs = ["attr_builders.bzl"], + deps = ["//python/private:attr_builders"], +) + +bzl_library( + name = "executables", + srcs = ["executables.bzl"], + deps = [ + "//python/private:py_binary_rule", + "//python/private:py_executable", + "//python/private:py_test_rule", + ], +) + +bzl_library( + name = "libraries", + srcs = ["libraries.bzl"], + deps = ["//python/private:py_library"], +) + +bzl_library( + name = "rule_builders", + srcs = ["rule_builders.bzl"], + deps = ["//python/private:rule_builders"], ) diff --git a/python/api/api.bzl b/python/api/api.bzl index c8fb921c12..d41ec739cd 100644 --- a/python/api/api.bzl +++ b/python/api/api.bzl @@ -1,4 +1,23 @@ -"""Public, analysis phase APIs for Python rules.""" +"""Public, analysis phase APIs for Python rules. + +To use the analyis-time API, add the attributes to your rule, then +use `py_common.get()` to get the api object: + +``` +load("@rules_python//python/api:api.bzl", "py_common") + +def _impl(ctx): + py_api = py_common.get(ctx) + +myrule = rule( + implementation = _impl, + attrs = {...} | py_common.API_ATTRS +) +``` + +:::{versionadded} 0.37.0 +::: +""" load("//python/private/api:api.bzl", _py_common = "py_common") diff --git a/python/api/attr_builders.bzl b/python/api/attr_builders.bzl new file mode 100644 index 0000000000..573f9c6bc1 --- /dev/null +++ b/python/api/attr_builders.bzl @@ -0,0 +1,5 @@ +"""Public, attribute building APIs for Python rules.""" + +load("//python/private:attr_builders.bzl", _attrb = "attrb") + +attrb = _attrb diff --git a/python/api/executables.bzl b/python/api/executables.bzl new file mode 100644 index 0000000000..99bb7cc603 --- /dev/null +++ b/python/api/executables.bzl @@ -0,0 +1,31 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +""" +{#python-apis-executables-bzl} +Loading-phase APIs specific to executables (binaries/tests). + +:::{versionadded} 1.3.0 +::: +""" + +load("//python/private:py_binary_rule.bzl", "create_py_binary_rule_builder") +load("//python/private:py_executable.bzl", "create_executable_rule_builder") +load("//python/private:py_test_rule.bzl", "create_py_test_rule_builder") + +executables = struct( + py_binary_rule_builder = create_py_binary_rule_builder, + py_test_rule_builder = create_py_test_rule_builder, + executable_rule_builder = create_executable_rule_builder, +) diff --git a/python/api/libraries.bzl b/python/api/libraries.bzl new file mode 100644 index 0000000000..0b470a9ad4 --- /dev/null +++ b/python/api/libraries.bzl @@ -0,0 +1,27 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +""" +{#python-apis-libraries-bzl} +Loading-phase APIs specific to libraries. + +:::{versionadded} 1.3.0 +::: +""" + +load("//python/private:py_library.bzl", "create_py_library_rule_builder") + +libraries = struct( + py_library_rule_builder = create_py_library_rule_builder, +) diff --git a/python/api/rule_builders.bzl b/python/api/rule_builders.bzl new file mode 100644 index 0000000000..13ec4d39ea --- /dev/null +++ b/python/api/rule_builders.bzl @@ -0,0 +1,5 @@ +"""Public, rule building APIs for Python rules.""" + +load("//python/private:rule_builders.bzl", _ruleb = "ruleb") + +ruleb = _ruleb diff --git a/python/bin/BUILD.bazel b/python/bin/BUILD.bazel new file mode 100644 index 0000000000..30af7d1b9f --- /dev/null +++ b/python/bin/BUILD.bazel @@ -0,0 +1,57 @@ +load("//python/private:interpreter.bzl", _interpreter_binary = "interpreter_binary") +load("//python/private:repl.bzl", "py_repl_binary") + +filegroup( + name = "distribution", + srcs = glob(["**"]), + visibility = ["//:__subpackages__"], +) + +_interpreter_binary( + name = "python", + binary = ":python_src", + target_compatible_with = select({ + "@platforms//os:windows": ["@platforms//:incompatible"], + "//conditions:default": [], + }), + visibility = ["//visibility:public"], +) + +# The user can modify this flag to source different interpreters for the +# `python` target above. +label_flag( + name = "python_src", + build_setting_default = "//python:none", +) + +py_repl_binary( + name = "repl", + stub = ":repl_stub", + visibility = ["//visibility:public"], + deps = [ + ":repl_dep", + ":repl_stub_dep", + ], +) + +# The user can replace this with their own stub. E.g. they can use this to +# import ipython instead of the default shell. +label_flag( + name = "repl_stub", + build_setting_default = "repl_stub.py", +) + +# The user can modify this flag to make an interpreter shell library available +# for the stub. E.g. if they switch the stub for an ipython-based one, then they +# can point this at their version of ipython. +label_flag( + name = "repl_stub_dep", + build_setting_default = "//python/private:empty", +) + +# The user can modify this flag to make arbitrary PyInfo targets available for +# import on the REPL. +label_flag( + name = "repl_dep", + build_setting_default = "//python/private:empty", +) diff --git a/python/bin/repl_stub.py b/python/bin/repl_stub.py new file mode 100644 index 0000000000..858cf810b9 --- /dev/null +++ b/python/bin/repl_stub.py @@ -0,0 +1,73 @@ +"""Simulates the REPL that Python spawns when invoking the binary with no arguments. + +The code module is responsible for the default shell. + +The import and `ocde.interact()` call here his is equivalent to doing: + + $ python3 -m code + Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] on linux + Type "help", "copyright", "credits" or "license" for more information. + (InteractiveConsole) + >>> + +The logic for PYTHONSTARTUP is handled in python/private/repl_template.py. +""" + +# Capture the globals from PYTHONSTARTUP so we can pass them on to the console. +console_locals = globals().copy() + +import code # noqa: E402 +import rlcompleter # noqa: E402 +import sys # noqa: E402 + + +class DynamicCompleter(rlcompleter.Completer): + """ + A custom completer that dynamically updates its namespace to include new + imports made within the interactive session. + """ + + def __init__(self, namespace): + # Store a reference to the namespace, not a copy, so that changes to the namespace are + # reflected. + self.namespace = namespace + + def complete(self, text, state): + # Update the completer's internal namespace with the current interactive session's locals + # and globals. This is the key to making new imports discoverable. + rlcompleter.Completer.__init__(self, self.namespace) + return super().complete(text, state) + + +if sys.stdin.isatty(): + # Use the default options. + exitmsg = None +else: + # On a non-interactive console, we want to suppress the >>> and the exit message. + exitmsg = "" + sys.ps1 = "" + sys.ps2 = "" + +# Set up tab completion. +try: + import readline + + completer = DynamicCompleter(console_locals) + readline.set_completer(completer.complete) + + # TODO(jpwoodbu): Use readline.backend instead of readline.__doc__ once we can depend on having + # Python >=3.13. + if "libedit" in readline.__doc__: # type: ignore + readline.parse_and_bind("bind ^I rl_complete") + elif "GNU readline" in readline.__doc__: # type: ignore + readline.parse_and_bind("tab: complete") + else: + print("Could not enable tab completion: unable to determine readline backend") +except ImportError: + print( + "Could not enable tab completion: " + "readline module not available on this platform" + ) + +# We set the banner to an empty string because the repl_template.py file already prints the banner. +code.interact(local=console_locals, banner="", exitmsg=exitmsg) diff --git a/python/cc/BUILD.bazel b/python/cc/BUILD.bazel index f4e4aeb00f..2fa03dcb5a 100644 --- a/python/cc/BUILD.bazel +++ b/python/cc/BUILD.bazel @@ -2,7 +2,7 @@ load("@bazel_skylib//:bzl_library.bzl", "bzl_library") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") -load("//python/private:current_py_cc_headers.bzl", "current_py_cc_headers") +load("//python/private:current_py_cc_headers.bzl", "current_py_cc_headers", "current_py_cc_headers_abi3") load("//python/private:current_py_cc_libs.bzl", "current_py_cc_libs") package( @@ -20,6 +20,17 @@ current_py_cc_headers( visibility = ["//visibility:public"], ) +# This target provides the C ABI3 headers for whatever the current toolchain is +# for the consuming rule. It basically acts like a cc_library by forwarding +# on the providers for the underlying cc_library that the toolchain is using. +current_py_cc_headers_abi3( + name = "current_py_cc_headers_abi3", + # Building this directly will fail unless a py cc toolchain is registered, + # and it's only under bzlmod that one is registered by default. + tags = [] if BZLMOD_ENABLED else ["manual"], + visibility = ["//visibility:public"], +) + # This target provides the C libraries for whatever the current toolchain is for # the consuming rule. It basically acts like a cc_library by forwarding on the # providers for the underlying cc_library that the toolchain is using. @@ -36,21 +47,42 @@ toolchain_type( visibility = ["//visibility:public"], ) +filegroup( + name = "distribution", + srcs = glob(["**"]), +) + bzl_library( - name = "py_cc_toolchain_bzl", + name = "py_cc_toolchain", srcs = ["py_cc_toolchain.bzl"], visibility = ["//visibility:public"], - deps = ["//python/private:py_cc_toolchain_macro_bzl"], + deps = ["//python/private:py_cc_toolchain_macro"], ) bzl_library( - name = "py_cc_toolchain_info_bzl", + name = "py_cc_toolchain_info", srcs = ["py_cc_toolchain_info.bzl"], visibility = ["//visibility:public"], - deps = ["//python/private:py_cc_toolchain_info_bzl"], + deps = ["//python/private:py_cc_toolchain_info"], ) -filegroup( - name = "distribution", - srcs = glob(["**"]), +alias( + name = "py_cc_toolchain_bzl", + actual = ":py_cc_toolchain", + deprecation = "Use //python/cc:py_cc_toolchain instead", + visibility = ["//visibility:public"], +) + +alias( + name = "py_cc_toolchain_info_bzl", + actual = ":py_cc_toolchain_info", + deprecation = "Use //python/cc:py_cc_toolchain_info instead", + visibility = ["//visibility:public"], +) + +bzl_library( + name = "py_extension", + srcs = ["py_extension.bzl"], + visibility = ["//visibility:public"], + deps = ["//python/private/cc:py_extension_macro"], ) diff --git a/python/cc/py_extension.bzl b/python/cc/py_extension.bzl new file mode 100644 index 0000000000..8e90517789 --- /dev/null +++ b/python/cc/py_extension.bzl @@ -0,0 +1,23 @@ +"""Rules for creating Python C extension modules. + +This module provides `py_extension` for building Python C extension modules +that can be imported by Python targets (`py_binary`, `py_test`, `py_library`). +It manages dynamic linking, symbol exports (`PyInit_*`), platform-specific link +flags, and target dependencies cleanly. + +See the [Python C API documentation](https://docs.python.org/3/c-api/index.html) +for information on writing C extension modules. + +:::{include} /_includes/experimental_api.md +::: + +:::{versionadded} 2.3.0 +::: +""" + +load( + "//python/private/cc:py_extension_macro.bzl", + _py_extension = "py_extension", +) + +py_extension = _py_extension diff --git a/python/config_settings/BUILD.bazel b/python/config_settings/BUILD.bazel index aa26e6e669..d92aa4261c 100644 --- a/python/config_settings/BUILD.bazel +++ b/python/config_settings/BUILD.bazel @@ -1,5 +1,6 @@ -load("@bazel_skylib//rules:common_settings.bzl", "string_flag") +load("@bazel_skylib//rules:common_settings.bzl", "bool_flag", "string_flag") load("@pythons_hub//:versions.bzl", "DEFAULT_PYTHON_VERSION", "MINOR_MAPPING", "PYTHON_VERSIONS") +load("@rules_python_internal//:rules_python_config.bzl", "config") load( "//python/private:flags.bzl", "AddSrcsToRunfilesFlag", @@ -9,15 +10,20 @@ load( "LibcFlag", "PrecompileFlag", "PrecompileSourceRetentionFlag", + "ValidateTestMainFlag", + "VenvsSitePackages", + "VenvsUseDeclareSymlinkFlag", + rp_string_flag = "string_flag", ) -load( - "//python/private/pypi:flags.bzl", - "UniversalWhlFlag", - "UseWhlFlag", - "define_pypi_internal_flags", -) +load("//python/private:visibility.bzl", "NOT_ACTUALLY_PUBLIC") # buildifier: disable=bzl-visibility +load("//python/private/pypi:flags.bzl", "define_pypi_internal_flags") load(":config_settings.bzl", "construct_config_settings") +# We don't generate bzl_library for these because they aren't public targets +# and should be moved +# gazelle:exclude config_settings.bzl +# gazelle:exclude transition.bzl + filegroup( name = "distribution", srcs = glob(["**"]) + [ @@ -29,6 +35,11 @@ filegroup( construct_config_settings( name = "construct_config_settings", default_version = DEFAULT_PYTHON_VERSION, + documented_flags = [ + ":pip_whl_osx_version", + ":py_freethreaded", + ":py_linux_libc", + ], minor_mapping = MINOR_MAPPING, versions = PYTHON_VERSIONS, ) @@ -38,7 +49,7 @@ string_flag( build_setting_default = AddSrcsToRunfilesFlag.AUTO, values = AddSrcsToRunfilesFlag.flag_values(), # NOTE: Only public because it is dependency of public rules. - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) string_flag( @@ -57,7 +68,7 @@ config_setting( }, # NOTE: Only public because it is used in py_toolchain_suite from toolchain # repositories - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) string_flag( @@ -65,7 +76,15 @@ string_flag( build_setting_default = PrecompileFlag.AUTO, values = sorted(PrecompileFlag.__members__.values()), # NOTE: Only public because it's an implicit dependency - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, +) + +string_flag( + name = "validate_test_main", + build_setting_default = ValidateTestMainFlag.AUTO, + values = ValidateTestMainFlag.flag_values(), + # NOTE: Only public because it's an implicit dependency of py_test. + visibility = NOT_ACTUALLY_PUBLIC, ) string_flag( @@ -73,24 +92,43 @@ string_flag( build_setting_default = PrecompileSourceRetentionFlag.AUTO, values = sorted(PrecompileSourceRetentionFlag.__members__.values()), # NOTE: Only public because it's an implicit dependency - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) -string_flag( +rp_string_flag( name = "bootstrap_impl", build_setting_default = BootstrapImplFlag.SYSTEM_PYTHON, + override = select({ + # Windows doesn't yet support bootstrap=script, so force disable it + ":_is_windows": BootstrapImplFlag.SYSTEM_PYTHON, + "//conditions:default": "", + }), values = sorted(BootstrapImplFlag.__members__.values()), # NOTE: Only public because it's an implicit dependency + visibility = NOT_ACTUALLY_PUBLIC, +) + +label_flag( + name = "debugger", + build_setting_default = "//python/private:empty", visibility = ["//visibility:public"], ) +# For some reason, @platforms//os:windows can't be directly used +# in the select() for the flag. But it can be used when put behind +# a config_setting(). +config_setting( + name = "_is_windows", + constraint_values = ["@platforms//os:windows"], +) + # This is used for pip and hermetic toolchain resolution. string_flag( name = "py_linux_libc", build_setting_default = LibcFlag.GLIBC, values = LibcFlag.flag_values(), # NOTE: Only public because it is used in pip hub and toolchain repos. - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) string_flag( @@ -100,78 +138,97 @@ string_flag( visibility = ["//visibility:public"], ) -config_setting( +alias( name = "is_py_freethreaded", - flag_values = {":py_freethreaded": FreeThreadedFlag.YES}, + actual = ":_is_py_freethreaded_yes", + deprecation = "not actually public, please create your own config_setting using the flag that rules_python exposes", + tags = ["manual"], visibility = ["//visibility:public"], ) -# pip.parse related flags +alias( + name = "is_py_non_freethreaded", + actual = ":_is_py_freethreaded_no", + deprecation = "not actually public, please create your own config_setting using the flag that rules_python exposes", + tags = ["manual"], + visibility = ["//visibility:public"], +) string_flag( - name = "pip_whl", - build_setting_default = UseWhlFlag.AUTO, - values = sorted(UseWhlFlag.__members__.values()), - # NOTE: Only public because it is used in pip hub repos. + name = "venvs_use_declare_symlink", + build_setting_default = VenvsUseDeclareSymlinkFlag.YES, + values = VenvsUseDeclareSymlinkFlag.flag_values(), visibility = ["//visibility:public"], ) -config_setting( - name = "is_pip_whl_auto", - flag_values = { - ":pip_whl": UseWhlFlag.AUTO, - }, +# pip.parse related flags + +string_flag( + name = "venv", + build_setting_default = "auto", + # NOTE: Only public because it is used in pip hub repos and executable transitions. + visibility = NOT_ACTUALLY_PUBLIC, +) + +string_flag( + name = "pip_whl_osx_version", + build_setting_default = "", # NOTE: Only public because it is used in pip hub repos. - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) -config_setting( - name = "is_pip_whl_no", - flag_values = { - ":pip_whl": UseWhlFlag.NO, - }, +string_flag( + name = "venvs_site_packages", + build_setting_default = VenvsSitePackages.NO, # NOTE: Only public because it is used in pip hub repos. - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) config_setting( - name = "is_pip_whl_only", + name = "_is_venvs_site_packages_yes", flag_values = { - ":pip_whl": UseWhlFlag.ONLY, + ":venvs_site_packages": VenvsSitePackages.YES, }, - # NOTE: Only public because it is used in pip hub repos. - visibility = ["//visibility:public"], + # NOTE: Only public because it is used in whl_library repos. + visibility = NOT_ACTUALLY_PUBLIC, ) -string_flag( - name = "pip_whl_osx_arch", - build_setting_default = UniversalWhlFlag.ARCH, - values = sorted(UniversalWhlFlag.__members__.values()), - # NOTE: Only public because it is used in pip hub repos. - visibility = ["//visibility:public"], +define_pypi_internal_flags( + name = "define_pypi_internal_flags", ) -string_flag( - name = "pip_whl_glibc_version", - build_setting_default = "", +label_flag( + name = "pip_env_marker_config", + build_setting_default = ":_pip_env_marker_default_config", # NOTE: Only public because it is used in pip hub repos. + visibility = NOT_ACTUALLY_PUBLIC, +) + +bool_flag( + name = "experimental_python_import_all_repositories", + build_setting_default = True, + scope = "universal", visibility = ["//visibility:public"], ) -string_flag( - name = "pip_whl_muslc_version", - build_setting_default = "", - # NOTE: Only public because it is used in pip hub repos. +bool_flag( + name = "build_python_zip", + build_setting_default = config.build_python_zip_default, + help = "Build python executable zip. Defaults to on on Windows, off on other platforms", + scope = "universal", visibility = ["//visibility:public"], ) -string_flag( - name = "pip_whl_osx_version", - build_setting_default = "", - # NOTE: Only public because it is used in pip hub repos. +bool_flag( + name = "incompatible_default_to_explicit_init_py", + build_setting_default = False, + scope = "universal", visibility = ["//visibility:public"], ) -define_pypi_internal_flags( - name = "define_pypi_internal_flags", +string_flag( + name = "python_path", + build_setting_default = "python", + scope = "universal", + visibility = ["//visibility:public"], ) diff --git a/python/config_settings/private/py_args.bzl b/python/config_settings/private/py_args.bzl deleted file mode 100644 index 09a26461b7..0000000000 --- a/python/config_settings/private/py_args.bzl +++ /dev/null @@ -1,42 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""A helper to extract default args for the transition rule.""" - -def py_args(name, kwargs): - """A helper to extract common py_binary and py_test args - - See https://bazel.build/reference/be/python#py_binary and - https://bazel.build/reference/be/python#py_test for the list - that should be returned - - Args: - name: The name of the target. - kwargs: The kwargs to be extracted from; MODIFIED IN-PLACE. - - Returns: - A dict with the extracted arguments - """ - return dict( - args = kwargs.pop("args", None), - data = kwargs.pop("data", None), - env = kwargs.pop("env", None), - srcs = kwargs.pop("srcs", None), - deps = kwargs.pop("deps", None), - # See https://bazel.build/reference/be/python#py_binary.main - # for default logic. - # NOTE: This doesn't match the exact way a regular py_binary searches for - # it's main amongst the srcs, but is close enough for most cases. - main = kwargs.pop("main", name + ".py"), - ) diff --git a/python/config_settings/transition.bzl b/python/config_settings/transition.bzl index a7646dcda3..937f33bb88 100644 --- a/python/config_settings/transition.bzl +++ b/python/config_settings/transition.bzl @@ -14,266 +14,41 @@ """The transition module contains the rule definitions to wrap py_binary and py_test and transition them to the desired target platform. + +:::{versionchanged} 1.1.0 +The `py_binary` and `py_test` symbols are aliases to the regular rules. Usages +of them should be changed to load the regular rules directly. +::: """ -load("@bazel_skylib//lib:dicts.bzl", "dicts") load("//python:py_binary.bzl", _py_binary = "py_binary") -load("//python:py_info.bzl", "PyInfo") -load("//python:py_runtime_info.bzl", "PyRuntimeInfo") load("//python:py_test.bzl", _py_test = "py_test") -load("//python/config_settings/private:py_args.bzl", "py_args") -load("//python/private:reexports.bzl", "BuiltinPyInfo", "BuiltinPyRuntimeInfo") - -def _transition_python_version_impl(_, attr): - return {"//python/config_settings:python_version": str(attr.python_version)} - -_transition_python_version = transition( - implementation = _transition_python_version_impl, - inputs = [], - outputs = ["//python/config_settings:python_version"], -) - -def _transition_py_impl(ctx): - target = ctx.attr.target - windows_constraint = ctx.attr._windows_constraint[platform_common.ConstraintValueInfo] - target_is_windows = ctx.target_platform_has_constraint(windows_constraint) - executable = ctx.actions.declare_file(ctx.attr.name + (".exe" if target_is_windows else "")) - ctx.actions.symlink( - is_executable = True, - output = executable, - target_file = target[DefaultInfo].files_to_run.executable, +load("//python/private:deprecation.bzl", "with_deprecation") +load("//python/private:text_util.bzl", "render") + +def _with_deprecation(kwargs, *, name, python_version): + kwargs["python_version"] = python_version + return with_deprecation.symbol( + kwargs, + symbol_name = name, + old_load = "@rules_python//python/config_settings:transition.bzl", + new_load = "@rules_python//python:{}.bzl".format(name), + snippet = render.call(name, **{k: repr(v) for k, v in kwargs.items()}), ) - default_outputs = [] - if target_is_windows: - # NOTE: Bazel 6 + host=linux + target=windows results in the .exe extension missing - inner_bootstrap_path = _strip_suffix(target[DefaultInfo].files_to_run.executable.short_path, ".exe") - inner_bootstrap = None - inner_zip_file_path = inner_bootstrap_path + ".zip" - inner_zip_file = None - for file in target[DefaultInfo].files.to_list(): - if file.short_path == inner_bootstrap_path: - inner_bootstrap = file - elif file.short_path == inner_zip_file_path: - inner_zip_file = file - - # TODO: Use `fragments.py.build_python_zip` once Bazel 6 support is dropped. - # Which file the Windows .exe looks for depends on the --build_python_zip file. - # Bazel 7+ has APIs to know the effective value of that flag, but not Bazel 6. - # To work around this, we treat the existence of a .zip in the default outputs - # to mean --build_python_zip=true. - if inner_zip_file: - suffix = ".zip" - underlying_launched_file = inner_zip_file - else: - suffix = "" - underlying_launched_file = inner_bootstrap - - if underlying_launched_file: - launched_file_symlink = ctx.actions.declare_file(ctx.attr.name + suffix) - ctx.actions.symlink( - is_executable = True, - output = launched_file_symlink, - target_file = underlying_launched_file, - ) - default_outputs.append(launched_file_symlink) - - env = {} - for k, v in ctx.attr.env.items(): - env[k] = ctx.expand_location(v) - - providers = [ - DefaultInfo( - executable = executable, - files = depset(default_outputs, transitive = [target[DefaultInfo].files]), - runfiles = ctx.runfiles(default_outputs).merge(target[DefaultInfo].default_runfiles), - ), - # Ensure that the binary we're wrapping is included in code coverage. - coverage_common.instrumented_files_info( - ctx, - dependency_attributes = ["target"], - ), - target[OutputGroupInfo], - # TODO(f0rmiga): testing.TestEnvironment is deprecated in favour of RunEnvironmentInfo but - # RunEnvironmentInfo is not exposed in Bazel < 5.3. - # https://github.com/bazelbuild/rules_python/issues/901 - # https://github.com/bazelbuild/bazel/commit/dbdfa07e92f99497be9c14265611ad2920161483 - testing.TestEnvironment(env), - ] - if PyInfo in target: - providers.append(target[PyInfo]) - if BuiltinPyInfo != None and BuiltinPyInfo in target and PyInfo != BuiltinPyInfo: - providers.append(target[BuiltinPyInfo]) - - if PyRuntimeInfo in target: - providers.append(target[PyRuntimeInfo]) - if BuiltinPyRuntimeInfo != None and BuiltinPyRuntimeInfo in target and PyRuntimeInfo != BuiltinPyRuntimeInfo: - providers.append(target[BuiltinPyRuntimeInfo]) - return providers - -_COMMON_ATTRS = { - "deps": attr.label_list( - mandatory = False, - ), - "env": attr.string_dict( - mandatory = False, - ), - "python_version": attr.string( - mandatory = True, - ), - "srcs": attr.label_list( - allow_files = True, - mandatory = False, - ), - "target": attr.label( - executable = True, - cfg = "target", - mandatory = True, - providers = [PyInfo], - ), - # "tools" is a hack here. It should be "data" but "data" is not included by default in the - # location expansion in the same way it is in the native Python rules. The difference on how - # the Bazel deals with those special attributes differ on the LocationExpander, e.g.: - # https://github.com/bazelbuild/bazel/blob/ce611646/src/main/java/com/google/devtools/build/lib/analysis/LocationExpander.java#L415-L429 - # - # Since the default LocationExpander used by ctx.expand_location is not the same as the native - # rules (it doesn't set "allowDataAttributeEntriesInLabel"), we use "tools" temporarily while a - # proper fix in Bazel happens. - # - # A fix for this was proposed in https://github.com/bazelbuild/bazel/pull/16381. - "tools": attr.label_list( - allow_files = True, - mandatory = False, - ), - # Required to Opt-in to the transitions feature. - "_allowlist_function_transition": attr.label( - default = "@bazel_tools//tools/allowlists/function_transition_allowlist", - ), - "_windows_constraint": attr.label( - default = "@platforms//os:windows", - ), -} -_PY_TEST_ATTRS = { - # Magic attribute to help C++ coverage work. There's no - # docs about this; see TestActionBuilder.java - "_collect_cc_coverage": attr.label( - default = "@bazel_tools//tools/test:collect_cc_coverage", - executable = True, - cfg = "exec", - ), - # Magic attribute to make coverage work. There's no - # docs about this; see TestActionBuilder.java - "_lcov_merger": attr.label( - default = configuration_field(fragment = "coverage", name = "output_generator"), - executable = True, - cfg = "exec", - ), -} +def py_binary(**kwargs): + """[DEPRECATED] Deprecated alias for py_binary. -_transition_py_binary = rule( - _transition_py_impl, - attrs = _COMMON_ATTRS | _PY_TEST_ATTRS, - cfg = _transition_python_version, - executable = True, - fragments = ["py"], -) - -_transition_py_test = rule( - _transition_py_impl, - attrs = _COMMON_ATTRS | _PY_TEST_ATTRS, - cfg = _transition_python_version, - test = True, - fragments = ["py"], -) - -def _py_rule(rule_impl, transition_rule, name, python_version, **kwargs): - pyargs = py_args(name, kwargs) - args = pyargs["args"] - data = pyargs["data"] - env = pyargs["env"] - srcs = pyargs["srcs"] - deps = pyargs["deps"] - main = pyargs["main"] - - # Attributes common to all build rules. - # https://bazel.build/reference/be/common-definitions#common-attributes - compatible_with = kwargs.pop("compatible_with", None) - deprecation = kwargs.pop("deprecation", None) - exec_compatible_with = kwargs.pop("exec_compatible_with", None) - exec_properties = kwargs.pop("exec_properties", None) - features = kwargs.pop("features", None) - restricted_to = kwargs.pop("restricted_to", None) - tags = kwargs.pop("tags", None) - target_compatible_with = kwargs.pop("target_compatible_with", None) - testonly = kwargs.pop("testonly", None) - toolchains = kwargs.pop("toolchains", None) - visibility = kwargs.pop("visibility", None) - - common_attrs = { - "compatible_with": compatible_with, - "deprecation": deprecation, - "exec_compatible_with": exec_compatible_with, - "exec_properties": exec_properties, - "features": features, - "restricted_to": restricted_to, - "target_compatible_with": target_compatible_with, - "testonly": testonly, - "toolchains": toolchains, - } - - # Test-specific extra attributes. - if "env_inherit" in kwargs: - common_attrs["env_inherit"] = kwargs.pop("env_inherit") - if "size" in kwargs: - common_attrs["size"] = kwargs.pop("size") - if "timeout" in kwargs: - common_attrs["timeout"] = kwargs.pop("timeout") - if "flaky" in kwargs: - common_attrs["flaky"] = kwargs.pop("flaky") - if "shard_count" in kwargs: - common_attrs["shard_count"] = kwargs.pop("shard_count") - if "local" in kwargs: - common_attrs["local"] = kwargs.pop("local") - - # Binary-specific extra attributes. - if "output_licenses" in kwargs: - common_attrs["output_licenses"] = kwargs.pop("output_licenses") - - rule_impl( - name = "_" + name, - args = args, - data = data, - deps = deps, - env = env, - srcs = srcs, - main = main, - tags = ["manual"] + (tags if tags else []), - visibility = ["//visibility:private"], - **dicts.add(common_attrs, kwargs) - ) - - return transition_rule( - name = name, - args = args, - deps = deps, - env = env, - python_version = python_version, - srcs = srcs, - tags = tags, - target = ":_" + name, - tools = data, - visibility = visibility, - **common_attrs - ) + Args: + **kwargs: keyword args forwarded onto {obj}`py_binary`. + """ -def py_binary(name, python_version, **kwargs): - return _py_rule(_py_binary, _transition_py_binary, name, python_version, **kwargs) + _py_binary(**_with_deprecation(kwargs, name = "py_binary", python_version = kwargs.get("python_version"))) -def py_test(name, python_version, **kwargs): - return _py_rule(_py_test, _transition_py_test, name, python_version, **kwargs) +def py_test(**kwargs): + """[DEPRECATED] Deprecated alias for py_test. -def _strip_suffix(s, suffix): - if s.endswith(suffix): - return s[:-len(suffix)] - else: - return s + Args: + **kwargs: keyword args forwarded onto {obj}`py_binary`. + """ + _py_test(**_with_deprecation(kwargs, name = "py_test", python_version = kwargs.get("python_version"))) diff --git a/python/current_py_toolchain.bzl b/python/current_py_toolchain.bzl index f3ff2ace07..4fb9eb9eed 100644 --- a/python/current_py_toolchain.bzl +++ b/python/current_py_toolchain.bzl @@ -27,11 +27,19 @@ def _current_py_toolchain_impl(ctx): direct.append(toolchain.py3_runtime.interpreter) transitive.append(toolchain.py3_runtime.files) vars["PYTHON3"] = toolchain.py3_runtime.interpreter.path + vars["PYTHON3_ROOTPATH"] = toolchain.py3_runtime.interpreter.short_path + elif toolchain.py3_runtime and toolchain.py3_runtime.interpreter_path: + vars["PYTHON3"] = toolchain.py3_runtime.interpreter_path + vars["PYTHON3_ROOTPATH"] = toolchain.py3_runtime.interpreter_path if toolchain.py2_runtime and toolchain.py2_runtime.interpreter: direct.append(toolchain.py2_runtime.interpreter) transitive.append(toolchain.py2_runtime.files) vars["PYTHON2"] = toolchain.py2_runtime.interpreter.path + vars["PYTHON2_ROOTPATH"] = toolchain.py2_runtime.interpreter.short_path + elif toolchain.py2_runtime and toolchain.py2_runtime.interpreter_path: + vars["PYTHON2"] = toolchain.py2_runtime.interpreter_path + vars["PYTHON2_ROOTPATH"] = toolchain.py2_runtime.interpreter_path files = depset(direct, transitive = transitive) return [ @@ -49,6 +57,11 @@ current_py_toolchain = rule( other rules, such as genrule. It allows exposing a python toolchain after toolchain resolution has happened, to a rule which expects a concrete implementation of a toolchain, rather than a toolchain_type which could be resolved to that toolchain. + + :::{versionchanged} 1.4.0 + From now on, we also expose `$(PYTHON2_ROOTPATH)` and `$(PYTHON3_ROOTPATH)` which are runfiles + locations equivalents of `$(PYTHON2)` and `$(PYTHON3) respectively. + ::: """, implementation = _current_py_toolchain_impl, attrs = { diff --git a/python/defs.bzl b/python/defs.bzl index bd89f5b1f2..bdf5dae2e4 100644 --- a/python/defs.bzl +++ b/python/defs.bzl @@ -13,7 +13,6 @@ # limitations under the License. """Core rules for building Python projects.""" -load("@bazel_tools//tools/python:srcs_version.bzl", _find_requirements = "find_requirements") load("//python:py_binary.bzl", _py_binary = "py_binary") load("//python:py_info.bzl", _PyInfo = "PyInfo") load("//python:py_library.bzl", _py_library = "py_library") @@ -34,12 +33,8 @@ current_py_toolchain = _current_py_toolchain py_import = _py_import -# Re-exports of Starlark-defined symbols in @bazel_tools//tools/python. - py_runtime_pair = _py_runtime_pair -find_requirements = _find_requirements - py_library = _py_library py_binary = _py_binary diff --git a/python/entry_points/BUILD.bazel b/python/entry_points/BUILD.bazel index 46dbd9298b..7cfd0cdf25 100644 --- a/python/entry_points/BUILD.bazel +++ b/python/entry_points/BUILD.bazel @@ -21,17 +21,22 @@ exports_files( visibility = ["//docs:__subpackages__"], ) -bzl_library( - name = "py_console_script_binary_bzl", - srcs = [":py_console_script_binary.bzl"], - visibility = ["//visibility:public"], - deps = [ - "//python/private:py_console_script_binary_bzl", - ], -) - filegroup( name = "distribution", srcs = glob(["**"]), visibility = ["//python:__subpackages__"], ) + +bzl_library( + name = "py_console_script_binary", + srcs = ["py_console_script_binary.bzl"], + visibility = ["//visibility:public"], + deps = ["//python/private:py_console_script_binary"], +) + +alias( + name = "py_console_script_binary_bzl", + actual = ":py_console_script_binary", + deprecation = "Use //python/entry_points:py_console_script_binary instead", + visibility = ["//visibility:public"], +) diff --git a/python/extensions/BUILD.bazel b/python/extensions/BUILD.bazel index e8a63d6d5b..af18fa15b7 100644 --- a/python/extensions/BUILD.bazel +++ b/python/extensions/BUILD.bazel @@ -25,17 +25,47 @@ filegroup( ) bzl_library( - name = "pip_bzl", + name = "config", + srcs = ["config.bzl"], + visibility = ["//:__subpackages__"], + deps = [ + "//python/private:internal_config_repo", + "//python/private/pypi:deps", + "@bazel_features//:features", + ], +) + +bzl_library( + name = "pip", srcs = ["pip.bzl"], visibility = ["//:__subpackages__"], - deps = ["//python/private/pypi:pip_bzl"], + deps = ["//python/private/pypi:pip"], ) bzl_library( - name = "python_bzl", + name = "python", srcs = ["python.bzl"], visibility = ["//:__subpackages__"], - deps = [ - "//python/private:python_bzl", - ], + deps = ["//python/private:python"], +) + +alias( + name = "config_bzl", + actual = ":config", + deprecation = "Use //python/extensions:config instead", + visibility = ["//:__subpackages__"], +) + +alias( + name = "pip_bzl", + actual = ":pip", + deprecation = "Use //python/extensions:pip instead", + visibility = ["//:__subpackages__"], +) + +alias( + name = "python_bzl", + actual = ":python", + deprecation = "Use //python/extensions:python instead", + visibility = ["//:__subpackages__"], ) diff --git a/python/extensions/config.bzl b/python/extensions/config.bzl new file mode 100644 index 0000000000..f19a07aaef --- /dev/null +++ b/python/extensions/config.bzl @@ -0,0 +1,59 @@ +"""Extension for configuring global settings of rules_python.""" + +load("@bazel_features//:features.bzl", "bazel_features") +load("//python/private:internal_config_repo.bzl", "internal_config_repo") +load("//python/private/pypi:deps.bzl", "pypi_deps") + +_add_transition_setting = tag_class( + doc = """ +Specify a build setting that terminal rules transition on by default. + +Terminal rules are rules such as py_binary, py_test, py_wheel, or similar +rules that represent some deployable unit. Settings added here can +then be used a keys with the {obj}`config_settings` attribute. + +:::{note} +This adds the label as a dependency of the Python rules. Take care to not refer +to repositories that are expensive to create or invalidate frequently. +::: +""", + attrs = { + "setting": attr.label(doc = "The build setting to add."), + }, +) + +def _config_impl(module_ctx): + transition_setting_generators = {} + transition_settings = [] + for mod in module_ctx.modules: + for tag in mod.tags.add_transition_setting: + setting = str(tag.setting) + if setting not in transition_setting_generators: + transition_setting_generators[setting] = [] + transition_settings.append(setting) + transition_setting_generators[setting].append(mod.name) + + internal_config_repo( + name = "rules_python_internal", + transition_setting_generators = transition_setting_generators, + transition_settings = transition_settings, + ) + + pypi_deps() + + if bazel_features.external_deps.extension_metadata_has_reproducible: + return module_ctx.extension_metadata(reproducible = True) + else: + return None + +config = module_extension( + doc = """Global settings for rules_python. + +:::{versionadded} 1.7.0 +::: +""", + implementation = _config_impl, + tag_classes = { + "add_transition_setting": _add_transition_setting, + }, +) diff --git a/python/extensions/python.bzl b/python/extensions/python.bzl index abd5080dd8..b8b755ebca 100644 --- a/python/extensions/python.bzl +++ b/python/extensions/python.bzl @@ -20,10 +20,8 @@ The simplest way to configure the toolchain with `rules_python` is as follows. ```starlark python = use_extension("@rules_python//python/extensions:python.bzl", "python") -python.toolchain( - is_default = True, - python_version = "3.11", -) +python.defaults(python_version = "3.11") +python.toolchain(python_version = "3.11") use_repo(python, "python_3_11") ``` diff --git a/python/features.bzl b/python/features.bzl index 90a1121909..99050a7ca5 100644 --- a/python/features.bzl +++ b/python/features.bzl @@ -17,7 +17,146 @@ # See https://git-scm.com/docs/git-archive/2.29.0#Documentation/git-archive.txt-export-subst _VERSION_PRIVATE = "$Format:%(describe:tags=true)$" +def _features_typedef(): + """Information about features rules_python has implemented. + + ::::{field} targets + :type: dict[str, bool] + + A map of public API targets available in rules_python for feature detection + purposes. + + :::{seealso} + * {obj}`features.loadable_symbols` + ::: + + :::{versionadded} 1.9.0 + ::: + :::: + + ::::{field} headers_abi3 + :type: bool + + True if the {obj}`@rules_python//python/cc:current_py_cc_headers_abi3` + target is available. + + :::{versionadded} 1.7.0 + ::: + :::: + + ::::{field} precompile + :type: bool + + True if the precompile attributes are available. + + :::{versionadded} 0.33.0 + ::: + :::: + + ::::{field} loadable_symbols + :type: dict[str, list[str]] + + A map of bzl paths to the list of public symbols they export. + + :::{seealso} + * {obj}`features.targets` + ::: + + :::{versionadded} 2.2.0 + ::: + :::: + + ::::{field} py_info_venv_symlinks + + True if the `PyInfo.venv_symlinks` field is available. + + :::{versionadded} 1.5.0 + ::: + :::: + + ::::{field} uses_builtin_rules + :type: bool + + True if the rules are using the Bazel-builtin implementation. + + :::{versionadded} 1.1.0 + ::: + :::: + + ::::{field} version + :type: str + + The rules_python version. This is a semver format, e.g. `X.Y.Z` with + optional trailing `-rcN`. For unreleased versions, it is an empty string. + :::{versionadded} 0.38.0 + :::: + + ::::{field} zipapp_rules + :type: bool + + Whether the rules_python version has the `py_zipapp_*` rules + + :::{versionadded} 1.9.0 + :::: + """ + +_TARGETS = { + "//command_line_option:build_runfile_links": True, + "//command_line_option:enable_runfiles": True, + "//command_line_option:extra_toolchains": True, + "//python/api:api": True, + "//python/api:executables": True, + "//python/api:libraries": True, + "//python/cc:current_py_cc_headers_abi3": True, + "//python/cc:py_cc_toolchain": True, + "//python/cc:py_cc_toolchain_info": True, + "//python/cc:py_extension": True, + "//python/config_settings:venv": True, + "//python/entry_points:py_console_script_binary": True, + "//python/local_toolchains:repos": True, + "//python:defs": True, + "//python:features": True, + "//python:packaging": True, + "//python:pip": True, + "//python:proto": True, + "//python:py_binary": True, + "//python:py_cc_link_params_info": True, + "//python:py_exec_tools_info": True, + "//python:py_exec_tools_toolchain": True, + "//python:py_executable_info": True, + "//python:py_import": True, + "//python:py_info": True, + "//python:py_library": True, + "//python:py_runtime": True, + "//python:py_runtime_info": True, + "//python:py_runtime_pair": True, + "//python:py_test": True, + "//python:repositories": True, + "//python:versions": True, +} + +_LOADABLE_SYMBOLS = { + "//python/cc:py_extension.bzl": [ + # keep sorted + "py_extension", + ], + "//python:py_info.bzl": [ + # keep sorted + "PyInfo", + "VenvSymlinkEntry", + "VenvSymlinkKind", + ], +} + features = struct( - version = _VERSION_PRIVATE if "$Format" not in _VERSION_PRIVATE else "", + TYPEDEF = _features_typedef, + # keep sorted + headers_abi3 = True, + loadable_symbols = _LOADABLE_SYMBOLS, precompile = True, + py_info_venv_symlinks = True, + targets = _TARGETS, + uses_builtin_rules = False, + version = _VERSION_PRIVATE if "$Format" not in _VERSION_PRIVATE else "", + zipapp_rules = True, ) diff --git a/python/local_toolchains/BUILD.bazel b/python/local_toolchains/BUILD.bazel new file mode 100644 index 0000000000..256dffb337 --- /dev/null +++ b/python/local_toolchains/BUILD.bazel @@ -0,0 +1,25 @@ +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") + +package(default_visibility = ["//:__subpackages__"]) + +filegroup( + name = "distribution", + srcs = glob(["**"]), +) + +bzl_library( + name = "repos", + srcs = ["repos.bzl"], + visibility = ["//visibility:public"], + deps = [ + "//python/private:local_runtime_repo", + "//python/private:local_runtime_toolchains_repo", + ], +) + +alias( + name = "repos_bzl", + actual = ":repos", + deprecation = "Use //python/local_toolchains:repos instead", + visibility = ["//visibility:public"], +) diff --git a/python/local_toolchains/repos.bzl b/python/local_toolchains/repos.bzl new file mode 100644 index 0000000000..320e503e1a --- /dev/null +++ b/python/local_toolchains/repos.bzl @@ -0,0 +1,18 @@ +"""Rules/macros for repository phase for local toolchains. + +:::{versionadded} 1.4.0 +::: +""" + +load( + "@rules_python//python/private:local_runtime_repo.bzl", + _local_runtime_repo = "local_runtime_repo", +) +load( + "@rules_python//python/private:local_runtime_toolchains_repo.bzl", + _local_runtime_toolchains_repo = "local_runtime_toolchains_repo", +) + +local_runtime_repo = _local_runtime_repo + +local_runtime_toolchains_repo = _local_runtime_toolchains_repo diff --git a/python/packaging.bzl b/python/packaging.bzl index 17f72a7d67..fb333ca72f 100644 --- a/python/packaging.bzl +++ b/python/packaging.bzl @@ -63,7 +63,7 @@ py_wheel_dist = rule( doc = """\ Prepare a dist/ folder, following Python's packaging standard practice. -See https://packaging.python.org/en/latest/tutorials/packaging-projects/#generating-distribution-archives +See which recommends a dist/ folder containing the wheel file(s), source distributions, etc. This also has the advantage that stamping information is included in the wheel's filename. @@ -94,13 +94,18 @@ def py_wheel( **kwargs): """Builds a Python Wheel. - Wheels are Python distribution format defined in https://www.python.org/dev/peps/pep-0427/. + Wheels are Python distribution format defined in {pep}`PEP 427`. This macro packages a set of targets into a single wheel. It wraps the [py_wheel rule](#py_wheel_rule). Currently only pure-python wheels are supported. + :::{versionchanged} 1.4.0 + From now on, an empty `requires_file` is treated as if it were omitted, resulting in a valid + `METADATA` file. + ::: + Examples: ```python @@ -139,7 +144,7 @@ def py_wheel( To publish the wheel to PyPI, the twine package is required and it is installed by default on `bzlmod` setups. On legacy `WORKSPACE`, `rules_python` doesn't provide `twine` itself - (see https://github.com/bazelbuild/rules_python/issues/1016), but + (see {gh-issue}`1016`), but you can install it with `pip_parse`, just like we do any other dependencies. Once you've installed twine, you can pass its label to the `twine` @@ -155,7 +160,7 @@ def py_wheel( ) ``` - Now you can run a command like the following, which publishes to https://test.pypi.org/ + Now you can run a command like the following, which publishes to ```sh % TWINE_USERNAME=__token__ TWINE_PASSWORD=pypi-*** \\ @@ -167,7 +172,7 @@ def py_wheel( name: A unique name for this target. twine: A label of the external location of the py_library target for twine twine_binary: A label of the external location of a binary target for twine. - publish_args: arguments passed to twine, e.g. ["--repository-url", "https://pypi.my.org/simple/"]. + publish_args: arguments passed to twine, e.g. `["--repository-url", "https://pypi.my.org/simple/"]`. These are subject to make var expansion, as with the `args` attribute. Note that you can also pass additional args to the bazel run command as in the example above. **kwargs: other named parameters passed to the underlying [py_wheel rule](#py_wheel_rule) @@ -211,19 +216,12 @@ def py_wheel( **copy_propagating_kwargs(kwargs) ) elif twine: - if not twine.endswith(":pkg"): - fail("twine label should look like @my_twine_repo//:pkg") - - twine_main = twine.replace(":pkg", ":rules_python_wheel_entry_point_twine.py") - py_binary( name = "{}.publish".format(name), - srcs = [twine_main], + deps = [twine], args = twine_args, data = [dist_target], - imports = ["."], - main = twine_main, - deps = [twine], + main_module = "twine", tags = manual_tags, visibility = kwargs.get("visibility"), **copy_propagating_kwargs(kwargs) diff --git a/python/pip_install/BUILD.bazel b/python/pip_install/BUILD.bazel index 09bc46eea7..b3c1ab1c03 100644 --- a/python/pip_install/BUILD.bazel +++ b/python/pip_install/BUILD.bazel @@ -18,23 +18,6 @@ package( default_visibility = ["//:__subpackages__"], ) -bzl_library( - name = "pip_repository_bzl", - srcs = ["pip_repository.bzl"], - deps = [ - "//python/private/pypi:group_library_bzl", - "//python/private/pypi:package_annotation_bzl", - "//python/private/pypi:pip_repository_bzl", - "//python/private/pypi:whl_library_bzl", - ], -) - -bzl_library( - name = "requirements_bzl", - srcs = ["requirements.bzl"], - deps = ["//python/private/pypi:pip_compile_bzl"], -) - filegroup( name = "distribution", srcs = glob(["**"]), @@ -51,3 +34,26 @@ exports_files( glob(["*.bzl"]), visibility = ["//docs:__pkg__"], ) + +bzl_library( + name = "pip_repository", + srcs = ["pip_repository.bzl"], + deps = [ + "//python/private/pypi:package_annotation", + "//python/private/pypi:pip_repository", + "//python/private/pypi:whl_config_repo", + "//python/private/pypi:whl_library", + ], +) + +bzl_library( + name = "requirements", + srcs = ["requirements.bzl"], + deps = ["//python/private/pypi:pip_compile"], +) + +bzl_library( + name = "requirements_parser", + srcs = ["requirements_parser.bzl"], + deps = ["//python/private/pypi:parse_requirements_txt"], +) diff --git a/python/pip_install/pip_repository.bzl b/python/pip_install/pip_repository.bzl index 18deee1993..f9c3c9fb56 100644 --- a/python/pip_install/pip_repository.bzl +++ b/python/pip_install/pip_repository.bzl @@ -14,13 +14,14 @@ "" -load("//python/private/pypi:group_library.bzl", _group_library = "group_library") load("//python/private/pypi:package_annotation.bzl", _package_annotation = "package_annotation") load("//python/private/pypi:pip_repository.bzl", _pip_repository = "pip_repository") +load("//python/private/pypi:whl_config_repo.bzl", _whl_config_repo = "whl_config_repo") load("//python/private/pypi:whl_library.bzl", _whl_library = "whl_library") # Re-exports for backwards compatibility -group_library = _group_library +group_library = _whl_config_repo pip_repository = _pip_repository whl_library = _whl_library +whl_config_repo = _whl_config_repo package_annotation = _package_annotation diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 9772089e97..0b0dc97f66 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -13,33 +13,46 @@ # limitations under the License. load("@bazel_skylib//:bzl_library.bzl", "bzl_library") -load("@bazel_skylib//rules:common_settings.bzl", "bool_setting") +load("@bazel_skylib//rules:common_settings.bzl", "bool_flag") load("//python:py_binary.bzl", "py_binary") load("//python:py_library.bzl", "py_library") -load("//python:versions.bzl", "print_toolchains_checksums") +load(":bazel_config_mode.bzl", "bazel_config_mode") load(":py_exec_tools_toolchain.bzl", "current_interpreter_executable") -load(":sentinel.bzl", "sentinel") -load(":stamp.bzl", "stamp_build_setting") +load(":py_interpreter_program.bzl", "py_interpreter_program") +load(":sentinel_impl.bzl", "sentinel") +load(":stamp_impl.bzl", "stamp_build_setting") +load(":uncachable_version_file.bzl", "define_uncachable_version_file") +load(":visibility.bzl", "NOT_ACTUALLY_PUBLIC") package( - default_visibility = ["//:__subpackages__"], + default_visibility = [ + "//:__subpackages__", + ], ) licenses(["notice"]) +exports_files([ + "runtime_env_toolchain_interpreter.sh", + "runtimes_manifest.txt", + "runtimes_manifest_workspace.bzl", +]) + filegroup( name = "distribution", srcs = glob(["**"]) + [ - "//python/private/proto:distribution", + "//python/private/api:distribution", + "//python/private/cc:distribution", "//python/private/pypi:distribution", "//python/private/whl_filegroup:distribution", + "//python/private/zipapp:distribution", "//tools/build_defs/python/private:distribution", ], visibility = ["//python:__pkg__"], ) filegroup( - name = "coverage_deps", + name = "coverage_deps_filegroup", srcs = ["coverage_deps.bzl"], visibility = ["//tools/private/update_deps:__pkg__"], ) @@ -51,734 +64,974 @@ filegroup( visibility = ["//python:__pkg__"], ) +alias( + name = "build_data_writer", + actual = select({ + "@platforms//os:windows": ":build_data_writer.ps1", + "//conditions:default": ":build_data_writer.sh", + }), + # Not actually public. Only public because it's an implicit dependency of + # rules built via py_binary_rule_builder() / py_test_rule_builder(). + visibility = ["//visibility:public"], +) + +define_uncachable_version_file( + name = "uncachable_version_file", + # Not actually public. Only public because it's an implicit dependency of + # rules built via py_binary_rule_builder() / py_test_rule_builder(). + visibility = ["//visibility:public"], +) + +# Needed to define bzl_library targets for docgen. (We don't define the +# bzl_library target here because it'd give our users a transitive dependency +# on Skylib.) +exports_files( + [ + "coverage.patch", + "py_package.bzl", + "py_wheel.bzl", + "version.bzl", + "reexports.bzl", + "stamp_impl.bzl", + "util.bzl", + ], + visibility = ["//:__subpackages__"], +) + +exports_files( + ["python_bootstrap_template.txt"], + # Not actually public. Only public because it's an implicit dependency of + # py_runtime. + visibility = NOT_ACTUALLY_PUBLIC, +) + +filegroup( + name = "stage1_bootstrap_template", + srcs = ["stage1_bootstrap_template.sh"], + # Not actually public. Only public because it's an implicit dependency of + # py_runtime. + visibility = NOT_ACTUALLY_PUBLIC, +) + +filegroup( + name = "stage2_bootstrap_template", + srcs = ["stage2_bootstrap_template.py"], + # Not actually public. Only public because it's an implicit dependency of + # py_runtime. + visibility = NOT_ACTUALLY_PUBLIC, +) + +filegroup( + name = "site_init_template", + srcs = ["site_init_template.py"], + # Not actually public. Only public because it's an implicit dependency of + # py_runtime. + visibility = NOT_ACTUALLY_PUBLIC, +) + +# NOTE: Windows builds don't use this bootstrap. Instead, a native Windows +# program locates some Python exe and runs `python.exe foo.zip` which +# runs the __main__.py in the zip file. +alias( + name = "bootstrap_template", + actual = select({ + ":is_script_bootstrap_enabled": "stage1_bootstrap_template.sh", + "//conditions:default": "python_bootstrap_template.txt", + }), + # Not actually public. Only public because it's an implicit dependency of + # py_runtime. + visibility = NOT_ACTUALLY_PUBLIC, +) + +# Used to determine the use of `--stamp` in Starlark rules +stamp_build_setting(name = "stamp") + +config_setting( + name = "is_script_bootstrap_enabled", + flag_values = { + "//python/config_settings:bootstrap_impl": "script", + }, +) + +config_setting( + name = "is_bazel_config_mode_target", + flag_values = { + "//python/private:bazel_config_mode": "target", + }, +) + +alias( + name = "debugger_if_target_config", + actual = select({ + ":is_bazel_config_mode_target": "//python/config_settings:debugger", + "//conditions:default": "//python/private:empty", + }), + # Not actually public. Only public because it's an implicit dependency of + # rules built via py_binary_rule_builder() / py_test_rule_builder(). + visibility = ["//visibility:public"], +) + +bazel_config_mode(name = "bazel_config_mode") + +# This should only be set by analysis tests to expose additional metadata to +# aid testing, so a setting instead of a flag. +bool_flag( + name = "visible_for_testing", + build_setting_default = False, + # This is only because it is an implicit dependency by the toolchains. + visibility = NOT_ACTUALLY_PUBLIC, +) + +# Used for py_console_script_gen rule +py_binary( + name = "py_console_script_gen_py", + srcs = ["py_console_script_gen.py"], + main = "py_console_script_gen.py", + visibility = [ + "//visibility:public", + ], +) + +py_binary( + name = "py_wheel_dist", + srcs = ["py_wheel_dist.py"], + visibility = ["//visibility:public"], +) + +py_library( + name = "py_console_script_gen_lib", + srcs = ["py_console_script_gen.py"], + imports = ["../.."], + visibility = [ + "//tests/entry_points:__pkg__", + ], +) + +# The current toolchain's interpreter as an excutable, usable with +# executable=True attributes. +current_interpreter_executable( + name = "current_interpreter_executable", + # Not actually public. Only public because it's an implicit dependency of + # py_exec_tools_toolchain. + visibility = NOT_ACTUALLY_PUBLIC, +) + +py_library( + name = "empty", + # Not actually public. Only public because it's the resolved default of + # debugger_if_target_config, an implicit dependency of rules built via + # py_binary_rule_builder() / py_test_rule_builder(). + visibility = ["//visibility:public"], +) + +sentinel( + name = "sentinel", + # Not actually public. Only public because it's the resolved default of + # uncachable_version_file, an implicit dependency of rules built via + # py_binary_rule_builder() / py_test_rule_builder(). + visibility = ["//visibility:public"], +) + +py_binary( + name = "sync_runtimes_manifest_workspace", + srcs = ["tools/sync_runtimes_manifest_workspace.py"], + visibility = ["//:__subpackages__"], +) + +# Tool used by py_test's validation action to statically check that the main +# module actually runs tests. See the validate_test_main config setting. +py_interpreter_program( + name = "py_test_main_validator", + main = "py_test_main_validator.py", + # Not actually public. Only public because it's an implicit dependency of + # the py_test rule. + visibility = NOT_ACTUALLY_PUBLIC, +) + +py_library( + name = "py_test_main_validator_lib", + srcs = ["py_test_main_validator.py"], + imports = ["../.."], + visibility = [ + "//tests/validate_test_main:__pkg__", + ], +) + bzl_library( - name = "attributes_bzl", + name = "attr_builders", + srcs = ["attr_builders.bzl"], + deps = [ + ":builders_util", + "@bazel_skylib//lib:types", + ], +) + +bzl_library( + name = "attributes", srcs = ["attributes.bzl"], deps = [ - ":common_bzl", - ":enum_bzl", - ":flags_bzl", - ":py_info_bzl", - ":py_internal_bzl", - ":reexports_bzl", - ":rules_cc_srcs_bzl", - ":semantics_bzl", + ":attr_builders", + ":common_labels", + ":enum", + ":flags", + ":py_info", + ":reexports", + ":rule_builders", + "@bazel_skylib//lib:dicts", "@bazel_skylib//rules:common_settings", + "@rules_cc//cc/common", ], ) bzl_library( - name = "auth_bzl", + name = "auth", srcs = ["auth.bzl"], - deps = [":bazel_tools_bzl"], + deps = ["//python/private:bazel_tools"], ) +# @bazel_tools can't define bzl_library itself, so we just put a wrapper around it. +# Keep this target because Gazelle does not automatically generate +# bzl_library targets for external repositories like @bazel_tools. +# keep bzl_library( - name = "runtime_env_toolchain_bzl", - srcs = ["runtime_env_toolchain.bzl"], - deps = [ - ":py_exec_tools_toolchain_bzl", - ":toolchain_types_bzl", - "//python:py_runtime_bzl", - "//python:py_runtime_pair_bzl", + name = "bazel_tools", + srcs = [ + # This set of sources is overly broad, but it's the only public + # target available across Bazel versions that has all the necessary + # sources. + "@bazel_tools//tools:bzl_srcs", ], ) bzl_library( - name = "builders_bzl", + name = "builders", srcs = ["builders.bzl"], - deps = [ - "@bazel_skylib//lib:types", - ], + deps = ["@bazel_skylib//lib:types"], ) bzl_library( - name = "bzlmod_enabled_bzl", - srcs = ["bzlmod_enabled.bzl"], + name = "builders_util", + srcs = ["builders_util.bzl"], + deps = [ + ":bzlmod_enabled", + "@bazel_skylib//lib:types", + ], ) bzl_library( - name = "cc_helper_bzl", + name = "cc_helper", srcs = ["cc_helper.bzl"], - deps = [":py_internal_bzl"], + deps = [":py_internal"], ) bzl_library( - name = "common_bazel_bzl", - srcs = ["common_bazel.bzl"], + name = "common", + srcs = ["common.bzl"], deps = [ - ":attributes_bzl", - ":common_bzl", - ":py_cc_link_params_info_bzl", - ":py_internal_bzl", - ":py_interpreter_program_bzl", - ":toolchain_types_bzl", + ":builders", + ":cc_helper", + ":py_cc_link_params_info", + ":py_info", + ":py_internal", + ":py_interpreter_program", + ":reexports", + ":toolchain_types", "@bazel_skylib//lib:paths", + "@rules_cc//cc/common", + "@rules_python_internal//:rules_python_config", ], ) bzl_library( - name = "common_bzl", - srcs = ["common.bzl"], + name = "config_settings", + srcs = ["config_settings.bzl"], deps = [ - ":cc_helper_bzl", - ":py_info_bzl", - ":py_internal_bzl", - ":reexports_bzl", - ":rules_cc_srcs_bzl", - ":semantics_bzl", + ":text_util", + ":version", + ":visibility", + "@bazel_skylib//lib:selects", + "@bazel_skylib//rules:common_settings", ], ) bzl_library( - name = "config_settings_bzl", - srcs = ["config_settings.bzl"], + name = "coverage_deps", + srcs = ["coverage_deps.bzl"], deps = [ - ":semver_bzl", - "@bazel_skylib//lib:selects", - "@bazel_skylib//rules:common_settings", + ":version_label", + "//python/private:bazel_tools", ], ) bzl_library( - name = "coverage_deps_bzl", - srcs = ["coverage_deps.bzl"], + name = "current_py_cc_headers", + srcs = ["current_py_cc_headers.bzl"], deps = [ - ":bazel_tools_bzl", - ":version_label_bzl", + ":toolchain_types", + "@rules_cc//cc/common", ], ) bzl_library( - name = "enum_bzl", - srcs = ["enum.bzl"], + name = "current_py_cc_libs", + srcs = ["current_py_cc_libs.bzl"], + deps = ["@rules_cc//cc/common"], ) bzl_library( - name = "envsubst_bzl", - srcs = ["envsubst.bzl"], + name = "deprecation", + srcs = ["deprecation.bzl"], + deps = ["@rules_python_internal//:rules_python_config"], ) bzl_library( - name = "flags_bzl", + name = "flags", srcs = ["flags.bzl"], deps = [ - ":enum_bzl", + ":enum", "@bazel_skylib//rules:common_settings", ], ) bzl_library( - name = "full_version_bzl", - srcs = ["full_version.bzl"], -) - -bzl_library( - name = "glob_excludes_bzl", - srcs = ["glob_excludes.bzl"], - deps = [":util_bzl"], + name = "hermetic_runtime_repo_setup", + srcs = ["hermetic_runtime_repo_setup.bzl"], + deps = [ + ":py_exec_tools_toolchain", + ":version", + "//python:py_runtime", + "//python:py_runtime_pair", + "//python/cc:py_cc_toolchain", + "@rules_cc//cc:core_rules", + ], ) bzl_library( - name = "internal_config_repo_bzl", + name = "internal_config_repo", srcs = ["internal_config_repo.bzl"], - deps = [":bzlmod_enabled_bzl"], -) - -bzl_library( - name = "is_standalone_interpreter_bzl", - srcs = ["is_standalone_interpreter.bzl"], deps = [ - ":repo_utils_bzl", + ":repo_utils", + ":text_util", ], ) bzl_library( - name = "normalize_name_bzl", - srcs = ["normalize_name.bzl"], + name = "interpreter", + srcs = ["interpreter.bzl"], + deps = [ + ":common", + ":sentinel_impl", + ":toolchain_types", + "//python:py_runtime_info", + "@bazel_skylib//lib:paths", + ], ) bzl_library( - name = "python_bzl", - srcs = ["python.bzl"], - deps = [ - ":full_version_bzl", - ":python_register_toolchains_bzl", - ":pythons_hub_bzl", - ":repo_utils_bzl", - ":semver_bzl", - ":toolchains_repo_bzl", - ":util_bzl", - "@bazel_features//:features", - ], + name = "is_standalone_interpreter", + srcs = ["is_standalone_interpreter.bzl"], + deps = [":repo_utils"], ) bzl_library( - name = "python_register_toolchains_bzl", - srcs = ["python_register_toolchains.bzl"], + name = "local_runtime_repo", + srcs = ["local_runtime_repo.bzl"], deps = [ - ":auth_bzl", - ":bazel_tools_bzl", - ":coverage_deps_bzl", - ":full_version_bzl", - ":internal_config_repo_bzl", - ":python_repository_bzl", - ":toolchains_repo_bzl", - "//python:versions_bzl", - "//python/private/pypi:deps_bzl", + ":enum", + ":repo_utils", ], ) bzl_library( - name = "python_repository_bzl", - srcs = ["python_repository.bzl"], + name = "local_runtime_repo_setup", + srcs = ["local_runtime_repo_setup.bzl"], deps = [ - ":auth_bzl", - ":repo_utils_bzl", - ":text_util_bzl", - "//python:versions_bzl", + "@bazel_skylib//lib:selects", + "@rules_cc//cc:core_rules", + "@rules_python//python:py_runtime", + "@rules_python//python:py_runtime_pair", + "@rules_python//python/cc:py_cc_toolchain", + "@rules_python//python/private:py_exec_tools_toolchain", ], ) bzl_library( - name = "python_register_multi_toolchains_bzl", - srcs = ["python_register_multi_toolchains.bzl"], + name = "local_runtime_toolchains_repo", + srcs = ["local_runtime_toolchains_repo.bzl"], deps = [ - ":python_register_toolchains_bzl", - ":toolchains_repo_bzl", - "//python:versions_bzl", + ":repo_utils", + ":text_util", ], ) bzl_library( - name = "pythons_hub_bzl", - srcs = ["pythons_hub.bzl"], + name = "precompile", + srcs = ["precompile.bzl"], deps = [ - ":py_toolchain_suite_bzl", - ":text_util_bzl", - "//python:versions_bzl", + ":attributes", + ":flags", + ":py_interpreter_program", + ":toolchain_types", + "@bazel_skylib//rules:common_settings", ], ) bzl_library( - name = "py_binary_macro_bzl", + name = "py_binary_macro", srcs = ["py_binary_macro.bzl"], deps = [ - ":common_bzl", - ":py_binary_rule_bzl", + ":py_binary_rule", + ":py_executable", ], ) bzl_library( - name = "py_binary_rule_bzl", + name = "py_binary_rule", srcs = ["py_binary_rule.bzl"], deps = [ - ":attributes_bzl", - ":py_executable_bazel_bzl", - ":semantics_bzl", - "@bazel_skylib//lib:dicts", + ":attributes", + ":py_executable", ], ) bzl_library( - name = "py_cc_link_params_info_bzl", + name = "py_cc_link_params_info", srcs = ["py_cc_link_params_info.bzl"], - deps = [ - ":rules_cc_srcs_bzl", - ":util_bzl", - ], + deps = ["@rules_cc//cc/common"], ) bzl_library( - name = "py_cc_toolchain_macro_bzl", + name = "py_cc_toolchain_macro", srcs = ["py_cc_toolchain_macro.bzl"], deps = [ - ":py_cc_toolchain_rule_bzl", + ":py_cc_toolchain_rule", + ":util", + "//python/private/pypi:pep508_env", ], ) bzl_library( - name = "py_cc_toolchain_rule_bzl", + name = "py_cc_toolchain_rule", srcs = ["py_cc_toolchain_rule.bzl"], deps = [ - ":py_cc_toolchain_info_bzl", - ":rules_cc_srcs_bzl", - ":util_bzl", + ":common_labels", + ":flags", + ":py_cc_toolchain_info", + ":sentinel_impl", "@bazel_skylib//rules:common_settings", + "@rules_cc//cc/common", ], ) bzl_library( - name = "py_cc_toolchain_info_bzl", - srcs = ["py_cc_toolchain_info.bzl"], -) - -bzl_library( - name = "py_console_script_binary_bzl", - srcs = [ - "py_console_script_binary.bzl", - "py_console_script_gen.bzl", - ], - visibility = ["//python/entry_points:__pkg__"], + name = "py_console_script_binary", + srcs = ["py_console_script_binary.bzl"], deps = [ - "//python:py_binary_bzl", + ":py_console_script_gen", + "//python:py_binary", ], ) bzl_library( - name = "py_exec_tools_info_bzl", - srcs = ["py_exec_tools_info.bzl"], -) - -bzl_library( - name = "py_exec_tools_toolchain_bzl", + name = "py_exec_tools_toolchain", srcs = ["py_exec_tools_toolchain.bzl"], deps = [ - ":py_exec_tools_info_bzl", - ":sentinel_bzl", - ":toolchain_types_bzl", + ":common_labels", + ":py_exec_tools_info", + ":sentinel_impl", + ":toolchain_types", "@bazel_skylib//lib:paths", "@bazel_skylib//rules:common_settings", ], ) bzl_library( - name = "py_executable_bazel_bzl", - srcs = ["py_executable_bazel.bzl"], - deps = [ - ":attributes_bzl", - ":common_bazel_bzl", - ":common_bzl", - ":py_executable_bzl", - ":py_internal_bzl", - ":py_runtime_info_bzl", - ":semantics_bzl", - ], -) - -bzl_library( - name = "py_executable_bzl", + name = "py_executable", srcs = ["py_executable.bzl"], deps = [ - ":attributes_bzl", - ":cc_helper_bzl", - ":common_bzl", - ":flags_bzl", - ":py_cc_link_params_info_bzl", - ":py_executable_info_bzl", - ":py_info_bzl", - ":py_internal_bzl", - ":py_runtime_info_bzl", - ":rules_cc_srcs_bzl", - ":toolchain_types_bzl", + ":attr_builders", + ":attributes", + ":builders", + ":cc_helper", + ":common", + ":common_labels", + ":flags", + ":precompile", + ":py_cc_link_params_info", + ":py_executable_info", + ":py_info", + ":py_internal", + ":py_runtime_info", + ":reexports", + ":rule_builders", + ":toolchain_types", + ":transition_labels", + ":venv_runfiles", "@bazel_skylib//lib:dicts", + "@bazel_skylib//lib:paths", "@bazel_skylib//lib:structs", "@bazel_skylib//rules:common_settings", + "@rules_cc//cc/common", + "@rules_python_internal//:rules_python_config", ], ) bzl_library( - name = "py_executable_info_bzl", - srcs = ["py_executable_info.bzl"], -) - -bzl_library( - name = "py_info_bzl", + name = "py_info", srcs = ["py_info.bzl"], deps = [ - ":builders_bzl", - ":reexports_bzl", - ":util_bzl", - "@rules_python_internal//:rules_python_config_bzl", + ":builders", + ":reexports", ], ) bzl_library( - name = "py_internal_bzl", + name = "py_internal", srcs = ["py_internal.bzl"], - deps = ["@rules_python_internal//:py_internal_bzl"], + deps = ["//tools/build_defs/python/private:py_internal_renamed"], ) bzl_library( - name = "py_interpreter_program_bzl", + name = "py_interpreter_program", srcs = ["py_interpreter_program.bzl"], - deps = ["@bazel_skylib//rules:common_settings"], + deps = [ + ":sentinel_impl", + "@bazel_skylib//rules:common_settings", + ], ) bzl_library( - name = "py_library_bzl", + name = "py_library", srcs = ["py_library.bzl"], deps = [ - ":attributes_bzl", - ":common_bzl", - ":flags_bzl", - ":py_cc_link_params_info_bzl", - ":py_internal_bzl", - ":toolchain_types_bzl", + ":attr_builders", + ":attributes", + ":builders", + ":common", + ":common_labels", + ":flags", + ":normalize_name", + ":precompile", + ":py_cc_link_params_info", + ":py_info", + ":reexports", + ":rule_builders", + ":toolchain_types", + ":venv_runfiles", + ":version", "@bazel_skylib//lib:dicts", + "@bazel_skylib//lib:paths", "@bazel_skylib//rules:common_settings", ], ) bzl_library( - name = "py_library_macro_bzl", + name = "py_library_macro", srcs = ["py_library_macro.bzl"], - deps = [":py_library_rule_bzl"], + deps = [":py_library_rule"], ) bzl_library( - name = "py_library_rule_bzl", + name = "py_library_rule", srcs = ["py_library_rule.bzl"], - deps = [ - ":common_bazel_bzl", - ":common_bzl", - ":py_library_bzl", - ], + deps = [":py_library"], ) bzl_library( - name = "py_package_bzl", + name = "py_package", srcs = ["py_package.bzl"], - visibility = ["//:__subpackages__"], deps = [ - ":builders_bzl", - ":py_info_bzl", + ":builders", + ":py_info", ], ) bzl_library( - name = "py_runtime_info_bzl", - srcs = ["py_runtime_info.bzl"], - deps = [":util_bzl"], -) - -bzl_library( - name = "py_repositories_bzl", + name = "py_repositories", srcs = ["py_repositories.bzl"], deps = [ - ":bazel_tools_bzl", - ":internal_config_repo_bzl", - ":pythons_hub_bzl", - "//python:versions_bzl", - "//python/private/pypi:deps_bzl", + ":internal_config_repo", + ":pythons_hub", + "//python:versions", + "//python/private:bazel_tools", + "//python/private/pypi:deps", ], ) bzl_library( - name = "py_runtime_macro_bzl", + name = "py_runtime_macro", srcs = ["py_runtime_macro.bzl"], - deps = [":py_runtime_rule_bzl"], + deps = [":py_runtime_rule"], ) bzl_library( - name = "py_runtime_rule_bzl", - srcs = ["py_runtime_rule.bzl"], - deps = [ - ":attributes_bzl", - ":flags_bzl", - ":py_internal_bzl", - ":py_runtime_info_bzl", - ":reexports_bzl", - ":util_bzl", - "@bazel_skylib//lib:dicts", - "@bazel_skylib//lib:paths", - "@bazel_skylib//rules:common_settings", - ], + name = "py_runtime_pair_macro", + srcs = ["py_runtime_pair_macro.bzl"], + deps = [":py_runtime_pair_rule"], ) bzl_library( - name = "py_runtime_pair_macro_bzl", - srcs = ["py_runtime_pair_macro.bzl"], - visibility = ["//:__subpackages__"], - deps = [":py_runtime_pair_rule_bzl"], + name = "py_runtime_pair_rule", + srcs = ["py_runtime_pair_rule.bzl"], + deps = [ + ":common_labels", + ":reexports", + "//python:py_runtime_info", + "@bazel_skylib//rules:common_settings", + ], ) bzl_library( - name = "py_runtime_pair_rule_bzl", - srcs = ["py_runtime_pair_rule.bzl"], + name = "py_runtime_rule", + srcs = ["py_runtime_rule.bzl"], deps = [ - "//python:py_runtime_bzl", - "//python:py_runtime_info_bzl", + ":common_labels", + ":flags", + ":py_internal", + ":py_runtime_info", + ":reexports", + ":version", + "@bazel_skylib//lib:dicts", + "@bazel_skylib//lib:paths", "@bazel_skylib//rules:common_settings", ], ) bzl_library( - name = "py_test_macro_bzl", + name = "py_test_macro", srcs = ["py_test_macro.bzl"], deps = [ - ":common_bazel_bzl", - ":py_test_rule_bzl", + ":py_executable", + ":py_test_rule", ], ) bzl_library( - name = "py_test_rule_bzl", + name = "py_test_rule", srcs = ["py_test_rule.bzl"], deps = [ - ":attributes_bzl", - ":common_bzl", - ":py_executable_bazel_bzl", - ":semantics_bzl", - "@bazel_skylib//lib:dicts", + ":attributes", + ":common", + ":py_executable", ], ) bzl_library( - name = "py_toolchain_suite_bzl", + name = "py_toolchain_suite", srcs = ["py_toolchain_suite.bzl"], deps = [ - ":config_settings_bzl", - ":text_util_bzl", - ":toolchain_types_bzl", + ":text_util", + ":toolchain_types", "@bazel_skylib//lib:selects", + "@platforms//host:constraints_lib", ], ) bzl_library( - name = "py_wheel_bzl", + name = "py_wheel", srcs = ["py_wheel.bzl"], - visibility = ["//:__subpackages__"], deps = [ - ":py_package_bzl", - ":stamp_bzl", + ":attributes", + ":py_info", + ":py_package", + ":rule_builders", + ":stamp_impl", + ":transition_labels", + ":version", ], ) bzl_library( - name = "reexports_bzl", - srcs = ["reexports.bzl"], - visibility = [ - "//:__subpackages__", + name = "python", + srcs = ["python.bzl"], + deps = [ + ":auth", + ":full_version", + ":pbs_manifest", + ":platform_info", + ":pyproject_utils", + ":python_register_toolchains", + ":pythons_hub", + ":repo_utils", + ":toolchains_repo", + ":version", + "//python:versions", + "@bazel_features//:features", ], +) + +bzl_library( + name = "python_register_multi_toolchains", + srcs = ["python_register_multi_toolchains.bzl"], deps = [ - ":bazel_tools_bzl", - "@rules_python_internal//:rules_python_config_bzl", + ":python_register_toolchains", + ":toolchains_repo", + "//python:versions", ], ) bzl_library( - name = "register_extension_info_bzl", - srcs = ["register_extension_info.bzl"], + name = "python_register_toolchains", + srcs = ["python_register_toolchains.bzl"], + deps = [ + ":coverage_deps", + ":full_version", + ":python_repository", + ":toolchains_repo", + "//python:versions", + ], ) bzl_library( - name = "repo_utils_bzl", - srcs = ["repo_utils.bzl"], + name = "python_repository", + srcs = ["python_repository.bzl"], + deps = [ + ":auth", + ":repo_utils", + ":text_util", + "//python:versions", + ], ) bzl_library( - name = "semver_bzl", - srcs = ["semver.bzl"], + name = "pythons_hub", + srcs = ["pythons_hub.bzl"], + deps = [ + ":pbs_manifest", + ":text_util", + ":toolchains_repo", + "//python:versions", + ], ) bzl_library( - name = "sentinel_bzl", - srcs = ["sentinel.bzl"], + name = "reexports", + srcs = ["reexports.bzl"], + deps = ["@rules_python_internal//:rules_python_config"], ) bzl_library( - name = "stamp_bzl", - srcs = ["stamp.bzl"], - visibility = ["//:__subpackages__"], + name = "repl", + srcs = ["repl.bzl"], + deps = ["//python:py_binary"], ) bzl_library( - name = "text_util_bzl", - srcs = ["text_util.bzl"], + name = "rule_builders", + srcs = ["rule_builders.bzl"], + deps = [ + ":builders_util", + "@bazel_skylib//lib:types", + ], ) +# Keep this target because Gazelle does not automatically generate or +# maintain composite bzl_library targets. We need this to group external +# dependency sources. +# keep bzl_library( - name = "toolchains_repo_bzl", - srcs = ["toolchains_repo.bzl"], + name = "rules_cc_srcs", + srcs = [ + # rules_cc 0.0.13 and earlier load cc_proto_libary (and thus protobuf@), + # but their bzl srcs targets don't transitively refer to protobuf. + "@com_google_protobuf//:bzl_srcs", + # NOTE: As of rules_cc 0.10, cc:bzl_srcs no longer contains + # everything and sub-targets must be used instead + "@rules_cc//cc:bzl_srcs", + "@rules_cc//cc/common", + "@rules_cc//cc/toolchains:toolchain_rules", + ], deps = [ - ":repo_utils_bzl", - ":text_util_bzl", - "//python:versions_bzl", + ":bazel_tools", + "@rules_cc//cc/common", ], ) bzl_library( - name = "toolchain_types_bzl", - srcs = ["toolchain_types.bzl"], + name = "runtime_env_repo", + srcs = ["runtime_env_repo.bzl"], + deps = [":repo_utils"], ) bzl_library( - name = "util_bzl", - srcs = ["util.bzl"], - visibility = [ - "//:__subpackages__", - ], + name = "runtime_env_toolchain", + srcs = ["runtime_env_toolchain.bzl"], deps = [ - "@bazel_skylib//lib:types", - "@rules_python_internal//:rules_python_config_bzl", + ":config_settings", + ":py_exec_tools_toolchain", + ":toolchain_types", + "//python:py_runtime", + "//python:py_runtime_pair", + "//python/cc:py_cc_toolchain", + "@rules_cc//cc:core_rules", ], ) bzl_library( - name = "version_label_bzl", - srcs = ["version_label.bzl"], + name = "toolchain_aliases", + srcs = ["toolchain_aliases.bzl"], + deps = [ + "//python:versions", + "@bazel_skylib//lib:selects", + ], ) -# @bazel_tools can't define bzl_library itself, so we just put a wrapper around it. bzl_library( - name = "bazel_tools_bzl", - srcs = [ - # This set of sources is overly broad, but it's the only public - # target available across Bazel versions that has all the necessary - # sources. - "@bazel_tools//tools:bzl_srcs", + name = "toolchains_repo", + srcs = ["toolchains_repo.bzl"], + deps = [ + ":repo_utils", + ":text_util", + "//python:versions", ], ) bzl_library( - name = "rules_cc_srcs_bzl", - srcs = [ - # rules_cc 0.0.13 and earlier load cc_proto_libary (and thus protobuf@), - # but their bzl srcs targets don't transitively refer to protobuf. - "@com_google_protobuf//:bzl_srcs", - # NOTE: As of rules_cc 0.10, cc:bzl_srcs no longer contains - # everything and sub-targets must be used instead - "@rules_cc//cc:bzl_srcs", - "@rules_cc//cc/common", - "@rules_cc//cc/toolchains:toolchain_rules", + name = "transition_labels", + srcs = ["transition_labels.bzl"], + deps = [ + ":common_labels", + "@bazel_skylib//lib:collections", + "@rules_python_internal//:extra_transition_settings", ], +) + +bzl_library( + name = "util", + srcs = ["util.bzl"], deps = [ - ":bazel_tools_bzl", - "@rules_cc//cc/common", + ":py_internal", + "@bazel_skylib//lib:types", ], ) bzl_library( - name = "semantics_bzl", - srcs = ["semantics.bzl"], + name = "venv_runfiles", + srcs = ["venv_runfiles.bzl"], + deps = [ + ":common", + ":py_info", + ":util", + "@bazel_skylib//lib:paths", + ], ) -# Needed to define bzl_library targets for docgen. (We don't define the -# bzl_library target here because it'd give our users a transitive dependency -# on Skylib.) -exports_files( - [ - "coverage.patch", - "repack_whl.py", - "py_package.bzl", - "py_wheel.bzl", - "py_wheel_normalize_pep440.bzl", - "reexports.bzl", - "stamp.bzl", - "util.bzl", +bzl_library( + name = "pyproject_utils", + srcs = ["pyproject_utils.bzl"], + deps = [ + ":version", + "@toml.bzl//:toml", ], - visibility = ["//:__subpackages__"], ) -exports_files( - ["python_bootstrap_template.txt"], - # Not actually public. Only public because it's an implicit dependency of - # py_runtime. - visibility = ["//visibility:public"], +bzl_library( + name = "stamp_impl", + srcs = ["stamp_impl.bzl"], + deps = [":visibility"], ) -filegroup( - name = "stage1_bootstrap_template", - srcs = ["stage1_bootstrap_template.sh"], - # Not actually public. Only public because it's an implicit dependency of - # py_runtime. - visibility = ["//visibility:public"], +bzl_library( + name = "bzlmod_enabled", + srcs = ["bzlmod_enabled.bzl"], ) -filegroup( - name = "stage2_bootstrap_template", - srcs = ["stage2_bootstrap_template.py"], - # Not actually public. Only public because it's an implicit dependency of - # py_runtime. - visibility = ["//visibility:public"], +bzl_library( + name = "common_labels", + srcs = ["common_labels.bzl"], ) -filegroup( - name = "zip_main_template", - srcs = ["zip_main_template.py"], - # Not actually public. Only public because it's an implicit dependency of - # py_runtime. - visibility = ["//visibility:public"], +bzl_library( + name = "enum", + srcs = ["enum.bzl"], ) -filegroup( - name = "site_init_template", - srcs = ["site_init_template.py"], - # Not actually public. Only public because it's an implicit dependency of - # py_runtime. - visibility = ["//visibility:public"], +bzl_library( + name = "envsubst", + srcs = ["envsubst.bzl"], ) -# NOTE: Windows builds don't use this bootstrap. Instead, a native Windows -# program locates some Python exe and runs `python.exe foo.zip` which -# runs the __main__.py in the zip file. -alias( - name = "bootstrap_template", - actual = select({ - ":is_script_bootstrap_enabled": "stage1_bootstrap_template.sh", - "//conditions:default": "python_bootstrap_template.txt", - }), - # Not actually public. Only public because it's an implicit dependency of - # py_runtime. - visibility = ["//visibility:public"], +bzl_library( + name = "full_version", + srcs = ["full_version.bzl"], ) -# Used to determine the use of `--stamp` in Starlark rules -stamp_build_setting(name = "stamp") +bzl_library( + name = "normalize_name", + srcs = ["normalize_name.bzl"], +) -config_setting( - name = "is_script_bootstrap_enabled", - flag_values = { - "//python/config_settings:bootstrap_impl": "script", - }, +bzl_library( + name = "pbs_manifest", + srcs = ["pbs_manifest.bzl"], ) -# This should only be set by analysis tests to expose additional metadata to -# aid testing, so a setting instead of a flag. -bool_setting( - name = "visible_for_testing", - build_setting_default = False, - # This is only because it is an implicit dependency by the toolchains. - visibility = ["//visibility:public"], +bzl_library( + name = "platform_info", + srcs = ["platform_info.bzl"], ) -print_toolchains_checksums(name = "print_toolchains_checksums") +bzl_library( + name = "py_cc_toolchain_info", + srcs = ["py_cc_toolchain_info.bzl"], +) -# Used for py_console_script_gen rule -py_binary( - name = "py_console_script_gen_py", - srcs = ["py_console_script_gen.py"], - main = "py_console_script_gen.py", - visibility = [ - "//visibility:public", - ], +bzl_library( + name = "py_console_script_gen", + srcs = ["py_console_script_gen.bzl"], ) -py_binary( - name = "py_wheel_dist", - srcs = ["py_wheel_dist.py"], - visibility = ["//visibility:public"], +bzl_library( + name = "py_exec_tools_info", + srcs = ["py_exec_tools_info.bzl"], ) -py_library( - name = "py_console_script_gen_lib", - srcs = ["py_console_script_gen.py"], - imports = ["../.."], - visibility = [ - "//tests/entry_points:__pkg__", - ], +bzl_library( + name = "py_executable_info", + srcs = ["py_executable_info.bzl"], ) -# The current toolchain's interpreter as an excutable, usable with -# executable=True attributes. -current_interpreter_executable( - name = "current_interpreter_executable", - # Not actually public. Only public because it's an implicit dependency of - # py_exec_tools_toolchain. - visibility = ["//visibility:public"], +bzl_library( + name = "py_runtime_info", + srcs = ["py_runtime_info.bzl"], ) -sentinel( - name = "sentinel", +bzl_library( + name = "repo_utils", + srcs = ["repo_utils.bzl"], +) + +bzl_library( + name = "runtimes_manifest_workspace", + srcs = ["runtimes_manifest_workspace.bzl"], +) + +bzl_library( + name = "sentinel_impl", + srcs = ["sentinel_impl.bzl"], +) + +bzl_library( + name = "text_util", + srcs = ["text_util.bzl"], +) + +bzl_library( + name = "toolchain_types", + srcs = ["toolchain_types.bzl"], +) + +bzl_library( + name = "version", + srcs = ["version.bzl"], +) + +bzl_library( + name = "version_label", + srcs = ["version_label.bzl"], +) + +bzl_library( + name = "visibility", + srcs = ["visibility.bzl"], ) diff --git a/python/private/api/BUILD.bazel b/python/private/api/BUILD.bazel index 9e97dc2b59..cd7eda1bd2 100644 --- a/python/private/api/BUILD.bazel +++ b/python/private/api/BUILD.bazel @@ -13,31 +13,34 @@ # limitations under the License. load("@bazel_skylib//:bzl_library.bzl", "bzl_library") +load("//python/private:visibility.bzl", "NOT_ACTUALLY_PUBLIC") load(":py_common_api.bzl", "py_common_api") package( default_visibility = ["//:__subpackages__"], ) +filegroup( + name = "distribution", + srcs = glob(["**"]), +) + py_common_api( - name = "py_common_api", + name = "py_common_api_impl", # NOTE: Not actually public. Implicit dependency of public rules. - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) bzl_library( - name = "api_bzl", - srcs = ["api.bzl"], + name = "py_common_api", + srcs = ["py_common_api.bzl"], deps = [ - "//python/private:py_info_bzl", + ":api", + "//python/private:py_info", ], ) bzl_library( - name = "py_common_api_bzl", - srcs = ["py_common_api.bzl"], - deps = [ - ":api_bzl", - "//python/private:py_info_bzl", - ], + name = "api", + srcs = ["api.bzl"], ) diff --git a/python/private/api/api.bzl b/python/private/api/api.bzl index 06fb7294b9..2a1f79fdf5 100644 --- a/python/private/api/api.bzl +++ b/python/private/api/api.bzl @@ -13,7 +13,7 @@ # limitations under the License. """Implementation of py_api.""" -_PY_COMMON_API_LABEL = Label("//python/private/api:py_common_api") +_PY_COMMON_API_LABEL = Label("//python/private/api:py_common_api_impl") ApiImplInfo = provider( doc = "Provider to hold an API implementation", @@ -27,6 +27,17 @@ will depend on the target that is providing the API struct. }, ) +def _py_common_typedef(): + """Typedef for py_common. + + :::{field} API_ATTRS + :type: dict[str, Attribute] + + The attributes that rules must have for `py_common.get()` to work. + ::: + + """ + def _py_common_get(ctx): """Get the py_common API instance. @@ -45,6 +56,7 @@ def _py_common_get(ctx): return ctx.attr._py_common_api[ApiImplInfo].impl py_common = struct( + TYPEDEF = _py_common_typedef, get = _py_common_get, API_ATTRS = { "_py_common_api": attr.label( diff --git a/python/private/api/py_common_api.bzl b/python/private/api/py_common_api.bzl index 401b35973e..6fed245257 100644 --- a/python/private/api/py_common_api.bzl +++ b/python/private/api/py_common_api.bzl @@ -22,17 +22,40 @@ def _py_common_api_impl(ctx): py_common_api = rule( implementation = _py_common_api_impl, - doc = "Rule implementing py_common API.", + doc = "Internal Rule implementing py_common API.", ) +def _py_common_api_typedef(): + """The py_common API implementation. + + An instance of this object is obtained using {obj}`py_common.get()` + """ + def _merge_py_infos(transitive, *, direct = []): - builder = PyInfoBuilder() + """Merge PyInfo objects into a single PyInfo. + + This is a convenience wrapper around {obj}`PyInfoBuilder.merge_all`. For + more control over merging PyInfo objects, use {obj}`PyInfoBuilder`. + + Args: + transitive: {type}`list[PyInfo]` The PyInfo objects with info + considered indirectly provided by something (e.g. via + its deps attribute). + direct: {type}`list[PyInfo]` The PyInfo objects that are + considered directly provided by something (e.g. via + the srcs attribute). + + Returns: + {type}`PyInfo` A PyInfo containing the merged values. + """ + builder = PyInfoBuilder.new() builder.merge_all(transitive, direct = direct) return builder.build() # Exposed for doc generation, not directly used. # buildifier: disable=name-conventions PyCommonApi = struct( + TYPEDEF = _py_common_api_typedef, merge_py_infos = _merge_py_infos, - PyInfoBuilder = PyInfoBuilder, + PyInfoBuilder = PyInfoBuilder.new, ) diff --git a/python/private/attr_builders.bzl b/python/private/attr_builders.bzl new file mode 100644 index 0000000000..8e8a690b8f --- /dev/null +++ b/python/private/attr_builders.bzl @@ -0,0 +1,1370 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Builders for creating attributes et al. + +:::{versionadded} 1.3.0 +::: +""" + +load("@bazel_skylib//lib:types.bzl", "types") +load( + ":builders_util.bzl", + "kwargs_getter", + "kwargs_getter_doc", + "kwargs_getter_mandatory", + "kwargs_set_default_doc", + "kwargs_set_default_ignore_none", + "kwargs_set_default_list", + "kwargs_set_default_mandatory", + "kwargs_setter", + "kwargs_setter_doc", + "kwargs_setter_mandatory", + "normalize_transition_in_out_values", + "to_label_maybe", +) + +# Various string constants for kwarg key names used across two or more +# functions, or in contexts with optional lookups (e.g. dict.dict, key in dict). +# Constants are used to reduce the chance of typos. +# NOTE: These keys are often part of function signature via `**kwargs`; they +# are not simply internal names. +_ALLOW_FILES = "allow_files" +_ALLOW_EMPTY = "allow_empty" +_ALLOW_SINGLE_FILE = "allow_single_file" +_DEFAULT = "default" +_INPUTS = "inputs" +_OUTPUTS = "outputs" +_CFG = "cfg" +_VALUES = "values" + +def _kwargs_set_default_allow_empty(kwargs): + existing = kwargs.get(_ALLOW_EMPTY) + if existing == None: + kwargs[_ALLOW_EMPTY] = True + +def _kwargs_getter_allow_empty(kwargs): + return kwargs_getter(kwargs, _ALLOW_EMPTY) + +def _kwargs_setter_allow_empty(kwargs): + return kwargs_setter(kwargs, _ALLOW_EMPTY) + +def _kwargs_set_default_allow_files(kwargs): + existing = kwargs.get(_ALLOW_FILES) + if existing == None: + kwargs[_ALLOW_FILES] = False + +def _kwargs_getter_allow_files(kwargs): + return kwargs_getter(kwargs, _ALLOW_FILES) + +def _kwargs_setter_allow_files(kwargs): + return kwargs_setter(kwargs, _ALLOW_FILES) + +def _kwargs_set_default_aspects(kwargs): + kwargs_set_default_list(kwargs, "aspects") + +def _kwargs_getter_aspects(kwargs): + return kwargs_getter(kwargs, "aspects") + +def _kwargs_getter_providers(kwargs): + return kwargs_getter(kwargs, "providers") + +def _kwargs_set_default_providers(kwargs): + kwargs_set_default_list(kwargs, "providers") + +def _common_label_build(self, attr_factory): + kwargs = dict(self.kwargs) + kwargs[_CFG] = self.cfg.build() + return attr_factory(**kwargs) + +def _WhichCfg_typedef(): + """Values returned by `AttrCfg.which_cfg` + + :::{field} TARGET + + Indicates the target config is set. + ::: + + :::{field} EXEC + + Indicates the exec config is set. + ::: + :::{field} NONE + + Indicates the "none" config is set (see {obj}`config.none`). + ::: + :::{field} IMPL + + Indicates a custom transition is set. + ::: + """ + +# buildifier: disable=name-conventions +_WhichCfg = struct( + TYPEDEF = _WhichCfg_typedef, + TARGET = "target", + EXEC = "exec", + NONE = "none", + IMPL = "impl", +) + +def _AttrCfg_typedef(): + """Builder for `cfg` arg of label attributes. + + :::{function} inputs() -> list[Label] + ::: + + :::{function} outputs() -> list[Label] + ::: + + :::{function} which_cfg() -> attrb.WhichCfg + + Tells which of the cfg modes is set. Will be one of: target, exec, none, + or implementation + ::: + """ + +_ATTR_CFG_WHICH = "which" +_ATTR_CFG_VALUE = "value" + +def _AttrCfg_new( + inputs = None, + outputs = None, + **kwargs): + """Creates a builder for the `attr.cfg` attribute. + + Args: + inputs: {type}`list[Label] | None` inputs to use for a transition + outputs: {type}`list[Label] | None` outputs to use for a transition + **kwargs: {type}`dict` Three different keyword args are supported. + The presence of a keyword arg will mark the respective mode + returned by `which_cfg`. + - `cfg`: string of either "target" or "exec" + - `exec_group`: string of an exec group name to use. None means + to use regular exec config (i.e. `config.exec()`) + - `implementation`: callable for a custom transition function. + + Returns: + {type}`AttrCfg` + """ + state = { + _INPUTS: inputs, + _OUTPUTS: outputs, + # Value depends on _ATTR_CFG_WHICH key. See associated setters. + _ATTR_CFG_VALUE: True, + # str: one of the _WhichCfg values + _ATTR_CFG_WHICH: _WhichCfg.TARGET, + } + kwargs_set_default_list(state, _INPUTS) + kwargs_set_default_list(state, _OUTPUTS) + normalize_transition_in_out_values("input", state[_INPUTS]) + normalize_transition_in_out_values("output", state[_OUTPUTS]) + + # buildifier: disable=uninitialized + self = struct( + # keep sorted + _state = state, + build = lambda: _AttrCfg_build(self), + exec_group = lambda: _AttrCfg_exec_group(self), + implementation = lambda: _AttrCfg_implementation(self), + inputs = kwargs_getter(state, _INPUTS), + none = lambda: _AttrCfg_none(self), + outputs = kwargs_getter(state, _OUTPUTS), + set_exec = lambda *a, **k: _AttrCfg_set_exec(self, *a, **k), + set_implementation = lambda *a, **k: _AttrCfg_set_implementation(self, *a, **k), + set_none = lambda: _AttrCfg_set_none(self), + set_target = lambda: _AttrCfg_set_target(self), + target = lambda: _AttrCfg_target(self), + which_cfg = kwargs_getter(state, _ATTR_CFG_WHICH), + ) + + # Only one of the three kwargs should be present. We just process anything + # we see because it's simpler. + if _CFG in kwargs: + cfg = kwargs.pop(_CFG) + if cfg == "target" or cfg == None: + self.set_target() + elif cfg == "exec": + self.set_exec() + elif cfg == "none": + self.set_none() + else: + self.set_implementation(cfg) + if "exec_group" in kwargs: + self.set_exec(kwargs.pop("exec_group")) + + if "implementation" in kwargs: + self.set_implementation(kwargs.pop("implementation")) + + return self + +def _AttrCfg_from_attr_kwargs_pop(attr_kwargs): + """Creates a `AttrCfg` from the cfg arg passed to an attribute bulider. + + Args: + attr_kwargs: dict of attr kwargs, it's "cfg" key will be removed. + + Returns: + {type}`AttrCfg` + """ + cfg = attr_kwargs.pop(_CFG, None) + if not types.is_dict(cfg): + kwargs = {_CFG: cfg} + else: + kwargs = cfg + return _AttrCfg_new(**kwargs) + +def _AttrCfg_implementation(self): + """Tells the custom transition function, if any and applicable. + + Returns: + {type}`callable | None` the custom transition function to use, if + any, or `None` if a different config mode is being used. + """ + return self._state[_ATTR_CFG_VALUE] if self._state[_ATTR_CFG_WHICH] == _WhichCfg.IMPL else None + +def _AttrCfg_none(self): + """Tells if none cfg (`config.none()`) is set. + + Returns: + {type}`bool` True if none cfg is set, False if not. + """ + return self._state[_ATTR_CFG_VALUE] if self._state[_ATTR_CFG_WHICH] == _WhichCfg.NONE else False + +def _AttrCfg_target(self): + """Tells if target cfg is set. + + Returns: + {type}`bool` True if target cfg is set, False if not. + """ + return self._state[_ATTR_CFG_VALUE] if self._state[_ATTR_CFG_WHICH] == _WhichCfg.TARGET else False + +def _AttrCfg_exec_group(self): + """Tells the exec group to use if an exec transition is being used. + + Args: + self: implicitly added. + + Returns: + {type}`str | None` the name of the exec group to use if any, + or `None` if `which_cfg` isn't `exec` + """ + return self._state[_ATTR_CFG_VALUE] if self._state[_ATTR_CFG_WHICH] == _WhichCfg.EXEC else None + +def _AttrCfg_set_implementation(self, impl): + """Sets a custom transition function to use. + + Args: + self: implicitly added. + impl: {type}`callable` a transition implementation function. + """ + self._state[_ATTR_CFG_WHICH] = _WhichCfg.IMPL + self._state[_ATTR_CFG_VALUE] = impl + +def _AttrCfg_set_none(self): + """Sets to use the "none" transition.""" + self._state[_ATTR_CFG_WHICH] = _WhichCfg.NONE + self._state[_ATTR_CFG_VALUE] = True + +def _AttrCfg_set_exec(self, exec_group = None): + """Sets to use an exec transition. + + Args: + self: implicitly added. + exec_group: {type}`str | None` the exec group name to use, if any. + """ + self._state[_ATTR_CFG_WHICH] = _WhichCfg.EXEC + self._state[_ATTR_CFG_VALUE] = exec_group + +def _AttrCfg_set_target(self): + """Sets to use the target transition.""" + self._state[_ATTR_CFG_WHICH] = _WhichCfg.TARGET + self._state[_ATTR_CFG_VALUE] = True + +def _AttrCfg_build(self): + which = self._state[_ATTR_CFG_WHICH] + value = self._state[_ATTR_CFG_VALUE] + if which == None: + return None + elif which == _WhichCfg.TARGET: + # config.target is Bazel 8+ + if hasattr(config, "target"): + return config.target() + else: + return "target" + elif which == _WhichCfg.EXEC: + return config.exec(value) + elif which == _WhichCfg.NONE: + return config.none() + elif types.is_function(value): + return transition( + implementation = value, + # Transitions only accept unique lists of strings. + inputs = {str(v): None for v in self._state[_INPUTS]}.keys(), + outputs = {str(v): None for v in self._state[_OUTPUTS]}.keys(), + ) + else: + # Otherwise, just assume the value is valid and whoever set it knows + # what they're doing. + return value + +# buildifier: disable=name-conventions +AttrCfg = struct( + TYPEDEF = _AttrCfg_typedef, + new = _AttrCfg_new, + # keep sorted + exec_group = _AttrCfg_exec_group, + implementation = _AttrCfg_implementation, + none = _AttrCfg_none, + set_exec = _AttrCfg_set_exec, + set_implementation = _AttrCfg_set_implementation, + set_none = _AttrCfg_set_none, + set_target = _AttrCfg_set_target, + target = _AttrCfg_target, +) + +def _Bool_typedef(): + """Builder for attr.bool. + + :::{function} build() -> attr.bool + ::: + + :::{function} default() -> bool. + ::: + + :::{function} doc() -> str + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} mandatory() -> bool + ::: + + :::{function} set_default(v: bool) + ::: + + :::{function} set_doc(v: str) + ::: + + :::{function} set_mandatory(v: bool) + ::: + + """ + +def _Bool_new(**kwargs): + """Creates a builder for `attr.bool`. + + Args: + **kwargs: Same kwargs as {obj}`attr.bool` + + Returns: + {type}`Bool` + """ + kwargs_set_default_ignore_none(kwargs, _DEFAULT, False) + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + + # buildifier: disable=uninitialized + self = struct( + # keep sorted + build = lambda: attr.bool(**self.kwargs), + default = kwargs_getter(kwargs, _DEFAULT), + doc = kwargs_getter_doc(kwargs), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + set_default = kwargs_setter(kwargs, _DEFAULT), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +# buildifier: disable=name-conventions +Bool = struct( + TYPEDEF = _Bool_typedef, + new = _Bool_new, +) + +def _Int_typedef(): + """Builder for attr.int. + + :::{function} build() -> attr.int + ::: + + :::{function} default() -> int + ::: + + :::{function} doc() -> str + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} mandatory() -> bool + ::: + + :::{function} values() -> list[int] + + The returned value is a mutable reference to the underlying list. + ::: + + :::{function} set_default(v: int) + ::: + + :::{function} set_doc(v: str) + ::: + + :::{function} set_mandatory(v: bool) + ::: + """ + +def _Int_new(**kwargs): + """Creates a builder for `attr.int`. + + Args: + **kwargs: Same kwargs as {obj}`attr.int` + + Returns: + {type}`Int` + """ + kwargs_set_default_ignore_none(kwargs, _DEFAULT, 0) + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + kwargs_set_default_list(kwargs, _VALUES) + + # buildifier: disable=uninitialized + self = struct( + build = lambda: attr.int(**self.kwargs), + default = kwargs_getter(kwargs, _DEFAULT), + doc = kwargs_getter_doc(kwargs), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + values = kwargs_getter(kwargs, _VALUES), + set_default = kwargs_setter(kwargs, _DEFAULT), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +# buildifier: disable=name-conventions +Int = struct( + TYPEDEF = _Int_typedef, + new = _Int_new, +) + +def _IntList_typedef(): + """Builder for attr.int_list. + + :::{function} allow_empty() -> bool + ::: + + :::{function} build() -> attr.int_list + ::: + + :::{function} default() -> list[int] + ::: + + :::{function} doc() -> str + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} mandatory() -> bool + ::: + + :::{function} set_allow_empty(v: bool) + ::: + + :::{function} set_doc(v: str) + ::: + + :::{function} set_mandatory(v: bool) + ::: + """ + +def _IntList_new(**kwargs): + """Creates a builder for `attr.int_list`. + + Args: + **kwargs: Same as {obj}`attr.int_list`. + + Returns: + {type}`IntList` + """ + kwargs_set_default_list(kwargs, _DEFAULT) + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + _kwargs_set_default_allow_empty(kwargs) + + # buildifier: disable=uninitialized + self = struct( + # keep sorted + allow_empty = _kwargs_getter_allow_empty(kwargs), + build = lambda: attr.int_list(**self.kwargs), + default = kwargs_getter(kwargs, _DEFAULT), + doc = kwargs_getter_doc(kwargs), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + set_allow_empty = _kwargs_setter_allow_empty(kwargs), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +# buildifier: disable=name-conventions +IntList = struct( + TYPEDEF = _IntList_typedef, + new = _IntList_new, +) + +def _Label_typedef(): + """Builder for `attr.label` objects. + + :::{function} allow_files() -> bool | list[str] | None + + Note that `allow_files` is mutually exclusive with `allow_single_file`. + Only one of the two can have a value set. + ::: + + :::{function} allow_single_file() -> bool | None + Note that `allow_single_file` is mutually exclusive with `allow_files`. + Only one of the two can have a value set. + ::: + + :::{function} aspects() -> list[aspect] + + The returned list is a mutable reference to the underlying list. + ::: + + :::{function} build() -> attr.label + ::: + + :::{field} cfg + :type: AttrCfg + ::: + + :::{function} default() -> str | label | configuration_field | None + ::: + + :::{function} doc() -> str + ::: + + :::{function} executable() -> bool + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} mandatory() -> bool + ::: + + + :::{function} providers() -> list[list[provider]] + The returned list is a mutable reference to the underlying list. + ::: + + :::{function} set_default(v: str | Label) + ::: + + :::{function} set_doc(v: str) + ::: + + :::{function} set_executable(v: bool) + ::: + + :::{function} set_mandatory(v: bool) + ::: + """ + +def _Label_new(**kwargs): + """Creates a builder for `attr.label`. + + Args: + **kwargs: The same as {obj}`attr.label`. + + Returns: + {type}`Label` + """ + kwargs_set_default_ignore_none(kwargs, "executable", False) + _kwargs_set_default_aspects(kwargs) + _kwargs_set_default_providers(kwargs) + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + + kwargs[_DEFAULT] = to_label_maybe(kwargs.get(_DEFAULT)) + + # buildifier: disable=uninitialized + self = struct( + # keep sorted + add_allow_files = lambda v: _Label_add_allow_files(self, v), + allow_files = _kwargs_getter_allow_files(kwargs), + allow_single_file = kwargs_getter(kwargs, _ALLOW_SINGLE_FILE), + aspects = _kwargs_getter_aspects(kwargs), + build = lambda: _common_label_build(self, attr.label), + cfg = _AttrCfg_from_attr_kwargs_pop(kwargs), + default = kwargs_getter(kwargs, _DEFAULT), + doc = kwargs_getter_doc(kwargs), + executable = kwargs_getter(kwargs, "executable"), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + providers = _kwargs_getter_providers(kwargs), + set_allow_files = lambda v: _Label_set_allow_files(self, v), + set_allow_single_file = lambda v: _Label_set_allow_single_file(self, v), + set_default = kwargs_setter(kwargs, _DEFAULT), + set_doc = kwargs_setter_doc(kwargs), + set_executable = kwargs_setter(kwargs, "executable"), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +def _Label_set_allow_files(self, v): + """Set the allow_files arg + + NOTE: Setting `allow_files` unsets `allow_single_file` + + Args: + self: implicitly added. + v: {type}`bool | list[str] | None` the value to set to. + If set to `None`, then `allow_files` is unset. + """ + if v == None: + self.kwargs.pop(_ALLOW_FILES, None) + else: + self.kwargs[_ALLOW_FILES] = v + self.kwargs.pop(_ALLOW_SINGLE_FILE, None) + +def _Label_add_allow_files(self, *values): + """Adds allowed file extensions + + NOTE: Add an allowed file extension unsets `allow_single_file` + + Args: + self: implicitly added. + *values: {type}`str` file extensions to allow (including dot) + """ + self.kwargs.pop(_ALLOW_SINGLE_FILE, None) + if not types.is_list(self.kwargs.get(_ALLOW_FILES)): + self.kwargs[_ALLOW_FILES] = [] + existing = self.kwargs[_ALLOW_FILES] + existing.extend([v for v in values if v not in existing]) + +def _Label_set_allow_single_file(self, v): + """Sets the allow_single_file arg. + + NOTE: Setting `allow_single_file` unsets `allow_file` + + Args: + self: implicitly added. + v: {type}`bool | None` the value to set to. + If set to `None`, then `allow_single_file` is unset. + """ + if v == None: + self.kwargs.pop(_ALLOW_SINGLE_FILE, None) + else: + self.kwargs[_ALLOW_SINGLE_FILE] = v + self.kwargs.pop(_ALLOW_FILES, None) + +# buildifier: disable=name-conventions +Label = struct( + TYPEDEF = _Label_typedef, + new = _Label_new, + set_allow_files = _Label_set_allow_files, + add_allow_files = _Label_add_allow_files, + set_allow_single_file = _Label_set_allow_single_file, +) + +def _LabelKeyedStringDict_typedef(): + """Builder for attr.label_keyed_string_dict. + + :::{function} aspects() -> list[aspect] + The returned list is a mutable reference to the underlying list. + ::: + + :::{function} allow_files() -> bool | list[str] + ::: + + :::{function} allow_empty() -> bool + ::: + + :::{field} cfg + :type: AttrCfg + ::: + + :::{function} default() -> dict[str | Label, str] | callable + ::: + + :::{function} doc() -> str + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} mandatory() -> bool + ::: + + :::{function} providers() -> list[provider | list[provider]] + + Returns a mutable reference to the underlying list. + ::: + + :::{function} set_mandatory(v: bool) + ::: + :::{function} set_allow_empty(v: bool) + ::: + :::{function} set_default(v: dict[str | Label, str] | callable) + ::: + :::{function} set_doc(v: str) + ::: + :::{function} set_allow_files(v: bool | list[str]) + ::: + """ + +def _LabelKeyedStringDict_new(**kwargs): + """Creates a builder for `attr.label_keyed_string_dict`. + + Args: + **kwargs: Same as {obj}`attr.label_keyed_string_dict`. + + Returns: + {type}`LabelKeyedStringDict` + """ + kwargs_set_default_ignore_none(kwargs, _DEFAULT, {}) + _kwargs_set_default_aspects(kwargs) + _kwargs_set_default_providers(kwargs) + _kwargs_set_default_allow_empty(kwargs) + _kwargs_set_default_allow_files(kwargs) + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + + # buildifier: disable=uninitialized + self = struct( + # keep sorted + add_allow_files = lambda *v: _LabelKeyedStringDict_add_allow_files(self, *v), + allow_empty = _kwargs_getter_allow_empty(kwargs), + allow_files = _kwargs_getter_allow_files(kwargs), + aspects = _kwargs_getter_aspects(kwargs), + build = lambda: _common_label_build(self, attr.label_keyed_string_dict), + cfg = _AttrCfg_from_attr_kwargs_pop(kwargs), + default = kwargs_getter(kwargs, _DEFAULT), + doc = kwargs_getter_doc(kwargs), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + providers = _kwargs_getter_providers(kwargs), + set_allow_empty = _kwargs_setter_allow_empty(kwargs), + set_allow_files = _kwargs_setter_allow_files(kwargs), + set_default = kwargs_setter(kwargs, _DEFAULT), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +def _LabelKeyedStringDict_add_allow_files(self, *values): + """Adds allowed file extensions + + Args: + self: implicitly added. + *values: {type}`str` file extensions to allow (including dot) + """ + if not types.is_list(self.kwargs.get(_ALLOW_FILES)): + self.kwargs[_ALLOW_FILES] = [] + existing = self.kwargs[_ALLOW_FILES] + existing.extend([v for v in values if v not in existing]) + +# buildifier: disable=name-conventions +LabelKeyedStringDict = struct( + TYPEDEF = _LabelKeyedStringDict_typedef, + new = _LabelKeyedStringDict_new, + add_allow_files = _LabelKeyedStringDict_add_allow_files, +) + +def _LabelList_typedef(): + """Builder for `attr.label_list` + + :::{function} aspects() -> list[aspect] + ::: + + :::{function} allow_files() -> bool | list[str] + ::: + + :::{function} allow_empty() -> bool + ::: + + :::{function} build() -> attr.label_list + ::: + + :::{field} cfg + :type: AttrCfg + ::: + + :::{function} default() -> list[str|Label] | configuration_field | callable + ::: + + :::{function} doc() -> str + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} mandatory() -> bool + ::: + + :::{function} providers() -> list[provider | list[provider]] + ::: + + :::{function} set_allow_empty(v: bool) + ::: + + :::{function} set_allow_files(v: bool | list[str]) + ::: + + :::{function} set_default(v: list[str|Label] | configuration_field | callable) + ::: + + :::{function} set_doc(v: str) + ::: + + :::{function} set_mandatory(v: bool) + ::: + """ + +def _LabelList_new(**kwargs): + """Creates a builder for `attr.label_list`. + + Args: + **kwargs: Same as {obj}`attr.label_list`. + + Returns: + {type}`LabelList` + """ + _kwargs_set_default_allow_empty(kwargs) + kwargs_set_default_mandatory(kwargs) + kwargs_set_default_doc(kwargs) + if kwargs.get(_ALLOW_FILES) == None: + kwargs[_ALLOW_FILES] = False + _kwargs_set_default_aspects(kwargs) + kwargs_set_default_list(kwargs, _DEFAULT) + _kwargs_set_default_providers(kwargs) + + # buildifier: disable=uninitialized + self = struct( + # keep sorted + allow_empty = _kwargs_getter_allow_empty(kwargs), + allow_files = _kwargs_getter_allow_files(kwargs), + aspects = _kwargs_getter_aspects(kwargs), + build = lambda: _common_label_build(self, attr.label_list), + cfg = _AttrCfg_from_attr_kwargs_pop(kwargs), + default = kwargs_getter(kwargs, _DEFAULT), + doc = kwargs_getter_doc(kwargs), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + providers = _kwargs_getter_providers(kwargs), + set_allow_empty = _kwargs_setter_allow_empty(kwargs), + set_allow_files = _kwargs_setter_allow_files(kwargs), + set_default = kwargs_setter(kwargs, _DEFAULT), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +# buildifier: disable=name-conventions +LabelList = struct( + TYPEDEF = _LabelList_typedef, + new = _LabelList_new, +) + +def _Output_typedef(): + """Builder for attr.output + + :::{function} build() -> attr.output + ::: + + :::{function} doc() -> str + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} mandatory() -> bool + ::: + + :::{function} set_doc(v: str) + ::: + + :::{function} set_mandatory(v: bool) + ::: + """ + +def _Output_new(**kwargs): + """Creates a builder for `attr.output`. + + Args: + **kwargs: Same as {obj}`attr.output`. + + Returns: + {type}`Output` + """ + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + + # buildifier: disable=uninitialized + self = struct( + # keep sorted + build = lambda: attr.output(**self.kwargs), + doc = kwargs_getter_doc(kwargs), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +# buildifier: disable=name-conventions +Output = struct( + TYPEDEF = _Output_typedef, + new = _Output_new, +) + +def _OutputList_typedef(): + """Builder for attr.output_list + + :::{function} allow_empty() -> bool + ::: + + :::{function} build() -> attr.output + ::: + + :::{function} doc() -> str + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} mandatory() -> bool + ::: + + :::{function} set_allow_empty(v: bool) + ::: + :::{function} set_doc(v: str) + ::: + :::{function} set_mandatory(v: bool) + ::: + """ + +def _OutputList_new(**kwargs): + """Creates a builder for `attr.output_list`. + + Args: + **kwargs: Same as {obj}`attr.output_list`. + + Returns: + {type}`OutputList` + """ + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + _kwargs_set_default_allow_empty(kwargs) + + # buildifier: disable=uninitialized + self = struct( + allow_empty = _kwargs_getter_allow_empty(kwargs), + build = lambda: attr.output_list(**self.kwargs), + doc = kwargs_getter_doc(kwargs), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + set_allow_empty = _kwargs_setter_allow_empty(kwargs), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +# buildifier: disable=name-conventions +OutputList = struct( + TYPEDEF = _OutputList_typedef, + new = _OutputList_new, +) + +def _String_typedef(): + """Builder for `attr.string` + + :::{function} build() -> attr.string + ::: + + :::{function} default() -> str | configuration_field + ::: + + :::{function} doc() -> str + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} mandatory() -> bool + ::: + + :::{function} values() -> list[str] + ::: + + :::{function} set_default(v: str | configuration_field) + ::: + + :::{function} set_doc(v: str) + ::: + + :::{function} set_mandatory(v: bool) + ::: + """ + +def _String_new(**kwargs): + """Creates a builder for `attr.string`. + + Args: + **kwargs: Same as {obj}`attr.string`. + + Returns: + {type}`String` + """ + kwargs_set_default_ignore_none(kwargs, _DEFAULT, "") + kwargs_set_default_list(kwargs, _VALUES) + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + + # buildifier: disable=uninitialized + self = struct( + default = kwargs_getter(kwargs, _DEFAULT), + doc = kwargs_getter_doc(kwargs), + mandatory = kwargs_getter_mandatory(kwargs), + build = lambda: attr.string(**self.kwargs), + kwargs = kwargs, + values = kwargs_getter(kwargs, _VALUES), + set_default = kwargs_setter(kwargs, _DEFAULT), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +# buildifier: disable=name-conventions +String = struct( + TYPEDEF = _String_typedef, + new = _String_new, +) + +def _StringDict_typedef(): + """Builder for `attr.string_dict` + + :::{function} default() -> dict[str, str] + ::: + + :::{function} doc() -> str + ::: + + :::{function} mandatory() -> bool + ::: + + :::{function} allow_empty() -> bool + ::: + + :::{function} build() -> attr.string_dict + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} set_doc(v: str) + ::: + :::{function} set_mandatory(v: bool) + ::: + :::{function} set_allow_empty(v: bool) + ::: + """ + +def _StringDict_new(**kwargs): + """Creates a builder for `attr.string_dict`. + + Args: + **kwargs: The same args as for `attr.string_dict`. + + Returns: + {type}`StringDict` + """ + kwargs_set_default_ignore_none(kwargs, _DEFAULT, {}) + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + _kwargs_set_default_allow_empty(kwargs) + + # buildifier: disable=uninitialized + self = struct( + allow_empty = _kwargs_getter_allow_empty(kwargs), + build = lambda: attr.string_dict(**self.kwargs), + default = kwargs_getter(kwargs, _DEFAULT), + doc = kwargs_getter_doc(kwargs), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + set_allow_empty = _kwargs_setter_allow_empty(kwargs), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +# buildifier: disable=name-conventions +StringDict = struct( + TYPEDEF = _StringDict_typedef, + new = _StringDict_new, +) + +def _StringKeyedLabelDict_typedef(): + """Builder for attr.string_keyed_label_dict. + + :::{function} allow_empty() -> bool + ::: + + :::{function} allow_files() -> bool | list[str] + ::: + + :::{function} aspects() -> list[aspect] + ::: + + :::{function} build() -> attr.string_list + ::: + + :::{field} cfg + :type: AttrCfg + ::: + + :::{function} default() -> dict[str, Label] | callable + ::: + + :::{function} doc() -> str + ::: + + :::{function} mandatory() -> bool + ::: + + :::{function} providers() -> list[list[provider]] + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} set_allow_empty(v: bool) + ::: + + :::{function} set_allow_files(v: bool | list[str]) + ::: + + :::{function} set_doc(v: str) + ::: + + :::{function} set_default(v: dict[str, Label] | callable) + ::: + + :::{function} set_mandatory(v: bool) + ::: + """ + +def _StringKeyedLabelDict_new(**kwargs): + """Creates a builder for `attr.string_keyed_label_dict`. + + Args: + **kwargs: Same as {obj}`attr.string_keyed_label_dict`. + + Returns: + {type}`StringKeyedLabelDict` + """ + kwargs_set_default_ignore_none(kwargs, _DEFAULT, {}) + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + _kwargs_set_default_allow_files(kwargs) + _kwargs_set_default_allow_empty(kwargs) + _kwargs_set_default_aspects(kwargs) + _kwargs_set_default_providers(kwargs) + + # buildifier: disable=uninitialized + self = struct( + allow_empty = _kwargs_getter_allow_empty(kwargs), + allow_files = _kwargs_getter_allow_files(kwargs), + build = lambda: _common_label_build(self, attr.string_keyed_label_dict), + cfg = _AttrCfg_from_attr_kwargs_pop(kwargs), + default = kwargs_getter(kwargs, _DEFAULT), + doc = kwargs_getter_doc(kwargs), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + set_allow_empty = _kwargs_setter_allow_empty(kwargs), + set_allow_files = _kwargs_setter_allow_files(kwargs), + set_default = kwargs_setter(kwargs, _DEFAULT), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + providers = _kwargs_getter_providers(kwargs), + aspects = _kwargs_getter_aspects(kwargs), + ) + return self + +# buildifier: disable=name-conventions +StringKeyedLabelDict = struct( + TYPEDEF = _StringKeyedLabelDict_typedef, + new = _StringKeyedLabelDict_new, +) + +def _StringList_typedef(): + """Builder for `attr.string_list` + + :::{function} allow_empty() -> bool + ::: + + :::{function} build() -> attr.string_list + ::: + + :::{field} default + :type: list[str] | configuration_field + ::: + + :::{function} doc() -> str + ::: + + :::{function} mandatory() -> bool + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} set_allow_empty(v: bool) + ::: + + :::{function} set_default(v: list[str] | configuration_field) + ::: + + :::{function} set_doc(v: str) + ::: + + :::{function} set_mandatory(v: bool) + ::: + """ + +def _StringList_new(**kwargs): + """Creates a builder for `attr.string_list`. + + Args: + **kwargs: Same as {obj}`attr.string_list`. + + Returns: + {type}`StringList` + """ + kwargs_set_default_ignore_none(kwargs, _DEFAULT, []) + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + _kwargs_set_default_allow_empty(kwargs) + + # buildifier: disable=uninitialized + self = struct( + allow_empty = _kwargs_getter_allow_empty(kwargs), + build = lambda: attr.string_list(**self.kwargs), + default = kwargs_getter(kwargs, _DEFAULT), + doc = kwargs_getter_doc(kwargs), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + set_allow_empty = _kwargs_setter_allow_empty(kwargs), + set_default = kwargs_setter(kwargs, _DEFAULT), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +# buildifier: disable=name-conventions +StringList = struct( + TYPEDEF = _StringList_typedef, + new = _StringList_new, +) + +def _StringListDict_typedef(): + """Builder for attr.string_list_dict. + + :::{function} allow_empty() -> bool + ::: + + :::{function} build() -> attr.string_list + ::: + + :::{function} default() -> dict[str, list[str]] + ::: + + :::{function} doc() -> str + ::: + + :::{function} mandatory() -> bool + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} set_allow_empty(v: bool) + ::: + + :::{function} set_doc(v: str) + ::: + + :::{function} set_mandatory(v: bool) + ::: + """ + +def _StringListDict_new(**kwargs): + """Creates a builder for `attr.string_list_dict`. + + Args: + **kwargs: Same as {obj}`attr.string_list_dict`. + + Returns: + {type}`StringListDict` + """ + kwargs_set_default_ignore_none(kwargs, _DEFAULT, {}) + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + _kwargs_set_default_allow_empty(kwargs) + + # buildifier: disable=uninitialized + self = struct( + allow_empty = _kwargs_getter_allow_empty(kwargs), + build = lambda: attr.string_list_dict(**self.kwargs), + default = kwargs_getter(kwargs, _DEFAULT), + doc = kwargs_getter_doc(kwargs), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + set_allow_empty = _kwargs_setter_allow_empty(kwargs), + set_default = kwargs_setter(kwargs, _DEFAULT), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +# buildifier: disable=name-conventions +StringListDict = struct( + TYPEDEF = _StringListDict_typedef, + new = _StringListDict_new, +) + +attrb = struct( + # keep sorted + Bool = _Bool_new, + Int = _Int_new, + IntList = _IntList_new, + Label = _Label_new, + LabelKeyedStringDict = _LabelKeyedStringDict_new, + LabelList = _LabelList_new, + Output = _Output_new, + OutputList = _OutputList_new, + String = _String_new, + StringDict = _StringDict_new, + StringKeyedLabelDict = _StringKeyedLabelDict_new, + StringList = _StringList_new, + StringListDict = _StringListDict_new, + WhichCfg = _WhichCfg, +) diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index e62abf9f71..5909d8872b 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -13,21 +13,16 @@ # limitations under the License. """Attributes for Python rules.""" +load("@bazel_skylib//lib:dicts.bzl", "dicts") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") -load(":common.bzl", "union_attrs") +load(":attr_builders.bzl", "attrb") +load(":common_labels.bzl", "labels") load(":enum.bzl", "enum") load(":flags.bzl", "PrecompileFlag", "PrecompileSourceRetentionFlag") load(":py_info.bzl", "PyInfo") -load(":py_internal.bzl", "py_internal") load(":reexports.bzl", "BuiltinPyInfo") -load( - ":semantics.bzl", - "DEPS_ATTR_ALLOW_RULES", - "SRCS_ATTR_ALLOW_FILES", -) - -_PackageSpecificationInfo = getattr(py_internal, "PackageSpecificationInfo", None) +load(":rule_builders.bzl", "ruleb") # Due to how the common exec_properties attribute works, rules must add exec # groups even if they don't actually use them. This is due to two interactions: @@ -41,12 +36,12 @@ _PackageSpecificationInfo = getattr(py_internal, "PackageSpecificationInfo", Non # NOTE: These are no-op/empty exec groups. If a rule *does* support an exec # group and needs custom settings, it should merge this dict with one that # overrides the supported key. -REQUIRED_EXEC_GROUPS = { +REQUIRED_EXEC_GROUP_BUILDERS = { # py_binary may invoke C++ linking, or py rules may be used in combination # with cc rules (e.g. within the same macro), so support that exec group. # This exec group is defined by rules_cc for the cc rules. - "cpp_link": exec_group(), - "py_precompile": exec_group(), + "cpp_link": lambda: ruleb.ExecGroup(), + "py_precompile": lambda: ruleb.ExecGroup(), } _STAMP_VALUES = [-1, 0, 1] @@ -139,59 +134,6 @@ PycCollectionAttr = enum( is_pyc_collection_enabled = _pyc_collection_attr_is_pyc_collection_enabled, ) -def create_stamp_attr(**kwargs): - return { - "stamp": attr.int( - values = _STAMP_VALUES, - doc = """ -Whether to encode build information into the binary. Possible values: - -* `stamp = 1`: Always stamp the build information into the binary, even in - `--nostamp` builds. **This setting should be avoided**, since it potentially kills - remote caching for the binary and any downstream actions that depend on it. -* `stamp = 0`: Always replace build information by constant values. This gives - good build result caching. -* `stamp = -1`: Embedding of build information is controlled by the - `--[no]stamp` flag. - -Stamped binaries are not rebuilt unless their dependencies change. - -WARNING: Stamping can harm build performance by reducing cache hits and should -be avoided if possible. -""", - **kwargs - ), - } - -def create_srcs_attr(*, mandatory): - return { - "srcs": attr.label_list( - # Google builds change the set of allowed files. - allow_files = SRCS_ATTR_ALLOW_FILES, - mandatory = mandatory, - # Necessary for --compile_one_dependency to work. - flags = ["DIRECT_COMPILE_TIME_INPUT"], - doc = """ -The list of Python source files that are processed to create the target. This -includes all your checked-in code and may include generated source files. The -`.py` files belong in `srcs` and library targets belong in `deps`. Other binary -files that may be needed at run time belong in `data`. -""", - ), - } - -SRCS_VERSION_ALL_VALUES = ["PY2", "PY2ONLY", "PY2AND3", "PY3", "PY3ONLY"] -SRCS_VERSION_NON_CONVERSION_VALUES = ["PY2AND3", "PY2ONLY", "PY3ONLY"] - -def create_srcs_version_attr(values): - return { - "srcs_version": attr.string( - default = "PY2AND3", - values = values, - doc = "Defunct, unused, does nothing.", - ), - } - def copy_common_binary_kwargs(kwargs): return { key: kwargs[key] @@ -206,17 +148,11 @@ def copy_common_test_kwargs(kwargs): if key in kwargs } -CC_TOOLCHAIN = { - # NOTE: The `cc_helper.find_cpp_toolchain()` function expects the attribute - # name to be this name. - "_cc_toolchain": attr.label(default = "@bazel_tools//tools/cpp:current_cc_toolchain"), -} - # The common "data" attribute definition. DATA_ATTRS = { # NOTE: The "flags" attribute is deprecated, but there isn't an alternative # way to specify that constraints should be ignored. - "data": attr.label_list( + "data": lambda: attrb.LabelList( allow_files = True, flags = ["SKIP_CONSTRAINTS_OVERRIDE"], doc = """ @@ -229,33 +165,9 @@ This is because Python has a concept of runtime resources. ), } -def _create_native_rules_allowlist_attrs(): - if py_internal: - # The fragment and name are validated when configuration_field is called - default = configuration_field( - fragment = "py", - name = "native_rules_allowlist", - ) - - # A None provider isn't allowed - providers = [_PackageSpecificationInfo] - else: - default = None - providers = [] - - return { - "_native_rules_allowlist": attr.label( - default = default, - providers = providers, - ), - } - -NATIVE_RULES_ALLOWLIST_ATTRS = _create_native_rules_allowlist_attrs() - # Attributes common to all rules. -COMMON_ATTRS = union_attrs( +COMMON_ATTRS = dicts.add( DATA_ATTRS, - NATIVE_RULES_ALLOWLIST_ATTRS, # buildifier: disable=attr-licenses { # NOTE: This attribute is deprecated and slated for removal. @@ -267,18 +179,24 @@ COMMON_ATTRS = union_attrs( # buildifier: disable=attr-license "licenses": attr.license() if hasattr(attr, "license") else attr.string_list(), }, - allow_none = True, ) IMPORTS_ATTRS = { - "imports": attr.string_list( + "imports": lambda: attrb.StringList( doc = """ List of import directories to be added to the PYTHONPATH. Subject to "Make variable" substitution. These import directories will be added for this rule and all rules that depend on it (note: not the rules this rule depends on. Each directory will be added to `PYTHONPATH` by `py_binary` rules -that depend on this rule. The strings are repo-runfiles-root relative, +that depend on this rule. + +The values are target-directory-relative runfiles-root paths. e.g. given target +`//foo/bar:baz`, `sys.path` will be affected as: +* `a/b` adds `$runfilesRoot/$repo/foo/bar/a/b` +* `../sibling` adds `$runfilesRoot/$repo/foo/sibling` +* `../../` adds `$runfilesRoot/$repo` +(where `$repo` is the name of the repository containing the target). Absolute paths (paths that start with `/`) and paths that references a path above the execution root are not allowed and will result in an error. @@ -289,16 +207,13 @@ above the execution root are not allowed and will result in an error. _MaybeBuiltinPyInfo = [[BuiltinPyInfo]] if BuiltinPyInfo != None else [] # Attributes common to rules accepting Python sources and deps. -PY_SRCS_ATTRS = union_attrs( +PY_SRCS_ATTRS = dicts.add( { - "deps": attr.label_list( + "deps": lambda: attrb.LabelList( providers = [ [PyInfo], [CcInfo], ] + _MaybeBuiltinPyInfo, - # TODO(b/228692666): Google-specific; remove these allowances once - # the depot is cleaned up. - allow_rules = DEPS_ATTR_ALLOW_RULES, doc = """ List of additional libraries to be linked in to the target. See comments about @@ -308,9 +223,20 @@ These are typically `py_library` rules. Targets that only provide data files used at runtime belong in the `data` attribute. + +:::{note} +The order of this list can matter because it affects the order that information +from dependencies is merged in, which can be relevant depending on the ordering +mode of depsets that are merged. + +* {obj}`PyInfo.venv_symlinks` uses default ordering. + +See {obj}`PyInfo` for more information about the ordering of its depsets and +how its fields are merged. +::: """, ), - "precompile": attr.string( + "precompile": lambda: attrb.String( doc = """ Whether py source files **for this target** should be precompiled. @@ -332,7 +258,7 @@ Values: default = PrecompileAttr.INHERIT, values = sorted(PrecompileAttr.__members__.values()), ), - "precompile_invalidation_mode": attr.string( + "precompile_invalidation_mode": lambda: attrb.String( doc = """ How precompiled files should be verified to be up-to-date with their associated source files. Possible values are: @@ -345,17 +271,17 @@ source files. Possible values are: the source file. This is most useful when the code won't be modified. For more information on pyc invalidation modes, see -https://docs.python.org/3/library/py_compile.html#py_compile.PycInvalidationMode + """, default = PrecompileInvalidationModeAttr.AUTO, values = sorted(PrecompileInvalidationModeAttr.__members__.values()), ), - "precompile_optimize_level": attr.int( + "precompile_optimize_level": lambda: attrb.Int( doc = """ The optimization level for precompiled files. For more information about optimization levels, see the `compile()` function's -`optimize` arg docs at https://docs.python.org/3/library/functions.html#compile +`optimize` arg docs at NOTE: The value `-1` means "current interpreter", which will be the interpreter used _at build time when pycs are generated_, not the interpreter used at @@ -363,7 +289,7 @@ runtime when the code actually runs. """, default = 0, ), - "precompile_source_retention": attr.string( + "precompile_source_retention": lambda: attrb.String( default = PrecompileSourceRetentionAttr.INHERIT, values = sorted(PrecompileSourceRetentionAttr.__members__.values()), doc = """ @@ -375,37 +301,168 @@ in the resulting output or not. Valid values are: * `omit_source`: Don't include the original py source. """, ), - # Required attribute, but details vary by rule. - # Use create_srcs_attr to create one. - "srcs": None, - # NOTE: In Google, this attribute is deprecated, and can only - # effectively be PY3 or PY3ONLY. Externally, with Bazel, this attribute - # has a separate story. - # Required attribute, but the details vary by rule. - # Use create_srcs_version_attr to create one. - "srcs_version": None, - "_precompile_flag": attr.label( - default = "//python/config_settings:precompile", + "pyi_deps": lambda: attrb.LabelList( + doc = """ +Dependencies providing type definitions the library needs. + +These are dependencies that satisfy imports guarded by `typing.TYPE_CHECKING`. +These are build-time only dependencies and not included as part of a runnable +program (packaging rules may include them, however). + +:::{versionadded} 1.1.0 +::: +""", + providers = [ + [PyInfo], + [CcInfo], + ] + _MaybeBuiltinPyInfo, + ), + "pyi_srcs": lambda: attrb.LabelList( + doc = """ +Type definition files for the library. + +These are typically `.pyi` files, but other file types for type-checker specific +formats are allowed. These files are build-time only dependencies and not included +as part of a runnable program (packaging rules may include them, however). + +:::{versionadded} 1.1.0 +::: +""", + allow_files = True, + ), + "srcs": lambda: attrb.LabelList( + allow_files = [".py", ".py3"], + # Necessary for --compile_one_dependency to work. + flags = ["DIRECT_COMPILE_TIME_INPUT"], + doc = """ +The list of Python source files that are processed to create the target. This +includes all your checked-in code and may include generated source files. The +`.py` files belong in `srcs` and library targets belong in `deps`. Other binary +files that may be needed at run time belong in `data`. +""", + ), + "srcs_version": lambda: attrb.String( + doc = "Defunct, unused, does nothing.", + ), + "_precompile_flag": lambda: attrb.Label( + default = labels.PRECOMPILE, providers = [BuildSettingInfo], ), - "_precompile_source_retention_flag": attr.label( - default = "//python/config_settings:precompile_source_retention", + "_precompile_source_retention_flag": lambda: attrb.Label( + default = labels.PRECOMPILE_SOURCE_RETENTION, providers = [BuildSettingInfo], ), # Force enabling auto exec groups, see # https://bazel.build/extending/auto-exec-groups#how-enable-particular-rule - "_use_auto_exec_groups": attr.bool(default = True), + "_use_auto_exec_groups": lambda: attrb.Bool( + default = True, + ), }, - allow_none = True, ) +COVERAGE_ATTRS = { + # Magic attribute to help C++ coverage work. There's no + # docs about this; see TestActionBuilder.java + "_collect_cc_coverage": lambda: attrb.Label( + default = "@bazel_tools//tools/test:collect_cc_coverage", + executable = True, + cfg = config.exec(exec_group = "test"), + ), + # Magic attribute to make coverage work. There's no + # docs about this; see TestActionBuilder.java + "_lcov_merger": lambda: attrb.Label( + default = configuration_field(fragment = "coverage", name = "output_generator"), + executable = True, + cfg = config.exec(exec_group = "test"), + ), +} + # Attributes specific to Python executable-equivalent rules. Such rules may not # accept Python sources (e.g. some packaged-version of a py_test/py_binary), but # still accept Python source-agnostic settings. -AGNOSTIC_EXECUTABLE_ATTRS = union_attrs( +CONFIG_SETTINGS_ATTR = { + "config_settings": lambda: attrb.LabelKeyedStringDict( + doc = """ +Config settings to change for this target. + +The keys are labels for settings, and the values are strings for the new value +to use. Pass `Label` objects or canonical label strings for the keys to ensure +they resolve as expected (canonical labels start with `@@` and can be +obtained by calling `str(Label(...))`). + +Most `@rules_python//python/config_setting` settings can be used here, which +allows, for example, making only a certain `py_binary` use +{obj}`--bootstrap_impl=script`. + +Additional or custom config settings can be registered using the +{obj}`add_transition_setting` API. This allows, for example, forcing a +particular CPU, or defining a custom setting that `select()` uses elsewhere +to pick between `pip.parse` hubs. See the [How to guide on multiple +versions of a library] for a more concrete example. + +:::{important} +Labels with package `command_line_option` are handled specially: they are treated +as the Bazel-builtin `//command_line_option:` psuedo-targets. + +e.g. `@foo//command_line_option:NAME` will attempt to transition +the Bazel-builtin `//command_line_option:NAME` setting. + +See the {obj}`@rules_python//command_line_option` package for some predefined +special targets, or define your own by putting them in your own `command_line_option` +directory. +::: + +:::{seealso} +* {obj}`//command_line_option:build_runfile_links` +* {obj}`//command_line_option:enable_runfiles` +::: + +:::{note} +These values are transitioned on, so will affect the analysis graph and the +associated memory overhead. The more unique configurations in your overall +build, the more memory and (often unnecessary) re-analysis and re-building +can occur. See + for +more information about risks and considerations. +::: + +:::{versionadded} 1.7.0 +::: +""", + ), +} + +def apply_config_settings_attr(settings, attr): + """Applies the config_settings attribute to the settings. + + Args: + settings: The settings dict to modify in-place. + attr: The rule attributes struct. + + Returns: + {type}`dict[str, object]` the input `settings` value. + """ + for key, value in attr.config_settings.items(): + if key.package == "command_line_option": + if value == "INHERIT": + continue + str_key = "//command_line_option:" + key.name + if key.name == "extra_toolchains": + if value == "": + value = [] + else: + value = [v.strip() for v in value.split(",") if v.strip()] + else: + str_key = str(key) + + settings[str_key] = value + return settings + +AGNOSTIC_EXECUTABLE_ATTRS = dicts.add( DATA_ATTRS, + CONFIG_SETTINGS_ATTR, { - "env": attr.string_dict( + "env": lambda: attrb.StringDict( doc = """\ Dictionary of strings; optional; values are subject to `$(location)` and "Make variable" substitution. @@ -414,22 +471,52 @@ Specifies additional environment variables to set when the target is executed by `test` or `run`. """, ), - # The value is required, but varies by rule and/or rule type. Use - # create_stamp_attr to create one. - "stamp": None, + "stamp": lambda: attrb.Int( + values = _STAMP_VALUES, + doc = """ +Whether to encode build information into the binary. Possible values: + +* `stamp = 1`: Always stamp the build information into the binary, even in + `--nostamp` builds. **This setting should be avoided**, since it potentially kills + remote caching for the binary and any downstream actions that depend on it. +* `stamp = 0`: Always replace build information by constant values. This gives + good build result caching. +* `stamp = -1`: Embedding of build information is controlled by the + `--[no]stamp` flag. + +Stamped binaries are not rebuilt unless their dependencies change. + +Stamped build information can accessed using the `bazel_binary_info` module. +See the [Accessing build information docs] for more information. + +:::{warning} +Stamping can harm build performance by reducing cache hits and should +be avoided if possible. + +In addition, this transitions the {flag}`--stamp ` flag, which can additional +config state overhead. +::: + +:::{note} +Stamping of build data output is always disabled for the exec config. +::: +""", + default = -1, + ), }, - allow_none = True, ) -# Attributes specific to Python test-equivalent executable rules. Such rules may -# not accept Python sources (e.g. some packaged-version of a py_test/py_binary), -# but still accept Python source-agnostic settings. -AGNOSTIC_TEST_ATTRS = union_attrs( - AGNOSTIC_EXECUTABLE_ATTRS, +def _init_agnostic_test_attrs(): + base_stamp = AGNOSTIC_EXECUTABLE_ATTRS["stamp"] + # Tests have stamping disabled by default. - create_stamp_attr(default = 0), - { - "env_inherit": attr.string_list( + def stamp_default_disabled(): + b = base_stamp() + b.set_default(0) + return b + + return dicts.add(AGNOSTIC_EXECUTABLE_ATTRS, { + "env_inherit": lambda: attrb.StringList( doc = """\ List of strings; optional @@ -437,8 +524,9 @@ Specifies additional environment variables to inherit from the external environment when the test is executed by bazel test. """, ), + "stamp": stamp_default_disabled, # TODO(b/176993122): Remove when Bazel automatically knows to run on darwin. - "_apple_constraints": attr.label_list( + "_apple_constraints": lambda: attrb.LabelList( default = [ "@platforms//os:ios", "@platforms//os:macos", @@ -447,16 +535,33 @@ environment when the test is executed by bazel test. "@platforms//os:watchos", ], ), - }, -) + "_validate_test_main": lambda: attrb.Label( + default = "//python/private:py_test_main_validator", + cfg = "exec", + ), + "_validate_test_main_flag": lambda: attrb.Label( + default = labels.VALIDATE_TEST_MAIN, + providers = [BuildSettingInfo], + ), + }) + +# Attributes specific to Python test-equivalent executable rules. Such rules may +# not accept Python sources (e.g. some packaged-version of a py_test/py_binary), +# but still accept Python source-agnostic settings. +AGNOSTIC_TEST_ATTRS = _init_agnostic_test_attrs() # Attributes specific to Python binary-equivalent executable rules. Such rules may # not accept Python sources (e.g. some packaged-version of a py_test/py_binary), # but still accept Python source-agnostic settings. -AGNOSTIC_BINARY_ATTRS = union_attrs( - AGNOSTIC_EXECUTABLE_ATTRS, - create_stamp_attr(default = -1), -) +AGNOSTIC_BINARY_ATTRS = dicts.add(AGNOSTIC_EXECUTABLE_ATTRS) + +WINDOWS_CONSTRAINTS_ATTRS = { + "_windows_constraints": lambda: attrb.LabelList( + default = [ + "@platforms//os:windows", + ], + ), +} # Attribute names common to all Python rules COMMON_ATTR_NAMES = [ diff --git a/python/private/auth.bzl b/python/private/auth.bzl index 6b612678c8..2bab92ef30 100644 --- a/python/private/auth.bzl +++ b/python/private/auth.bzl @@ -95,11 +95,7 @@ def get_auth(ctx, urls, ctx_attr = None): if ctx_attr.netrc: netrc = read_netrc(ctx, ctx_attr.netrc) elif "NETRC" in ctx.os.environ: - # This can be used on newer bazel versions - if hasattr(ctx, "getenv"): - netrc = read_netrc(ctx, ctx.getenv("NETRC")) - else: - netrc = read_netrc(ctx, ctx.os.environ["NETRC"]) + netrc = read_netrc(ctx, ctx.getenv("NETRC")) else: netrc = read_user_netrc(ctx) diff --git a/python/private/bazel_config_mode.bzl b/python/private/bazel_config_mode.bzl new file mode 100644 index 0000000000..ec6be5c83b --- /dev/null +++ b/python/private/bazel_config_mode.bzl @@ -0,0 +1,12 @@ +"""Flag to tell if exec or target mode is active.""" + +load(":py_internal.bzl", "py_internal") + +def _bazel_config_mode_impl(ctx): + return [config_common.FeatureFlagInfo( + value = "exec" if py_internal.is_tool_configuration(ctx) else "target", + )] + +bazel_config_mode = rule( + implementation = _bazel_config_mode_impl, +) diff --git a/python/private/build_data_writer.ps1 b/python/private/build_data_writer.ps1 new file mode 100644 index 0000000000..05c49fef9c --- /dev/null +++ b/python/private/build_data_writer.ps1 @@ -0,0 +1,37 @@ +$OutputPath = $env:OUTPUT +$Lines = @( + "TARGET $env:TARGET", + "STAMPED $env:STAMPED" +) + +$VersionFilePath = $env:VERSION_FILE +if (-not [string]::IsNullOrEmpty($VersionFilePath) -and (Test-Path $VersionFilePath)) { + $Lines += Get-Content -Path $VersionFilePath +} + +$InfoFilePath = $env:INFO_FILE +if (-not [string]::IsNullOrEmpty($InfoFilePath) -and (Test-Path $InfoFilePath)) { + $Lines += Get-Content -Path $InfoFilePath +} + +# Use .NET to write file to avoid PowerShell encoding/locking quirks +# We use UTF8 without BOM for compatibility with how the bash script writes (and +# what consumers expect). +$Utf8NoBom = New-Object System.Text.UTF8Encoding $False +[System.IO.File]::WriteAllLines($OutputPath, $Lines, $Utf8NoBom) + +$Acl = Get-Acl $OutputPath +# We use WorldSid because the "Everyone" name is locale-dependent. +$EveryoneSid = New-Object System.Security.Principal.SecurityIdentifier( + [System.Security.Principal.WellKnownSidType]::WorldSid, + $null +) +$AccessRule = New-Object System.Security.AccessControl.FileSystemAccessRule( + $EveryoneSid, + "Read", + "Allow" +) +$Acl.SetAccessRule($AccessRule) +Set-Acl $OutputPath $Acl + +exit 0 diff --git a/python/private/build_data_writer.sh b/python/private/build_data_writer.sh new file mode 100755 index 0000000000..4af98c6269 --- /dev/null +++ b/python/private/build_data_writer.sh @@ -0,0 +1,11 @@ +#!/bin/sh + +echo "TARGET $TARGET" >> $OUTPUT +echo "STAMPED $STAMPED" >> $OUTPUT +if [ -n "$VERSION_FILE" ]; then + cat "$VERSION_FILE" >> "$OUTPUT" +fi +if [ -n "$INFO_FILE" ]; then + cat "$INFO_FILE" >> "$OUTPUT" +fi +exit 0 diff --git a/python/private/builders.bzl b/python/private/builders.bzl index 50aa3ed91a..54d46c2af2 100644 --- a/python/private/builders.bzl +++ b/python/private/builders.bzl @@ -15,12 +15,19 @@ load("@bazel_skylib//lib:types.bzl", "types") -def _DepsetBuilder(): - """Create a builder for a depset.""" +def _DepsetBuilder(order = None): + """Create a builder for a depset. + + Args: + order: {type}`str | None` The order to initialize the depset to, if any. + + Returns: + {type}`DepsetBuilder` + """ # buildifier: disable=uninitialized self = struct( - _order = [None], + _order = [order], add = lambda *a, **k: _DepsetBuilder_add(self, *a, **k), build = lambda *a, **k: _DepsetBuilder_build(self, *a, **k), direct = [], diff --git a/python/private/builders_util.bzl b/python/private/builders_util.bzl new file mode 100644 index 0000000000..7710383cb1 --- /dev/null +++ b/python/private/builders_util.bzl @@ -0,0 +1,153 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Utilities for builders.""" + +load("@bazel_skylib//lib:types.bzl", "types") +load(":bzlmod_enabled.bzl", "BZLMOD_ENABLED") + +def normalize_transition_in_out_values(arg_name, values): + """Normalize transition inputs/outputs to canonical label strings.""" + for i, value in enumerate(values): + values[i] = normalize_transition_in_out_value(arg_name, value) + +def normalize_transition_in_out_value(arg_name, value): + """Normalize a transition input/output value to a canonical label string. + + Args: + arg_name: {type}`str` the transition arg name, "input" or "output" + value: A label-like value to normalize. + + Returns: + {type}`str` the canonical label string. + """ + if is_label(value): + return str(value) + elif types.is_string(value): + if value.startswith("//command_line_option:"): + return value + if value.startswith("@@" if BZLMOD_ENABLED else "@"): + return value + else: + fail("transition {arg_name} invalid: non-canonical string '{value}'".format( + arg_name = arg_name, + value = value, + )) + else: + fail("transition {arg_name} invalid: ({type}) {value}".format( + arg_name = arg_name, + type = type(value), + value = repr(value), + )) + +def to_label_maybe(value): + """Converts `value` to a `Label`, maybe. + + The "maybe" qualification is because invalid values for `Label()` + are returned as-is (e.g. None, or special values that might be + used with e.g. the `default` attribute arg). + + Args: + value: {type}`str | Label | None | object` the value to turn into a label, + or return as-is. + + Returns: + {type}`Label | input_value` + """ + if value == None: + return None + if is_label(value): + return value + if types.is_string(value): + return Label(value) + return value + +def is_label(obj): + """Tell if an object is a `Label`.""" + return type(obj) == "Label" + +def kwargs_set_default_ignore_none(kwargs, key, default): + """Normalize None/missing to `default`.""" + existing = kwargs.get(key) + if existing == None: + kwargs[key] = default + +def kwargs_set_default_list(kwargs, key): + """Normalizes None/missing to list.""" + existing = kwargs.get(key) + if existing == None: + kwargs[key] = [] + +def kwargs_set_default_dict(kwargs, key): + """Normalizes None/missing to list.""" + existing = kwargs.get(key) + if existing == None: + kwargs[key] = {} + +def kwargs_set_default_doc(kwargs): + """Sets the `doc` arg default.""" + existing = kwargs.get("doc") + if existing == None: + kwargs["doc"] = "" + +def kwargs_set_default_mandatory(kwargs): + """Sets `False` as the `mandatory` arg default.""" + existing = kwargs.get("mandatory") + if existing == None: + kwargs["mandatory"] = False + +def kwargs_getter(kwargs, key): + """Create a function to get `key` from `kwargs`.""" + return lambda: kwargs.get(key) + +def kwargs_setter(kwargs, key): + """Create a function to set `key` in `kwargs`.""" + + def setter(v): + kwargs[key] = v + + return setter + +def kwargs_getter_doc(kwargs): + """Creates a `kwargs_getter` for the `doc` key.""" + return kwargs_getter(kwargs, "doc") + +def kwargs_setter_doc(kwargs): + """Creates a `kwargs_setter` for the `doc` key.""" + return kwargs_setter(kwargs, "doc") + +def kwargs_getter_mandatory(kwargs): + """Creates a `kwargs_getter` for the `mandatory` key.""" + return kwargs_getter(kwargs, "mandatory") + +def kwargs_setter_mandatory(kwargs): + """Creates a `kwargs_setter` for the `mandatory` key.""" + return kwargs_setter(kwargs, "mandatory") + +def list_add_unique(add_to, others, convert = None): + """Bulk add values to a list if not already present. + + Args: + add_to: {type}`list[T]` the list to add values to. It is modified + in-place. + others: {type}`collection[collection[T]]` collection of collections of + the values to add. + convert: {type}`callable | None` function to convert the values to add. + """ + existing = {v: None for v in add_to} + for values in others: + for value in values: + value = convert(value) if convert else value + if value not in existing: + add_to.append(value) diff --git a/python/private/cc/BUILD.bazel b/python/private/cc/BUILD.bazel new file mode 100644 index 0000000000..087d406932 --- /dev/null +++ b/python/private/cc/BUILD.bazel @@ -0,0 +1,66 @@ +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") +load("@rules_cc//cc:cc_library.bzl", "cc_library") +load("//python/private:visibility.bzl", "NOT_ACTUALLY_PUBLIC") + +package( + default_visibility = ["//:__subpackages__"], +) + +licenses(["notice"]) + +filegroup( + name = "distribution", + srcs = glob(["**"]), +) + +# An empty cc target for use when a cc target is needed to satisfy +# Bazel, but its contents don't matter. +cc_library( + name = "empty", + visibility = NOT_ACTUALLY_PUBLIC, +) + +# Private alias to avoid "duplicate dependency label" errors when users +# explicitly pass //python/cc:current_py_cc_headers or //python/cc:current_py_cc_libs +# in the deps attribute (including when deps is a select() expression). +alias( + name = "current_py_cc_headers_private_alias", + actual = "//python/cc:current_py_cc_headers", + visibility = NOT_ACTUALLY_PUBLIC, +) + +# Private alias to avoid "duplicate dependency label" errors when users +# explicitly pass //python/cc:current_py_cc_headers or //python/cc:current_py_cc_libs +# in the deps attribute (including when deps is a select() expression). +alias( + name = "current_py_cc_libs_private_alias", + actual = "//python/cc:current_py_cc_libs", + visibility = NOT_ACTUALLY_PUBLIC, +) + +bzl_library( + name = "py_extension_macro", + srcs = ["py_extension_macro.bzl"], + deps = [ + ":py_extension_rule", + "//python/private:common_labels", + "//python/private:util", + "@rules_cc//cc:core_rules", + ], +) + +bzl_library( + name = "py_extension_rule", + srcs = ["py_extension_rule.bzl"], + deps = [ + "//python/private:attr_builders", + "//python/private:attributes", + "//python/private:builders", + "//python/private:common", + "//python/private:py_info", + "//python/private:reexports", + "//python/private:rule_builders", + "//python/private:toolchain_types", + "@bazel_skylib//lib:dicts", + ], +) diff --git a/python/private/cc/py_extension_macro.bzl b/python/private/cc/py_extension_macro.bzl new file mode 100644 index 0000000000..b76685113a --- /dev/null +++ b/python/private/cc/py_extension_macro.bzl @@ -0,0 +1,201 @@ +"""Macro for creating Python extensions. + +:::{include} /_includes/experimental_api.md +::: +""" + +load("@rules_cc//cc:cc_library.bzl", "cc_library") +load("@rules_cc//cc:cc_shared_library.bzl", "cc_shared_library") +load("//python/private:common_labels.bzl", "labels") +load("//python/private:util.bzl", "add_tag", "copy_propagating_kwargs") +load(":py_extension_rule.bzl", "py_extension_libs", "py_extension_wrapper") + +_EMPTY_CANONICAL_TARGET = str(Label("//python/private/cc:empty")) + +_PY_CC_HEADERS_ALIAS_BASE_TARGET = "//python/private/cc:current_py_cc_headers_private_alias" +_PY_CC_HEADERS_ALIAS_CANONICAL_TARGET = str(Label(_PY_CC_HEADERS_ALIAS_BASE_TARGET)) + +_PY_CC_LIBS_ALIAS_BASE_TARGET = "//python/private/cc:current_py_cc_libs_private_alias" +_PY_CC_LIBS_ALIAS_CANONICAL_TARGET = str(Label(_PY_CC_LIBS_ALIAS_BASE_TARGET)) + +def py_extension( + name, + srcs = None, + hdrs = None, + copts = None, + defines = None, + local_defines = None, + includes = None, + linkopts = None, + deps = None, + dynamic_deps = None, + exports_filter = None, + user_link_flags = None, + additional_linker_inputs = None, + module_name = None, + py_limited_api = None, + **kwargs): + """Creates a Python extension module. + + :::{include} /_includes/experimental_api.md + ::: + + By default, extensions are created within their workspace package directory + (e.g., `pkg/ext.so`) and imported using standard Python package paths + (e.g., `from pkg import ext`). + + To customize import path behavior: + - `imports`: Pass `imports = ["..."]` to append custom search directories to + `sys.path` (matching {attr}`py_library.imports`). + - `module_name`: Pass `module_name = "custom_name"` to override the base + module filename. + + :::{versionadded} 2.3.0 + ::: + + Args: + name: {type}`str` Target name. + srcs: {type}`list[Label | str] | None` C/C++ source files to compile + directly for this extension. + hdrs: {type}`list[Label | str] | None` Header files for `srcs`. + copts: {type}`list[str] | None` Compiler flags for `srcs`. + defines: {type}`list[str] | None` Preprocessor defines for `srcs`. + local_defines: {type}`list[str] | None` Preprocessor defines for `srcs` + passed to internal `cc_library`. + includes: {type}`list[str] | None` Header include search paths passed + to internal `cc_library`. + linkopts: {type}`list[str] | None` Link options passed to internal + `cc_library` created for `srcs`/`hdrs`. To pass linker flags to + `cc_shared_library`, use `user_link_flags`. + deps: {type}`list[Label | str] | None` `cc_library` targets to + statically link into the extension. + dynamic_deps: {type}`list[Label | str] | None` `cc_shared_library` + targets to dynamically link. + exports_filter: {type}`list[str] | None` Filter for exported symbols + passed to `cc_shared_library`. + user_link_flags: {type}`list[str] | None` Additional link flags passed + to `cc_shared_library`. To pass linker flags that apply to `srcs`, + use `linkopts`. + additional_linker_inputs: {type}`list[Label | str] | None` Additional + linker inputs passed to `cc_shared_library`. + module_name: {type}`str | None` Custom Python module name. If not set, + defaults to `name`. + py_limited_api: {type}`str | None` Python limited API version string + (e.g., `"3.8"`). + **kwargs: {type}`dict` Additional arguments passed to the underlying + wrapper rule. + """ + add_tag(kwargs, "@rules_python//python/cc:py_extension") + additional_linker_inputs = additional_linker_inputs or [] + copts = copts or [] + deps = deps or [] + user_link_flags = user_link_flags or [] + + csl_deps = [] + + copts = copts + select({ + # -fPIC (Position Independent Code) is required when compiling C/C++ sources into + # dynamic/shared libraries (.so/.dylib/.pyd) so code can be loaded at arbitrary addresses. + # MSVC on Windows does not support or require -fPIC. + labels.PLATFORMS_OS_WINDOWS: [], + "//conditions:default": ["-fPIC"], + }) + + # Private alias targets are appended to avoid "duplicate dependency label" errors + # if a user explicitly passes //python/cc:current_py_cc_headers or //python/cc:current_py_cc_libs + # in their deps attribute (including when deps is a select() expression). + deps = deps + [_PY_CC_HEADERS_ALIAS_CANONICAL_TARGET] + + # 1. If srcs or hdrs are specified, create an implicit cc_library for them + if srcs or hdrs: + impl_lib_name = "_" + name + "_impl" + impl_lib_kwargs = copy_propagating_kwargs(kwargs) + if includes: + impl_lib_kwargs["includes"] = includes + if linkopts: + impl_lib_kwargs["linkopts"] = linkopts + cc_library( + name = impl_lib_name, + srcs = srcs, + hdrs = hdrs, + copts = copts, + defines = defines, + local_defines = local_defines, + deps = deps, + visibility = ["//visibility:private"], + **impl_lib_kwargs + ) + csl_deps.append(":" + impl_lib_name) + + if not csl_deps: + csl_deps = deps + if not csl_deps: + # cc_shared_library requires a dependency, so use an empty library when none are given. + csl_deps = [_EMPTY_CANONICAL_TARGET] + + # 4. Create the underlying cc_shared_library + csl_name = "_" + name + "_csl" + csl_kwargs = copy_propagating_kwargs(kwargs) + + if exports_filter != None: + csl_kwargs["exports_filter"] = exports_filter + + user_link_flags = user_link_flags + select({ + # On macOS, Apple's ld64 linker requires '-undefined dynamic_lookup' so CPython + # C-API symbols (e.g. PyModule_Create) remain unresolved at link time and are + # dynamically resolved at runtime when CPython loads the shared library (.so). + labels.PLATFORMS_OS_MACOS: ["-undefined", "dynamic_lookup"], + "//conditions:default": [], + }) + + win_libs_name = "_" + name + "_win_libs" + + # On Windows, create a private target using toolchain resolution to extract .lib files + # from CcInfo in py_cc_toolchain because system_provided=True in cc_import leaves DefaultInfo empty. + py_extension_libs( + name = win_libs_name, + tags = ["manual"], + visibility = ["//visibility:private"], + ) + + # Windows-specific CPython linking requirements: + # 1. Windows requires .lib files when linking, so they must be added to deps. + # 2. CPython import libraries (python3xx.lib) are declared with system_provided = True + # in cc_import, suppressing automatic propagation of the .lib file path to link.exe. + # We explicitly pass $(locations ...) from win_libs_name to provide the path to MSVC link.exe. + # 3. We pass win_libs_name as an additional linker input to ensure the .lib file is available to the link action. + deps = deps + select({ + labels.PLATFORMS_OS_WINDOWS: [_PY_CC_LIBS_ALIAS_CANONICAL_TARGET], + "//conditions:default": [], + }) + user_link_flags = user_link_flags + select({ + labels.PLATFORMS_OS_WINDOWS: ["$(locations :" + win_libs_name + ")"], + "//conditions:default": [], + }) + additional_linker_inputs = additional_linker_inputs + select({ + labels.PLATFORMS_OS_WINDOWS: [":" + win_libs_name], + "//conditions:default": [], + }) + + cc_shared_library( + name = csl_name, + deps = csl_deps, + additional_linker_inputs = additional_linker_inputs, + dynamic_deps = dynamic_deps, + user_link_flags = user_link_flags, + visibility = ["//visibility:private"], + **csl_kwargs + ) + + # 5. Filter out C++ specific compilation/linking attributes before invoking wrapper rule + for cc_attr in ("includes", "linkopts", "linkshared", "linkstatic", "features"): + kwargs.pop(cc_attr, None) + + # 6. Wrap with py_extension_wrapper for PEP 3149 naming & PyInfo + py_extension_wrapper( + name = name, + src = ":" + csl_name, + module_name = module_name, + py_limited_api = py_limited_api, + **kwargs + ) diff --git a/python/private/cc/py_extension_rule.bzl b/python/private/cc/py_extension_rule.bzl new file mode 100644 index 0000000000..e167969d5b --- /dev/null +++ b/python/private/cc/py_extension_rule.bzl @@ -0,0 +1,135 @@ +"""Implementation of rules supporting py_extension. + +:::{include} /_includes/experimental_api.md +::: +""" + +load("@bazel_skylib//lib:dicts.bzl", "dicts") +load("//python/private:attr_builders.bzl", "attrb") +load("//python/private:attributes.bzl", "COMMON_ATTRS", "IMPORTS_ATTRS", "WINDOWS_CONSTRAINTS_ATTRS") +load("//python/private:builders.bzl", "builders") +load("//python/private:common.bzl", "get_imports", "is_windows_platform") +load("//python/private:py_info.bzl", "PyInfo", "PyInfoBuilder") +load("//python/private:rule_builders.bzl", "ruleb") +load("//python/private:toolchain_types.bzl", "CC_TOOLCHAIN_TYPE", "PY_CC_TOOLCHAIN_TYPE") + +def _py_extension_wrapper_impl(ctx): + module_name = ctx.attr.module_name or ctx.label.name + + ext = _get_extension(ctx) + use_py_limited_api = bool(ctx.attr.py_limited_api) + if use_py_limited_api: + output_filename = "{module_name}.abi3.{ext}".format( + module_name = module_name, + ext = ext, + ) + else: + py_toolchain = ctx.toolchains[PY_CC_TOOLCHAIN_TYPE] + py_cc_toolchain = py_toolchain.py_cc_toolchain + output_filename = "{module_name}.{soabi}.{ext}".format( + module_name = module_name, + soabi = py_cc_toolchain.soabi, + ext = ext, + ) + + py_dso = ctx.actions.declare_file(output_filename) + + # Symlink the cc_shared_library output to the PEP 3149 / abi3 filename + csl_target = ctx.attr.src + csl_file = csl_target[DefaultInfo].files.to_list()[0] + ctx.actions.symlink( + output = py_dso, + target_file = csl_file, + ) + + runfiles_builder = builders.RunfilesBuilder() + runfiles_builder.add(py_dso) + runfiles_builder.add(ctx.files.data) + runfiles_builder.add_targets(ctx.attr.data) + runfiles_builder.add(csl_target[DefaultInfo].default_runfiles) + runfiles = runfiles_builder.build(ctx) + + py_info_builder = PyInfoBuilder.new() + py_info_builder.transitive_sources.add(py_dso) + py_info_builder.imports.add(get_imports(ctx)) + + return [ + DefaultInfo( + files = depset([py_dso]), + runfiles = runfiles, + ), + py_info_builder.build(), + ] + +PY_EXTENSION_WRAPPER_ATTRS = dicts.add( + COMMON_ATTRS, + IMPORTS_ATTRS, + WINDOWS_CONSTRAINTS_ATTRS, + { + "module_name": lambda: attrb.String( + doc = "Custom Python module name. If not set, defaults to name.", + ), + "py_limited_api": lambda: attrb.String( + default = "", + doc = "Python limited API version string (e.g., '3.8').", + ), + "src": lambda: attrb.Label( + mandatory = True, + doc = "The cc_shared_library target to wrap.", + ), + }, +) + +def create_py_extension_wrapper_rule_builder(): + """Create a rule builder for the private internal wrapper rule.""" + return ruleb.Rule( + doc = "Private internal helper rule for py_extension targets.", + implementation = _py_extension_wrapper_impl, + attrs = PY_EXTENSION_WRAPPER_ATTRS, + provides = [PyInfo], + toolchains = [ + ruleb.ToolchainType(PY_CC_TOOLCHAIN_TYPE), + ruleb.ToolchainType(CC_TOOLCHAIN_TYPE), + ], + fragments = ["cpp"], + ) + +py_extension_wrapper = create_py_extension_wrapper_rule_builder().build() + +def _get_extension(ctx): + """Derives the appropriate file extension for C extensions from target platform. + + Note: On macOS, CPython C extensions use .so (PEP 3149), not .dylib. + Windows uses .pyd. + + Args: + ctx: The rule context. + + Returns: + The extension, e.g. "so" or "pyd" + """ + return "pyd" if is_windows_platform(ctx) else "so" + +def _py_extension_libs_impl(ctx): + py_toolchain = ctx.toolchains[PY_CC_TOOLCHAIN_TYPE] + py_cc_toolchain = py_toolchain.py_cc_toolchain + cc_info = py_cc_toolchain.libs.providers_map["CcInfo"] + files = [] + for input in cc_info.linking_context.linker_inputs.to_list(): + for lib in input.libraries: + if lib.interface_library: + files.append(lib.interface_library) + elif lib.static_library: + files.append(lib.static_library) + elif lib.dynamic_library: + files.append(lib.dynamic_library) + link_files = [f for f in files if not f.path.endswith(".dll")] + return [DefaultInfo(files = depset(link_files))] + +py_extension_libs = rule( + implementation = _py_extension_libs_impl, + toolchains = [PY_CC_TOOLCHAIN_TYPE], + doc = """\ +Private internal helper rule for extracting Windows C/C++ library files from toolchain. +""", +) diff --git a/python/private/common.bzl b/python/private/common.bzl index 97fabcebcb..7e8c6decf3 100644 --- a/python/private/common.bzl +++ b/python/private/common.bzl @@ -11,9 +11,17 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. -"""Various things common to Bazel and Google rule implementations.""" - +"""Various things common to rule implementations.""" + +load("@bazel_skylib//lib:paths.bzl", "paths") +load("@rules_cc//cc/common:cc_common.bzl", "cc_common") +load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") +load("@rules_python_internal//:rules_python_config.bzl", "config") +load("//python/private:py_interpreter_program.bzl", "PyInterpreterProgramInfo") +load("//python/private:toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", "LAUNCHER_MAKER_TOOLCHAIN_TYPE") +load(":builders.bzl", "builders") load(":cc_helper.bzl", "cc_helper") +load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") load(":py_info.bzl", "PyInfo", "PyInfoBuilder") load(":py_internal.bzl", "py_internal") load(":reexports.bzl", "BuiltinPyInfo") @@ -26,116 +34,119 @@ PackageSpecificationInfo = getattr(py_internal, "PackageSpecificationInfo", None # Extensions without the dot _PYTHON_SOURCE_EXTENSIONS = ["py"] +# Extensions that mean a file is relevant to Python +PYTHON_FILE_EXTENSIONS = [ + "dll", # Python C modules, Windows specific + "dylib", # Python C modules, Mac specific + "py", + "pyc", + "pth", # import 'pth' files + "pyi", + "so", # Python C modules, usually Linux +] + +BUILTIN_BUILD_PYTHON_ZIP = [] if config.bazel_10_or_later else [ + "//command_line_option:build_python_zip", +] + +# Not an actual provider. Provider used for memory efficiency. +# buildifier: disable=name-conventions +ExplicitSymlink = provider( + doc = """ +A runfile that should be created as a symlink pointing to a specific location. + +This is only needed on Windows, where Bazel doesn't preserve declare_symlink +with relative paths. This is basically manually captures what using +declare_symlink(), symlink() and runfiles like so would capture: + +``` +link = declare_symlink(...) +link_to_path = relative_path(from=link, to=target) +symlink(output=link, target_path=link_to_path) +runfiles.add([link, target]) +``` +""", + fields = { + "files": "depset[File] of files that should be included if this symlink is used", + "link_to_path": "Path the symlink should point to", + "runfiles_path": "runfiles-root-relative path for the symlink", + "venv_path": "venv-root-relative path for the symlink", + }, +) + +def maybe_builtin_build_python_zip(value, settings = None): + settings = settings or {} + if not config.bazel_10_or_later: + settings["//command_line_option:build_python_zip"] = value + + return settings + +def _find_launcher_maker(ctx): + if config.bazel_9_or_later: + return (ctx.toolchains[LAUNCHER_MAKER_TOOLCHAIN_TYPE].binary, LAUNCHER_MAKER_TOOLCHAIN_TYPE) + return (ctx.executable._windows_launcher_maker, None) + +def create_windows_exe_launcher( + ctx, + *, + output, + python_binary_path, + use_zip_file): + """Creates a Windows exe launcher. + + Args: + ctx: The rule context. + output: The output file for the launcher. + python_binary_path: The path to the Python binary. + use_zip_file: Whether to use a zip file. + """ + launch_info = ctx.actions.args() + launch_info.use_param_file("%s", use_always = True) + launch_info.set_param_file_format("multiline") + launch_info.add("binary_type=Python") + launch_info.add(ctx.workspace_name, format = "workspace_name=%s") + launch_info.add( + "1" if py_internal.runfiles_enabled(ctx) else "0", + format = "symlink_runfiles_enabled=%s", + ) + launch_info.add(python_binary_path, format = "python_bin_path=%s") + launch_info.add("1" if use_zip_file else "0", format = "use_zip_file=%s") + + launcher = ctx.attr._launcher[DefaultInfo].files_to_run.executable + executable, toolchain = _find_launcher_maker(ctx) + ctx.actions.run( + executable = executable, + arguments = [launcher.path, launch_info, output.path], + inputs = [launcher], + outputs = [output], + mnemonic = "PyBuildLauncher", + progress_message = "Creating launcher for %{label}", + # Needed to inherit PATH when using non-MSVC compilers like MinGW + use_default_shell_env = True, + toolchain = toolchain, + ) + def create_binary_semantics_struct( *, - create_executable, - get_cc_details_for_binary, - get_central_uncachable_version_file, - get_coverage_deps, - get_debugger_deps, - get_extra_common_runfiles_for_binary, - get_extra_providers, - get_extra_write_build_data_env, - get_interpreter_path, - get_imports, get_native_deps_dso_name, - get_native_deps_user_link_flags, - get_stamp_flag, - maybe_precompile, - should_build_native_deps_dso, - should_create_init_files, - should_include_build_data): + should_build_native_deps_dso): """Helper to ensure a semantics struct has all necessary fields. Call this instead of a raw call to `struct(...)`; it'll help ensure all the necessary functions are being correctly provided. Args: - create_executable: Callable; creates a binary's executable output. See - py_executable.bzl#py_executable_base_impl for details. - get_cc_details_for_binary: Callable that returns a `CcDetails` struct; see - `create_cc_detail_struct`. - get_central_uncachable_version_file: Callable that returns an optional - Artifact; this artifact is special: it is never cached and is a copy - of `ctx.version_file`; see py_builtins.copy_without_caching - get_coverage_deps: Callable that returns a list of Targets for making - coverage work; only called if coverage is enabled. - get_debugger_deps: Callable that returns a list of Targets that provide - custom debugger support; only called for target-configuration. - get_extra_common_runfiles_for_binary: Callable that returns a runfiles - object of extra runfiles a binary should include. - get_extra_providers: Callable that returns extra providers; see - py_executable.bzl#_create_providers for details. - get_extra_write_build_data_env: Callable that returns a dict[str, str] - of additional environment variable to pass to build data generation. - get_interpreter_path: Callable that returns an optional string, which is - the path to the Python interpreter to use for running the binary. - get_imports: Callable that returns a list of the target's import - paths (from the `imports` attribute, so just the target's own import - path strings, not from dependencies). get_native_deps_dso_name: Callable that returns a string, which is the basename (with extension) of the native deps DSO library. - get_native_deps_user_link_flags: Callable that returns a list of strings, - which are any extra linker flags to pass onto the native deps DSO - linking action. - get_stamp_flag: Callable that returns bool of if the --stamp flag was - enabled or not. - maybe_precompile: Callable that may optional precompile the input `.py` - sources and returns the full set of desired outputs derived from - the source files (e.g., both py and pyc, only one of them, etc). should_build_native_deps_dso: Callable that returns bool; True if building a native deps DSO is supported, False if not. - should_create_init_files: Callable that returns bool; True if - `__init__.py` files should be generated, False if not. - should_include_build_data: Callable that returns bool; True if - build data should be generated, False if not. Returns: A "BinarySemantics" struct. """ return struct( # keep-sorted - create_executable = create_executable, - get_cc_details_for_binary = get_cc_details_for_binary, - get_central_uncachable_version_file = get_central_uncachable_version_file, - get_coverage_deps = get_coverage_deps, - get_debugger_deps = get_debugger_deps, - get_extra_common_runfiles_for_binary = get_extra_common_runfiles_for_binary, - get_extra_providers = get_extra_providers, - get_extra_write_build_data_env = get_extra_write_build_data_env, - get_imports = get_imports, - get_interpreter_path = get_interpreter_path, get_native_deps_dso_name = get_native_deps_dso_name, - get_native_deps_user_link_flags = get_native_deps_user_link_flags, - get_stamp_flag = get_stamp_flag, - maybe_precompile = maybe_precompile, should_build_native_deps_dso = should_build_native_deps_dso, - should_create_init_files = should_create_init_files, - should_include_build_data = should_include_build_data, - ) - -def create_library_semantics_struct( - *, - get_cc_info_for_library, - get_imports, - maybe_precompile): - """Create a `LibrarySemantics` struct. - - Call this instead of a raw call to `struct(...)`; it'll help ensure all - the necessary functions are being correctly provided. - - Args: - get_cc_info_for_library: Callable that returns a CcInfo for the library; - see py_library_impl for arg details. - get_imports: Callable; see create_binary_semantics_struct. - maybe_precompile: Callable; see create_binary_semantics_struct. - Returns: - a `LibrarySemantics` struct. - """ - return struct( - # keep sorted - get_cc_info_for_library = get_cc_info_for_library, - get_imports = get_imports, - maybe_precompile = maybe_precompile, ) def create_cc_details_struct( @@ -183,73 +194,6 @@ def create_cc_details_struct( **kwargs ) -def create_executable_result_struct(*, extra_files_to_build, output_groups, extra_runfiles = None): - """Creates a `CreateExecutableResult` struct. - - This is the return value type of the semantics create_executable function. - - Args: - extra_files_to_build: depset of File; additional files that should be - included as default outputs. - output_groups: dict[str, depset[File]]; additional output groups that - should be returned. - extra_runfiles: A runfiles object of additional runfiles to include. - - Returns: - A `CreateExecutableResult` struct. - """ - return struct( - extra_files_to_build = extra_files_to_build, - output_groups = output_groups, - extra_runfiles = extra_runfiles, - ) - -def union_attrs(*attr_dicts, allow_none = False): - """Helper for combining and building attriute dicts for rules. - - Similar to dict.update, except: - * Duplicate keys raise an error if they aren't equal. This is to prevent - unintentionally replacing an attribute with a potentially incompatible - definition. - * None values are special: They mean the attribute is required, but the - value should be provided by another attribute dict (depending on the - `allow_none` arg). - Args: - *attr_dicts: The dicts to combine. - allow_none: bool, if True, then None values are allowed. If False, - then one of `attrs_dicts` must set a non-None value for keys - with a None value. - - Returns: - dict of attributes. - """ - result = {} - missing = {} - for attr_dict in attr_dicts: - for attr_name, value in attr_dict.items(): - if value == None and not allow_none: - if attr_name not in result: - missing[attr_name] = None - else: - if attr_name in missing: - missing.pop(attr_name) - - if attr_name not in result or result[attr_name] == None: - result[attr_name] = value - elif value != None and result[attr_name] != value: - fail("Duplicate attribute name: '{}': existing={}, new={}".format( - attr_name, - result[attr_name], - value, - )) - - # Else, they're equal, so do nothing. This allows merging dicts - # that both define the same key from a common place. - - if missing and not allow_none: - fail("Required attributes missing: " + csv(missing.keys())) - return result - def csv(values): """Convert a list of strings to comma separated value string.""" return ", ".join(sorted(values)) @@ -262,23 +206,73 @@ def filter_to_py_srcs(srcs): # as a valid extension. return [f for f in srcs if f.extension == "py"] -def collect_imports(ctx, semantics): +def collect_cc_info(ctx, extra_deps = []): + """Collect C++ information from dependencies for Bazel. + + Args: + ctx: Rule ctx; must have `deps` attribute. + extra_deps: list of Target to also collect C+ information from. + + Returns: + CcInfo provider of merged information. + """ + cc_infos = [] + for dep in collect_deps(ctx, extra_deps): + if CcInfo in dep: + cc_infos.append(dep[CcInfo]) + + if PyCcLinkParamsInfo in dep: + cc_infos.append(dep[PyCcLinkParamsInfo].cc_info) + + return cc_common.merge_cc_infos(cc_infos = cc_infos) + +def collect_imports(ctx, extra_deps = []): """Collect the direct and transitive `imports` strings. Args: ctx: {type}`ctx` the current target ctx - semantics: semantics object for fetching direct imports. + extra_deps: list of Target to also collect imports from. Returns: {type}`depset[str]` of import paths """ + transitive = [] - for dep in ctx.attr.deps: + for dep in collect_deps(ctx, extra_deps): if PyInfo in dep: transitive.append(dep[PyInfo].imports) if BuiltinPyInfo != None and BuiltinPyInfo in dep: transitive.append(dep[BuiltinPyInfo].imports) - return depset(direct = semantics.get_imports(ctx), transitive = transitive) + return depset(direct = get_imports(ctx), transitive = transitive) + +def get_imports(ctx): + """Gets the imports from a rule's `imports` attribute. + + Args: + ctx: Rule ctx. + + Returns: + List of strings. + """ + prefix = "{}/{}".format( + ctx.workspace_name, + py_internal.get_label_repo_runfiles_path(ctx.label), + ) + result = [] + for import_str in ctx.attr.imports: + import_str = ctx.expand_make_variables("imports", import_str, {}) + if import_str.startswith("/"): + continue + + # To prevent "escaping" out of the runfiles tree, we normalize + # the path and ensure it doesn't have up-level references. + import_path = paths.normalize("{}/{}".format(prefix, import_str)) + if import_path.startswith("../") or import_path == "..": + fail("Path '{}' references a path above the execution root".format( + import_str, + )) + result.append(import_path) + return result def collect_runfiles(ctx, files = depset()): """Collects the necessary files from the rule's context. @@ -308,7 +302,7 @@ def collect_runfiles(ctx, files = depset()): # If the target is a File, then add that file to the runfiles. # Otherwise, add the target's **data runfiles** to the runfiles. # - # Note that, contray to best practice, the default outputs of the + # Note that, contrary to best practice, the default outputs of the # targets in `data` are *not* added, nor are the default runfiles. # # This ends up being important for several reasons, some of which are @@ -349,15 +343,18 @@ def collect_runfiles(ctx, files = depset()): def create_py_info( ctx, *, + original_sources, required_py_files, required_pyc_files, implicit_pyc_files, implicit_pyc_source_files, - imports): + imports, + venv_symlinks = []): """Create PyInfo provider. Args: ctx: rule ctx. + original_sources: `depset[File]`; the original input sources from `srcs` required_py_files: `depset[File]`; the direct, `.py` sources for the target that **must** be included by downstream targets. This should only be Python source files. It should not include pyc files. @@ -370,16 +367,22 @@ def create_py_info( implicit_pyc_files: {type}`depset[File]` Implicitly generated pyc files that a binary can choose to include. imports: depset of strings; the import path values to propagate. + venv_symlinks: {type}`list[VenvSymlinkEntry]` instances for + symlinks to create in the consuming binary's venv. Returns: A tuple of the PyInfo instance and a depset of the transitive sources collected from dependencies (the latter is only necessary for deprecated extra actions support). """ - - py_info = PyInfoBuilder() + py_info = PyInfoBuilder.new() + py_info.venv_symlinks.add(venv_symlinks) + py_info.direct_original_sources.add(original_sources) py_info.direct_pyc_files.add(required_pyc_files) + py_info.direct_pyi_files.add(ctx.files.pyi_srcs) + py_info.transitive_original_sources.add(original_sources) py_info.transitive_pyc_files.add(required_pyc_files) + py_info.transitive_pyi_files.add(ctx.files.pyi_srcs) py_info.transitive_implicit_pyc_files.add(implicit_pyc_files) py_info.transitive_implicit_pyc_source_files.add(implicit_pyc_source_files) py_info.imports.add(imports) @@ -393,13 +396,16 @@ def create_py_info( else: # TODO(b/228692666): Remove this once non-PyInfo targets are no # longer supported in `deps`. - files = target.files.to_list() + files = target[DefaultInfo].files.to_list() for f in files: if f.extension == "py": py_info.transitive_sources.add(f) py_info.merge_uses_shared_libraries(cc_helper.is_valid_shared_library_artifact(f)) + for target in ctx.attr.pyi_deps: + # PyInfo may not be present e.g. cc_library rules. + if PyInfo in target or (BuiltinPyInfo != None and BuiltinPyInfo in target): + py_info.merge(_get_py_info(target)) - deps_transitive_sources = py_info.transitive_sources.build() py_info.transitive_sources.add(required_py_files) # We only look at data to calculate uses_shared_libraries, if it's already @@ -413,7 +419,7 @@ def create_py_info( info = _get_py_info(target) py_info.merge_uses_shared_libraries(info.uses_shared_libraries) else: - files = target.files.to_list() + files = target[DefaultInfo].files.to_list() for f in files: py_info.merge_uses_shared_libraries(cc_helper.is_valid_shared_library_artifact(f)) if py_info.get_uses_shared_libraries(): @@ -421,7 +427,7 @@ def create_py_info( if py_info.get_uses_shared_libraries(): break - return py_info.build(), deps_transitive_sources, py_info.build_builtin_py_info() + return py_info.build(), py_info.build_builtin_py_info() def _get_py_info(target): return target[PyInfo] if PyInfo in target or BuiltinPyInfo == None else target[BuiltinPyInfo] @@ -460,6 +466,9 @@ _BOOL_TYPE = type(True) def is_bool(v): return type(v) == _BOOL_TYPE +def is_file(v): + return type(v) == "File" + def target_platform_has_any_constraint(ctx, constraints): """Check if target platform has any of a list of constraints. @@ -475,3 +484,211 @@ def target_platform_has_any_constraint(ctx, constraints): if ctx.target_platform_has_constraint(constraint_value): return True return False + +def is_windows_platform(ctx): + """Check if target platform is windows. + + Args: + ctx: rule context. + + Returns: + True if target platform is windows. + """ + return target_platform_has_any_constraint(ctx, ctx.attr._windows_constraints) + +def relative_path(from_, to): + """Compute a relative path from one path to another. + + Args: + from_: {type}`str` the starting directory. Note that it should be + a directory because relative-symlinks are relative to the + directory the symlink resides in. + to: {type}`str` the path that `from_` wants to point to + + Returns: + {type}`str` a relative path + """ + from_parts = from_.split("/") + to_parts = to.split("/") + + # Strip common leading parts from both paths + n = min(len(from_parts), len(to_parts)) + for _ in range(n): + if from_parts[0] == to_parts[0]: + from_parts.pop(0) + to_parts.pop(0) + else: + break + + # Impossible to compute a relative path without knowing what ".." is + if from_parts and from_parts[0] == "..": + fail("cannot compute relative path from '%s' to '%s'", from_, to) + + parts = ([".."] * len(from_parts)) + to_parts + return paths.join(*parts) + +def runfiles_root_path(ctx, short_path): + """Compute a runfiles-root relative path from `File.short_path` + + Args: + ctx: current target ctx + short_path: str, a main-repo relative path from `File.short_path` + + Returns: + {type}`str`, a runflies-root relative path + """ + + # The ../ comes from short_path is for files in other repos. + if short_path.startswith("../"): + return short_path[3:] + else: + return "{}/{}".format(ctx.workspace_name, short_path) + +def collect_deps(ctx, extra_deps = []): + """Collect the dependencies from the rule's context. + + Args: + ctx: rule ctx + extra_deps: list of Target to also collect dependencies from. + + Returns: + list of Target + """ + deps = ctx.attr.deps + if extra_deps: + deps = list(deps) + deps.extend(extra_deps) + return deps + +def maybe_create_repo_mapping(ctx, *, runfiles): + """Creates a repo mapping manifest if bzlmod is enabled. + + There isn't a way to reference the repo mapping Bazel implicitly + creates, so we have to manually create it ourselves. + + Args: + ctx: rule ctx. + runfiles: runfiles object to generate mapping for. + + Returns: + File object if the repo mapping manifest was created, None otherwise. + """ + if not py_internal.is_bzlmod_enabled(ctx): + return None + + # We have to add `.custom` because `{name}.repo_mapping` is used by Bazel + # internally. + repo_mapping_manifest = ctx.actions.declare_file(ctx.label.name + ".custom.repo_mapping") + py_internal.create_repo_mapping_manifest( + ctx = ctx, + runfiles = runfiles, + output = repo_mapping_manifest, + ) + return repo_mapping_manifest + +def actions_run( + ctx, + *, + executable, + toolchain = None, + **kwargs): + """Runs a tool as an action, supporting py_interpreter_program targets. + + This is wrapper around `ctx.actions.run()` that sets some useful defaults, + supports handling `py_interpreter_program` targets, and some other features + to let the target being run influence the action invocation. + + Args: + ctx: The rule context. The rule must have the + `//python:exec_tools_toolchain_type` toolchain available. + executable: The executable to run. This can be a target that provides + `PyInterpreterProgramInfo` or a regular executable target. If it + provides `testing.ExecutionInfo`, the requirements will be added to + the execution requirements. + toolchain: The toolchain type to use. Must be None or + `//python:exec_tools_toolchain_type`. + **kwargs: Additional arguments to pass to `ctx.actions.run()`. + `mnemonic` and `progress_message` are required. + """ + mnemonic = kwargs.pop("mnemonic", None) + if not mnemonic: + fail("actions_run: missing required argument 'mnemonic'") + + progress_message = kwargs.pop("progress_message", None) + if not progress_message: + fail("actions_run: missing required argument 'progress_message'") + + tools = kwargs.pop("tools", None) + tools = list(tools) if tools else [] + + action_arguments = [] + action_env = { + "PYTHONHASHSEED": "0", # Helps avoid non-deterministic behavior + "PYTHONNOUSERSITE": "1", # Helps avoid non-deterministic behavior + "PYTHONSAFEPATH": "1", # Helps avoid incorrect import issues + } + default_info = executable[DefaultInfo] + action_inputs = builders.DepsetBuilder() + action_inputs.add(kwargs.pop("inputs", None) or []) + if PyInterpreterProgramInfo in executable: + if toolchain and toolchain != EXEC_TOOLS_TOOLCHAIN_TYPE: + fail(("Action {}: tool {} provides PyInterpreterProgramInfo, which " + + "requires the `toolchain` arg be " + + "None or {}, got: {}").format( + mnemonic, + executable, + EXEC_TOOLS_TOOLCHAIN_TYPE, + toolchain, + )) + exec_runtime = ctx.toolchains[EXEC_TOOLS_TOOLCHAIN_TYPE].exec_tools.exec_runtime + if exec_runtime.interpreter: + action_exe = exec_runtime.interpreter + action_inputs.add(exec_runtime.files) + elif exec_runtime.interpreter_path: + action_exe = exec_runtime.interpreter_path + else: + fail(("Action {}: PyRuntimeInfo from exec tools toolchain is " + + "malformed: requires one of `interpreter` or " + + "`interpreter_path` set").format( + mnemonic, + )) + + program_info = executable[PyInterpreterProgramInfo] + + interpreter_args = ctx.actions.args() + interpreter_args.add_all(program_info.interpreter_args) + interpreter_args.add(default_info.files_to_run.executable) + action_arguments.append(interpreter_args) + + action_env.update(program_info.env) + + tools.append(default_info.files_to_run) + toolchain = EXEC_TOOLS_TOOLCHAIN_TYPE + else: + action_exe = executable[DefaultInfo].files_to_run + + execution_requirements = {} + if testing.ExecutionInfo in executable: + execution_requirements.update(executable[testing.ExecutionInfo].requirements) + + # Give precedence to caller's execution requirements. + execution_requirements.update(kwargs.pop("execution_requirements", None) or {}) + + # Give precedence to caller's env. + action_env.update(kwargs.pop("env", None) or {}) + + # Handle arguments=None + action_arguments.extend(list(kwargs.pop("arguments", None) or [])) + + ctx.actions.run( + executable = action_exe, + arguments = action_arguments, + tools = tools, + env = action_env, + execution_requirements = execution_requirements, + toolchain = toolchain, + mnemonic = mnemonic, + progress_message = progress_message, + inputs = action_inputs.build(), + **kwargs + ) diff --git a/python/private/common_labels.bzl b/python/private/common_labels.bzl new file mode 100644 index 0000000000..db4a00ba0a --- /dev/null +++ b/python/private/common_labels.bzl @@ -0,0 +1,38 @@ +"""Constants for common labels used in the codebase.""" + +# NOTE: str() is called because some APIs don't accept Label objects +# (e.g. transition inputs/outputs or the transition settings return dict) + +labels = struct( + # keep sorted + ADD_SRCS_TO_RUNFILES = str(Label("//python/config_settings:add_srcs_to_runfiles")), + BOOTSTRAP_IMPL = str(Label("//python/config_settings:bootstrap_impl")), + BUILD_PYTHON_ZIP = str(Label("//python/config_settings:build_python_zip")), + # NOTE: Special target; see definition for details. + BUILD_RUNFILE_LINKS = str(Label("//command_line_option:build_runfile_links")), + DEBUGGER = str(Label("//python/config_settings:debugger")), + # NOTE: Special target; see definition for details. + ENABLE_RUNFILES = str(Label("//command_line_option:enable_runfiles")), + EXEC_TOOLS_TOOLCHAIN = str(Label("//python/config_settings:exec_tools_toolchain")), + EXTRA_TOOLCHAINS = str(Label("//command_line_option:extra_toolchains")), + NONE = str(Label("//python:none")), + PIP_ENV_MARKER_CONFIG = str(Label("//python/config_settings:pip_env_marker_config")), + PIP_WHL_OSX_VERSION = str(Label("//python/config_settings:pip_whl_osx_version")), + PLATFORMS_OS_MACOS = str(Label("@platforms//os:macos")), + PLATFORMS_OS_WINDOWS = str(Label("@platforms//os:windows")), + PRECOMPILE = str(Label("//python/config_settings:precompile")), + PRECOMPILE_SOURCE_RETENTION = str(Label("//python/config_settings:precompile_source_retention")), + PYC_COLLECTION = str(Label("//python/config_settings:pyc_collection")), + PYTHON_IMPORT_ALL_REPOSITORIES = str(Label("//python/config_settings:experimental_python_import_all_repositories")), + PYTHON_SRC = str(Label("//python/bin:python_src")), + PYTHON_VERSION = str(Label("//python/config_settings:python_version")), + PYTHON_VERSION_MAJOR_MINOR = str(Label("//python/config_settings:python_version_major_minor")), + PY_FREETHREADED = str(Label("//python/config_settings:py_freethreaded")), + PY_LINUX_LIBC = str(Label("//python/config_settings:py_linux_libc")), + REPL_DEP = str(Label("//python/bin:repl_dep")), + VALIDATE_TEST_MAIN = str(Label("//python/config_settings:validate_test_main")), + VENV = str(Label("//python/config_settings:venv")), + VENVS_SITE_PACKAGES = str(Label("//python/config_settings:venvs_site_packages")), + VENVS_USE_DECLARE_SYMLINK = str(Label("//python/config_settings:venvs_use_declare_symlink")), + VISIBLE_FOR_TESTING = str(Label("//python/private:visible_for_testing")), +) diff --git a/python/private/config_settings.bzl b/python/private/config_settings.bzl index 10b4d686a7..9cd729aff2 100644 --- a/python/private/config_settings.bzl +++ b/python/private/config_settings.bzl @@ -17,12 +17,29 @@ load("@bazel_skylib//lib:selects.bzl", "selects") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") -load(":semver.bzl", "semver") +load(":text_util.bzl", "render") +load(":version.bzl", "version") +load(":visibility.bzl", "NOT_ACTUALLY_PUBLIC") _PYTHON_VERSION_FLAG = Label("//python/config_settings:python_version") _PYTHON_VERSION_MAJOR_MINOR_FLAG = Label("//python/config_settings:python_version_major_minor") -def construct_config_settings(*, name, default_version, versions, minor_mapping): # buildifier: disable=function-docstring +_DEBUG_ENV_MESSAGE_TEMPLATE = """\ +The current configuration rules_python config flags is: + {flags} + +If the value is missing, then the default value is being used, see documentation: +{docs_url}/python/config_settings +""" + +def construct_config_settings( + *, + name, + default_version, + versions, + minor_mapping, + compat_lowest_version = "3.8", + documented_flags): # buildifier: disable=function-docstring """Create a 'python_version' config flag and construct all config settings used in rules_python. This mainly includes the targets that are used in the toolchain and pip hub @@ -33,6 +50,10 @@ def construct_config_settings(*, name, default_version, versions, minor_mapping) default_version: {type}`str` the default value for the `python_version` flag. versions: {type}`list[str]` A list of versions to build constraint settings for. minor_mapping: {type}`dict[str, str]` A mapping from `X.Y` to `X.Y.Z` python versions. + compat_lowest_version: {type}`str` The version that we should use as the lowest available + version for `is_python_3.X` flags. + documented_flags: {type}`list[str]` The labels of the documented settings + that affect build configuration. """ _ = name # @unused _python_version_flag( @@ -54,22 +75,22 @@ def construct_config_settings(*, name, default_version, versions, minor_mapping) ) _reverse_minor_mapping = {full: minor for minor, full in minor_mapping.items()} - for version in versions: - minor_version = _reverse_minor_mapping.get(version) + for ver in versions: + minor_version = _reverse_minor_mapping.get(ver) if not minor_version: native.config_setting( - name = "is_python_{}".format(version), - flag_values = {":python_version": version}, + name = "is_python_{}".format(ver), + flag_values = {":python_version": ver}, visibility = ["//visibility:public"], ) continue # Also need to match the minor version when using - name = "is_python_{}".format(version) + name = "is_python_{}".format(ver) native.config_setting( name = "_" + name, - flag_values = {":python_version": version}, - visibility = ["//visibility:public"], + flag_values = {":python_version": ver}, + visibility = NOT_ACTUALLY_PUBLIC, ) # An alias pointing to an underscore-prefixed config_setting_group @@ -79,7 +100,7 @@ def construct_config_settings(*, name, default_version, versions, minor_mapping) selects.config_setting_group( name = "_{}_group".format(name), match_any = [ - ":_is_python_{}".format(version), + ":_is_python_{}".format(ver), ":is_python_{}".format(minor_version), ], visibility = ["//visibility:private"], @@ -87,20 +108,77 @@ def construct_config_settings(*, name, default_version, versions, minor_mapping) native.alias( name = name, actual = "_{}_group".format(name), - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) # This matches the raw flag value, e.g. --//python/config_settings:python_version=3.8 # It's private because matching the concept of e.g. "3.8" value is done # using the `is_python_X.Y` config setting group, which is aware of the # minor versions that could match instead. + first_minor = None for minor in minor_mapping.keys(): + ver = version.parse(minor) + if first_minor == None or version.is_lt(ver, first_minor): + first_minor = ver + native.config_setting( name = "is_python_{}".format(minor), flag_values = {_PYTHON_VERSION_MAJOR_MINOR_FLAG: minor}, visibility = ["//visibility:public"], ) + # This is a compatibility layer to ensure that `select` statements don't break out right + # when the toolchains for EOL minor versions are no longer registered. + compat_lowest_version = version.parse(compat_lowest_version) + for minor in range(compat_lowest_version.release[-1], first_minor.release[-1]): + native.alias( + name = "is_python_3.{}".format(minor), + actual = "@platforms//:incompatible", + visibility = ["//visibility:public"], + ) + + _current_config( + name = "current_config", + build_setting_default = "", + settings = documented_flags + [_PYTHON_VERSION_FLAG.name], + visibility = ["//visibility:private"], + ) + native.config_setting( + name = "is_not_matching_current_config", + # We use the rule above instead of @platforms//:incompatible so that the + # printing of the current env always happens when the _current_config rule + # is executed. + # + # NOTE: This should in practise only happen if there is a missing compatible + # `whl_library` in the hub repo created by `pip.parse`. + flag_values = {"current_config": "will-never-match"}, + # Only public so that PyPI hub repo can access it + visibility = NOT_ACTUALLY_PUBLIC, + ) + + libc = Label("//python/config_settings:py_linux_libc") + native.config_setting( + name = "_is_py_linux_libc_glibc", + flag_values = {libc: "glibc"}, + visibility = NOT_ACTUALLY_PUBLIC, + ) + native.config_setting( + name = "_is_py_linux_libc_musl", + flag_values = {libc: "musl"}, + visibility = NOT_ACTUALLY_PUBLIC, + ) + freethreaded = Label("//python/config_settings:py_freethreaded") + native.config_setting( + name = "_is_py_freethreaded_yes", + flag_values = {freethreaded: "yes"}, + visibility = NOT_ACTUALLY_PUBLIC, + ) + native.config_setting( + name = "_is_py_freethreaded_no", + flag_values = {freethreaded: "no"}, + visibility = NOT_ACTUALLY_PUBLIC, + ) + def _python_version_flag_impl(ctx): value = ctx.build_setting_value return [ @@ -122,10 +200,10 @@ _python_version_flag = rule( ) def _python_version_major_minor_flag_impl(ctx): - input = ctx.attr._python_version_flag[config_common.FeatureFlagInfo].value + input = _flag_value(ctx.attr._python_version_flag) if input: - version = semver(input) - value = "{}.{}".format(version.major, version.minor) + ver = version.parse(input) + value = "{}.{}".format(ver.release[0], ver.release[1]) else: value = "" @@ -140,3 +218,81 @@ _python_version_major_minor_flag = rule( ), }, ) + +def _flag_value(s): + if config_common.FeatureFlagInfo in s: + return s[config_common.FeatureFlagInfo].value + else: + return s[BuildSettingInfo].value + +def _print_current_config_impl(ctx): + flags = "\n".join([ + "{}: \"{}\"".format(k, v) + for k, v in sorted({ + str(setting.label): _flag_value(setting) + for setting in ctx.attr.settings + }.items()) + ]) + + msg = ctx.attr._template.format( + docs_url = "https://rules-python.readthedocs.io/en/latest/api/rules_python", + flags = render.indent(flags).lstrip(), + ) + if ctx.build_setting_value and ctx.build_setting_value != "fail": + fail("Only 'fail' and empty build setting values are allowed for {}".format( + str(ctx.label), + )) + elif ctx.build_setting_value: + fail(msg) + else: + print(msg) # buildifier: disable=print + + return [config_common.FeatureFlagInfo(value = "")] + +_current_config = rule( + implementation = _print_current_config_impl, + build_setting = config.string(flag = True), + attrs = { + "settings": attr.label_list(mandatory = True), + "_template": attr.string(default = _DEBUG_ENV_MESSAGE_TEMPLATE), + }, +) + +def is_python_version_at_least(name, **kwargs): + flag_name = "_{}_flag".format(name) + native.config_setting( + name = name, + flag_values = { + flag_name: "yes", + }, + ) + _python_version_at_least( + name = flag_name, + visibility = ["//visibility:private"], + **kwargs + ) + +def _python_version_at_least_impl(ctx): + flag_value = ctx.attr._major_minor[config_common.FeatureFlagInfo].value + + # CI is, somehow, getting an empty string for the current flag value. + # How isn't clear. + if not flag_value: + return [config_common.FeatureFlagInfo(value = "no")] + + current = tuple([ + int(x) + for x in flag_value.split(".") + ]) + at_least = tuple([int(x) for x in ctx.attr.at_least.split(".")]) + + value = "yes" if current >= at_least else "no" + return [config_common.FeatureFlagInfo(value = value)] + +_python_version_at_least = rule( + implementation = _python_version_at_least_impl, + attrs = { + "at_least": attr.string(mandatory = True), + "_major_minor": attr.label(default = _PYTHON_VERSION_MAJOR_MINOR_FLAG), + }, +) diff --git a/python/private/coverage.patch b/python/private/coverage.patch index 051f7fc543..9b3830332c 100644 --- a/python/private/coverage.patch +++ b/python/private/coverage.patch @@ -8,10 +8,6 @@ diff --git a/coverage/__main__.py b/coverage/__main__.py index ce2d8db..7d7d0a0 100644 --- a/coverage/__main__.py +++ b/coverage/__main__.py -@@ -6,5 +6,6 @@ - from __future__ import annotations - +@@ -8,1 +8,2 @@ import sys +sys.path.append(sys.path.pop(0)) - from coverage.cmdline import main - sys.exit(main()) diff --git a/python/private/coverage_deps.bzl b/python/private/coverage_deps.bzl index e80e8ee910..001146c3d4 100644 --- a/python/private/coverage_deps.bzl +++ b/python/private/coverage_deps.bzl @@ -20,121 +20,149 @@ load("@bazel_tools//tools/build_defs/repo:utils.bzl", "maybe") load("//python/private:version_label.bzl", "version_label") # START: maintained by 'bazel run //tools/private/update_deps:update_coverage_deps ' +_default = ( + "https://files.pythonhosted.org/packages/ec/16/114df1c291c22cac3b0c127a73e0af5c12ed7bbb6558d310429a0ae24023/coverage-7.10.7-py3-none-any.whl", + "f7941f6f2fe6dd6807a1208737b8a0cbcf1cc6d7b07d24998ad2d63590868260", +) _coverage_deps = { "cp310": { "aarch64-apple-darwin": ( - "https://files.pythonhosted.org/packages/7d/73/041928e434442bd3afde5584bdc3f932fb4562b1597629f537387cec6f3d/coverage-7.6.1-cp310-cp310-macosx_11_0_arm64.whl", - "cf4b19715bccd7ee27b6b120e7e9dd56037b9c0681dcc1adc9ba9db3d417fa36", + "https://files.pythonhosted.org/packages/03/94/952d30f180b1a916c11a56f5c22d3535e943aa22430e9e3322447e520e1c/coverage-7.10.7-cp310-cp310-macosx_11_0_arm64.whl", + "e201e015644e207139f7e2351980feb7040e6f4b2c2978892f3e3789d1c125e5", ), "aarch64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/c7/c8/6ca52b5147828e45ad0242388477fdb90df2c6cbb9a441701a12b3c71bc8/coverage-7.6.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - "e61c0abb4c85b095a784ef23fdd4aede7a2628478e7baba7c5e3deba61070a02", + "https://files.pythonhosted.org/packages/60/83/5c283cff3d41285f8eab897651585db908a909c572bdc014bcfaf8a8b6ae/coverage-7.10.7-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", + "6be8ed3039ae7f7ac5ce058c308484787c86e8437e72b30bf5e88b8ea10f3c87", ), "x86_64-apple-darwin": ( - "https://files.pythonhosted.org/packages/7e/61/eb7ce5ed62bacf21beca4937a90fe32545c91a3c8a42a30c6616d48fc70d/coverage-7.6.1-cp310-cp310-macosx_10_9_x86_64.whl", - "b06079abebbc0e89e6163b8e8f0e16270124c154dc6e4a47b413dd538859af16", + "https://files.pythonhosted.org/packages/e5/6c/3a3f7a46888e69d18abe3ccc6fe4cb16cccb1e6a2f99698931dafca489e6/coverage-7.10.7-cp310-cp310-macosx_10_9_x86_64.whl", + "fc04cc7a3db33664e0c2d10eb8990ff6b3536f6842c9590ae8da4c614b9ed05a", ), "x86_64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/53/23/9e2c114d0178abc42b6d8d5281f651a8e6519abfa0ef460a00a91f80879d/coverage-7.6.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "8f59d57baca39b32db42b83b2a7ba6f47ad9c394ec2076b084c3f029b7afca23", + "https://files.pythonhosted.org/packages/19/20/d0384ac06a6f908783d9b6aa6135e41b093971499ec488e47279f5b846e6/coverage-7.10.7-cp310-cp310-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", + "8421e088bc051361b01c4b3a50fd39a4b9133079a2229978d9d30511fd05231b", ), }, "cp311": { "aarch64-apple-darwin": ( - "https://files.pythonhosted.org/packages/e1/0e/e52332389e057daa2e03be1fbfef25bb4d626b37d12ed42ae6281d0a274c/coverage-7.6.1-cp311-cp311-macosx_11_0_arm64.whl", - "ed37bd3c3b063412f7620464a9ac1314d33100329f39799255fb8d3027da50d3", + "https://files.pythonhosted.org/packages/54/f0/514dcf4b4e3698b9a9077f084429681bf3aad2b4a72578f89d7f643eb506/coverage-7.10.7-cp311-cp311-macosx_11_0_arm64.whl", + "65646bb0359386e07639c367a22cf9b5bf6304e8630b565d0626e2bdf329227a", ), "aarch64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/aa/cd/766b45fb6e090f20f8927d9c7cb34237d41c73a939358bc881883fd3a40d/coverage-7.6.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - "d85f5e9a5f8b73e2350097c3756ef7e785f55bd71205defa0bfdaf96c31616ff", + "https://files.pythonhosted.org/packages/a5/b6/bf054de41ec948b151ae2b79a55c107f5760979538f5fb80c195f2517718/coverage-7.10.7-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", + "4da86b6d62a496e908ac2898243920c7992499c1712ff7c2b6d837cc69d9467e", ), "x86_64-apple-darwin": ( - "https://files.pythonhosted.org/packages/ad/5f/67af7d60d7e8ce61a4e2ddcd1bd5fb787180c8d0ae0fbd073f903b3dd95d/coverage-7.6.1-cp311-cp311-macosx_10_9_x86_64.whl", - "7dea0889685db8550f839fa202744652e87c60015029ce3f60e006f8c4462c93", + "https://files.pythonhosted.org/packages/d2/5d/c1a17867b0456f2e9ce2d8d4708a4c3a089947d0bec9c66cdf60c9e7739f/coverage-7.10.7-cp311-cp311-macosx_10_9_x86_64.whl", + "a609f9c93113be646f44c2a0256d6ea375ad047005d7f57a5c15f614dc1b2f59", ), "x86_64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/14/6f/8351b465febb4dbc1ca9929505202db909c5a635c6fdf33e089bbc3d7d85/coverage-7.6.1-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "0c0420b573964c760df9e9e86d1a9a622d0d27f417e1a949a8a66dd7bcee7bc6", + "https://files.pythonhosted.org/packages/b0/ef/bd8e719c2f7417ba03239052e099b76ea1130ac0cbb183ee1fcaa58aaff3/coverage-7.10.7-cp311-cp311-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", + "35f5e3f9e455bb17831876048355dca0f758b6df22f49258cb5a91da23ef437d", ), }, "cp312": { "aarch64-apple-darwin": ( - "https://files.pythonhosted.org/packages/e1/ab/6bf00de5327ecb8db205f9ae596885417a31535eeda6e7b99463108782e1/coverage-7.6.1-cp312-cp312-macosx_11_0_arm64.whl", - "5621a9175cf9d0b0c84c2ef2b12e9f5f5071357c4d2ea6ca1cf01814f45d2391", + "https://files.pythonhosted.org/packages/37/66/593f9be12fc19fb36711f19a5371af79a718537204d16ea1d36f16bd78d2/coverage-7.10.7-cp312-cp312-macosx_11_0_arm64.whl", + "18afb24843cbc175687225cab1138c95d262337f5473512010e46831aa0c2973", ), "aarch64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/92/8f/2ead05e735022d1a7f3a0a683ac7f737de14850395a826192f0288703472/coverage-7.6.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - "260933720fdcd75340e7dbe9060655aff3af1f0c5d20f46b57f262ab6c86a5e8", + "https://files.pythonhosted.org/packages/98/2e/2dda59afd6103b342e096f246ebc5f87a3363b5412609946c120f4e7750d/coverage-7.10.7-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", + "c41e71c9cfb854789dee6fc51e46743a6d138b1803fab6cb860af43265b42ea6", ), "x86_64-apple-darwin": ( - "https://files.pythonhosted.org/packages/7e/d4/300fc921dff243cd518c7db3a4c614b7e4b2431b0d1145c1e274fd99bd70/coverage-7.6.1-cp312-cp312-macosx_10_9_x86_64.whl", - "95cae0efeb032af8458fc27d191f85d1717b1d4e49f7cb226cf526ff28179778", + "https://files.pythonhosted.org/packages/13/e4/eb12450f71b542a53972d19117ea5a5cea1cab3ac9e31b0b5d498df1bd5a/coverage-7.10.7-cp312-cp312-macosx_10_13_x86_64.whl", + "7bb3b9ddb87ef7725056572368040c32775036472d5a033679d1fa6c8dc08417", ), "x86_64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/1f/0f/c890339dd605f3ebc269543247bdd43b703cce6825b5ed42ff5f2d6122c7/coverage-7.6.1-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "c44fee9975f04b33331cb8eb272827111efc8930cfd582e0320613263ca849ca", + "https://files.pythonhosted.org/packages/a6/90/a64aaacab3b37a17aaedd83e8000142561a29eb262cede42d94a67f7556b/coverage-7.10.7-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", + "314f2c326ded3f4b09be11bc282eb2fc861184bc95748ae67b360ac962770be7", ), }, "cp313": { "aarch64-apple-darwin": ( - "https://files.pythonhosted.org/packages/b9/67/e1413d5a8591622a46dd04ff80873b04c849268831ed5c304c16433e7e30/coverage-7.6.1-cp313-cp313-macosx_11_0_arm64.whl", - "a6d3adcf24b624a7b778533480e32434a39ad8fa30c315208f6d3e5542aeb6e9", + "https://files.pythonhosted.org/packages/72/4f/732fff31c119bb73b35236dd333030f32c4bfe909f445b423e6c7594f9a2/coverage-7.10.7-cp313-cp313-macosx_11_0_arm64.whl", + "73ab1601f84dc804f7812dc297e93cd99381162da39c47040a827d4e8dafe63b", ), "aarch64-apple-darwin-freethreaded": ( - "https://files.pythonhosted.org/packages/c4/ae/b5d58dff26cade02ada6ca612a76447acd69dccdbb3a478e9e088eb3d4b9/coverage-7.6.1-cp313-cp313t-macosx_11_0_arm64.whl", - "502753043567491d3ff6d08629270127e0c31d4184c4c8d98f92c26f65019962", + "https://files.pythonhosted.org/packages/11/0b/91128e099035ece15da3445d9015e4b4153a6059403452d324cbb0a575fa/coverage-7.10.7-cp313-cp313t-macosx_11_0_arm64.whl", + "dd5e856ebb7bfb7672b0086846db5afb4567a7b9714b8a0ebafd211ec7ce6a15", ), "aarch64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/14/5b/9dec847b305e44a5634d0fb8498d135ab1d88330482b74065fcec0622224/coverage-7.6.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - "d0c212c49b6c10e6951362f7c6df3329f04c2b1c28499563d4035d964ab8e08c", + "https://files.pythonhosted.org/packages/b1/20/b6ea4f69bbb52dac0aebd62157ba6a9dddbfe664f5af8122dac296c3ee15/coverage-7.10.7-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", + "c79124f70465a150e89340de5963f936ee97097d2ef76c869708c4248c63ca49", ), "aarch64-unknown-linux-gnu-freethreaded": ( - "https://files.pythonhosted.org/packages/b8/d7/62095e355ec0613b08dfb19206ce3033a0eedb6f4a67af5ed267a8800642/coverage-7.6.1-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - "6a89ecca80709d4076b95f89f308544ec8f7b4727e8a547913a35f16717856cb", + "https://files.pythonhosted.org/packages/f7/08/16bee2c433e60913c610ea200b276e8eeef084b0d200bdcff69920bd5828/coverage-7.10.7-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", + "83082a57783239717ceb0ad584de3c69cf581b2a95ed6bf81ea66034f00401c0", + ), + "x86_64-apple-darwin": ( + "https://files.pythonhosted.org/packages/9a/94/b765c1abcb613d103b64fcf10395f54d69b0ef8be6a0dd9c524384892cc7/coverage-7.10.7-cp313-cp313-macosx_10_13_x86_64.whl", + "981a651f543f2854abd3b5fcb3263aac581b18209be49863ba575de6edf4c14d", + ), + "x86_64-apple-darwin-freethreaded": ( + "https://files.pythonhosted.org/packages/bb/22/e04514bf2a735d8b0add31d2b4ab636fc02370730787c576bb995390d2d5/coverage-7.10.7-cp313-cp313t-macosx_10_13_x86_64.whl", + "a0ec07fd264d0745ee396b666d47cef20875f4ff2375d7c4f58235886cc1ef0c", ), "x86_64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/f7/95/d2fd31f1d638df806cae59d7daea5abf2b15b5234016a5ebb502c2f3f7ee/coverage-7.6.1-cp313-cp313-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "78b260de9790fd81e69401c2dc8b17da47c8038176a79092a89cb2b7d945d060", + "https://files.pythonhosted.org/packages/a2/77/8c6d22bf61921a59bce5471c2f1f7ac30cd4ac50aadde72b8c48d5727902/coverage-7.10.7-cp313-cp313-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", + "10b6ba00ab1132a0ce4428ff68cf50a25efd6840a42cdf4239c9b99aad83be8b", ), "x86_64-unknown-linux-gnu-freethreaded": ( - "https://files.pythonhosted.org/packages/8b/61/a7a6a55dd266007ed3b1df7a3386a0d760d014542d72f7c2c6938483b7bd/coverage-7.6.1-cp313-cp313t-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "13b0a73a0896988f053e4fbb7de6d93388e6dd292b0d87ee51d106f2c11b465b", + "https://files.pythonhosted.org/packages/5d/22/9b8d458c2881b22df3db5bb3e7369e63d527d986decb6c11a591ba2364f7/coverage-7.10.7-cp313-cp313t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", + "1ef2319dd15a0b009667301a3f84452a4dc6fddfd06b0c5c53ea472d3989fbf0", ), }, - "cp38": { + "cp314": { "aarch64-apple-darwin": ( - "https://files.pythonhosted.org/packages/38/ea/cab2dc248d9f45b2b7f9f1f596a4d75a435cb364437c61b51d2eb33ceb0e/coverage-7.6.1-cp38-cp38-macosx_11_0_arm64.whl", - "f1adfc8ac319e1a348af294106bc6a8458a0f1633cc62a1446aebc30c5fa186a", + "https://files.pythonhosted.org/packages/f0/89/673f6514b0961d1f0e20ddc242e9342f6da21eaba3489901b565c0689f34/coverage-7.10.7-cp314-cp314-macosx_11_0_arm64.whl", + "212f8f2e0612778f09c55dd4872cb1f64a1f2b074393d139278ce902064d5b32", + ), + "aarch64-apple-darwin-freethreaded": ( + "https://files.pythonhosted.org/packages/f5/6f/f58d46f33db9f2e3647b2d0764704548c184e6f5e014bef528b7f979ef84/coverage-7.10.7-cp314-cp314t-macosx_11_0_arm64.whl", + "9fa6e4dd51fe15d8738708a973470f67a855ca50002294852e9571cdbd9433f2", ), "aarch64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/ca/6f/f82f9a500c7c5722368978a5390c418d2a4d083ef955309a8748ecaa8920/coverage-7.6.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - "a95324a9de9650a729239daea117df21f4b9868ce32e63f8b650ebe6cef5595b", + "https://files.pythonhosted.org/packages/ff/49/07f00db9ac6478e4358165a08fb41b469a1b053212e8a00cb02f0d27a05f/coverage-7.10.7-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", + "813922f35bd800dca9994c5971883cbc0d291128a5de6b167c7aa697fcf59360", + ), + "aarch64-unknown-linux-gnu-freethreaded": ( + "https://files.pythonhosted.org/packages/84/fd/193a8fb132acfc0a901f72020e54be5e48021e1575bb327d8ee1097a28fd/coverage-7.10.7-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", + "6e16e07d85ca0cf8bafe5f5d23a0b850064e8e945d5677492b06bbe6f09cc699", ), "x86_64-apple-darwin": ( - "https://files.pythonhosted.org/packages/81/d0/d9e3d554e38beea5a2e22178ddb16587dbcbe9a1ef3211f55733924bf7fa/coverage-7.6.1-cp38-cp38-macosx_10_9_x86_64.whl", - "6db04803b6c7291985a761004e9060b2bca08da6d04f26a7f2294b8623a0c1a0", + "https://files.pythonhosted.org/packages/23/9c/5844ab4ca6a4dd97a1850e030a15ec7d292b5c5cb93082979225126e35dd/coverage-7.10.7-cp314-cp314-macosx_10_13_x86_64.whl", + "b06f260b16ead11643a5a9f955bd4b5fd76c1a4c6796aeade8520095b75de520", + ), + "x86_64-apple-darwin-freethreaded": ( + "https://files.pythonhosted.org/packages/62/09/9a5608d319fa3eba7a2019addeacb8c746fb50872b57a724c9f79f146969/coverage-7.10.7-cp314-cp314t-macosx_10_13_x86_64.whl", + "a62c6ef0d50e6de320c270ff91d9dd0a05e7250cac2a800b7784bae474506e63", ), "x86_64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/e4/6e/885bcd787d9dd674de4a7d8ec83faf729534c63d05d51d45d4fa168f7102/coverage-7.6.1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "8929543a7192c13d177b770008bc4e8119f2e1f881d563fc6b6305d2d0ebe9de", + "https://files.pythonhosted.org/packages/82/62/14ed6546d0207e6eda876434e3e8475a3e9adbe32110ce896c9e0c06bb9a/coverage-7.10.7-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", + "bb45474711ba385c46a0bfe696c695a929ae69ac636cda8f532be9e8c93d720a", + ), + "x86_64-unknown-linux-gnu-freethreaded": ( + "https://files.pythonhosted.org/packages/0f/48/71a8abe9c1ad7e97548835e3cc1adbf361e743e9d60310c5f75c9e7bf847/coverage-7.10.7-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", + "affef7c76a9ef259187ef31599a9260330e0335a3011732c4b9effa01e1cd6e0", ), }, "cp39": { "aarch64-apple-darwin": ( - "https://files.pythonhosted.org/packages/a5/fe/137d5dca72e4a258b1bc17bb04f2e0196898fe495843402ce826a7419fe3/coverage-7.6.1-cp39-cp39-macosx_11_0_arm64.whl", - "547f45fa1a93154bd82050a7f3cddbc1a7a4dd2a9bf5cb7d06f4ae29fe94eaf8", + "https://files.pythonhosted.org/packages/52/2f/b9f9daa39b80ece0b9548bbb723381e29bc664822d9a12c2135f8922c22b/coverage-7.10.7-cp39-cp39-macosx_11_0_arm64.whl", + "bc91b314cef27742da486d6839b677b3f2793dfe52b51bbbb7cf736d5c29281c", ), "aarch64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/78/5b/a0a796983f3201ff5485323b225d7c8b74ce30c11f456017e23d8e8d1945/coverage-7.6.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - "645786266c8f18a931b65bfcefdbf6952dd0dea98feee39bd188607a9d307ed2", + "https://files.pythonhosted.org/packages/6a/92/1c1c5a9e8677ce56d42b97bdaca337b2d4d9ebe703d8c174ede52dbabd5f/coverage-7.10.7-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", + "c7315339eae3b24c2d2fa1ed7d7a38654cba34a13ef19fbcb9425da46d3dc594", ), "x86_64-apple-darwin": ( - "https://files.pythonhosted.org/packages/19/d3/d54c5aa83268779d54c86deb39c1c4566e5d45c155369ca152765f8db413/coverage-7.6.1-cp39-cp39-macosx_10_9_x86_64.whl", - "abd5fd0db5f4dc9289408aaf34908072f805ff7792632250dcb36dc591d24255", + "https://files.pythonhosted.org/packages/a3/ad/d1c25053764b4c42eb294aae92ab617d2e4f803397f9c7c8295caa77a260/coverage-7.10.7-cp39-cp39-macosx_10_9_x86_64.whl", + "fff7b9c3f19957020cac546c70025331113d2e61537f6e2441bc7657913de7d3", ), "x86_64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/9a/6f/eef79b779a540326fee9520e5542a8b428cc3bfa8b7c8f1022c1ee4fc66c/coverage-7.6.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "609b06f178fe8e9f89ef676532760ec0b4deea15e9969bf754b37f7c40326dbc", + "https://files.pythonhosted.org/packages/b0/49/8a070782ce7e6b94ff6a0b6d7c65ba6bc3091d92a92cef4cd4eb0767965c/coverage-7.10.7-cp39-cp39-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", + "2af88deffcc8a4d5974cf2d502251bc3b2db8461f0b66d80a449c33757aa9f40", ), }, } @@ -156,16 +184,14 @@ def coverage_dep(name, python_version, platform, visibility): """ if "windows" in platform: # NOTE @aignas 2023-01-19: currently we do not support windows as the - # upstream coverage wrapper is written in shell. Do not log any warning - # for now as it is not actionable. + # upstream coverage wrapper is written in shell. return None abi = "cp" + version_label(python_version) - url, sha256 = _coverage_deps.get(abi, {}).get(platform, (None, "")) + url, sha256 = _coverage_deps.get(abi, {}).get(platform, _default) - if url == None: - # Some wheels are not present for some builds, so let's silently ignore those. - return None + # NOTE @aignas 2026-07-27: if the default is matched, then the same file may be extracted + # multiple times. The wheel is small enough to not matter in most cases. maybe( http_archive, diff --git a/python/private/current_py_cc_headers.bzl b/python/private/current_py_cc_headers.bzl index 217904c22f..f7fcd8d738 100644 --- a/python/private/current_py_cc_headers.bzl +++ b/python/private/current_py_cc_headers.bzl @@ -12,19 +12,21 @@ # See the License for the specific language governing permissions and # limitations under the License. -"""Implementation of current_py_cc_headers rule.""" +"""Implementation of current_py_cc_headers and current_py_cc_headers_abi3 rules. +""" load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") +load("//python/private:toolchain_types.bzl", "PY_CC_TOOLCHAIN_TYPE") def _current_py_cc_headers_impl(ctx): - py_cc_toolchain = ctx.toolchains["//python/cc:toolchain_type"].py_cc_toolchain + py_cc_toolchain = ctx.toolchains[PY_CC_TOOLCHAIN_TYPE].py_cc_toolchain return py_cc_toolchain.headers.providers_map.values() current_py_cc_headers = rule( implementation = _current_py_cc_headers_impl, - toolchains = ["//python/cc:toolchain_type"], + toolchains = [PY_CC_TOOLCHAIN_TYPE], provides = [CcInfo], - doc = """\ + doc = """ Provides the currently active Python toolchain's C headers. This is a wrapper around the underlying `cc_library()` for the @@ -41,3 +43,40 @@ cc_library( ``` """, ) + +def _current_py_cc_headers_abi3_impl(ctx): + py_cc_toolchain = ctx.toolchains[PY_CC_TOOLCHAIN_TYPE].py_cc_toolchain + if not py_cc_toolchain.headers_abi3: + fail(( + "The resolved {} toolchain does not provide abi3 headers. " + + "Verify the toolchain sets `.headers_abi3`, or use the " + + "`:current_py_cc_headers` target." + ).format( + PY_CC_TOOLCHAIN_TYPE, + )) + return py_cc_toolchain.headers_abi3.providers_map.values() + +current_py_cc_headers_abi3 = rule( + implementation = _current_py_cc_headers_abi3_impl, + toolchains = [PY_CC_TOOLCHAIN_TYPE], + provides = [CcInfo], + doc = """ +Provides the currently active Python toolchain's C ABI3 headers. + +This is a wrapper around the underlying `cc_library()` for the +C ABI3 headers for the consuming target's currently active Python toolchain. + +To use, simply depend on this target where you would have wanted the +toolchain's underlying `:python_headers_abi3` target: + +```starlark +cc_library( + name = "foo", + deps = ["@rules_python//python/cc:current_py_cc_headers_abi3"] +) +``` + +:::{versionadded} 1.7.0 +::: +""", +) diff --git a/python/private/deprecation.bzl b/python/private/deprecation.bzl new file mode 100644 index 0000000000..70461c2fa1 --- /dev/null +++ b/python/private/deprecation.bzl @@ -0,0 +1,59 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Helper functions to deprecation utilities. +""" + +load("@rules_python_internal//:rules_python_config.bzl", "config") + +_DEPRECATION_MESSAGE = """ +The '{name}' symbol in '{old_load}' +is deprecated. It is an alias to the regular rule; use it directly instead: + +load("{new_load}", "{name}") + +{snippet} +""" + +def _symbol(kwargs, *, symbol_name, new_load, old_load, snippet = ""): + """An internal function to propagate the deprecation warning. + + This is not an API that should be used outside `rules_python`. + + Args: + kwargs: Arguments to modify. + symbol_name: {type}`str` the symbol name that is deprecated. + new_load: {type}`str` the new load location under `//`. + old_load: {type}`str` the symbol import location that we are deprecating. + snippet: {type}`str` the usage snippet of the new symbol. + + Returns: + The kwargs to be used in the macro creation. + """ + + if config.enable_deprecation_warnings: + deprecation = _DEPRECATION_MESSAGE.format( + name = symbol_name, + old_load = old_load, + new_load = new_load, + snippet = snippet, + ) + if kwargs.get("deprecation"): + deprecation = kwargs.get("deprecation") + "\n\n" + deprecation + kwargs["deprecation"] = deprecation + return kwargs + +with_deprecation = struct( + symbol = _symbol, +) diff --git a/python/private/enum.bzl b/python/private/enum.bzl index d71442e3b5..4d0fb10699 100644 --- a/python/private/enum.bzl +++ b/python/private/enum.bzl @@ -43,3 +43,23 @@ def enum(methods = {}, **kwargs): self = struct(__members__ = members, **kwargs) return self + +def _FlagEnum_flag_values(self): + return sorted(self.__members__.values()) + +def FlagEnum(**kwargs): + """Define an enum specialized for flags. + + Args: + **kwargs: members of the enum. + + Returns: + {type}`FlagEnum` struct. This is an enum with the following extras: + * `flag_values`: A function that returns a sorted list of the + flag values (enum `__members__`). Useful for passing to the + `values` attribute for string flags. + """ + return enum( + methods = dict(flag_values = _FlagEnum_flag_values), + **kwargs + ) diff --git a/python/private/envsubst.bzl b/python/private/envsubst.bzl index b2fdb99e1e..98dc72a818 100644 --- a/python/private/envsubst.bzl +++ b/python/private/envsubst.bzl @@ -20,9 +20,7 @@ def envsubst(template_string, varnames, getenv): Supports `$VARNAME`, `${VARNAME}` and `${VARNAME:-default}` syntaxes in the `template_string`, looking up each `VARNAME` listed in the `varnames` list in the environment defined by the - `getenv` function. Typically called with `getenv = rctx.getenv` - (if it is available) or `getenv = rctx.os.environ.get` (on e.g. - Bazel 6 or Bazel 7, which don't have `rctx.getenv` yet). + `getenv` function. Typically called with `getenv = rctx.getenv`. Limitations: Unlike the shell, we don't support `${VARNAME}` and `${VARNAME:-default}` in the default expression for a different diff --git a/python/private/flags.bzl b/python/private/flags.bzl index 9070f113ac..042e4e9838 100644 --- a/python/private/flags.bzl +++ b/python/private/flags.bzl @@ -19,27 +19,59 @@ unnecessary files when all that are needed are flag definitions. """ load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") -load(":enum.bzl", "enum") +load(":enum.bzl", "FlagEnum", "enum") -def _FlagEnum_flag_values(self): - return sorted(self.__members__.values()) +# Maps "--myflag" to a tuple of: +# +# - the flag's ctx.fragments native API accessor +# -"native|starlark": which definition to use if the flag is available both +# from ctx.fragments and Starlark +# +# Builds that set --incompatible_remove_ctx_py_fragment or +# --incompatible_remove_ctx_bazel_py_fragment disable ctx.fragments. These +# builds assume flags are solely defined in Starlark. +# +# The "native|starlark" override is only for devs who are testing flag +# Starlarkification. If ctx.fragments.[py|bazel_py] is available and +# a flag is set to "starlark", we exclusively read its starlark version. +# +# See https://github.com/bazel-contrib/rules_python/issues/3252. +_POSSIBLY_NATIVE_FLAGS = { + "build_python_zip": (lambda ctx: ctx.fragments.py.build_python_zip, "native"), + "default_to_explicit_init_py": (lambda ctx: ctx.fragments.py.default_to_explicit_init_py, "native"), + "python_import_all_repositories": (lambda ctx: ctx.fragments.bazel_py.python_import_all_repositories, "native"), + "python_path": (lambda ctx: ctx.fragments.bazel_py.python_path, "native"), +} -def FlagEnum(**kwargs): - """Define an enum specialized for flags. +def read_possibly_native_flag(ctx, flag_name): + """ + Canonical API for reading a Python build flag. + + Flags might be defined in Starlark or native-Bazel. This function reasd flags + from tbe correct source based on supporting Bazel version and --incompatible* + flags that disable native references. Args: - **kwargs: members of the enum. + ctx: Rule's configuration context. + flag_name: Name of the flag to read, without preceding "--". Returns: - {type}`FlagEnum` struct. This is an enum with the following extras: - * `flag_values`: A function that returns a sorted list of the - flag values (enum `__members__`). Useful for passing to the - `values` attribute for string flags. + The flag's value. """ - return enum( - methods = dict(flag_values = _FlagEnum_flag_values), - **kwargs - ) + + # Bazel 9.0+ can disable these fragments with --incompatible_remove_ctx_py_fragment and + # --incompatible_remove_ctx_bazel_py_fragment. Disabling them means bazel expects + # Python to read Starlark flags. + use_native_def = hasattr(ctx.fragments, "py") and hasattr(ctx.fragments, "bazel_py") + + # Developer override to force the Starlark definition for testing. + if _POSSIBLY_NATIVE_FLAGS[flag_name][1] == "starlark": + use_native_def = False + if use_native_def: + return _POSSIBLY_NATIVE_FLAGS[flag_name][0](ctx) + else: + # Starlark definition of "--foo" is assumed to be a label dependency named "_foo". + return getattr(ctx.attr, "_" + flag_name + "_flag")[BuildSettingInfo].value def _AddSrcsToRunfilesFlag_is_enabled(ctx): value = ctx.attr._add_srcs_to_runfiles_flag[BuildSettingInfo].value @@ -55,8 +87,59 @@ AddSrcsToRunfilesFlag = FlagEnum( is_enabled = _AddSrcsToRunfilesFlag_is_enabled, ) +def _ValidateTestMainFlag_is_enabled(ctx): + value = ctx.attr._validate_test_main_flag[BuildSettingInfo].value + if value == ValidateTestMainFlag.AUTO: + # Default off; intended to be flipped to enabled in a future major + # version (e.g. rules_python 3.0). + value = ValidateTestMainFlag.DISABLED + return value == ValidateTestMainFlag.ENABLED + +# Determines if py_test runs a validation action that statically checks the +# main module actually runs tests (instead of silently passing). +# buildifier: disable=name-conventions +ValidateTestMainFlag = FlagEnum( + # Automatically decide the effective value; currently resolves to disabled. + AUTO = "auto", + # Run the validation action. + ENABLED = "enabled", + # Don't run the validation action. + DISABLED = "disabled", + is_enabled = _ValidateTestMainFlag_is_enabled, +) + +def _string_flag_impl(ctx): + if ctx.attr.override: + value = ctx.attr.override + else: + value = ctx.build_setting_value + + if value not in ctx.attr.values: + fail(( + "Invalid value for {name}: got {value}, must " + + "be one of {allowed}" + ).format( + name = ctx.label, + value = value, + allowed = ctx.attr.values, + )) + + return [ + BuildSettingInfo(value = value), + config_common.FeatureFlagInfo(value = value), + ] + +string_flag = rule( + implementation = _string_flag_impl, + build_setting = config.string(flag = True), + attrs = { + "override": attr.string(), + "values": attr.string_list(), + }, +) + def _bootstrap_impl_flag_get_value(ctx): - return ctx.attr._bootstrap_impl_flag[BuildSettingInfo].value + return ctx.attr._bootstrap_impl_flag[config_common.FeatureFlagInfo].value # buildifier: disable=name-conventions BootstrapImplFlag = enum( @@ -123,6 +206,37 @@ PrecompileSourceRetentionFlag = enum( get_effective_value = _precompile_source_retention_flag_get_effective_value, ) +def _venvs_use_declare_symlink_flag_get_value(ctx): + return ctx.attr._venvs_use_declare_symlink_flag[BuildSettingInfo].value + +# Decides if the venv created by bootstrap=script uses declare_file() to +# create relative symlinks. Workaround for #2489 (packaging rules not supporting +# declare_link() files). +# buildifier: disable=name-conventions +VenvsUseDeclareSymlinkFlag = FlagEnum( + # Use declare_file() and relative symlinks in the venv + YES = "yes", + # Do not use declare_file() and relative symlinks in the venv + NO = "no", + get_value = _venvs_use_declare_symlink_flag_get_value, +) + +def _venvs_site_packages_is_enabled(ctx): + if not ctx.attr.experimental_venvs_site_packages: + return False + flag_value = ctx.attr.experimental_venvs_site_packages[BuildSettingInfo].value + return flag_value == VenvsSitePackages.YES + +# Decides if libraries try to use a site-packages layout using venv_symlinks +# buildifier: disable=name-conventions +VenvsSitePackages = FlagEnum( + # Use venv_symlinks + YES = "yes", + # Don't use venv_symlinks + NO = "no", + is_enabled = _venvs_site_packages_is_enabled, +) + # Used for matching freethreaded toolchains and would have to be used in wheels # as well. # buildifier: disable=name-conventions @@ -133,6 +247,9 @@ FreeThreadedFlag = enum( NO = "no", ) +def _libc_flag_get_value(ctx): + return ctx.attr._py_linux_libc_flag[BuildSettingInfo].value + # Determines which libc flavor is preferred when selecting the toolchain and # linux whl distributions. # @@ -142,4 +259,5 @@ LibcFlag = FlagEnum( GLIBC = "glibc", # Prefer musl wheels (e.g. musllinux_2_17_x86_64) MUSL = "musl", + get_value = _libc_flag_get_value, ) diff --git a/python/private/full_version.bzl b/python/private/full_version.bzl index 0292d6c77d..0be5b44daf 100644 --- a/python/private/full_version.bzl +++ b/python/private/full_version.bzl @@ -14,12 +14,13 @@ """A small helper to ensure that we are working with full versions.""" -def full_version(*, version, minor_mapping): +def full_version(*, version, minor_mapping, fail_on_err = True): """Return a full version. Args: version: {type}`str` the version in `X.Y` or `X.Y.Z` format. minor_mapping: {type}`dict[str, str]` mapping between `X.Y` to `X.Y.Z` format. + fail_on_err: {type}`bool` whether to fail on error or return `None` instead. Returns: a full version given the version string. If the string is already a @@ -31,6 +32,8 @@ def full_version(*, version, minor_mapping): parts = version.split(".") if len(parts) == 3: return version + elif not fail_on_err: + return None elif len(parts) == 2: fail( "Unknown Python version '{}', available values are: {}".format( diff --git a/python/private/get_local_runtime_info.py b/python/private/get_local_runtime_info.py index 0207f56bef..a4f622700b 100644 --- a/python/private/get_local_runtime_info.py +++ b/python/private/get_local_runtime_info.py @@ -11,10 +11,254 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. +"""Returns information about the local Python runtime as JSON.""" +import glob import json +import os +import platform import sys import sysconfig +from typing import Any + +_IS_WINDOWS = sys.platform == "win32" +_IS_DARWIN = sys.platform == "darwin" + + +def _get_abi_flags(get_config) -> str: + """Returns the ABI flags for the Python runtime.""" + # sys.abiflags may not exist, but it still may be set in the config. + abi_flags = getattr(sys, "abiflags", None) + if abi_flags is None: + abi_flags = get_config("ABIFLAGS") or get_config("abiflags") or "" + return abi_flags + + +def _search_directories(get_config, base_executable) -> list[str]: + """Returns a list of library directories to search for shared libraries.""" + # There's several types of libraries with different names and a plethora + # of settings, and many different config variables to check: + # + # LIBPL is used in python-config when shared library is not enabled: + # https://github.com/python/cpython/blob/v3.12.0/Misc/python-config.in#L63 + # + # LIBDIR may also be the python directory with library files. + # https://stackoverflow.com/questions/47423246/get-pythons-lib-path + # See also: MULTIARCH + # + # On MacOS, the LDLIBRARY may be a relative path under /Library/Frameworks, + # such as "Python.framework/Versions/3.12/Python", not a file under the + # LIBDIR/LIBPL directory, so include PYTHONFRAMEWORKPREFIX. + lib_dirs = [get_config(x) for x in ("PYTHONFRAMEWORKPREFIX", "LIBPL", "LIBDIR")] + + # On Debian, with multiarch enabled, prior to Python 3.10, `LIBDIR` didn't + # tell the location of the libs, just the base directory. The `MULTIARCH` + # sysconfig variable tells the subdirectory within it with the libs. + # See: + # https://wiki.debian.org/Python/MultiArch + # https://git.launchpad.net/ubuntu/+source/python3.12/tree/debian/changelog#n842 + multiarch = get_config("MULTIARCH") + if multiarch: + for x in ("LIBPL", "LIBDIR"): + config_value = get_config(x) + if config_value and not config_value.endswith(multiarch): + lib_dirs.append(os.path.join(config_value, multiarch)) + + if not _IS_DARWIN: + for exec_dir in ( + os.path.dirname(base_executable) if base_executable else None, + get_config("BINDIR"), + ): + if not exec_dir: + continue + if _IS_WINDOWS: + # On Windows DLLs go in the same directory as the executable, while .lib + # files live in the lib/ or libs/ subdirectory. + lib_dirs.append(exec_dir) + lib_dirs.append(os.path.join(exec_dir, "lib")) + lib_dirs.append(os.path.join(exec_dir, "libs")) + else: + # On most non-windows systems the executable is in a bin/ directory and + # the libraries are in a sibling lib/ directory. + lib_dirs.append(os.path.join(os.path.dirname(exec_dir), "lib")) + + # Dedup and remove empty values, keeping the order. + return list(dict.fromkeys(d for d in lib_dirs if d)) + + +def _default_library_names(version, abi_flags) -> tuple[str, ...]: + """Returns a list of default library files to search for shared libraries.""" + if _IS_WINDOWS: + return ( + f"python{version}{abi_flags}.dll", + f"python{version}.dll", + ) + elif _IS_DARWIN: + return ( + f"libpython{version}{abi_flags}.dylib", + f"libpython{version}.dylib", + ) + else: + return ( + f"libpython{version}{abi_flags}.so", + f"libpython{version}.so", + f"libpython{version}{abi_flags}.so.1.0", + f"libpython{version}.so.1.0", + ) + + +def _search_library_names(get_config, version, abi_flags) -> list[str]: + """Returns a list of library files to search for shared libraries.""" + # Quoting configure.ac in the cpython code base: + # "INSTSONAME is the name of the shared library that will be use to install + # on the system - some systems like version suffix, others don't."" + # + # A typical INSTSONAME is 'libpython3.8.so.1.0' on Linux, or + # 'Python.framework/Versions/3.9/Python' on MacOS. + # + # A typical LDLIBRARY is 'libpythonX.Y.so' on Linux, or 'pythonXY.dll' on + # Windows, or 'Python.framework/Versions/3.9/Python' on MacOS. + # + # A typical LIBRARY is 'libpythonX.Y.a' on Linux. + lib_names = [ + get_config(x) + for x in ( + "LDLIBRARY", + "INSTSONAME", + "PY3LIBRARY", + "LIBRARY", + "DLLLIBRARY", + ) + ] + + # Include the default libraries for the system. + lib_names.extend(_default_library_names(version, abi_flags)) + + # Also include the abi3 libraries for the system. + lib_names.extend(_default_library_names(sys.version_info.major, abi_flags)) + + # Uniqify, preserving order. + return list(dict.fromkeys(k for k in lib_names if k)) + + +def _get_python_library_info(base_executable) -> dict[str, Any]: + """Returns a dictionary with the static and dynamic python libraries.""" + config_vars = sysconfig.get_config_vars() + + # VERSION is X.Y in Linux/macOS and XY in Windows. This is used to + # construct library paths such as python3.12, so ensure it exists. + version = config_vars.get("VERSION") + if not version: + if _IS_WINDOWS: + version = f"{sys.version_info.major}{sys.version_info.minor}" + else: + version = f"{sys.version_info.major}.{sys.version_info.minor}" + + defines = [] + if config_vars.get("Py_GIL_DISABLED", "0") == "1": + defines.append("Py_GIL_DISABLED") + + # Avoid automatically linking the libraries on windows via pydefine.h + # pragma comment(lib ...) + if _IS_WINDOWS: + defines.append("Py_NO_LINK_LIB") + + # sys.abiflags may not exist, but it still may be set in the config. + abi_flags = _get_abi_flags(config_vars.get) + + search_directories = _search_directories(config_vars.get, base_executable) + search_libnames = _search_library_names(config_vars.get, version, abi_flags) + + # Used to test whether the library is an abi3 library or a full api library. + abi3_libraries = _default_library_names(sys.version_info.major, abi_flags) + + # Found libraries + static_libraries: dict[str, None] = {} + dynamic_libraries: dict[str, None] = {} + interface_libraries: dict[str, None] = {} + abi_dynamic_libraries: dict[str, None] = {} + abi_interface_libraries: dict[str, None] = {} + + for root_dir in search_directories: + for libname in search_libnames: + composed_path = os.path.join(root_dir, libname) + is_abi3_file = os.path.basename(composed_path) in abi3_libraries + + # Check whether the library exists and add it to the appropriate list. + if os.path.exists(composed_path) or os.path.isdir(composed_path): + if is_abi3_file: + if not libname.endswith(".a"): + abi_dynamic_libraries[composed_path] = None + elif libname.endswith(".a"): + static_libraries[composed_path] = None + else: + dynamic_libraries[composed_path] = None + + interface_path = None + if libname.endswith(".dll"): + # On windows a .lib file may be an "import library" or a static + # library. The file could be inspected to determine which it is; + # typically python is used as a shared library. + # + # On Windows, extensions should link with the pythonXY.lib interface + # libraries. + # + # See: https://docs.python.org/3/extending/windows.html + # https://learn.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-creation + interface_path = os.path.join(root_dir, libname[:-3] + "lib") + elif libname.endswith(".so"): + # It's possible, though unlikely, that interface stubs (.ifso) exist. + interface_path = os.path.join(root_dir, libname[:-2] + "ifso") + + # Check whether an interface library exists. + if interface_path and os.path.exists(interface_path): + if is_abi3_file: + abi_interface_libraries[interface_path] = None + else: + interface_libraries[interface_path] = None + + # Additional DLLs are needed on Windows to link properly. + dlls = [] + if _IS_WINDOWS: + dlls.extend(glob.glob(os.path.join(os.path.dirname(base_executable), "*.dll"))) + dlls = [ + x + for x in dlls + if x not in dynamic_libraries and x not in abi_dynamic_libraries + ] + + def _unique_basenames(inputs: dict[str, None]) -> list[str]: + """Returns a list of paths, keeping only the first path for each basename.""" + result = [] + seen = set() + for k in inputs: + b = os.path.basename(k) + if b not in seen: + seen.add(b) + result.append(k) + return result + + # When no libraries are found it's likely that the python interpreter is not + # configured to use shared or static libraries (minilinux). If this seems + # suspicious try running `uv tool run find_libpython --list-all -v` + return { + "dynamic_libraries": _unique_basenames(dynamic_libraries), + "static_libraries": _unique_basenames(static_libraries), + "interface_libraries": _unique_basenames(interface_libraries), + "abi_dynamic_libraries": _unique_basenames(abi_dynamic_libraries), + "abi_interface_libraries": _unique_basenames(abi_interface_libraries), + "abi_flags": abi_flags, + "soabi": config_vars.get("SOABI") or "", + "shlib_suffix": ".dylib" if _IS_DARWIN else "", + "additional_dlls": dlls, + "defines": defines, + } + + +def _get_base_executable() -> str: + """Returns the base executable path.""" + return getattr(sys, "_base_executable", None) or sys.executable + data = { "major": sys.version_info.major, @@ -22,28 +266,11 @@ "micro": sys.version_info.micro, "include": sysconfig.get_path("include"), "implementation_name": sys.implementation.name, + "base_executable": _get_base_executable(), + "sys_platform": sys.platform, + # Normalize to lowercase: on Windows, platform.machine() returns uppercase + # "AMD64" / "ARM64", whereas PEP 508 and toolchains expect lowercase. + "platform_machine": platform.machine().lower(), } - -config_vars = [ - # The libpythonX.Y.so file. Usually? - # It might be a static archive (.a) file instead. - "LDLIBRARY", - # The directory with library files. Supposedly. - # It's not entirely clear how to get the directory with libraries. - # There's several types of libraries with different names and a plethora - # of settings. - # https://stackoverflow.com/questions/47423246/get-pythons-lib-path - # For now, it seems LIBDIR has what is needed, so just use that. - "LIBDIR", - # The versioned libpythonX.Y.so.N file. Usually? - # It might be a static archive (.a) file instead. - "INSTSONAME", - # The libpythonX.so file. Usually? - # It might be a static archive (a.) file instead. - "PY3LIBRARY", - # The platform-specific filename suffix for library files. - # Includes the dot, e.g. `.so` - "SHLIB_SUFFIX", -] -data.update(zip(config_vars, sysconfig.get_config_vars(*config_vars))) +data.update(_get_python_library_info(_get_base_executable())) print(json.dumps(data)) diff --git a/python/private/glob_excludes.bzl b/python/private/glob_excludes.bzl deleted file mode 100644 index c98afe0ae2..0000000000 --- a/python/private/glob_excludes.bzl +++ /dev/null @@ -1,32 +0,0 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"Utilities for glob exclusions." - -load(":util.bzl", "IS_BAZEL_7_4_OR_HIGHER") - -def _version_dependent_exclusions(): - """Returns glob exclusions that are sensitive to Bazel version. - - Returns: - a list of glob exclusion patterns - """ - if IS_BAZEL_7_4_OR_HIGHER: - return [] - else: - return ["**/* *"] - -glob_excludes = struct( - version_dependent_exclusions = _version_dependent_exclusions, -) diff --git a/python/private/hermetic_runtime_repo_setup.bzl b/python/private/hermetic_runtime_repo_setup.bzl index 64d721ecad..20b0324894 100644 --- a/python/private/hermetic_runtime_repo_setup.bzl +++ b/python/private/hermetic_runtime_repo_setup.bzl @@ -18,11 +18,11 @@ load("@rules_cc//cc:cc_library.bzl", "cc_library") load("//python:py_runtime.bzl", "py_runtime") load("//python:py_runtime_pair.bzl", "py_runtime_pair") load("//python/cc:py_cc_toolchain.bzl", "py_cc_toolchain") -load(":glob_excludes.bzl", "glob_excludes") load(":py_exec_tools_toolchain.bzl", "py_exec_tools_toolchain") -load(":semver.bzl", "semver") +load(":version.bzl", "version") -_IS_FREETHREADED = Label("//python/config_settings:is_py_freethreaded") +_IS_FREETHREADED_YES = Label("//python/config_settings:_is_py_freethreaded_yes") +_IS_FREETHREADED_NO = Label("//python/config_settings:_is_py_freethreaded_no") def define_hermetic_runtime_toolchain_impl( *, @@ -53,47 +53,55 @@ def define_hermetic_runtime_toolchain_impl( use. """ _ = name # @unused - version_info = semver(python_version) - version_dict = version_info.to_dict() + version_info = version.parse(python_version) + version_dict = { + "major": version_info.release[0], + "minor": version_info.release[1], + } + files_include = [ + "bin/**", + "extensions/**", + "include/**", + "libs/**", + "share/**", + ] + files_include += extra_files_glob_include + files_exclude = [ + # Unused shared libraries. `python` executable and the `:libpython` target + # depend on `libpython{python_version}.so.1.0`. + "lib/libpython{major}.{minor}*.so".format(**version_dict), + # static libraries + "lib/**/*.a", + # tests for the standard libraries. + "lib/python{major}.{minor}*/**/test/**".format(**version_dict), + "lib/python{major}.{minor}*/**/tests/**".format(**version_dict), + # During pyc creation, temp files named *.pyc.NNN are created + "**/__pycache__/*.pyc.*", + ] + files_exclude += extra_files_glob_exclude + native.filegroup( name = "files", srcs = native.glob( - include = [ - "bin/**", - "extensions/**", - "include/**", - "libs/**", - "share/**", - ] + extra_files_glob_include, + include = files_include, # Platform-agnostic filegroup can't match on all patterns. allow_empty = True, - exclude = [ - # Unused shared libraries. `python` executable and the `:libpython` target - # depend on `libpython{python_version}.so.1.0`. - "lib/libpython{major}.{minor}*.so".format(**version_dict), - # static libraries - "lib/**/*.a", - # tests for the standard libraries. - "lib/python{major}.{minor}*/**/test/**".format(**version_dict), - "lib/python{major}.{minor}*/**/tests/**".format(**version_dict), - # During pyc creation, temp files named *.pyc.NNN are created - "**/__pycache__/*.pyc.*", - ] + glob_excludes.version_dependent_exclusions() + extra_files_glob_exclude, + exclude = files_exclude, ), ) cc_import( name = "interface", interface_library = select({ - _IS_FREETHREADED: "libs/python{major}{minor}t.lib".format(**version_dict), - "//conditions:default": "libs/python{major}{minor}.lib".format(**version_dict), + _IS_FREETHREADED_YES: "libs/python{major}{minor}t.lib".format(**version_dict), + _IS_FREETHREADED_NO: "libs/python{major}{minor}.lib".format(**version_dict), }), system_provided = True, ) cc_import( name = "abi3_interface", interface_library = select({ - _IS_FREETHREADED: "libs/python3t.lib", - "//conditions:default": "libs/python3.lib", + _IS_FREETHREADED_YES: "libs/python3t.lib", + _IS_FREETHREADED_NO: "libs/python3.lib", }), system_provided = True, ) @@ -103,24 +111,32 @@ def define_hermetic_runtime_toolchain_impl( srcs = native.glob(["include/**/*.h"]), ) cc_library( - name = "python_headers", + name = "python_headers_abi3", deps = select({ - "@bazel_tools//src/conditions:windows": [":interface", ":abi3_interface"], + "@bazel_tools//src/conditions:windows": [":abi3_interface"], "//conditions:default": None, }), hdrs = [":includes"], includes = [ "include", ] + select({ - _IS_FREETHREADED: [ + _IS_FREETHREADED_YES: [ "include/python{major}.{minor}t".format(**version_dict), ], - "//conditions:default": [ + _IS_FREETHREADED_NO: [ "include/python{major}.{minor}".format(**version_dict), "include/python{major}.{minor}m".format(**version_dict), ], }), ) + cc_library( + name = "python_headers", + hdrs = [":includes"], + deps = [":python_headers_abi3"] + select({ + "@bazel_tools//src/conditions:windows": [":interface"], + "//conditions:default": [], + }), + ) native.config_setting( name = "is_freethreaded_linux", flag_values = { @@ -169,10 +185,6 @@ def define_hermetic_runtime_toolchain_impl( "libs/python{major}{minor}t.lib".format(**version_dict), "libs/python3t.lib", ], - "@platforms//os:linux": [ - "lib/libpython{major}.{minor}.so".format(**version_dict), - "lib/libpython{major}.{minor}.so.1.0".format(**version_dict), - ], "@platforms//os:macos": ["lib/libpython{major}.{minor}.dylib".format(**version_dict)], "@platforms//os:windows": [ "python3.dll", @@ -180,6 +192,10 @@ def define_hermetic_runtime_toolchain_impl( "libs/python{major}{minor}.lib".format(**version_dict), "libs/python3.lib", ], + "//conditions:default": [ + "lib/libpython{major}.{minor}.so".format(**version_dict), + "lib/libpython{major}.{minor}.so.1.0".format(**version_dict), + ], }), ) @@ -192,15 +208,25 @@ def define_hermetic_runtime_toolchain_impl( values = {"collect_code_coverage": "true"}, visibility = ["//visibility:private"], ) + if not version_info.pre: + releaselevel = "final" + else: + releaselevel = { + "a": "alpha", + "b": "beta", + "rc": "candidate", + }.get(version_info.pre[0]) py_runtime( name = "py3_runtime", files = [":files"], interpreter = python_bin, interpreter_version_info = { - "major": str(version_info.major), - "micro": str(version_info.patch), - "minor": str(version_info.minor), + "major": str(version_info.release[0]), + "micro": str(version_info.release[2]), + "minor": str(version_info.release[1]), + "releaselevel": releaselevel, + "serial": str(version_info.pre[1]) if version_info.pre else "0", }, coverage_tool = select({ # Convert empty string to None @@ -211,8 +237,20 @@ def define_hermetic_runtime_toolchain_impl( implementation_name = "cpython", # See https://peps.python.org/pep-3147/ for pyc tag infix format pyc_tag = select({ - _IS_FREETHREADED: "cpython-{major}{minor}t".format(**version_dict), - "//conditions:default": "cpython-{major}{minor}".format(**version_dict), + _IS_FREETHREADED_YES: "cpython-{major}{minor}t".format(**version_dict), + _IS_FREETHREADED_NO: "cpython-{major}{minor}".format(**version_dict), + }), + # On Windows, a symlink-style venv requires supporting .dll files. + venv_bin_files = select({ + "@platforms//os:windows": native.glob( + include = [ + "*.dll", + ], + # This must be true because glob empty-ness is checked + # during loading phase, before select() filters it out. + allow_empty = True, + ), + "//conditions:default": [], }), ) @@ -225,6 +263,8 @@ def define_hermetic_runtime_toolchain_impl( py_cc_toolchain( name = "py_cc_toolchain", headers = ":python_headers", + headers_abi3 = ":python_headers_abi3", + # TODO #3155: add libctl, libtk libs = ":libpython", python_version = python_version, ) diff --git a/python/private/internal_config_repo.bzl b/python/private/internal_config_repo.bzl index 7b6869e9a5..8ee6a2d017 100644 --- a/python/private/internal_config_repo.bzl +++ b/python/private/internal_config_repo.bzl @@ -18,26 +18,29 @@ such as globals available to Bazel versions, or propagating user environment settings for rules to later use. """ -_ENABLE_PYSTAR_ENVVAR_NAME = "RULES_PYTHON_ENABLE_PYSTAR" -_ENABLE_PYSTAR_DEFAULT = "1" +load("//python/private:text_util.bzl", "render") +load(":repo_utils.bzl", "repo_utils") -_CONFIG_TEMPLATE = """\ +_ENABLE_DEPRECATION_WARNINGS_ENVVAR_NAME = "RULES_PYTHON_DEPRECATION_WARNINGS" +_ENABLE_DEPRECATION_WARNINGS_DEFAULT = "0" + +_CONFIG_TEMPLATE = """ config = struct( - enable_pystar = {enable_pystar}, + build_python_zip_default = {build_python_zip_default}, + supports_whl_extraction = {supports_whl_extraction}, + enable_pystar = True, + enable_deprecation_warnings = {enable_deprecation_warnings}, + extract_needs_chmod = {extract_needs_chmod}, + bazel_8_or_later = {bazel_8_or_later}, + bazel_9_or_later = {bazel_9_or_later}, + bazel_10_or_later = {bazel_10_or_later}, BuiltinPyInfo = getattr(getattr(native, "legacy_globals", None), "PyInfo", {builtin_py_info_symbol}), BuiltinPyRuntimeInfo = getattr(getattr(native, "legacy_globals", None), "PyRuntimeInfo", {builtin_py_runtime_info_symbol}), BuiltinPyCcLinkParamsProvider = getattr(getattr(native, "legacy_globals", None), "PyCcLinkParamsProvider", {builtin_py_cc_link_params_provider}), ) """ -# The py_internal symbol is only accessible from within @rules_python, so we have to -# load it from there and re-export it so that rules_python can later load it. -_PY_INTERNAL_SHIM = """\ -load("@rules_python//tools/build_defs/python/private:py_internal_renamed.bzl", "py_internal_renamed") -py_internal_impl = py_internal_renamed -""" - -ROOT_BUILD_TEMPLATE = """\ +ROOT_BUILD_TEMPLATE = """ load("@bazel_skylib//:bzl_library.bzl", "bzl_library") package( @@ -47,28 +50,49 @@ package( ) bzl_library( - name = "rules_python_config_bzl", - srcs = ["rules_python_config.bzl"] + name = "extra_transition_settings", + srcs = ["extra_transition_settings.bzl"], ) bzl_library( - name = "py_internal_bzl", - srcs = ["py_internal.bzl"], - deps = [{py_internal_dep}], + name = "rules_python_config", + srcs = ["rules_python_config.bzl"], ) """ -def _internal_config_repo_impl(rctx): - pystar_requested = _bool_from_environ(rctx, _ENABLE_PYSTAR_ENVVAR_NAME, _ENABLE_PYSTAR_DEFAULT) +_EXTRA_TRANSITIONS_TEMPLATE = """ +# Generated by @rules_python//python/private:internal_config_repo.bzl +# +# For a list of what modules added what labels, see +# transition_settings_debug.txt + +EXTRA_TRANSITION_SETTINGS = {labels} +""" + +_TRANSITION_SETTINGS_DEBUG_TEMPLATE = """ +# Generated by @rules_python//python/private:internal_config_repo.bzl + +{lines} +""" - # Bazel 7+ (dev and later) has native.starlark_doc_extract, and thus the - # py_internal global, which are necessary for the pystar implementation. - if pystar_requested and hasattr(native, "starlark_doc_extract"): - enable_pystar = pystar_requested +def _internal_config_repo_impl(rctx): + # An empty version signifies a development build, which is treated as + # the latest version. + if native.bazel_version: + version_parts = native.bazel_version.split(".") + bazel_major_version = int(version_parts[0]) + bazel_minor_version = int(version_parts[1]) else: - enable_pystar = False + bazel_major_version = 99999 + bazel_minor_version = 99999 + + supports_whl_extraction = False + extract_needs_chmod = bazel_major_version < 8 or (bazel_major_version == 8 and bazel_minor_version < 6) + if bazel_major_version >= 8: + # Extracting .whl files requires Bazel 8.3.0 or later. + if bazel_major_version > 8 or bazel_minor_version >= 3: + supports_whl_extraction = True - if not native.bazel_version or int(native.bazel_version.split(".")[0]) >= 8: builtin_py_info_symbol = "None" builtin_py_runtime_info_symbol = "None" builtin_py_cc_link_params_provider = "None" @@ -78,38 +102,48 @@ def _internal_config_repo_impl(rctx): builtin_py_cc_link_params_provider = "PyCcLinkParamsProvider" rctx.file("rules_python_config.bzl", _CONFIG_TEMPLATE.format( - enable_pystar = enable_pystar, + build_python_zip_default = repo_utils.get_platforms_os_name(rctx) == "windows", + enable_deprecation_warnings = _bool_from_environ(rctx, _ENABLE_DEPRECATION_WARNINGS_ENVVAR_NAME, _ENABLE_DEPRECATION_WARNINGS_DEFAULT), builtin_py_info_symbol = builtin_py_info_symbol, builtin_py_runtime_info_symbol = builtin_py_runtime_info_symbol, + supports_whl_extraction = str(supports_whl_extraction), + extract_needs_chmod = str(extract_needs_chmod), builtin_py_cc_link_params_provider = builtin_py_cc_link_params_provider, + bazel_8_or_later = str(bazel_major_version >= 8), + bazel_9_or_later = str(bazel_major_version >= 9), + bazel_10_or_later = str(bazel_major_version > 9), )) - if enable_pystar: - shim_content = _PY_INTERNAL_SHIM - py_internal_dep = '"@rules_python//tools/build_defs/python/private:py_internal_renamed_bzl"' - else: - shim_content = "py_internal_impl = None\n" - py_internal_dep = "" - - # Bazel 5 doesn't support repository visibility, so just use public - # as a stand-in - if native.bazel_version.startswith("5."): - visibility = "//visibility:public" - else: - visibility = "@rules_python//:__subpackages__" - rctx.file("BUILD", ROOT_BUILD_TEMPLATE.format( - py_internal_dep = py_internal_dep, - visibility = visibility, + visibility = "@rules_python//:__subpackages__", )) - rctx.file("py_internal.bzl", shim_content) + + rctx.file( + "extra_transition_settings.bzl", + _EXTRA_TRANSITIONS_TEMPLATE.format( + labels = render.list(rctx.attr.transition_settings), + ), + ) + debug_lines = [ + "{} added by modules: {}".format(setting, ", ".join(sorted(requesters))) + for setting, requesters in rctx.attr.transition_setting_generators.items() + ] + rctx.file( + "transition_settings_debug.txt", + _TRANSITION_SETTINGS_DEBUG_TEMPLATE.format(lines = "\n".join(debug_lines)), + ) + return None internal_config_repo = repository_rule( implementation = _internal_config_repo_impl, configure = True, - environ = [_ENABLE_PYSTAR_ENVVAR_NAME], + environ = [], + attrs = { + "transition_setting_generators": attr.string_list_dict(), + "transition_settings": attr.string_list(), + }, ) def _bool_from_environ(rctx, key, default): - return bool(int(rctx.os.environ.get(key, default))) + return bool(int(rctx.getenv(key, default))) diff --git a/python/private/internal_deps.bzl b/python/private/internal_deps.bzl deleted file mode 100644 index 6ea3fa40c7..0000000000 --- a/python/private/internal_deps.bzl +++ /dev/null @@ -1,22 +0,0 @@ -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"Python toolchain module extension for internal rule use" - -load("@bazel_skylib//lib:modules.bzl", "modules") -load("//python/private/pypi:deps.bzl", "pypi_deps") -load(":internal_config_repo.bzl", "internal_config_repo") - -def _internal_deps(): - internal_config_repo(name = "rules_python_internal") - pypi_deps() - -internal_deps = modules.as_extension( - _internal_deps, - doc = "This extension registers internal rules_python dependencies.", -) diff --git a/python/private/internal_dev_deps.bzl b/python/private/internal_dev_deps.bzl index 2a3b84e7df..5bbee232f9 100644 --- a/python/private/internal_dev_deps.bzl +++ b/python/private/internal_dev_deps.bzl @@ -14,28 +14,151 @@ """Module extension for internal dev_dependency=True setup.""" load("@bazel_ci_rules//:rbe_repo.bzl", "rbe_preconfig") -load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_file") +load("//python/private/pypi:whl_library.bzl", "whl_library") +load("//tests/support/whl_from_dir:whl_from_dir_repo.bzl", "whl_from_dir_repo") +load(":runtime_env_repo.bzl", "runtime_env_repo") def _internal_dev_deps_impl(mctx): _ = mctx # @unused - # This wheel is purely here to validate the wheel extraction code. It's not - # intended for anything else. - http_file( - name = "wheel_for_testing", - downloaded_file_path = "numpy-1.25.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - sha256 = "0d60fbae8e0019865fc4784745814cff1c421df5afee233db6d88ab4f14655a2", - urls = [ - "https://files.pythonhosted.org/packages/50/67/3e966d99a07d60a21a21d7ec016e9e4c2642a86fea251ec68677daf71d4d/numpy-1.25.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - ], - ) - # Creates a default toolchain config for RBE. # Use this as is if you are using the rbe_ubuntu16_04 container, # otherwise refer to RBE docs. rbe_preconfig( name = "buildkite_config", - toolchain = "ubuntu1804-bazel-java11", + toolchain = "ubuntu2404", + ) + runtime_env_repo(name = "rules_python_runtime_env_tc_info") + + # Setup for //tests/whl_with_build_files + whl_from_dir_repo( + name = "whl_with_build_files", + root = "//tests/whl_with_build_files:testdata/BUILD.bazel", + output = "somepkg-1.0-any-none-any.whl", + ) + whl_library( + name = "somepkg_with_build_files", + whl_file = "@whl_with_build_files//:somepkg-1.0-any-none-any.whl", + requirement = "somepkg", + ) + + # Setup for //tests/implicit_namespace_packages + whl_from_dir_repo( + name = "implicit_namespace_ns_sub1_whl", + root = "//tests/implicit_namespace_packages:testdata/ns-sub1/BUILD.bazel", + output = "ns_sub1-1.0-any-none-any.whl", + ) + whl_library( + name = "implicit_namespace_ns_sub1", + whl_file = "@implicit_namespace_ns_sub1_whl//:ns_sub1-1.0-any-none-any.whl", + requirement = "ns-sub1", + enable_implicit_namespace_pkgs = False, + ) + + whl_from_dir_repo( + name = "implicit_namespace_ns_sub2_whl", + root = "//tests/implicit_namespace_packages:testdata/ns-sub2/BUILD.bazel", + output = "ns_sub2-1.0-any-none-any.whl", + ) + whl_library( + name = "implicit_namespace_ns_sub2", + whl_file = "@implicit_namespace_ns_sub2_whl//:ns_sub2-1.0-any-none-any.whl", + requirement = "ns-sub2", + enable_implicit_namespace_pkgs = False, + ) + + whl_from_dir_repo( + name = "pkgutil_nspkg1_whl", + root = "//tests/repos/pkgutil_nspkg1:BUILD.bazel", + output = "pkgutil_nspkg1-1.0-any-none-any.whl", + ) + whl_library( + name = "pkgutil_nspkg1", + whl_file = "@pkgutil_nspkg1_whl//:pkgutil_nspkg1-1.0-any-none-any.whl", + requirement = "pkgutil_nspkg1", + enable_implicit_namespace_pkgs = False, + ) + + whl_from_dir_repo( + name = "pkgutil_nspkg2_whl", + root = "//tests/repos/pkgutil_nspkg2:BUILD.bazel", + output = "pkgutil_nspkg2-1.0-any-none-any.whl", + ) + whl_library( + name = "pkgutil_nspkg2", + whl_file = "@pkgutil_nspkg2_whl//:pkgutil_nspkg2-1.0-any-none-any.whl", + requirement = "pkgutil_nspkg2", + enable_implicit_namespace_pkgs = False, + ) + + whl_from_dir_repo( + name = "whl_with_data1_whl", + root = "//tests/repos/whl_with_data1:BUILD.bazel", + output = "whl_with_data1-1.0-any-none-any.whl", + ) + whl_library( + name = "whl_with_data1", + whl_file = "@whl_with_data1_whl//:whl_with_data1-1.0-any-none-any.whl", + requirement = "whl-with-data1", + ) + + whl_from_dir_repo( + name = "whl_with_data2_whl", + root = "//tests/repos/whl_with_data2:BUILD.bazel", + output = "whl_with_data2-1.0-any-none-any.whl", + ) + whl_library( + name = "whl_with_data2", + whl_file = "@whl_with_data2_whl//:whl_with_data2-1.0-any-none-any.whl", + requirement = "whl-with-data2", + ) + + _whl_library_from_dir( + name = "whl_library_extras_direct_dep", + root = "//tests/pypi/whl_library/testdata/pkg:BUILD.bazel", + output = "pkg-1.0-any-none-any.whl", + requirement = "pkg[optional]", + # The following is necessary to enable pipstar and make tests faster + config_load = "@rules_python//tests/pypi/whl_library/testdata:packages.bzl", + dep_template = "@whl_library_extras_{name}//:{target}", + ) + _whl_library_from_dir( + name = "whl_library_extras_optional_dep", + root = "//tests/pypi/whl_library/testdata/optional_dep:BUILD.bazel", + output = "optional_dep-1.0-any-none-any.whl", + requirement = "optional_dep", + # The following is necessary to enable pipstar and make tests faster + config_load = "@rules_python//tests/pypi/whl_library/testdata:packages.bzl", + ) + + # Setup for //tests/pypi/patch_whl/patch_whl_patch_test.py + whl_from_dir_repo( + name = "patch_whl_pkg_whl", + root = "//tests/pypi/patch_whl/testdata/pkg:BUILD.bazel", + output = "pkg-1.0-any-none-any.whl", + ) + whl_library( + name = "patch_whl_pkg", + whl_file = "@patch_whl_pkg_whl//:pkg-1.0-any-none-any.whl", + requirement = "pkg", + whl_patches = { + "//tests/pypi/patch_whl/testdata/patches:modify_pkg.patch": json.encode({ + "patch_strip": 0, + "whls": ["pkg-1.0-any-none-any.whl"], + }), + }, + ) + +def _whl_library_from_dir(*, name, output, root, **kwargs): + whl_from_dir_repo( + name = "{}_whl".format(name), + root = root, + output = output, + ) + whl_library( + name = name, + whl_file = "@{}_whl//:{}".format(name, output), + **kwargs ) internal_dev_deps = module_extension( diff --git a/python/private/interpreter.bzl b/python/private/interpreter.bzl new file mode 100644 index 0000000000..b281be9639 --- /dev/null +++ b/python/private/interpreter.bzl @@ -0,0 +1,82 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Implementation of the rules to access the underlying Python interpreter.""" + +load("@bazel_skylib//lib:paths.bzl", "paths") +load("//python:py_runtime_info.bzl", "PyRuntimeInfo") +load(":common.bzl", "runfiles_root_path") +load(":sentinel_impl.bzl", "SentinelInfo") +load(":toolchain_types.bzl", "TARGET_TOOLCHAIN_TYPE") + +def _interpreter_binary_impl(ctx): + if SentinelInfo in ctx.attr.binary: + toolchain = ctx.toolchains[TARGET_TOOLCHAIN_TYPE] + runtime = toolchain.py3_runtime + else: + runtime = ctx.attr.binary[PyRuntimeInfo] + + # NOTE: We name the output filename after the underlying file name + # because of things like pyenv: they use $0 to determine what to + # re-exec. If it's not a recognized name, then they fail. + if runtime.interpreter: + # In order for this to work both locally and remotely, we create a + # shell script here that re-exec's into the real interpreter. Ideally, + # we'd just use a symlink, but that breaks under certain conditions. If + # we use a ctx.actions.symlink(target=...) then it fails under remote + # execution. If we use ctx.actions.symlink(target_path=...) then it + # behaves differently inside the runfiles tree and outside the runfiles + # tree. + # + # This currently does not work on Windows. Need to find a way to enable + # that. + executable = ctx.actions.declare_file(runtime.interpreter.basename) + ctx.actions.expand_template( + template = ctx.file._template, + output = executable, + substitutions = { + "%target_file%": runfiles_root_path(ctx, runtime.interpreter.short_path), + }, + is_executable = True, + ) + else: + executable = ctx.actions.declare_symlink(paths.basename(runtime.interpreter_path)) + ctx.actions.symlink(output = executable, target_path = runtime.interpreter_path) + + return [ + DefaultInfo( + executable = executable, + runfiles = ctx.runfiles([executable], transitive_files = runtime.files).merge_all([ + ctx.attr._bash_runfiles[DefaultInfo].default_runfiles, + ]), + ), + ] + +interpreter_binary = rule( + implementation = _interpreter_binary_impl, + toolchains = [TARGET_TOOLCHAIN_TYPE], + executable = True, + attrs = { + "binary": attr.label( + mandatory = True, + ), + "_bash_runfiles": attr.label( + default = "@bazel_tools//tools/bash/runfiles", + ), + "_template": attr.label( + default = "//python/private:interpreter_tmpl.sh", + allow_single_file = True, + ), + }, +) diff --git a/python/private/interpreter_tmpl.sh b/python/private/interpreter_tmpl.sh new file mode 100644 index 0000000000..c4e87fbb43 --- /dev/null +++ b/python/private/interpreter_tmpl.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash + +# --- begin runfiles.bash initialization v3 --- +# Copy-pasted from the Bazel Bash runfiles library v3. +set -uo pipefail; set +e; f=bazel_tools/tools/bash/runfiles/runfiles.bash +# shellcheck disable=SC1090 +source "${RUNFILES_DIR:-/dev/null}/$f" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "${RUNFILES_MANIFEST_FILE:-/dev/null}" | cut -f2- -d' ')" 2>/dev/null || \ + source "$0.runfiles/$f" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "$0.runfiles_manifest" | cut -f2- -d' ')" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "$0.exe.runfiles_manifest" | cut -f2- -d' ')" 2>/dev/null || \ + { echo>&2 "ERROR: cannot find $f"; exit 1; }; f=; set -e +# --- end runfiles.bash initialization v3 --- + +set +e # allow us to check for errors more easily +readonly TARGET_FILE="%target_file%" +MAIN_BIN=$(rlocation "$TARGET_FILE") + +if [[ -z "$MAIN_BIN" || ! -e "$MAIN_BIN" ]]; then + echo "ERROR: interpreter executable not found: $MAIN_BIN (from $TARGET_FILE)" + exit 1 +fi +exec "${MAIN_BIN}" "$@" diff --git a/python/private/local_runtime_repo.bzl b/python/private/local_runtime_repo.bzl index fb1a8e29ac..f5597db613 100644 --- a/python/private/local_runtime_repo.bzl +++ b/python/private/local_runtime_repo.bzl @@ -31,27 +31,101 @@ load("@rules_python//python/private:local_runtime_repo_setup.bzl", "define_local define_local_runtime_toolchain_impl( name = "local_runtime", - lib_ext = "{lib_ext}", major = "{major}", minor = "{minor}", micro = "{micro}", - interpreter_path = "{interpreter_path}", - implementation_name = "{implementation_name}", + abi_flags = "{abi_flags}", + soabi = "{soabi}", os = "{os}", + implementation_name = "{implementation_name}", + interpreter_path = "{interpreter_path}", + interface_library = {interface_library}, + libraries = {libraries}, + defines = {defines}, + abi3_interface_library = {abi3_interface_library}, + abi3_libraries = {abi3_libraries}, + additional_dlls = {additional_dlls}, + sys_platform = "{sys_platform}", + platform_machine = "{platform_machine}", ) """ +def _expand_incompatible_template(): + return _TOOLCHAIN_IMPL_TEMPLATE.format( + major = "0", + minor = "0", + micro = "0", + abi_flags = "", + soabi = "", + os = "@platforms//:incompatible", + implementation_name = "incompatible", + interpreter_path = "/incompatible", + interface_library = "None", + libraries = "[]", + defines = "[]", + abi3_interface_library = "None", + abi3_libraries = "[]", + additional_dlls = "[]", + sys_platform = "", + platform_machine = "", + ) + +def _norm_path(path): + """Returns a path using '/' separators and no trailing slash.""" + path = path.replace("\\", "/") + if path[-1] == "/": + path = path[:-1] + return path + +def _symlink_libraries(rctx, logger, libraries, shlib_suffix): + """Symlinks the shared libraries into the lib/ directory. + + Individual files are symlinked instead of the whole directory because + shared_lib_dirs contains multiple search paths for the shared libraries, + and the python files may be missing from any of those directories, and + any of those directories may include non-python runtime libraries, + as would be the case if LIBDIR were, for example, /usr/lib. + + Args: + rctx: A repository_ctx object + logger: A repo_utils.logger object + libraries: paths to libraries to attempt to symlink. + shlib_suffix: Optional. Ensure that the generated symlinks end with this suffix. + Returns: + A list of library paths (under lib/) linked by the action. + """ + result = [] + for source in libraries: + origin = rctx.path(source) + if not origin.exists: + # The reported names don't always exist; it depends on the particulars + # of the runtime installation. + continue + if shlib_suffix and not origin.basename.endswith(shlib_suffix): + target = "lib/{}{}".format(origin.basename, shlib_suffix) + else: + target = "lib/{}".format(origin.basename) + logger.debug(lambda: "Symlinking {} to {}".format(origin, target)) + rctx.watch(origin) + rctx.symlink(origin, target) + result.append(target) + return result + def _local_runtime_repo_impl(rctx): logger = repo_utils.logger(rctx) on_failure = rctx.attr.on_failure - result = _resolve_interpreter_path(rctx) - if not result.resolved_path: + def _emit_log(msg): if on_failure == "fail": - fail("interpreter not found: {}".format(result.describe_failure())) + logger.fail(msg) + elif on_failure == "warn": + logger.warn(msg) + else: + logger.debug(msg) - if on_failure == "warn": - logger.warn(lambda: "interpreter not found: {}".format(result.describe_failure())) + result = _resolve_interpreter_path(rctx) + if not result.resolved_path: + _emit_log(lambda: "interpreter not found: {}".format(result.describe_failure())) # else, on_failure must be skip rctx.file("BUILD.bazel", _expand_incompatible_template()) @@ -72,10 +146,7 @@ def _local_runtime_repo_impl(rctx): logger = logger, ) if exec_result.return_code != 0: - if on_failure == "fail": - fail("GetPythonInfo failed: {}".format(exec_result.describe_failure())) - if on_failure == "warn": - logger.warn(lambda: "GetPythonInfo failed: {}".format(exec_result.describe_failure())) + _emit_log(lambda: "GetPythonInfo failed: {}".format(exec_result.describe_failure())) # else, on_failure must be skip rctx.file("BUILD.bazel", _expand_incompatible_template()) @@ -84,53 +155,86 @@ def _local_runtime_repo_impl(rctx): info = json.decode(exec_result.stdout) logger.info(lambda: _format_get_info_result(info)) + # We use base_executable because we want the path within a Python + # installation directory ("PYTHONHOME"). The problems with sys.executable + # are: + # * If we're in an activated venv, then we don't want the venv's + # `bin/python3` path to be used -- it isn't an actual Python installation. + # * If sys.executable is a wrapper (e.g. pyenv), then (1) it may not be + # located within an actual Python installation directory, and (2) it + # can interfer with Python recognizing when it's within a venv. + # + # In some cases, it may be a symlink (usually e.g. `python3->python3.12`), + # but we don't realpath() it to respect what it has decided is the + # appropriate path. + interpreter_path = info["base_executable"] + # NOTE: Keep in sync with recursive glob in define_local_runtime_toolchain_impl - repo_utils.watch_tree(rctx, rctx.path(info["include"])) + include_path = rctx.path(info["include"]) + + # The reported include path may not exist, and watching a non-existant + # path is an error. Silently skip, since includes are only necessary + # if C extensions are built. + if include_path.exists and include_path.is_dir: + rctx.watch_tree(include_path) + else: + pass # The cc_library.includes values have to be non-absolute paths, otherwise # the toolchain will give an error. Work around this error by making them # appear as part of this repo. - rctx.symlink(info["include"], "include") - - shared_lib_names = [ - info["PY3LIBRARY"], - info["LDLIBRARY"], - info["INSTSONAME"], - ] + logger.debug(lambda: "Symlinking {} to include".format(include_path)) + rctx.symlink(include_path, "include") - # In some cases, the value may be empty. Not clear why. - shared_lib_names = [v for v in shared_lib_names if v] - - # In some cases, the same value is returned for multiple keys. Not clear why. - shared_lib_names = {v: None for v in shared_lib_names}.keys() - shared_lib_dir = info["LIBDIR"] - - # The specific files are symlinked instead of the whole directory - # because it can point to a directory that has more than just - # the Python runtime shared libraries, e.g. /usr/lib, or a Python - # specific directory with pip-installed shared libraries. rctx.report_progress("Symlinking external Python shared libraries") - for name in shared_lib_names: - origin = rctx.path("{}/{}".format(shared_lib_dir, name)) - # The reported names don't always exist; it depends on the particulars - # of the runtime installation. - if origin.exists: - repo_utils.watch(rctx, origin) - rctx.symlink(origin, "lib/" + name) + interface_library = None + if info["dynamic_libraries"]: + libraries = _symlink_libraries(rctx, logger, info["dynamic_libraries"][:1], info["shlib_suffix"]) + symlinked = _symlink_libraries(rctx, logger, info["interface_libraries"][:1], None) + if symlinked: + interface_library = symlinked[0] + else: + libraries = _symlink_libraries(rctx, logger, info["static_libraries"], None) + if not libraries: + logger.info("No python libraries found.") + + abi3_interface_library = None + if info["abi_dynamic_libraries"]: + abi3_libraries = _symlink_libraries(rctx, logger, info["abi_dynamic_libraries"][:1], info["shlib_suffix"]) + symlinked = _symlink_libraries(rctx, logger, info["abi_interface_libraries"][:1], None) + if symlinked: + abi3_interface_library = symlinked[0] + else: + abi3_libraries = [] + logger.info("No abi3 python libraries found.") - rctx.file("WORKSPACE", "") - rctx.file("MODULE.bazel", "") - rctx.file("REPO.bazel", "") - rctx.file("BUILD.bazel", _TOOLCHAIN_IMPL_TEMPLATE.format( + additional_dlls = _symlink_libraries(rctx, logger, info["additional_dlls"], None) + + build_bazel = _TOOLCHAIN_IMPL_TEMPLATE.format( major = info["major"], minor = info["minor"], micro = info["micro"], - interpreter_path = interpreter_path, - lib_ext = info["SHLIB_SUFFIX"], - implementation_name = info["implementation_name"], + abi_flags = info["abi_flags"], + soabi = info["soabi"], os = "@platforms//os:{}".format(repo_utils.get_platforms_os_name(rctx)), - )) + implementation_name = info["implementation_name"], + interpreter_path = _norm_path(interpreter_path), + interface_library = repr(interface_library), + libraries = repr(libraries), + defines = repr(info["defines"]), + abi3_interface_library = repr(abi3_interface_library), + abi3_libraries = repr(abi3_libraries), + additional_dlls = repr(additional_dlls), + sys_platform = info["sys_platform"], + platform_machine = info["platform_machine"], + ) + logger.debug(lambda: "BUILD.bazel\n{}".format(build_bazel)) + + rctx.file("WORKSPACE", "") + rctx.file("MODULE.bazel", "") + rctx.file("REPO.bazel", "") + rctx.file("BUILD.bazel", build_bazel) local_runtime_repo = repository_rule( implementation = _local_runtime_repo_impl, @@ -151,13 +255,37 @@ a system having the necessary Python installed. doc = """ An absolute path or program name on the `PATH` env var. +*Mutually exclusive with `interpreter_target`.* + Values with slashes are assumed to be the path to a program. Otherwise, it is treated as something to search for on `PATH` Note that, when a plain program name is used, the path to the interpreter is resolved at repository evalution time, not runtime of any resulting binaries. + +If not set, defaults to `python3`. + +:::{seealso} +The {obj}`interpreter_target` attribute for getting the interpreter from +a label +::: +""", + default = "", + ), + "interpreter_target": attr.label( + doc = """ +A label to a Python interpreter executable. + +*Mutually exclusive with `interpreter_path`.* + +On Windows, if the path doesn't exist, various suffixes will be tried to +find a usable path. + +:::{seealso} +The {obj}`interpreter_path` attribute for getting the interpreter from +a path or PATH environment lookup. +::: """, - default = "python3", ), "on_failure": attr.string( default = _OnFailure.SKIP, @@ -183,19 +311,39 @@ How to handle errors when trying to automatically determine settings. ), "_rule_name": attr.string(default = "local_runtime_repo"), }, - environ = ["PATH", REPO_DEBUG_ENV_VAR], + environ = ["PATH", REPO_DEBUG_ENV_VAR, "DEVELOPER_DIR", "XCODE_VERSION"], ) -def _expand_incompatible_template(): - return _TOOLCHAIN_IMPL_TEMPLATE.format( - interpreter_path = "/incompatible", - implementation_name = "incompatible", - lib_ext = "incompatible", - major = "0", - minor = "0", - micro = "0", - os = "@platforms//:incompatible", +def _find_python_exe_from_target(rctx): + base_path = rctx.path(rctx.attr.interpreter_target) + if base_path.exists: + return base_path, None + attempted_paths = [base_path] + + # Try to convert a unix-y path to a Windows path. On Linux/Mac, + # the path is usually `bin/python3`. On Windows, it's simply + # `python.exe`. + basename = base_path.basename.rstrip("3") + path = base_path.dirname.dirname.get_child(basename) + path = rctx.path("{}.exe".format(path)) + if path.exists: + return path, None + attempted_paths.append(path) + + # Try adding .exe to the base path + path = rctx.path("{}.exe".format(base_path)) + if path.exists: + return path, None + attempted_paths.append(path) + + describe_failure = lambda: ( + "Target '{target}' could not be resolved to a valid path. " + + "Attempted paths: {paths}" + ).format( + target = rctx.attr.interpreter_target, + paths = "\n".join([str(p) for p in attempted_paths]), ) + return None, describe_failure def _resolve_interpreter_path(rctx): """Find the absolute path for an interpreter. @@ -210,20 +358,27 @@ def _resolve_interpreter_path(rctx): returns a description of why it couldn't be resolved A path object or None. The path may not exist. """ - if "/" not in rctx.attr.interpreter_path and "\\" not in rctx.attr.interpreter_path: - # Provide a bit nicer integration with pyenv: recalculate the runtime if the - # user changes the python version using e.g. `pyenv shell` - repo_utils.getenv(rctx, "PYENV_VERSION") - result = repo_utils.which_unchecked(rctx, rctx.attr.interpreter_path) - resolved_path = result.binary - describe_failure = result.describe_failure + if rctx.attr.interpreter_path and rctx.attr.interpreter_target: + fail("interpreter_path and interpreter_target are mutually exclusive") + + if rctx.attr.interpreter_target: + resolved_path, describe_failure = _find_python_exe_from_target(rctx) else: - repo_utils.watch(rctx, rctx.attr.interpreter_path) - resolved_path = rctx.path(rctx.attr.interpreter_path) - if not resolved_path.exists: - describe_failure = lambda: "Path not found: {}".format(repr(rctx.attr.interpreter_path)) + interpreter_path = rctx.attr.interpreter_path or "python3" + if "/" not in interpreter_path and "\\" not in interpreter_path: + # Provide a bit nicer integration with pyenv: recalculate the runtime if the + # user changes the python version using e.g. `pyenv shell` + rctx.getenv("PYENV_VERSION") + result = repo_utils.which_unchecked(rctx, interpreter_path) + resolved_path = result.binary + describe_failure = result.describe_failure else: - describe_failure = None + rctx.watch(interpreter_path) + resolved_path = rctx.path(interpreter_path) + if not resolved_path.exists: + describe_failure = lambda: "Path not found: {}".format(repr(interpreter_path)) + else: + describe_failure = None return struct( resolved_path = resolved_path, diff --git a/python/private/local_runtime_repo_setup.bzl b/python/private/local_runtime_repo_setup.bzl index 37eab59575..05ca6228e0 100644 --- a/python/private/local_runtime_repo_setup.bzl +++ b/python/private/local_runtime_repo_setup.bzl @@ -11,10 +11,10 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. - """Setup code called by the code generated by `local_runtime_repo`.""" load("@bazel_skylib//lib:selects.bzl", "selects") +load("@rules_cc//cc:cc_import.bzl", "cc_import") load("@rules_cc//cc:cc_library.bzl", "cc_library") load("@rules_python//python:py_runtime.bzl", "py_runtime") load("@rules_python//python:py_runtime_pair.bzl", "py_runtime_pair") @@ -25,13 +25,22 @@ _PYTHON_VERSION_FLAG = Label("@rules_python//python/config_settings:python_versi def define_local_runtime_toolchain_impl( name, - lib_ext, major, minor, micro, - interpreter_path, + abi_flags, + os, implementation_name, - os): + interpreter_path, + interface_library, + libraries, + defines, + abi3_interface_library, + abi3_libraries, + additional_dlls, + sys_platform = "", + platform_machine = "", + soabi = ""): """Defines a toolchain implementation for a local Python runtime. Generates public targets: @@ -45,47 +54,109 @@ def define_local_runtime_toolchain_impl( Args: name: `str` Only present to satisfy tooling - lib_ext: `str` The file extension for the `libpython` shared libraries major: `str` The major Python version, e.g. `3` of `3.9.1`. minor: `str` The minor Python version, e.g. `9` of `3.9.1`. micro: `str` The micro Python version, e.g. "1" of `3.9.1`. - interpreter_path: `str` Absolute path to the interpreter. - implementation_name: `str` The implementation name, as returned by - `sys.implementation.name`. + abi_flags: `str` The abi flags, as returned by `sys.abiflags`. os: `str` A label to the OS constraint (e.g. `@platforms//os:linux`) for this runtime. + implementation_name: `str` The implementation name, as returned by + `sys.implementation.name`. + interpreter_path: `str` Absolute path to the interpreter. + interface_library: `str` Path to the interface library. + e.g. "lib/python312.lib" + libraries: `list[str]` Path[s] to the python libraries. + e.g. ["lib/python312.dll"] or ["lib/python312.so"] + defines: `list[str]` List of additional defines. + abi3_interface_library: `str` Path to the interface library. + e.g. "lib/python3.lib" + abi3_libraries: `list[str]` Path[s] to the python libraries. + e.g. ["lib/python3.dll"] or ["lib/python3.so"] + additional_dlls: `list[str]` Path[s] to additional DLLs. + e.g. ["lib/msvcrt123.dll"] + sys_platform: `str` The PEP 508 `sys_platform` marker, e.g. 'linux', 'darwin', 'win32'. + platform_machine: `str` The PEP 508 `platform_machine` marker, e.g. 'x86_64', 'aarch64'. + soabi: `str` The SOABI tag for extension modules, e.g. 'cpython-311-x86_64-linux-gnu'. """ major_minor = "{}.{}".format(major, minor) major_minor_micro = "{}.{}".format(major_minor, micro) - cc_library( - name = "_python_headers", - # NOTE: Keep in sync with watch_tree() called in local_runtime_repo + # To build Python C/C++ extension on Windows, we need to link to python import library pythonXY.lib + # See https://docs.python.org/3/extending/windows.html + # However not all python installations (such as manylinux) include shared or static libraries, + # so only create the import library when interface_library is set. + if interface_library: + cc_import( + name = "interface", + interface_library = interface_library, + system_provided = True, + ) + + if abi3_interface_library: + cc_import( + name = "abi3_interface", + interface_library = abi3_interface_library, + system_provided = True, + ) + + native.filegroup( + name = "includes", srcs = native.glob( - ["include/**/*.h"], - # A Python install may not have C headers - allow_empty = True, + include = ["include/**/*.h"], + exclude = ["include/numpy/**"], # numpy headers are handled separately + allow_empty = True, # A Python install may not have C headers ), + ) + + # header libraries. + cc_library( + name = "python_headers_abi3", + hdrs = [":includes"], includes = ["include"], + defines = defines, # NOTE: Users should define Py_LIMITED_API=3 + deps = select({ + "@bazel_tools//src/conditions:windows": [":abi3_interface"], + "//conditions:default": [], + }), ) cc_library( - name = "_libpython", - # Don't use a recursive glob because the lib/ directory usually contains - # a subdirectory of the stdlib -- lots of unrelated files - srcs = native.glob( - [ - "lib/*{}".format(lib_ext), # Match libpython*.so - "lib/*{}*".format(lib_ext), # Also match libpython*.so.1.0 - ], - # A Python install may not have shared libraries. - allow_empty = True, - ), - hdrs = [":_python_headers"], + name = "python_headers", + hdrs = [":includes"], + includes = ["include"], + defines = defines, + deps = select({ + "@bazel_tools//src/conditions:windows": [":interface"], + "//conditions:default": [], + }), + ) + + # python libraries + cc_library( + name = "libpython_abi3", + hdrs = [":includes"], + defines = defines, # NOTE: Users should define Py_LIMITED_API=3 + srcs = abi3_libraries + additional_dlls, + deps = select({ + "@bazel_tools//src/conditions:windows": [":abi3_interface"], + "//conditions:default": [], + }), + ) + + cc_library( + name = "libpython", + hdrs = [":includes"], + defines = defines, + srcs = libraries + additional_dlls, + deps = select({ + "@bazel_tools//src/conditions:windows": [":interface"], + "//conditions:default": [], + }), ) + # runtime configuration py_runtime( - name = "_py3_runtime", + name = "py3_runtime", interpreter_path = interpreter_path, python_version = "PY3", interpreter_version_info = { @@ -94,12 +165,27 @@ def define_local_runtime_toolchain_impl( "minor": minor, }, implementation_name = implementation_name, + abi_flags = abi_flags, + pyc_tag = "{}-{}{}{}".format(implementation_name, major, minor, abi_flags), + venv_bin_files = select({ + "@platforms//os:windows": native.glob( + include = [ + "lib/*.dll", + # The pdb files just provide debugging information + "lib/*.pdb", + ], + # This must be true because glob empty-ness is checked + # during loading phase, before select() filters it out. + allow_empty = True, + ), + "//conditions:default": [], + }), ) py_runtime_pair( name = "python_runtimes", py2_runtime = None, - py3_runtime = ":_py3_runtime", + py3_runtime = ":py3_runtime", visibility = ["//visibility:public"], ) @@ -111,9 +197,14 @@ def define_local_runtime_toolchain_impl( py_cc_toolchain( name = "py_cc_toolchain", - headers = ":_python_headers", - libs = ":_libpython", + abi_flags = abi_flags, + soabi = soabi, + headers = ":python_headers", + headers_abi3 = ":python_headers_abi3", + libs = ":libpython", + platform_machine = platform_machine, python_version = major_minor_micro, + sys_platform = sys_platform, visibility = ["//visibility:public"], ) diff --git a/python/private/local_runtime_toolchains_repo.bzl b/python/private/local_runtime_toolchains_repo.bzl index adb3bb560d..8ef5ee9728 100644 --- a/python/private/local_runtime_toolchains_repo.bzl +++ b/python/private/local_runtime_toolchains_repo.bzl @@ -26,6 +26,9 @@ define_local_toolchain_suites( name = "toolchains", version_aware_repo_names = {version_aware_names}, version_unaware_repo_names = {version_unaware_names}, + repo_exec_compatible_with = {repo_exec_compatible_with}, + repo_target_compatible_with = {repo_target_compatible_with}, + repo_target_settings = {repo_target_settings}, ) """ @@ -39,6 +42,9 @@ def _local_runtime_toolchains_repo(rctx): rctx.file("BUILD.bazel", _TOOLCHAIN_TEMPLATE.format( version_aware_names = render.list(rctx.attr.runtimes), + repo_target_settings = render.string_list_dict(rctx.attr.target_settings), + repo_target_compatible_with = render.string_list_dict(rctx.attr.target_compatible_with), + repo_exec_compatible_with = render.string_list_dict(rctx.attr.exec_compatible_with), version_unaware_names = render.list(rctx.attr.default_runtimes or rctx.attr.runtimes), )) @@ -62,8 +68,36 @@ These will be defined as *version-unaware* toolchains. This means they will match any Python version. As such, they are registered after the version-aware toolchains defined by the `runtimes` attribute. +If not set, then the `runtimes` values will be used. + Note that order matters: it determines the toolchain priority within the package. +""", + ), + "exec_compatible_with": attr.string_list_dict( + doc = """ +Constraints that must be satisfied by an exec platform for a toolchain to be used. + +This is a `dict[str, list[str]]`, where the keys are repo names from the +`runtimes` or `default_runtimes` args, and the values are constraint +target labels (e.g. OS, CPU, etc). + +:::{note} +Specify `@//foo:bar`, not simply `//foo:bar` or `:bar`. The additional `@` is +needed because the strings are evaluated in a different context than where +they originate. +::: + +The list of settings become the {obj}`toolchain.exec_compatible_with` value for +each respective repo. + +This allows a local toolchain to only be used if certain exec platform +conditions are met, typically values from `@platforms`. + +See the [Local toolchains] docs for examples and further information. + +:::{versionadded} 1.5.0 +::: """, ), "runtimes": attr.string_list( @@ -76,6 +110,81 @@ are registered before `default_runtimes`. Note that order matters: it determines the toolchain priority within the package. +""", + ), + "target_compatible_with": attr.string_list_dict( + doc = """ +Constraints that must be satisfied for a toolchain to be used. + + +This is a `dict[str, list[str]]`, where the keys are repo names from the +`runtimes` or `default_runtimes` args, and the values are constraint +target labels (e.g. OS, CPU, etc), or the special string `"HOST_CONSTRAINTS"` +(which will be replaced with the current Bazel hosts's constraints). + +If a repo's entry is missing or empty, it defaults to the supported OS the +underlying runtime repository detects as compatible. + +:::{note} +Specify `@//foo:bar`, not simply `//foo:bar` or `:bar`. The additional `@` is +needed because the strings are evaluated in a different context than where +they originate. +::: + +The list of settings **becomes the** the {obj}`toolchain.target_compatible_with` +value for each respective repo; i.e. they _replace_ the auto-detected values +the local runtime itself computes. + +This allows a local toolchain to only be used if certain target platform +conditions are met, typically values from `@platforms`. + +See the [Local toolchains] docs for examples and further information. + +:::{seealso} +The `target_settings` attribute, which handles `config_setting` values, +instead of constraints. +::: + +:::{versionadded} 1.5.0 +::: +""", + ), + "target_settings": attr.string_list_dict( + doc = """ +Config settings that must be satisfied for a toolchain to be used. + +This is a `dict[str, list[str]]`, where the keys are repo names from the +`runtimes` or `default_runtimes` args, and the values are {obj}`config_setting()` +target labels. + +If a repo's entry is missing or empty, it will default to +`@//:is_match_python_version` (for repos in `runtimes`) or an empty list +(for repos in `default_runtimes`). + +:::{note} +Specify `@//foo:bar`, not simply `//foo:bar` or `:bar`. The additional `@` is +needed because the strings are evaluated in a different context than where +they originate. +::: + +The list of settings will be applied atop of any of the local runtime's +settings that are used for {obj}`toolchain.target_settings`. i.e. they are +evaluated first and guard the checking of the local runtime's auto-detected +conditions. + +This allows a local toolchain to only be used if certain flags or +config setting conditions are met. Such conditions can include user-defined +flags, platform constraints, etc. + +See the [Local toolchains] docs for examples and further information. + +:::{seealso} +The `target_compatible_with` attribute, which handles *constraint* values, +instead of `config_settings`. +::: + +:::{versionadded} 1.5.0 +::: """, ), "_rule_name": attr.string(default = "local_toolchains_repo"), diff --git a/python/private/pbs_manifest.bzl b/python/private/pbs_manifest.bzl new file mode 100644 index 0000000000..86434dc0ea --- /dev/null +++ b/python/private/pbs_manifest.bzl @@ -0,0 +1,152 @@ +"""Helper functions to parse python-build-standalone manifests.""" + +def parse_filename(filename): + """Parses a python-build-standalone filename (or URL) into its components. + + See https://gregoryszorc.com/docs/python-build-standalone/main/running.html + + Example: cpython-3.10.20+20260414-x86_64_v2-unknown-linux-musl-lto-full.tar.zst + + Args: + filename: The filename or URL of the python-build-standalone release asset. + + Returns: + A dictionary of parsed components if parsed successfully, else None. + """ + basename = filename.rpartition("/")[-1] + if basename.endswith(".tar.zst"): + name = basename.removesuffix(".tar.zst") + elif basename.endswith(".tar.gz"): + name = basename.removesuffix(".tar.gz") + else: + return None + + if not name.startswith("cpython-"): + return None + name = name.removeprefix("cpython-") + + left, plus, tail = name.partition("+") + if plus: + python_version = left + build_version, sep, rest = tail.partition("-") + if not sep: + return None + else: + python_version, sep, rest = left.partition("-") + if not sep: + return None + build_version = "" + + arch, sep, rest = rest.partition("-") + if not sep: + return None + + microarch = "" + arch_base, sep_v, microarch_num = arch.partition("_v") + if sep_v: + arch = arch_base + microarch = "v" + microarch_num + + vendor, sep, rest = rest.partition("-") + if not sep: + return None + + os, sep, rest = rest.partition("-") + if not sep: + return None + + libc = "" + next_part, _, remaining = rest.partition("-") + if os == "linux" and next_part in ["gnu", "musl"]: + libc = next_part + flavor = remaining + elif os == "windows" and next_part == "msvc": + libc = next_part + flavor = remaining + else: + libc = "" + flavor = rest + + freethreaded = False + if flavor.startswith("freethreaded+"): + freethreaded = True + flavor = flavor.removeprefix("freethreaded+") + elif flavor.startswith("freethreaded-"): + freethreaded = True + flavor = flavor.removeprefix("freethreaded-") + elif flavor == "freethreaded": + freethreaded = True + flavor = "" + + archive_flavor = "" + if flavor.endswith("-full"): + archive_flavor = "full" + flavor = flavor.removesuffix("-full") + elif flavor == "full": + archive_flavor = "full" + flavor = "" + elif flavor.endswith("-install_only_stripped"): + archive_flavor = "install_only_stripped" + flavor = flavor.removesuffix("-install_only_stripped") + elif flavor == "install_only_stripped": + archive_flavor = "install_only_stripped" + flavor = "" + elif flavor.endswith("-install_only"): + archive_flavor = "install_only" + flavor = flavor.removesuffix("-install_only") + elif flavor == "install_only": + archive_flavor = "install_only" + flavor = "" + + return { + "arch": arch, + "archive_flavor": archive_flavor, + "build_flavor": flavor, + "build_version": build_version, + "freethreaded": freethreaded, + "libc": libc, + "location": filename, + "microarch": microarch, + "os": os, + "python_version": python_version, + "vendor": vendor, + } + +def parse_runtime_manifest(content): + """Parses the SHA256SUMS file content into a list of structs. + + Args: + content: The raw content of the manifest file. + + Returns: + A list of structs capturing the parsed components of each valid entry. + Each struct contains the following fields: + - arch: CPU architecture (e.g., "x86_64"). + - archive_flavor: Release asset archive type (e.g., "full", "install_only"). + - build_version: Standalone release date (e.g., "20260414"). + - location: Full package filename or URL (e.g., "cpython-3.11.15..." or "https://..."). + - build_flavor: Build configuration flavor (e.g., "debug", "pgo+lto"). + - freethreaded: Whether the build is free-threaded (boolean). + - libc: C library type (e.g., "gnu", "musl", "msvc", or ""). + - microarch: Microarchitecture level (e.g., "v2", "v3", or ""). + - os: Operating system (e.g., "linux", "darwin", "windows"). + - python_version: Python semver version (e.g., "3.11.15"). + - sha256: SHA256 integrity hash of the release asset. + - vendor: Platform vendor (e.g., "unknown", "apple"). + """ + results = [] + for line in content.split("\n"): + line = line.strip() + if not line or line.startswith("#"): + continue + parts = [p for p in line.split(" ") if p] + if len(parts) != 2: + continue + sha256, filename = parts + parsed = parse_filename(filename) + if parsed: + results.append(struct( + sha256 = sha256, + **parsed + )) + return results diff --git a/python/private/platform_info.bzl b/python/private/platform_info.bzl new file mode 100644 index 0000000000..3f7dc00165 --- /dev/null +++ b/python/private/platform_info.bzl @@ -0,0 +1,34 @@ +"""Helper to define a struct used to define platform metadata.""" + +def platform_info( + *, + compatible_with = [], + flag_values = {}, + target_settings = [], + os_name, + arch): + """Creates a struct of platform metadata. + + This is just a helper to ensure structs are created the same and + the meaning/values are documented. + + Args: + compatible_with: list[str], where the values are string labels. These + are the target_compatible_with values to use with the toolchain + flag_values: dict[str|Label, Any] of config_setting.flag_values + compatible values. DEPRECATED -- use target_settings instead + target_settings: list[str], where the values are string labels. These + are the target_settings values to use with the toolchain. + os_name: str, the os name; must match the name used in `@platfroms//os` + arch: str, the cpu name; must match the name used in `@platforms//cpu` + + Returns: + A struct with attributes and values matching the args. + """ + return struct( + compatible_with = compatible_with, + flag_values = flag_values, + target_settings = target_settings, + os_name = os_name, + arch = arch, + ) diff --git a/python/private/common_bazel.bzl b/python/private/precompile.bzl similarity index 77% rename from python/private/common_bazel.bzl rename to python/private/precompile.bzl index efbebd0252..c12882bf82 100644 --- a/python/private/common_bazel.bzl +++ b/python/private/precompile.bzl @@ -13,44 +13,12 @@ # limitations under the License. """Common functions that are specific to Bazel rule implementation""" -load("@bazel_skylib//lib:paths.bzl", "paths") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") -load("@rules_cc//cc/common:cc_common.bzl", "cc_common") -load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") load(":attributes.bzl", "PrecompileAttr", "PrecompileInvalidationModeAttr", "PrecompileSourceRetentionAttr") -load(":common.bzl", "is_bool") load(":flags.bzl", "PrecompileFlag") -load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") -load(":py_internal.bzl", "py_internal") load(":py_interpreter_program.bzl", "PyInterpreterProgramInfo") load(":toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", "TARGET_TOOLCHAIN_TYPE") -_py_builtins = py_internal - -def collect_cc_info(ctx, extra_deps = []): - """Collect C++ information from dependencies for Bazel. - - Args: - ctx: Rule ctx; must have `deps` attribute. - extra_deps: list of Target to also collect C+ information from. - - Returns: - CcInfo provider of merged information. - """ - deps = ctx.attr.deps - if extra_deps: - deps = list(deps) - deps.extend(extra_deps) - cc_infos = [] - for dep in deps: - if CcInfo in dep: - cc_infos.append(dep[CcInfo]) - - if PyCcLinkParamsInfo in dep: - cc_infos.append(dep[PyCcLinkParamsInfo].cc_info) - - return cc_common.merge_cc_infos(cc_infos = cc_infos) - def maybe_precompile(ctx, srcs): """Computes all the outputs (maybe precompiled) from the input srcs. @@ -214,7 +182,7 @@ def _precompile(ctx, src, *, use_pycache): # have the repo name, which is likely to contain extraneous info. precompile_request_args.add("--src_name", src.short_path) precompile_request_args.add("--pyc", pyc) - precompile_request_args.add("--optimize", ctx.attr.precompile_optimize_level) + precompile_request_args.add("--optimize", str(ctx.attr.precompile_optimize_level)) version_info = target_toolchain.interpreter_version_info python_version = "{}.{}".format(version_info.major, version_info.minor) @@ -237,44 +205,3 @@ def _precompile(ctx, src, *, use_pycache): toolchain = EXEC_TOOLS_TOOLCHAIN_TYPE, ) return pyc - -def get_imports(ctx): - """Gets the imports from a rule's `imports` attribute. - - See create_binary_semantics_struct for details about this function. - - Args: - ctx: Rule ctx. - - Returns: - List of strings. - """ - prefix = "{}/{}".format( - ctx.workspace_name, - _py_builtins.get_label_repo_runfiles_path(ctx.label), - ) - result = [] - for import_str in ctx.attr.imports: - import_str = ctx.expand_make_variables("imports", import_str, {}) - if import_str.startswith("/"): - continue - - # To prevent "escaping" out of the runfiles tree, we normalize - # the path and ensure it doesn't have up-level references. - import_path = paths.normalize("{}/{}".format(prefix, import_str)) - if import_path.startswith("../") or import_path == "..": - fail("Path '{}' references a path above the execution root".format( - import_str, - )) - result.append(import_path) - return result - -def convert_legacy_create_init_to_int(kwargs): - """Convert "legacy_create_init" key to int, in-place. - - Args: - kwargs: The kwargs to modify. The key "legacy_create_init", if present - and bool, will be converted to its integer value, in place. - """ - if is_bool(kwargs.get("legacy_create_init")): - kwargs["legacy_create_init"] = 1 if kwargs["legacy_create_init"] else 0 diff --git a/python/private/proto/BUILD.bazel b/python/private/proto/BUILD.bazel deleted file mode 100644 index 65c09444f7..0000000000 --- a/python/private/proto/BUILD.bazel +++ /dev/null @@ -1,46 +0,0 @@ -# Copyright 2022 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -load("@bazel_skylib//:bzl_library.bzl", "bzl_library") -load("@rules_proto//proto:defs.bzl", "proto_lang_toolchain") - -package(default_visibility = ["//visibility:private"]) - -licenses(["notice"]) - -filegroup( - name = "distribution", - srcs = glob(["**"]), - visibility = ["//python/private:__pkg__"], -) - -bzl_library( - name = "py_proto_library_bzl", - srcs = ["py_proto_library.bzl"], - visibility = ["//python:__pkg__"], - deps = [ - "//python:defs_bzl", - "@rules_proto//proto:defs", - ], -) - -proto_lang_toolchain( - name = "python_toolchain", - command_line = "--python_out=%s", - progress_message = "Generating Python proto_library %{label}", - runtime = "@com_google_protobuf//:protobuf_python", - # NOTE: This isn't *actually* public. It's an implicit dependency of py_proto_library, - # so must be public so user usages of the rule can reference it. - visibility = ["//visibility:public"], -) diff --git a/python/private/proto/py_proto_library.bzl b/python/private/proto/py_proto_library.bzl deleted file mode 100644 index ecb0938bcd..0000000000 --- a/python/private/proto/py_proto_library.bzl +++ /dev/null @@ -1,238 +0,0 @@ -# Copyright 2022 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""The implementation of the `py_proto_library` rule and its aspect.""" - -load("@rules_proto//proto:defs.bzl", "ProtoInfo", "proto_common") -load("//python:defs.bzl", "PyInfo") -load("//python/api:api.bzl", _py_common = "py_common") - -PY_PROTO_TOOLCHAIN = "@rules_python//python/proto:toolchain_type" - -_PyProtoInfo = provider( - doc = "Encapsulates information needed by the Python proto rules.", - fields = { - "imports": """ - (depset[str]) The field forwarding PyInfo.imports coming from - the proto language runtime dependency.""", - "py_info": "PyInfo from proto runtime (or other deps) to propagate.", - "runfiles_from_proto_deps": """ - (depset[File]) Files from the transitive closure implicit proto - dependencies""", - "transitive_sources": """(depset[File]) The Python sources.""", - }, -) - -def _filter_provider(provider, *attrs): - return [dep[provider] for attr in attrs for dep in attr if provider in dep] - -def _incompatible_toolchains_enabled(): - return getattr(proto_common, "INCOMPATIBLE_ENABLE_PROTO_TOOLCHAIN_RESOLUTION", False) - -def _py_proto_aspect_impl(target, ctx): - """Generates and compiles Python code for a proto_library. - - The function runs protobuf compiler on the `proto_library` target generating - a .py file for each .proto file. - - Args: - target: (Target) A target providing `ProtoInfo`. Usually this means a - `proto_library` target, but not always; you must expect to visit - non-`proto_library` targets, too. - ctx: (RuleContext) The rule context. - - Returns: - ([_PyProtoInfo]) Providers collecting transitive information about - generated files. - """ - _proto_library = ctx.rule.attr - - # Check Proto file names - for proto in target[ProtoInfo].direct_sources: - if proto.is_source and "-" in proto.dirname: - fail("Cannot generate Python code for a .proto whose path contains '-' ({}).".format( - proto.path, - )) - - if _incompatible_toolchains_enabled(): - toolchain = ctx.toolchains[PY_PROTO_TOOLCHAIN] - if not toolchain: - fail("No toolchains registered for '%s'." % PY_PROTO_TOOLCHAIN) - proto_lang_toolchain_info = toolchain.proto - else: - proto_lang_toolchain_info = getattr(ctx.attr, "_aspect_proto_toolchain")[proto_common.ProtoLangToolchainInfo] - - py_common = _py_common.get(ctx) - py_info = py_common.PyInfoBuilder().merge_target( - proto_lang_toolchain_info.runtime, - ).build() - - api_deps = [proto_lang_toolchain_info.runtime] - - generated_sources = [] - proto_info = target[ProtoInfo] - proto_root = proto_info.proto_source_root - if proto_info.direct_sources: - # Generate py files - generated_sources = proto_common.declare_generated_files( - actions = ctx.actions, - proto_info = proto_info, - extension = "_pb2.py", - name_mapper = lambda name: name.replace("-", "_").replace(".", "/"), - ) - - # Handles multiple repository and virtual import cases - if proto_root.startswith(ctx.bin_dir.path): - proto_root = proto_root[len(ctx.bin_dir.path) + 1:] - - plugin_output = ctx.bin_dir.path + "/" + proto_root - proto_root = ctx.workspace_name + "/" + proto_root - - proto_common.compile( - actions = ctx.actions, - proto_info = proto_info, - proto_lang_toolchain_info = proto_lang_toolchain_info, - generated_files = generated_sources, - plugin_output = plugin_output, - ) - - # Generated sources == Python sources - python_sources = generated_sources - - deps = _filter_provider(_PyProtoInfo, getattr(_proto_library, "deps", [])) - runfiles_from_proto_deps = depset( - transitive = [dep[DefaultInfo].default_runfiles.files for dep in api_deps] + - [dep.runfiles_from_proto_deps for dep in deps], - ) - transitive_sources = depset( - direct = python_sources, - transitive = [dep.transitive_sources for dep in deps], - ) - - return [ - _PyProtoInfo( - imports = depset( - # Adding to PYTHONPATH so the generated modules can be - # imported. This is necessary when there is - # strip_import_prefix, the Python modules are generated under - # _virtual_imports. But it's undesirable otherwise, because it - # will put the repo root at the top of the PYTHONPATH, ahead of - # directories added through `imports` attributes. - [proto_root] if "_virtual_imports" in proto_root else [], - transitive = [dep[PyInfo].imports for dep in api_deps] + [dep.imports for dep in deps], - ), - runfiles_from_proto_deps = runfiles_from_proto_deps, - transitive_sources = transitive_sources, - py_info = py_info, - ), - ] - -_py_proto_aspect = aspect( - implementation = _py_proto_aspect_impl, - attrs = _py_common.API_ATTRS | ( - {} if _incompatible_toolchains_enabled() else { - "_aspect_proto_toolchain": attr.label( - default = ":python_toolchain", - ), - } - ), - attr_aspects = ["deps"], - required_providers = [ProtoInfo], - provides = [_PyProtoInfo], - toolchains = [PY_PROTO_TOOLCHAIN] if _incompatible_toolchains_enabled() else [], -) - -def _py_proto_library_rule(ctx): - """Merges results of `py_proto_aspect` in `deps`. - - Args: - ctx: (RuleContext) The rule context. - Returns: - ([PyInfo, DefaultInfo, OutputGroupInfo]) - """ - if not ctx.attr.deps: - fail("'deps' attribute mustn't be empty.") - - pyproto_infos = _filter_provider(_PyProtoInfo, ctx.attr.deps) - default_outputs = depset( - transitive = [info.transitive_sources for info in pyproto_infos], - ) - - py_common = _py_common.get(ctx) - - py_info = py_common.PyInfoBuilder() - py_info.set_has_py2_only_sources(False) - py_info.set_has_py3_only_sources(False) - py_info.transitive_sources.add(default_outputs) - py_info.imports.add([info.imports for info in pyproto_infos]) - py_info.merge_all([ - pyproto_info.py_info - for pyproto_info in pyproto_infos - ]) - return [ - DefaultInfo( - files = default_outputs, - default_runfiles = ctx.runfiles(transitive_files = depset( - transitive = - [default_outputs] + - [info.runfiles_from_proto_deps for info in pyproto_infos], - )), - ), - OutputGroupInfo( - default = depset(), - ), - py_info.build(), - ] - -py_proto_library = rule( - implementation = _py_proto_library_rule, - doc = """ - Use `py_proto_library` to generate Python libraries from `.proto` files. - - The convention is to name the `py_proto_library` rule `foo_py_pb2`, - when it is wrapping `proto_library` rule `foo_proto`. - - `deps` must point to a `proto_library` rule. - - Example: - -```starlark -py_library( - name = "lib", - deps = [":foo_py_pb2"], -) - -py_proto_library( - name = "foo_py_pb2", - deps = [":foo_proto"], -) - -proto_library( - name = "foo_proto", - srcs = ["foo.proto"], -) -```""", - attrs = { - "deps": attr.label_list( - doc = """ - The list of `proto_library` rules to generate Python libraries for. - - Usually this is just the one target: the proto library of interest. - It can be any target providing `ProtoInfo`.""", - providers = [ProtoInfo], - aspects = [_py_proto_aspect], - ), - } | _py_common.API_ATTRS, - provides = [PyInfo], -) diff --git a/python/private/py_binary_macro.bzl b/python/private/py_binary_macro.bzl index 83b3c18677..c7840eff55 100644 --- a/python/private/py_binary_macro.bzl +++ b/python/private/py_binary_macro.bzl @@ -13,9 +13,12 @@ # limitations under the License. """Implementation of macro-half of py_binary rule.""" -load(":common_bazel.bzl", "convert_legacy_create_init_to_int") load(":py_binary_rule.bzl", py_binary_rule = "py_binary") +load(":py_executable.bzl", "common_executable_macro_kwargs_setup") def py_binary(**kwargs): - convert_legacy_create_init_to_int(kwargs) - py_binary_rule(**kwargs) + py_binary_macro(py_binary_rule, **kwargs) + +def py_binary_macro(py_rule, **kwargs): + common_executable_macro_kwargs_setup(kwargs) + py_rule(**kwargs) diff --git a/python/private/py_binary_rule.bzl b/python/private/py_binary_rule.bzl index 9ce0726c5e..356af5d9f8 100644 --- a/python/private/py_binary_rule.bzl +++ b/python/private/py_binary_rule.bzl @@ -13,40 +13,39 @@ # limitations under the License. """Rule implementation of py_binary for Bazel.""" -load("@bazel_skylib//lib:dicts.bzl", "dicts") load(":attributes.bzl", "AGNOSTIC_BINARY_ATTRS") load( - ":py_executable_bazel.bzl", - "create_executable_rule", - "py_executable_bazel_impl", + ":py_executable.bzl", + "create_executable_rule_builder", + "py_executable_impl", ) -_PY_TEST_ATTRS = { - # Magic attribute to help C++ coverage work. There's no - # docs about this; see TestActionBuilder.java - "_collect_cc_coverage": attr.label( - default = "@bazel_tools//tools/test:collect_cc_coverage", - executable = True, - cfg = "exec", - ), - # Magic attribute to make coverage work. There's no - # docs about this; see TestActionBuilder.java - "_lcov_merger": attr.label( - default = configuration_field(fragment = "coverage", name = "output_generator"), - executable = True, - cfg = "exec", - ), -} - def _py_binary_impl(ctx): - return py_executable_bazel_impl( + return py_executable_impl( ctx = ctx, is_test = False, inherited_environment = [], ) -py_binary = create_executable_rule( - implementation = _py_binary_impl, - attrs = dicts.add(AGNOSTIC_BINARY_ATTRS, _PY_TEST_ATTRS), - executable = True, -) +# NOTE: Exported publicly +def create_py_binary_rule_builder(): + """Create a rule builder for a py_binary. + + :::{include} /_includes/volatile_api.md + ::: + + :::{versionadded} 1.3.0 + ::: + + Returns: + {obj}`ruleb.Rule` with the necessary settings + for creating a `py_binary` rule. + """ + builder = create_executable_rule_builder( + implementation = _py_binary_impl, + executable = True, + ) + builder.attrs.update(AGNOSTIC_BINARY_ATTRS) + return builder + +py_binary = create_py_binary_rule_builder().build() diff --git a/python/private/py_cc_link_params_info.bzl b/python/private/py_cc_link_params_info.bzl index bfa2de5978..2fbd255eed 100644 --- a/python/private/py_cc_link_params_info.bzl +++ b/python/private/py_cc_link_params_info.bzl @@ -14,291 +14,6 @@ """Providers for Python rules.""" load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") -load(":util.bzl", "define_bazel_6_provider") - -DEFAULT_STUB_SHEBANG = "#!/usr/bin/env python3" - -DEFAULT_BOOTSTRAP_TEMPLATE = Label("//python/private:bootstrap_template") - -_PYTHON_VERSION_VALUES = ["PY2", "PY3"] - -def _optional_int(value): - return int(value) if value != None else None - -def interpreter_version_info_struct_from_dict(info_dict): - """Create a struct of interpreter version info from a dict from an attribute. - - Args: - info_dict: (dict | None) of version info fields. See interpreter_version_info - provider field docs. - - Returns: - struct of version info; see interpreter_version_info provider field docs. - """ - info_dict = dict(info_dict or {}) # Copy in case the original is frozen - if info_dict: - if not ("major" in info_dict and "minor" in info_dict): - fail("interpreter_version_info must have at least two keys, 'major' and 'minor'") - version_info_struct = struct( - major = _optional_int(info_dict.pop("major", None)), - minor = _optional_int(info_dict.pop("minor", None)), - micro = _optional_int(info_dict.pop("micro", None)), - releaselevel = str(info_dict.pop("releaselevel")) if "releaselevel" in info_dict else None, - serial = _optional_int(info_dict.pop("serial", None)), - ) - - if len(info_dict.keys()) > 0: - fail("unexpected keys {} in interpreter_version_info".format( - str(info_dict.keys()), - )) - - return version_info_struct - -def _PyRuntimeInfo_init( - *, - implementation_name = None, - interpreter_path = None, - interpreter = None, - files = None, - coverage_tool = None, - coverage_files = None, - pyc_tag = None, - python_version, - stub_shebang = None, - bootstrap_template = None, - interpreter_version_info = None, - stage2_bootstrap_template = None, - zip_main_template = None): - if (interpreter_path and interpreter) or (not interpreter_path and not interpreter): - fail("exactly one of interpreter or interpreter_path must be specified") - - if interpreter_path and files != None: - fail("cannot specify 'files' if 'interpreter_path' is given") - - if (coverage_tool and not coverage_files) or (not coverage_tool and coverage_files): - fail( - "coverage_tool and coverage_files must both be set or neither must be set, " + - "got coverage_tool={}, coverage_files={}".format( - coverage_tool, - coverage_files, - ), - ) - - if python_version not in _PYTHON_VERSION_VALUES: - fail("invalid python_version: '{}'; must be one of {}".format( - python_version, - _PYTHON_VERSION_VALUES, - )) - - if files != None and type(files) != type(depset()): - fail("invalid files: got value of type {}, want depset".format(type(files))) - - if interpreter: - if files == None: - files = depset() - else: - files = None - - if coverage_files == None: - coverage_files = depset() - - if not stub_shebang: - stub_shebang = DEFAULT_STUB_SHEBANG - - return { - "bootstrap_template": bootstrap_template, - "coverage_files": coverage_files, - "coverage_tool": coverage_tool, - "files": files, - "implementation_name": implementation_name, - "interpreter": interpreter, - "interpreter_path": interpreter_path, - "interpreter_version_info": interpreter_version_info_struct_from_dict(interpreter_version_info), - "pyc_tag": pyc_tag, - "python_version": python_version, - "stage2_bootstrap_template": stage2_bootstrap_template, - "stub_shebang": stub_shebang, - "zip_main_template": zip_main_template, - } - -PyRuntimeInfo, _unused_raw_py_runtime_info_ctor = define_bazel_6_provider( - doc = """Contains information about a Python runtime, as returned by the `py_runtime` -rule. - -A Python runtime describes either a *platform runtime* or an *in-build runtime*. -A platform runtime accesses a system-installed interpreter at a known path, -whereas an in-build runtime points to a `File` that acts as the interpreter. In -both cases, an "interpreter" is really any executable binary or wrapper script -that is capable of running a Python script passed on the command line, following -the same conventions as the standard CPython interpreter. -""", - init = _PyRuntimeInfo_init, - fields = { - "bootstrap_template": """ -:type: File - -A template of code responsible for the initial startup of a program. - -This code is responsible for: - -* Locating the target interpreter. Typically it is in runfiles, but not always. -* Setting necessary environment variables, command line flags, or other - configuration that can't be modified after the interpreter starts. -* Invoking the appropriate entry point. This is usually a second-stage bootstrap - that performs additional setup prior to running a program's actual entry point. - -The {obj}`--bootstrap_impl` flag affects how this stage 1 bootstrap -is expected to behave and the substutitions performed. - -* `--bootstrap_impl=system_python` substitutions: `%is_zipfile%`, `%python_binary%`, - `%target%`, `%workspace_name`, `%coverage_tool%`, `%import_all%`, `%imports%`, - `%main%`, `%shebang%` -* `--bootstrap_impl=script` substititions: `%is_zipfile%`, `%python_binary%`, - `%target%`, `%workspace_name`, `%shebang%, `%stage2_bootstrap%` - -Substitution definitions: - -* `%shebang%`: The shebang to use with the bootstrap; the bootstrap template - may choose to ignore this. -* `%stage2_bootstrap%`: A runfiles-relative path to the stage 2 bootstrap. -* `%python_binary%`: The path to the target Python interpreter. There are three - types of paths: - * An absolute path to a system interpreter (e.g. begins with `/`). - * A runfiles-relative path to an interpreter (e.g. `somerepo/bin/python3`) - * A program to search for on PATH, i.e. a word without spaces, e.g. `python3`. -* `%workspace_name%`: The name of the workspace the target belongs to. -* `%is_zipfile%`: The string `1` if this template is prepended to a zipfile to - create a self-executable zip file. The string `0` otherwise. - -For the other substitution definitions, see the {obj}`stage2_bootstrap_template` -docs. - -:::{versionchanged} 0.33.0 -The set of substitutions depends on {obj}`--bootstrap_impl` -::: -""", - "coverage_files": """ -:type: depset[File] | None - -The files required at runtime for using `coverage_tool`. Will be `None` if no -`coverage_tool` was provided. -""", - "coverage_tool": """ -:type: File | None - -If set, this field is a `File` representing tool used for collecting code -coverage information from python tests. Otherwise, this is `None`. -""", - "files": """ -:type: depset[File] | None - -If this is an in-build runtime, this field is a `depset` of `File`s that need to -be added to the runfiles of an executable target that uses this runtime (in -particular, files needed by `interpreter`). The value of `interpreter` need not -be included in this field. If this is a platform runtime then this field is -`None`. -""", - "implementation_name": """ -:type: str | None - -The Python implementation name (`sys.implementation.name`) -""", - "interpreter": """ -:type: File | None - -If this is an in-build runtime, this field is a `File` representing the -interpreter. Otherwise, this is `None`. Note that an in-build runtime can use -either a prebuilt, checked-in interpreter or an interpreter built from source. -""", - "interpreter_path": """ -:type: str | None - -If this is a platform runtime, this field is the absolute filesystem path to the -interpreter on the target platform. Otherwise, this is `None`. -""", - "interpreter_version_info": """ -:type: struct - -Version information about the interpreter this runtime provides. -It should match the format given by `sys.version_info`, however -for simplicity, the micro, releaselevel, and serial values are -optional. -A struct with the following fields: -* `major`: {type}`int`, the major version number -* `minor`: {type}`int`, the minor version number -* `micro`: {type}`int | None`, the micro version number -* `releaselevel`: {type}`str | None`, the release level -* `serial`: {type}`int | None`, the serial number of the release -""", - "pyc_tag": """ -:type: str | None - -The tag portion of a pyc filename, e.g. the `cpython-39` infix -of `foo.cpython-39.pyc`. See PEP 3147. If not specified, it will be computed -from {obj}`implementation_name` and {obj}`interpreter_version_info`. If no -pyc_tag is available, then only source-less pyc generation will function -correctly. -""", - "python_version": """ -:type: str - -Indicates whether this runtime uses Python major version 2 or 3. Valid values -are (only) `"PY2"` and `"PY3"`. -""", - "stage2_bootstrap_template": """ -:type: File - -A template of Python code that runs under the desired interpreter and is -responsible for orchestrating calling the program's actual main code. This -bootstrap is responsible for affecting the current runtime's state, such as -import paths or enabling coverage, so that, when it runs the program's actual -main code, it works properly under Bazel. - -The following substitutions are made during template expansion: -* `%main%`: A runfiles-relative path to the program's actual main file. This - can be a `.py` or `.pyc` file, depending on precompile settings. -* `%coverage_tool%`: Runfiles-relative path to the coverage library's entry point. - If coverage is not enabled or available, an empty string. -* `%import_all%`: The string `True` if all repositories in the runfiles should - be added to sys.path. The string `False` otherwise. -* `%imports%`: A colon-delimited string of runfiles-relative paths to add to - sys.path. -* `%target%`: The name of the target this is for. -* `%workspace_name%`: The name of the workspace the target belongs to. - -:::{versionadded} 0.33.0 -::: -""", - "stub_shebang": """ -:type: str - -"Shebang" expression prepended to the bootstrapping Python stub -script used when executing {obj}`py_binary` targets. Does not -apply to Windows. -""", - "zip_main_template": """ -:type: File - -A template of Python code that becomes a zip file's top-level `__main__.py` -file. The top-level `__main__.py` file is used when the zip file is explicitly -passed to a Python interpreter. See PEP 441 for more information about zipapp -support. Note that py_binary-generated zip files are self-executing and -skip calling `__main__.py`. - -The following substitutions are made during template expansion: -* `%stage2_bootstrap%`: A runfiles-relative string to the stage 2 bootstrap file. -* `%python_binary%`: The path to the target Python interpreter. There are three - types of paths: - * An absolute path to a system interpreter (e.g. begins with `/`). - * A runfiles-relative path to an interpreter (e.g. `somerepo/bin/python3`) - * A program to search for on PATH, i.e. a word without spaces, e.g. `python3`. -* `%workspace_name%`: The name of the workspace for the built target. - -:::{versionadded} 0.33.0 -::: -""", - }, -) def _PyCcLinkParamsInfo_init(cc_info): return { @@ -306,7 +21,7 @@ def _PyCcLinkParamsInfo_init(cc_info): } # buildifier: disable=name-conventions -PyCcLinkParamsInfo, _unused_raw_py_cc_link_params_provider_ctor = define_bazel_6_provider( +PyCcLinkParamsInfo, _unused_raw_py_cc_link_params_provider_ctor = provider( doc = ("Python-wrapper to forward {obj}`CcInfo.linking_context`. This is to " + "allow Python targets to propagate C++ linking information, but " + "without the Python target appearing to be a valid C++ rule dependency"), diff --git a/python/private/py_cc_toolchain_info.bzl b/python/private/py_cc_toolchain_info.bzl index c5cdbd9d84..a56ae7c960 100644 --- a/python/private/py_cc_toolchain_info.bzl +++ b/python/private/py_cc_toolchain_info.bzl @@ -17,6 +17,14 @@ PyCcToolchainInfo = provider( doc = "C/C++ information about the Python runtime.", fields = { + "abi_flags": """\ +:type: str + +The runtime's ABI flags, i.e. `sys.abiflags` (e.g. 't' for free-threaded builds). + +:::{versionadded} 2.3.0 +::: +""", "headers": """\ :type: struct @@ -40,7 +48,36 @@ Information about the header files, struct with fields: e.g. `:current_py_cc_headers` to act as the underlying headers target it represents). """, - "libs": """\ + "headers_abi3": """ +:type: struct | None + +If available, information about ABI3 (stable ABI) header files, struct with +fields: + * providers_map: a dict of string to provider instances. The key should be + a fully qualified name (e.g. `@rules_foo//bar:baz.bzl#MyInfo`) of the + provider to uniquely identify its type. + + The following keys are always present: + * CcInfo: the CcInfo provider instance for the headers. + * DefaultInfo: the DefaultInfo provider instance for the headers. + + A map is used to allow additional providers from the originating headers + target (typically a `cc_library`) to be propagated to consumers (directly + exposing a Target object can cause memory issues and is an anti-pattern). + + When consuming this map, it's suggested to use `providers_map.values()` to + return all providers; or copy the map and filter out or replace keys as + appropriate. Note that any keys beginning with `_` (underscore) are + considered private and should be forward along as-is (this better allows + e.g. `:current_py_cc_headers` to act as the underlying headers target it + represents). + +:::{versionadded} 1.7.0 +The {obj}`features.headers_abi3` attribute can be used to detect if this +attribute is available or not. +::: +""", + "libs": """ :type: struct | None If available, information about C libraries, struct with fields: @@ -62,11 +99,48 @@ If available, information about C libraries, struct with fields: considered private and should be forward along as-is (this better allows e.g. `:current_py_cc_headers` to act as the underlying headers target it represents). +""", + "platform_machine": """ +:type: str + +The {pep}`PEP 508` `platform_machine` marker +value for the target architecture, e.g. 'x86_64', 'aarch64'. + +:::{versionadded} 2.3.0 +::: +""", + "platform_tag": """\ +:type: str | None + +The PEP 3149 / PEP 425 platform tag for extension modules, e.g. +'x86_64-linux-gnu', 'darwin', or 'win_amd64'. + +:::{versionadded} 2.3.0 +::: """, "python_version": """ :type: str The Python Major.Minor version. +""", + "soabi": """\ +:type: str + +The SOABI tag for extension modules (see +[PEP 3149](https://peps.python.org/pep-3149/)), e.g. +'cpython-311-x86_64-linux-gnu' or 'cp311'. + +:::{versionadded} 2.3.0 +::: +""", + "sys_platform": """ +:type: str + +The {pep}`0508` `sys_platform` marker value +for the target OS, e.g. 'linux', 'darwin', 'win32'. + +:::{versionadded} 2.2.0 +::: """, }, ) diff --git a/python/private/py_cc_toolchain_macro.bzl b/python/private/py_cc_toolchain_macro.bzl index 416caac2ab..88223c747c 100644 --- a/python/private/py_cc_toolchain_macro.bzl +++ b/python/private/py_cc_toolchain_macro.bzl @@ -14,6 +14,7 @@ """Fronting macro for the py_cc_toolchain rule.""" +load("//python/private/pypi:pep508_env.bzl", "platform_machine_select_map", "sys_platform_select_map") load(":py_cc_toolchain_rule.bzl", _py_cc_toolchain = "py_cc_toolchain") load(":util.bzl", "add_tag") @@ -30,4 +31,10 @@ def py_cc_toolchain(**kwargs): # This tag is added to easily identify usages through other macros. add_tag(kwargs, "@rules_python//python:py_cc_toolchain") + + if kwargs.get("sys_platform") == None: + kwargs["sys_platform"] = select(sys_platform_select_map) + if kwargs.get("platform_machine") == None: + kwargs["platform_machine"] = select(platform_machine_select_map) + _py_cc_toolchain(**kwargs) diff --git a/python/private/py_cc_toolchain_rule.bzl b/python/private/py_cc_toolchain_rule.bzl index d5f3b685a4..405a8afe56 100644 --- a/python/private/py_cc_toolchain_rule.bzl +++ b/python/private/py_cc_toolchain_rule.bzl @@ -15,12 +15,51 @@ """Implementation of py_cc_toolchain rule. NOTE: This is a beta-quality feature. APIs subject to change until -https://github.com/bazelbuild/rules_python/issues/824 is considered done. +{gh-issue}`824` is considered done. """ load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") +load(":common_labels.bzl", "labels") +load(":flags.bzl", "FreeThreadedFlag", "LibcFlag") load(":py_cc_toolchain_info.bzl", "PyCcToolchainInfo") +load(":sentinel_impl.bzl", "SentinelInfo") + +def _get_platform_tag(sys_platform, platform_machine, libc): + """Derives the PEP 3149 platform tag string. + + Note that these are platform tags for C extension filenames, not + PEP 425 tags for wheels. + + Linux platform tags are standardized here: + - https://peps.python.org/pep-3149/ + Windows platform tags, such as they are, are defined in this issue and + commit (treated as a de facto standard): + - https://github.com/python/cpython/issues/67169 + - https://github.com/python/cpython/commit/03a144bb6ac3d7631a3bdb895e2a1f2d021fb08b + Apple platform tag is always just "darwin", discussed briefly here: + - https://github.com/python/cpython/commit/3b8124884c3655b4cf2629d741b18c1a38181805 + + Args: + sys_platform: Target PEP 508 OS marker, e.g. "win32", "darwin", "linux" + platform_machine: Target PEP 508 CPU marker, e.g. "x86_64", "aarch64", "x86_32" + libc: Target C library variant, e.g. "glibc", "musl" + + Returns: + The platform tag, e.g. "x86_64-linux-gnu", "darwin", or "win_amd64" + """ + if sys_platform == "win32": + if platform_machine in ("x86_64", "amd64"): + return "win_amd64" + if platform_machine in ("aarch64", "arm64"): + return "win_arm64" + return "win32" + if sys_platform == "darwin": + return "darwin" + + machine_val = platform_machine if platform_machine else "x86_64" + libc_val = "gnu" if libc == LibcFlag.GLIBC else libc + return "{}-linux-{}".format(machine_val, libc_val) def _py_cc_toolchain_impl(ctx): if ctx.attr.libs: @@ -33,15 +72,56 @@ def _py_cc_toolchain_impl(ctx): else: libs = None + if ctx.attr.headers_abi3 and SentinelInfo not in ctx.attr.headers_abi3: + headers_abi3 = struct( + providers_map = { + "CcInfo": ctx.attr.headers_abi3[CcInfo], + "DefaultInfo": ctx.attr.headers_abi3[DefaultInfo], + }, + ) + else: + headers_abi3 = None + + abi_flags = ctx.attr.abi_flags + if abi_flags == "": + abi_flags = "" + if ctx.attr._py_freethreaded_flag[BuildSettingInfo].value == FreeThreadedFlag.YES: + abi_flags += "t" + + libc = ctx.attr.libc or LibcFlag.get_value(ctx) + + platform_tag = _get_platform_tag( + sys_platform = ctx.attr.sys_platform, + platform_machine = ctx.attr.platform_machine, + libc = libc, + ) + + soabi = ctx.attr.soabi + if not soabi: + # Derive default SOABI tag according to PEP 3149: + # On POSIX: cpython-XX[t]- + # On Windows: cpXX[t]- (CPython issue #67169) + version_parts = ctx.attr.python_version.split(".") + prefix = "cp" if ctx.attr.sys_platform == "win32" else "cpython-" + soabi = "{}{}{}{}".format(prefix, version_parts[0], version_parts[1], abi_flags) + if platform_tag: + soabi = "{}-{}".format(soabi, platform_tag) + py_cc_toolchain = PyCcToolchainInfo( + abi_flags = abi_flags, headers = struct( providers_map = { "CcInfo": ctx.attr.headers[CcInfo], "DefaultInfo": ctx.attr.headers[DefaultInfo], }, ), + headers_abi3 = headers_abi3, libs = libs, + platform_machine = ctx.attr.platform_machine, + platform_tag = platform_tag, python_version = ctx.attr.python_version, + soabi = soabi, + sys_platform = ctx.attr.sys_platform, ) extra_kwargs = {} if ctx.attr._visible_for_testing[BuildSettingInfo].value: @@ -54,26 +134,96 @@ def _py_cc_toolchain_impl(ctx): py_cc_toolchain = rule( implementation = _py_cc_toolchain_impl, attrs = { + "abi_flags": attr.string( + default = "", + doc = """ +The runtime's ABI flags, i.e. `sys.abiflags`. + +If not set, or set to ``, the ABI flags are automatically derived +from `--//python/config_settings:py_freethreaded` (e.g., `'t'` when +free-threaded is enabled, or `''` otherwise). + +:::{versionadded} 2.3.0 +::: +""", + ), "headers": attr.label( doc = ("Target that provides the Python headers. Typically this " + "is a cc_library target."), providers = [CcInfo], mandatory = True, ), + "headers_abi3": attr.label( + doc = """ +Target that provides the Python ABI3 (stable abi) headers. + +Typically this is a cc_library target. + +:::{versionadded} 1.7.0 +The {obj}`features.headers_abi3` attribute can be used to detect if this +attribute is available or not. +::: +""", + default = "//python:none", + providers = [[SentinelInfo], [CcInfo]], + ), + "libc": attr.string( + doc = """\ +Target C library variant, e.g. 'glibc', 'musl'. + +:::{versionadded} 2.3.0 +::: +""", + default = "", + ), "libs": attr.label( doc = ("Target that provides the Python runtime libraries for linking. " + "Typically this is a cc_library target of `.so` files."), providers = [CcInfo], ), + "platform_machine": attr.string( + doc = """ +Target architecture as a PEP 508 `platform_machine` marker, e.g. 'x86_64', 'aarch64', 'x86_32'. + +:::{versionadded} 2.3.0 +::: +""", + default = "", + ), "python_version": attr.string( doc = "The Major.minor Python version, e.g. 3.11", mandatory = True, ), + "soabi": attr.string( + doc = """\ +The SOABI tag for extension modules (see PEP 3149), e.g. +'cpython-311-x86_64-linux-gnu' or 'cp311'. + +:::{versionadded} 2.3.0 +::: +""", + default = "", + ), + "sys_platform": attr.string( + doc = """ +Target OS as a PEP 508 `sys_platform` marker, e.g. 'linux', 'darwin', 'win32'. + +:::{versionadded} 2.2.0 +::: +""", + default = "", + ), + "_py_freethreaded_flag": attr.label( + default = labels.PY_FREETHREADED, + ), + "_py_linux_libc_flag": attr.label( + default = labels.PY_LINUX_LIBC, + ), "_visible_for_testing": attr.label( - default = "//python/private:visible_for_testing", + default = labels.VISIBLE_FOR_TESTING, ), }, - doc = """\ + doc = """ A toolchain for a Python runtime's C/C++ information (e.g. headers) This rule carries information about the C/C++ side of a Python runtime, e.g. diff --git a/python/private/py_console_script_binary.bzl b/python/private/py_console_script_binary.bzl index 7347ebe16a..0dda44e7b9 100644 --- a/python/private/py_console_script_binary.bzl +++ b/python/private/py_console_script_binary.bzl @@ -52,35 +52,42 @@ def py_console_script_binary( entry_points_txt = None, script = None, binary_rule = py_binary, + shebang = "", + main = None, **kwargs): """Generate a py_binary for a console_script entry_point. Args: - name: [`target-name`] The name of the resulting target. - pkg: {any}`simple label` the package for which to generate the script. - entry_points_txt: optional [`label`], the entry_points.txt file to parse + name: {type}`Name` The name of the resulting target. + pkg: {type}`Label` the package for which to generate the script. + entry_points_txt: {type}`label | None`, the entry_points.txt file to parse for available console_script values. It may be a single file, or a group of files, but must contain a file named `entry_points.txt`. If not specified, defaults to the `dist_info` target in the same package as the `pkg` Label. - script: [`str`], The console script name that the py_binary is going to be + script: {type}`str`, The console script name that the py_binary is going to be generated for. Defaults to the normalized name attribute. - binary_rule: {any}`rule callable`, The rule/macro to use to instantiate - the target. It's expected to behave like {any}`py_binary`. - Defaults to {any}`py_binary`. + main: {type}`str`, the python file to be generated, defaults to `_entry_point.py` to + be compatible with symbolic macros. + binary_rule: {type}`callable`, The rule/macro to use to instantiate + the target. It's expected to behave like {obj}`py_binary`. + Defaults to {obj}`py_binary`. + shebang: {type}`str`, The shebang to use for the entry point python file. + Defaults to empty string. **kwargs: Extra parameters forwarded to `binary_rule`. """ - main = "rules_python_entry_point_{}.py".format(name) + main = main or name + "_entry_point.py" if kwargs.pop("srcs", None): fail("passing 'srcs' attribute to py_console_script_binary is unsupported") py_console_script_gen( - name = "_{}_gen".format(name), + name = name + "_gen__", entry_points_txt = entry_points_txt or _dist_info(pkg), out = main, console_script = script, console_script_guess = name, + shebang = shebang, visibility = ["//visibility:private"], ) diff --git a/python/private/py_console_script_gen.bzl b/python/private/py_console_script_gen.bzl index 7dd4dd2dad..de016036b2 100644 --- a/python/private/py_console_script_gen.bzl +++ b/python/private/py_console_script_gen.bzl @@ -42,6 +42,7 @@ def _py_console_script_gen_impl(ctx): args = ctx.actions.args() args.add("--console-script", ctx.attr.console_script) args.add("--console-script-guess", ctx.attr.console_script_guess) + args.add("--shebang", ctx.attr.shebang) args.add(entry_points_txt) args.add(ctx.outputs.out) @@ -81,6 +82,10 @@ py_console_script_gen = rule( doc = "Output file location.", mandatory = True, ), + "shebang": attr.string( + doc = "The shebang to use for the entry point python file.", + default = "", + ), "_tool": attr.label( default = ":py_console_script_gen_py", executable = True, diff --git a/python/private/py_console_script_gen.py b/python/private/py_console_script_gen.py index 64ebea6ab7..2be986f732 100644 --- a/python/private/py_console_script_gen.py +++ b/python/private/py_console_script_gen.py @@ -17,7 +17,7 @@ For Python versions earlier than 3.11 and for earlier bazel versions than 7.0 we need to workaround the issue of sys.path[0] breaking out of the runfiles tree see the following for more context: -* https://github.com/bazelbuild/rules_python/issues/382 +* https://github.com/bazel-contrib/rules_python/issues/382 * https://github.com/bazelbuild/bazel/pull/15701 In affected bazel and Python versions we see in programs such as `flake8`, `pylint` or `pytest` errors because the @@ -37,14 +37,11 @@ import argparse import configparser import pathlib -import re -import sys -import textwrap _ENTRY_POINTS_TXT = "entry_points.txt" _TEMPLATE = """\ -import sys +{shebang}import sys # See @rules_python//python/private:py_console_script_gen.py for explanation if getattr(sys.flags, "safe_path", False): @@ -62,7 +59,7 @@ raise if __name__ == "__main__": - sys.exit({entry_point}()) + sys.exit({entry_point}()) # type: ignore """ @@ -75,7 +72,7 @@ class EntryPointsParser(configparser.ConfigParser): optionxform = staticmethod(str) -def _guess_entry_point(guess: str, console_scripts: dict[string, string]) -> str | None: +def _guess_entry_point(guess: str, console_scripts: dict[string, string]) -> str | None: # noqa: F821 for key, candidate in console_scripts.items(): if guess == key: return candidate @@ -87,6 +84,7 @@ def run( out: pathlib.Path, console_script: str, console_script_guess: str, + shebang: str, ): """Run the generator @@ -94,6 +92,8 @@ def run( entry_points: The entry_points.txt file to be parsed. out: The output file. console_script: The console_script entry in the entry_points.txt file. + console_script_guess: The string used for guessing the console_script if it is not provided. + shebang: The shebang to use for the entry point python file. Defaults to empty string (no shebang). """ config = EntryPointsParser() config.read(entry_points) @@ -130,12 +130,13 @@ def run( module, _, entry_point = entry_point.rpartition(":") attr, _, _ = entry_point.partition(".") # TODO: handle 'extras' in entry_point generation - # See https://github.com/bazelbuild/rules_python/issues/1383 + # See https://github.com/bazel-contrib/rules_python/issues/1383 # See https://packaging.python.org/en/latest/specifications/entry-points/ with open(out, "w") as f: f.write( _TEMPLATE.format( + shebang=f"{shebang}\n" if shebang else "", module=module, attr=attr, entry_point=entry_point, @@ -154,6 +155,10 @@ def main(): required=True, help="The string used for guessing the console_script if it is not provided.", ) + parser.add_argument( + "--shebang", + help="The shebang to use for the entry point python file.", + ) parser.add_argument( "entry_points", metavar="ENTRY_POINTS_TXT", @@ -173,6 +178,7 @@ def main(): out=args.out, console_script=args.console_script, console_script_guess=args.console_script_guess, + shebang=args.shebang, ) diff --git a/python/private/py_exec_tools_info.bzl b/python/private/py_exec_tools_info.bzl index b74f480fab..43213d5a13 100644 --- a/python/private/py_exec_tools_info.bzl +++ b/python/private/py_exec_tools_info.bzl @@ -24,15 +24,42 @@ When running it in an action, use `DefaultInfo.files_to_run` to ensure all its files are appropriately available. An exec interpreter may not be available, e.g. if all the exec tools are prebuilt binaries. -NOTE: this interpreter is really only for use when a build tool cannot use -the Python toolchain itself. When possible, prefeer to define a `py_binary` +:::{note} +this interpreter is really only for use when a build tool cannot use +the Python toolchain itself. When possible, prefer to define a `py_binary` instead and use it via a `cfg=exec` attribute; this makes it much easier to setup the runtime environment for the binary. See also: `py_interpreter_program` rule. +::: -NOTE: What interpreter is used depends on the toolchain constraints. Ensure -the proper target constraints are being applied when obtaining this from -the toolchain. +:::{note} +What interpreter is used depends on the toolchain constraints. Ensure the +proper target constraints are being applied when obtaining this from the +toolchain. +::: + +:::{warning} +This does not work correctly with RBE. Use {obj}`exec_runtime` instead. + +Once [bazelbuild/bazel#23620](https://github.com/bazelbuild/bazel/issues/23620) is resolved this warning +may be removed. +::: +""", + "exec_runtime": """ +:type: PyRuntimeInfo | None + +The Python runtime to use for the exec configuration. + +:::{versionadded} 1.9.0 + +In prior versions, the equivalent can be obtained using: +``` +exec_runtime = ( + ctx.toolchains["@rules_python//python:exec_tools_toolchain_type"]. + exec_tools.exec_interpreter[platform_common.ToolchainInfo].py3_runtime + ) +``` +::: """, "precompiler": """ :type: Target | None diff --git a/python/private/py_exec_tools_toolchain.bzl b/python/private/py_exec_tools_toolchain.bzl index edf9159759..d126262033 100644 --- a/python/private/py_exec_tools_toolchain.bzl +++ b/python/private/py_exec_tools_toolchain.bzl @@ -16,8 +16,9 @@ load("@bazel_skylib//lib:paths.bzl", "paths") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") +load(":common_labels.bzl", "labels") load(":py_exec_tools_info.bzl", "PyExecToolsInfo") -load(":sentinel.bzl", "SentinelInfo") +load(":sentinel_impl.bzl", "SentinelInfo") load(":toolchain_types.bzl", "TARGET_TOOLCHAIN_TYPE") def _py_exec_tools_toolchain_impl(ctx): @@ -29,13 +30,21 @@ def _py_exec_tools_toolchain_impl(ctx): if SentinelInfo in ctx.attr.exec_interpreter: exec_interpreter = None - return [platform_common.ToolchainInfo( - exec_tools = PyExecToolsInfo( - exec_interpreter = exec_interpreter, - precompiler = ctx.attr.precompiler, + exec_runtime = None + if exec_interpreter != None and platform_common.ToolchainInfo in exec_interpreter: + tc = exec_interpreter[platform_common.ToolchainInfo] + exec_runtime = getattr(tc, "py3_runtime", None) + + return [ + platform_common.ToolchainInfo( + exec_tools = PyExecToolsInfo( + exec_interpreter = exec_interpreter, + precompiler = ctx.attr.precompiler, + exec_runtime = exec_runtime, + ), + **extra_kwargs ), - **extra_kwargs - )] + ] py_exec_tools_toolchain = rule( implementation = _py_exec_tools_toolchain_impl, @@ -51,6 +60,11 @@ This provides `ToolchainInfo` with the following attributes: attrs = { "exec_interpreter": attr.label( default = "//python/private:current_interpreter_executable", + providers = [ + DefaultInfo, + # Add the toolchain provider so that we can forward provider fields. + platform_common.ToolchainInfo, + ], cfg = "exec", doc = """ An interpreter that is directly usable in the exec configuration @@ -69,6 +83,11 @@ handle all the necessary transitions and runtime setup to invoke a program. ::: See {obj}`PyExecToolsInfo.exec_interpreter` for further docs. + +:::{versionchanged} 1.4.0 +From now on the provided target also needs to provide `platform_common.ToolchainInfo` +so that the toolchain `py_runtime` field can be correctly forwarded. +::: """, ), "precompiler": attr.label( @@ -77,7 +96,7 @@ See {obj}`PyExecToolsInfo.exec_interpreter` for further docs. doc = "See {obj}`PyExecToolsInfo.precompiler`", ), "_visible_for_testing": attr.label( - default = "//python/private:visible_for_testing", + default = labels.VISIBLE_FOR_TESTING, ), }, ) @@ -91,6 +110,12 @@ def _current_interpreter_executable_impl(ctx): # re-exec. If it's not a recognized name, then they fail. if runtime.interpreter: executable = ctx.actions.declare_file(runtime.interpreter.basename) + + # NOTE: Using ctx.actions.symlink() here doesn't always work with RBE + # because it's not guaranteed that it will materialize as a symlink, but + # we rely on it being a symlink so that Python can find its actual + # PYTHONHOME. + # See https://github.com/bazelbuild/bazel/issues/23620 ctx.actions.symlink(output = executable, target_file = runtime.interpreter, is_executable = True) else: executable = ctx.actions.declare_symlink(paths.basename(runtime.interpreter_path)) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 8c0487d6a1..2ac2423b86 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -14,83 +14,152 @@ """Common functionality between test/binary executables.""" load("@bazel_skylib//lib:dicts.bzl", "dicts") +load("@bazel_skylib//lib:paths.bzl", "paths") load("@bazel_skylib//lib:structs.bzl", "structs") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load("@rules_cc//cc/common:cc_common.bzl", "cc_common") +load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") +load(":attr_builders.bzl", "attrb") load( ":attributes.bzl", "AGNOSTIC_EXECUTABLE_ATTRS", "COMMON_ATTRS", + "COVERAGE_ATTRS", + "IMPORTS_ATTRS", "PY_SRCS_ATTRS", "PrecompileAttr", "PycCollectionAttr", - "REQUIRED_EXEC_GROUPS", - "SRCS_VERSION_ALL_VALUES", - "create_srcs_attr", - "create_srcs_version_attr", + "REQUIRED_EXEC_GROUP_BUILDERS", + "WINDOWS_CONSTRAINTS_ATTRS", + "apply_config_settings_attr", ) load(":builders.bzl", "builders") load(":cc_helper.bzl", "cc_helper") load( ":common.bzl", + "ExplicitSymlink", + "actions_run", + "collect_cc_info", + "collect_deps", "collect_imports", "collect_runfiles", + "create_binary_semantics_struct", + "create_cc_details_struct", "create_instrumented_files_info", "create_output_group_info", "create_py_info", + "create_windows_exe_launcher", "csv", "filter_to_py_srcs", - "target_platform_has_any_constraint", - "union_attrs", + "is_bool", + "is_windows_platform", + "maybe_create_repo_mapping", + "relative_path", + "runfiles_root_path", ) +load(":common_labels.bzl", "labels") +load(":flags.bzl", "BootstrapImplFlag", "ValidateTestMainFlag", "VenvsUseDeclareSymlinkFlag", "read_possibly_native_flag") +load(":precompile.bzl", "maybe_precompile") load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") load(":py_executable_info.bzl", "PyExecutableInfo") -load(":py_info.bzl", "PyInfo") +load(":py_info.bzl", "PyInfo", "VenvSymlinkKind") load(":py_internal.bzl", "py_internal") -load(":py_runtime_info.bzl", "PyRuntimeInfo") +load(":py_interpreter_program.bzl", "PyInterpreterProgramInfo") +load(":py_runtime_info.bzl", "DEFAULT_STUB_SHEBANG") load(":reexports.bzl", "BuiltinPyInfo", "BuiltinPyRuntimeInfo") -load( - ":semantics.bzl", - "ALLOWED_MAIN_EXTENSIONS", - "BUILD_DATA_SYMLINK_PATH", - "IS_BAZEL", - "PY_RUNTIME_ATTR_NAME", -) -load( - ":toolchain_types.bzl", - "EXEC_TOOLS_TOOLCHAIN_TYPE", - TOOLCHAIN_TYPE = "TARGET_TOOLCHAIN_TYPE", -) +load(":rule_builders.bzl", "ruleb") +load(":toolchain_types.bzl", "CC_TOOLCHAIN_TYPE", "EXEC_TOOLS_TOOLCHAIN_TYPE", "LAUNCHER_MAKER_TOOLCHAIN_TYPE", TOOLCHAIN_TYPE = "TARGET_TOOLCHAIN_TYPE") +load(":transition_labels.bzl", "TRANSITION_LABELS") +load(":venv_runfiles.bzl", "create_venv_app_files") _py_builtins = py_internal - -# Bazel 5.4 doesn't have config_common.toolchain_type -_CC_TOOLCHAINS = [config_common.toolchain_type( - "@bazel_tools//tools/cpp:toolchain_type", - mandatory = False, -)] if hasattr(config_common, "toolchain_type") else [] +_EXTERNAL_PATH_PREFIX = "external" +_ZIP_RUNFILES_DIRECTORY_NAME = "runfiles" +_INIT_PY = "__init__.py" # Non-Google-specific attributes for executables # These attributes are for rules that accept Python sources. -EXECUTABLE_ATTRS = union_attrs( +EXECUTABLE_ATTRS = dicts.add( COMMON_ATTRS, AGNOSTIC_EXECUTABLE_ATTRS, PY_SRCS_ATTRS, + IMPORTS_ATTRS, + WINDOWS_CONSTRAINTS_ATTRS, + # starlark flags attributes + { + "_build_python_zip_flag": attr.label(default = "//python/config_settings:build_python_zip"), + "_default_to_explicit_init_py_flag": attr.label(default = "//python/config_settings:incompatible_default_to_explicit_init_py"), + "_python_import_all_repositories_flag": attr.label(default = "//python/config_settings:experimental_python_import_all_repositories"), + "_python_path_flag": attr.label(default = "//python/config_settings:python_path"), + }, { + "interpreter_args": lambda: attrb.StringList( + doc = """ +Arguments that are only applicable to the interpreter. + +The args an interpreter supports are specific to the interpreter. For +CPython, see . + +:::{note} +Only supported for {obj}`--bootstrap_impl=script`. Ignored otherwise. +::: + +:::{seealso} +The {any}`RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` environment variable +::: + +:::{versionadded} 1.3.0 +::: +""", + ), + "legacy_create_init": lambda: attrb.Int( + default = -1, + values = [-1, 0, 1], + doc = """\ +Whether to implicitly create empty `__init__.py` files in the runfiles tree. +These are created in every directory containing Python source code or shared +libraries, and every parent directory of those directories, excluding the repo +root directory. The default, `-1` (auto), means true unless +`--incompatible_default_to_explicit_init_py` is used. If false, the user is +responsible for creating (possibly empty) `__init__.py` files and adding them to +the `srcs` of Python targets as required. + """, + ), # TODO(b/203567235): In the Java impl, any file is allowed. While marked # label, it is more treated as a string, and doesn't have to refer to # anything that exists because it gets treated as suffix-search string # over `srcs`. - "main": attr.label( + "main": lambda: attrb.Label( allow_single_file = True, doc = """\ Optional; the name of the source file that is the main entry point of the application. This file must also be listed in `srcs`. If left unspecified, `name`, with `.py` appended, is used instead. If `name` does not match any filename in `srcs`, `main` must be specified. + +This is mutually exclusive with {obj}`main_module`. """, ), - "pyc_collection": attr.string( + "main_module": lambda: attrb.String( + doc = """ +Module name to execute as the main program. + +When set, `srcs` is not required, and it is assumed the module is +provided by a dependency. + +See for more +information about running modules as the main program. + +This is mutually exclusive with {obj}`main`. + +:::{versionadded} 1.3.0 +::: +:::{versionchanged} 1.7.0 +Support added for {obj}`--bootstrap_impl=system_python`. +::: +""", + ), + "pyc_collection": lambda: attrb.String( default = PycCollectionAttr.INHERIT, values = sorted(PycCollectionAttr.__members__.values()), doc = """ @@ -105,32 +174,986 @@ Valid values are: target level. """, ), - # TODO(b/203567235): In Google, this attribute is deprecated, and can - # only effectively be PY3. Externally, with Bazel, this attribute has - # a separate story. - "python_version": attr.string( + "python_version": lambda: attrb.String( # TODO(b/203567235): In the Java impl, the default comes from # --python_version. Not clear what the Starlark equivalent is. - default = "PY3", - # NOTE: Some tests care about the order of these values. - values = ["PY2", "PY3"], - doc = "Defunct, unused, does nothing.", + doc = """ +The Python version this target should use. + +The value should be in `X.Y` or `X.Y.Z` (or compatible) format. If empty or +unspecified, the incoming configuration's {obj}`--python_version` flag is +inherited. For backwards compatibility, the values `PY2` and `PY3` are +accepted, but treated as an empty/unspecified value. + +:::{note} +In order for the requested version to be used, there must be a +toolchain configured to match the Python version. If there isn't, then it +may be silently ignored, or an error may occur, depending on the toolchain +configuration. +::: + +:::{versionchanged} 1.1.0 + +This attribute was changed from only accepting `PY2` and `PY3` values to +accepting arbitrary Python versions. +::: +""", + ), + # Required to opt-in to the transition feature. + "_allowlist_function_transition": lambda: attrb.Label( + default = "@bazel_tools//tools/allowlists/function_transition_allowlist", ), - "_bootstrap_impl_flag": attr.label( - default = "//python/config_settings:bootstrap_impl", + "_bootstrap_impl_flag": lambda: attrb.Label( + default = labels.BOOTSTRAP_IMPL, providers = [BuildSettingInfo], ), - "_windows_constraints": attr.label_list( - default = [ - "@platforms//os:windows", - ], + "_bootstrap_template": lambda: attrb.Label( + allow_single_file = True, + default = "@bazel_tools//tools/python:python_bootstrap_template.txt", + ), + "_build_data_writer": lambda: attrb.Label( + default = "//python/private:build_data_writer", + allow_files = True, + cfg = "exec", + ), + "_debugger_flag": lambda: attrb.Label( + default = "//python/private:debugger_if_target_config", + providers = [PyInfo], + ), + "_exe_zip_maker": lambda: attrb.Label( + cfg = "exec", + default = "//tools/private/zipapp:exe_zip_maker", + ), + "_launcher": lambda: attrb.Label( + cfg = "target", + # NOTE: This is an executable, but is only used for Windows. It + # can't have executable=True because the backing target is an + # empty target for other platforms. + default = "//tools/launcher:launcher", + ), + "_python_version_flag": lambda: attrb.Label( + default = labels.PYTHON_VERSION, + ), + "_uncachable_version_file": lambda: attrb.Label( + default = "//python/private:uncachable_version_file", + allow_files = True, + ), + "_venvs_use_declare_symlink_flag": lambda: attrb.Label( + default = labels.VENVS_USE_DECLARE_SYMLINK, + providers = [BuildSettingInfo], + ), + "_zipper": lambda: attrb.Label( + cfg = "exec", + executable = True, + default = "@bazel_tools//tools/zip:zipper", ), }, - create_srcs_version_attr(values = SRCS_VERSION_ALL_VALUES), - create_srcs_attr(mandatory = True), - allow_none = True, + { + "_windows_launcher_maker": lambda: attrb.Label( + default = "@bazel_tools//tools/launcher:launcher_maker", + cfg = "exec", + executable = True, + ), + } if not rp_config.bazel_9_or_later else {}, ) +def convert_legacy_create_init_to_int(kwargs): + """Convert "legacy_create_init" key to int, in-place. + + Args: + kwargs: The kwargs to modify. The key "legacy_create_init", if present + and bool, will be converted to its integer value, in place. + """ + if is_bool(kwargs.get("legacy_create_init")): + kwargs["legacy_create_init"] = 1 if kwargs["legacy_create_init"] else 0 + +def py_executable_impl(ctx, *, is_test, inherited_environment): + return py_executable_base_impl( + ctx = ctx, + semantics = create_binary_semantics(), + is_test = is_test, + inherited_environment = inherited_environment, + ) + +def create_binary_semantics(): + return create_binary_semantics_struct( + # keep-sorted start + get_native_deps_dso_name = _get_native_deps_dso_name, + should_build_native_deps_dso = lambda ctx: False, + # keep-sorted end + ) + +def _should_create_init_files(ctx): + if ctx.attr.legacy_create_init == -1: + return not read_possibly_native_flag(ctx, "default_to_explicit_init_py") + else: + return bool(ctx.attr.legacy_create_init) + +def _create_executable( + ctx, + *, + executable, + main_py, + imports, + is_test, + runtime_details, + cc_details, + native_deps_details, + runfiles_details, + extra_deps): + _ = is_test, cc_details, native_deps_details # @unused + + is_windows = is_windows_platform(ctx) + + if is_windows: + if not executable.extension == "exe": + fail("Should not happen: somehow we are generating a non-.exe file on windows") + base_executable_name = executable.basename[0:-4] + else: + base_executable_name = executable.basename + + # Venv outputs are package-relative, so preserve the full target name to + # avoid collisions between targets like foo/tool, bar/tool, and foo_tool. + venv_output_prefix = ctx.label.name + venv = None + + # The check for stage2_bootstrap_template is to support legacy + # BuiltinPyRuntimeInfo providers, which is likely to come from + # @bazel_tools//tools/python:autodetecting_toolchain, the toolchain used + # for workspace builds when no rules_python toolchain is configured. + if ( + runtime_details.effective_runtime and + hasattr(runtime_details.effective_runtime, "stage2_bootstrap_template") + ): + venv = _create_venv( + ctx, + output_prefix = venv_output_prefix, + imports = imports, + runtime_details = runtime_details, + add_runfiles_root_to_sys_path = ( + "1" if BootstrapImplFlag.get_value(ctx) == BootstrapImplFlag.SYSTEM_PYTHON else "0" + ), + extra_deps = extra_deps, + ) + + stage2_bootstrap = _create_stage2_bootstrap( + ctx, + output_prefix = base_executable_name, + output_sibling = executable, + main_py = main_py, + imports = imports, + runtime_details = runtime_details, + venv = venv, + build_data_file = runfiles_details.build_data_file, + ) + extra_runfiles = ctx.runfiles( + [stage2_bootstrap] + ( + venv.files_without_interpreter if venv else [] + ), + ).merge(venv.lib_runfiles) + zip_main = _create_zip_main( + ctx, + stage2_bootstrap = stage2_bootstrap, + runtime_details = runtime_details, + venv = venv, + ) + else: + stage2_bootstrap = None + extra_runfiles = ctx.runfiles() + zip_main = ctx.actions.declare_file(base_executable_name + ".temp", sibling = executable) + _create_stage1_bootstrap( + ctx, + output = zip_main, + main_py = main_py, + imports = imports, + is_for_zip = True, + runtime_details = runtime_details, + ) + + zip_file = ctx.actions.declare_file(base_executable_name + ".zip", sibling = executable) + _create_zip_file( + ctx, + output = zip_file, + zip_main = zip_main, + runfiles = runfiles_details.runfiles_without_exe.merge(extra_runfiles), + ) + + extra_default_outputs = [] + + # NOTE: --build_python_zip defaults to true on Windows + build_zip_enabled = read_possibly_native_flag(ctx, "build_python_zip") and not is_windows + if is_windows: + # The legacy build_python_zip codepath isn't compatible with full venvs on Windows. + build_zip_enabled = False + + # When --build_python_zip is enabled, then the zip file becomes + # one of the default outputs. + if build_zip_enabled: + if not is_windows: + # buildifier: disable=print + print( + """ +====================================================================== +WARNING: Target: {} + The `--build_python_zip` flag and implicit zipapp output of `py_binary` + and `py_test` is deprecated and will be removed in a future release. + Switch to `py_zipapp_binary` or `py_zipapp_test`. For migration + instructions and guide, see: + + https://github.com/bazel-contrib/rules_python/issues/3567 +====================================================================== + """.rstrip().format(ctx.label), + ) + + extra_default_outputs.append(zip_file) + + # The logic here is a bit convoluted. Essentially, there are 3 types of + # executables produced: + # 1. (non-Windows) A bootstrap template based program. + # 2. (non-Windows) A self-executable zip file of a bootstrap template based program. + # 3. (Windows) A native Windows executable that finds and launches + # the actual underlying Bazel program (one of the above). Note that + # it implicitly assumes one of the above is located next to it, and + # that --build_python_zip defaults to true for Windows. + + should_create_executable_zip = False + bootstrap_output = None + if not is_windows: + if build_zip_enabled: + should_create_executable_zip = True + else: + bootstrap_output = executable + else: + create_windows_exe_launcher( + ctx, + output = executable, + use_zip_file = build_zip_enabled, + python_binary_path = runtime_details.executable_interpreter_path, + ) + + # On Windows, the main executable has an "exe" extension, so + # here we re-use the un-extensioned name for the bootstrap output. + bootstrap_output = ctx.actions.declare_file(base_executable_name, sibling = executable) + + # The launcher looks for the non-zip executable next to + # itself, so add it to the default outputs. + extra_default_outputs.append(bootstrap_output) + + if should_create_executable_zip: + if bootstrap_output != None: + fail("Should not occur: bootstrap_output should not be used " + + "when creating an executable zip") + _create_executable_zip_file( + ctx, + output = executable, + zip_file = zip_file, + stage2_bootstrap = stage2_bootstrap, + runtime_details = runtime_details, + venv = venv, + ) + elif bootstrap_output: + _create_stage1_bootstrap( + ctx, + output = bootstrap_output, + stage2_bootstrap = stage2_bootstrap, + runtime_details = runtime_details, + is_for_zip = False, + imports = imports, + main_py = main_py, + venv = venv, + ) + else: + # Otherwise, this should be the Windows case of launcher + zip. + # Double check this just to make sure. + if not is_windows or not build_zip_enabled: + fail(("Should not occur: The non-executable-zip and " + + "non-bootstrap-template case should have windows and zip " + + "both true, but got " + + "is_windows={is_windows} " + + "build_zip_enabled={build_zip_enabled}").format( + is_windows = is_windows, + build_zip_enabled = build_zip_enabled, + )) + + app_runfiles = builders.RunfilesBuilder() + app_runfiles.add(runfiles_details.app_runfiles) + if venv: + app_runfiles.add(venv.files_without_interpreter) + app_runfiles.add(venv.lib_runfiles) + + # The interpreter is added this late in the process so that it isn't + # added to the zipped files. + if venv and venv.interpreter_runfiles: + extra_runfiles = extra_runfiles.merge(venv.interpreter_runfiles) + return struct( + # depset[File] of additional files that should be included as default + # outputs. + extra_default_outputs = depset(extra_default_outputs), + # dict[str, depset[File]]; additional output groups that should be + # returned. + output_groups = {"python_zip_file": depset([zip_file])}, + # runfiles; additional runfiles to include. + extra_runfiles = extra_runfiles, + # File|None; the stage2 bootstrap file, if any + stage2_bootstrap = stage2_bootstrap, + # runfiles; runfiles for the app itself (e.g its deps, but no Python + # runtime files) + app_runfiles = app_runfiles.build(ctx), + # depset[ExplicitSymlink]None; symlinks that should be created in + # the venv to augment app_runfiles + venv_app_symlinks = venv.lib_symlinks if venv else None, + # File|None; the venv `bin/python3` file, if any. + venv_python_exe = venv.interpreter if venv else None, + # runfiles|None; runfiles in the venv for the interpreter + venv_interpreter_runfiles = venv.interpreter_runfiles if venv else None, + # depset[ExplicitSymlink]|None; symlinks that should be created + venv_interpreter_symlinks = venv.interpreter_symlinks if venv else None, + ) + +def _create_zip_main(ctx, *, stage2_bootstrap, runtime_details, venv): + if venv.interpreter: + python_binary = runfiles_root_path(ctx, venv.interpreter.short_path) + else: + python_binary = "" + python_binary_actual = venv.interpreter_actual_path + + # The location of this file doesn't really matter. It's added to + # the zip file as the top-level __main__.py file and not included + # elsewhere. + output = ctx.actions.declare_file(ctx.label.name + "_zip__main__.py") + ctx.actions.expand_template( + template = runtime_details.effective_runtime.zip_main_template, + output = output, + substitutions = { + "%python_binary%": python_binary, + "%python_binary_actual%": python_binary_actual, + "%stage2_bootstrap%": runfiles_root_path(ctx, stage2_bootstrap.short_path), + "%workspace_name%": ctx.workspace_name, + }, + ) + return output + +# Create a venv the executable can use. +# For venv details and the venv startup process, see: +# * https://docs.python.org/3/library/venv.html +# * https://snarky.ca/how-virtual-environments-work/ +# * https://github.com/python/cpython/blob/main/Modules/getpath.py +# * https://github.com/python/cpython/blob/main/Lib/site.py +def _create_venv(ctx, output_prefix, imports, runtime_details, add_runfiles_root_to_sys_path, extra_deps): + venv_ctx_rel_root = "_{}.venv".format(output_prefix.lstrip("_")) + runtime = runtime_details.effective_runtime + if runtime.interpreter: + interpreter_actual_path = runfiles_root_path(ctx, runtime.interpreter.short_path) + else: + interpreter_actual_path = runtime.interpreter_path + + is_windows = is_windows_platform(ctx) + if is_windows: + venv_details = _create_venv_windows( + ctx, + venv_ctx_rel_root = venv_ctx_rel_root, + interpreter_actual_path = interpreter_actual_path, + runtime = runtime, + ) + else: + venv_details = _create_venv_unixy( + ctx, + venv_ctx_rel_root = venv_ctx_rel_root, + interpreter_actual_path = interpreter_actual_path, + runtime = runtime, + ) + + site_packages = "{}/{}".format(venv_ctx_rel_root, venv_details.site_packages) + + pth = ctx.actions.declare_file("{}/bazel.pth".format(site_packages)) + ctx.actions.write(pth, "import _bazel_site_init\n") + + site_init = ctx.actions.declare_file("{}/_bazel_site_init.py".format(site_packages)) + computed_subs = ctx.actions.template_dict() + computed_subs.add_joined("%imports%", imports, join_with = ":", map_each = _map_each_identity) + ctx.actions.expand_template( + template = runtime.site_init_template, + output = site_init, + substitutions = { + "%add_runfiles_root_to_sys_path%": add_runfiles_root_to_sys_path, + "%coverage_tool%": _get_coverage_tool_runfiles_path(ctx, runtime), + "%import_all%": "True" if read_possibly_native_flag(ctx, "python_import_all_repositories") else "False", + "%site_init_runfiles_path%": runfiles_root_path(ctx, site_init.short_path), + "%workspace_name%": ctx.workspace_name, + }, + computed_substitutions = computed_subs, + ) + + # See https://docs.python.org/3/library/sysconfig.html#posix-prefix + # for how schemes map under the venv. + venv_dir_map = { + VenvSymlinkKind.BIN: "{}/{}".format(venv_ctx_rel_root, venv_details.bin_dir), + VenvSymlinkKind.LIB: site_packages, + VenvSymlinkKind.INCLUDE: "{}/{}".format(venv_ctx_rel_root, venv_details.include_dir), + VenvSymlinkKind.DATA: venv_ctx_rel_root, + } + venv_app_files = create_venv_app_files( + ctx, + deps = collect_deps(ctx, extra_deps), + venv_dir_map = venv_dir_map, + ) + + files_without_interpreter = [pth, site_init] + venv_app_files.venv_files + if venv_details.pyvenv_cfg: + files_without_interpreter.append(venv_details.pyvenv_cfg) + + return struct( + # File or None; the `bin/python3` executable in the venv. + # None if a full venv isn't created. + interpreter = venv_details.interpreter, + # Files in the venv that need to be created for the interpreter to work + interpreter_runfiles = venv_details.interpreter_runfiles, + # depset[ExplicitSymlink] of symlinks to create. + # This is only used when declare_symlink() can't be used to represent + # creating such a link (i.e Windows) + interpreter_symlinks = venv_details.interpreter_symlinks, + # bool; True if the venv should be recreated at runtime + recreate_venv_at_runtime = venv_details.recreate_venv_at_runtime, + # Runfiles root relative path or absolute path + interpreter_actual_path = interpreter_actual_path, + files_without_interpreter = files_without_interpreter, + # string; venv-relative path to the site-packages directory. + venv_site_packages = venv_details.site_packages, + # string; runfiles-root relative path to venv root. + venv_root = runfiles_root_path( + ctx, + paths.join( + py_internal.get_label_repo_runfiles_path(ctx.label), + venv_ctx_rel_root, + ), + ), + # venv files for user library dependencies (files that are specific + # to the executable bootstrap and python runtime aren't here). + # `root_symlinks` should be used, otherwise, with symlinks files always go + # to `_main` prefix, and binaries from non-root module become broken. + lib_runfiles = ctx.runfiles( + root_symlinks = venv_app_files.runfiles_symlinks, + ), + lib_symlinks = venv_app_files.explicit_symlinks, + ) + +def _create_venv_unixy(ctx, *, venv_ctx_rel_root, runtime, interpreter_actual_path): + interpreter_runfiles = builders.RunfilesBuilder() + is_bootstrap_script = BootstrapImplFlag.get_value(ctx) == BootstrapImplFlag.SCRIPT + create_full_venv = True + + # The legacy build_python_zip codepath (enabled by default on windows) isn't + # compatible with full venv. + # TODO: Use non-build_python_zip codepath for Windows + if not rp_config.bazel_8_or_later and not is_bootstrap_script: + # Full venv for Bazel 7 + system_python is disabled because packaging + # it using build_python_zip=true or rules_pkg breaks. + # * Using build_python_zip=true breaks because the legacy zipapp support + # doesn't handle symlinks correctly. + # * Using rules_pkg breaks for two reasons: + # 1. It requires rules_pkg 1.2, which crashes under Bazel 7 + # 2. It requires File.is_symlink, which is a Bazel 8+ API. + # While bootstrap=script has the same problems, it has always been like + # that. + create_full_venv = False + + if create_full_venv: + # The pyvenv.cfg file must be present to trigger the venv site hooks. + # Because it's paths are expected to be absolute paths, we can't reliably + # put much in it. See https://github.com/python/cpython/issues/83650 + pyvenv_cfg = ctx.actions.declare_file("{}/pyvenv.cfg".format(venv_ctx_rel_root)) + ctx.actions.write(pyvenv_cfg, "") + else: + pyvenv_cfg = None + + venvs_use_declare_symlink_enabled = ( + VenvsUseDeclareSymlinkFlag.get_value(ctx) == VenvsUseDeclareSymlinkFlag.YES + ) + + recreate_venv_at_runtime = False + + venv_bin_ctx_rel_path = "{}/bin".format(venv_ctx_rel_root) + if create_full_venv: + # Some wrappers around the interpreter (e.g. pyenv) use the program + # name to decide what to do, so preserve the name. + py_exe_basename = paths.basename(interpreter_actual_path) + + if not venvs_use_declare_symlink_enabled or not runtime.supports_build_time_venv: + recreate_venv_at_runtime = True + + # When the venv symlinks are disabled, the $venv/bin/python3 file isn't + # needed or used at runtime. However, the zip code uses the interpreter + # File object to figure out some paths. + interpreter = ctx.actions.declare_file("{}/{}".format(venv_bin_ctx_rel_path, py_exe_basename)) + ctx.actions.write(interpreter, "actual:{}".format(interpreter_actual_path)) + + elif runtime.interpreter: + # Even though ctx.actions.symlink() is used, using + # declare_symlink() is required to ensure that the resulting file + # in runfiles is always a symlink. An RBE implementation, for example, + # may choose to write what symlink() points to instead. + interpreter = ctx.actions.declare_symlink("{}/{}".format(venv_bin_ctx_rel_path, py_exe_basename)) + interpreter_runfiles.add(interpreter) + + rel_path = relative_path( + # dirname is necessary because a relative symlink is relative to + # the directory the symlink resides within. + from_ = paths.dirname(runfiles_root_path(ctx, interpreter.short_path)), + to = interpreter_actual_path, + ) + ctx.actions.symlink(output = interpreter, target_path = rel_path) + else: + interpreter = ctx.actions.declare_symlink("{}/{}".format(venv_bin_ctx_rel_path, py_exe_basename)) + interpreter_runfiles.add(interpreter) + ctx.actions.symlink(output = interpreter, target_path = runtime.interpreter_path) + else: + interpreter = None + + if runtime.interpreter_version_info: + version = "{}.{}".format( + runtime.interpreter_version_info.major, + runtime.interpreter_version_info.minor, + ) + else: + version_flag = ctx.attr._python_version_flag[config_common.FeatureFlagInfo].value + version_flag_parts = version_flag.split(".")[0:2] + version = "{}.{}".format(*version_flag_parts) + + # See site.py logic: free-threaded builds append "t" to the venv lib dir name + if "t" in runtime.abi_flags: + version += "t" + + site_packages = "lib/python{}/site-packages".format(version) + return _venv_details( + interpreter = interpreter, + pyvenv_cfg = pyvenv_cfg, + site_packages = site_packages, + bin_dir = "bin", + include_dir = "include", + recreate_venv_at_runtime = recreate_venv_at_runtime, + interpreter_runfiles = interpreter_runfiles.build(ctx), + interpreter_symlinks = depset(), + ) + +def _create_venv_windows(ctx, *, venv_ctx_rel_root, runtime, interpreter_actual_path): + interpreter_runfiles = builders.RunfilesBuilder() + interpreter_symlinks = builders.DepsetBuilder() + + # Some wrappers around the interpreter (e.g. pyenv) use the program + # name to decide what to do, so preserve the name. + py_exe_basename = paths.basename(interpreter_actual_path) + venv_bin_rel_path = "Scripts" + venv_bin_ctx_rel_path = "{}/{}".format(venv_ctx_rel_root, venv_bin_rel_path) + if runtime.interpreter: + venv_rel_path = paths.join(venv_bin_rel_path, py_exe_basename) + venv_ctx_rel_path = paths.join(venv_ctx_rel_root, venv_rel_path) + interpreter = ctx.actions.declare_file(venv_ctx_rel_path) + interpreter_runfiles.add(interpreter) + ctx.actions.symlink(output = interpreter, target_file = runtime.interpreter) + + rf_path = runfiles_root_path(ctx, interpreter.short_path) + interpreter_symlinks.add(ExplicitSymlink( + runfiles_path = rf_path, + venv_path = venv_rel_path, + link_to_path = interpreter_actual_path, + files = depset([runtime.interpreter]), + )) + + # This isn't strictly correct, but should work ok. + interpreter_symlinks.add(ExplicitSymlink( + runfiles_path = paths.join(paths.dirname(rf_path), "pythonw.exe"), + venv_path = paths.join(paths.dirname(venv_rel_path), "pythonw.exe"), + link_to_path = paths.join(paths.dirname(interpreter_actual_path), "pythonw.exe"), + files = depset(), + )) + else: + # It's OK to use declare_symlink here because an absolute path + # will be written to it, so Bazel won't mangle it. + interpreter = ctx.actions.declare_symlink("{}/{}".format(venv_bin_ctx_rel_path, py_exe_basename)) + interpreter_runfiles.add(interpreter) + ctx.actions.symlink(output = interpreter, target_path = runtime.interpreter_path) + + # NOTE: The .dll files must exist, however, they may not be known at build time + # if the interpreter is resolved at runtime. + for f in runtime.venv_bin_files: + venv_rel_path = paths.join(venv_bin_rel_path, f.basename) + venv_ctx_rel_path = paths.join(venv_ctx_rel_root, venv_rel_path) + + venv_file = ctx.actions.declare_file(venv_ctx_rel_path) + ctx.actions.symlink(output = venv_file, target_file = f) + + interpreter_runfiles.add(venv_file) + + rf_path = runfiles_root_path(ctx, venv_file.short_path) + interpreter_symlinks.add(ExplicitSymlink( + runfiles_path = rf_path, + venv_path = venv_rel_path, + link_to_path = runfiles_root_path(ctx, f.short_path), + files = depset([f]), + )) + + # See site.py logic: Windows uses a version/build agnostic site-packages path + site_packages = "Lib/site-packages" + + return _venv_details( + interpreter = interpreter, + pyvenv_cfg = None, + site_packages = site_packages, + bin_dir = venv_bin_rel_path, + include_dir = "Include", + recreate_venv_at_runtime = True, + interpreter_runfiles = interpreter_runfiles.build(ctx), + interpreter_symlinks = interpreter_symlinks.build(), + ) + +def _venv_details( + *, + interpreter, + pyvenv_cfg, + site_packages, + bin_dir, + include_dir, + recreate_venv_at_runtime, + interpreter_runfiles, + interpreter_symlinks): + """Helper to create a struct of platform-specific venv details.""" + return struct( + # File; the `bin/python` executable (or equivalent) within the venv. + interpreter = interpreter, + # File|None; the pyvenv.cfg file, if any. May be none, in which case, + # it's expected that one will be created at runtime. + pyvenv_cfg = pyvenv_cfg, + # str; venv-relative path to the site-packages directory + site_packages = site_packages, + # str; venv-relative path to the venv's bin directory. + bin_dir = bin_dir, + # str; venv-relative-path to the venv's include directory. + include_dir = include_dir, + # bool; True if the venv needs to be recreated at runtime (because the + # build-time construction isn't sufficient). False if the build-time + # constructed venv is sufficient. + recreate_venv_at_runtime = recreate_venv_at_runtime, + # runfiles; runfiles for interpreter-specific files in the venv. + interpreter_runfiles = interpreter_runfiles, + # depset[ExplicitSymlink] of symlinks specific + # to the interpreter. Only used for Windows. + interpreter_symlinks = interpreter_symlinks, + ) + +def _map_each_identity(v): + return v + +def _get_coverage_tool_runfiles_path(ctx, runtime): + if (ctx.configuration.coverage_enabled and + runtime and + runtime.coverage_tool): + return runfiles_root_path(ctx, runtime.coverage_tool.short_path) + else: + return "" + +def _create_stage2_bootstrap( + ctx, + *, + output_prefix, + output_sibling, + main_py, + imports, + runtime_details, + build_data_file, + venv): + output = ctx.actions.declare_file( + # Prepend with underscore to prevent pytest from trying to + # process the bootstrap for files starting with `test_` + "_{}_stage2_bootstrap.py".format(output_prefix), + sibling = output_sibling, + ) + runtime = runtime_details.effective_runtime + + template = runtime.stage2_bootstrap_template + + if main_py: + main_py_path = runfiles_root_path(ctx, main_py.short_path) + else: + main_py_path = "" + + ctx.actions.expand_template( + template = template, + output = output, + substitutions = { + "%build_data_file%": runfiles_root_path(ctx, build_data_file.short_path), + "%coverage_instrumented%": str(int(ctx.configuration.coverage_enabled and ctx.coverage_instrumented())), + "%coverage_tool%": _get_coverage_tool_runfiles_path(ctx, runtime), + "%import_all%": "True" if read_possibly_native_flag(ctx, "python_import_all_repositories") else "False", + "%imports%": ":".join(imports.to_list()), + "%main%": main_py_path, + "%main_module%": ctx.attr.main_module, + "%target%": str(ctx.label), + "%venv_rel_site_packages%": venv.venv_site_packages, + "%venv_root%": venv.venv_root, + "%workspace_name%": ctx.workspace_name, + }, + is_executable = True, + ) + return output + +def _create_stage1_bootstrap( + ctx, + *, + output, + main_py = None, + stage2_bootstrap = None, + imports = None, + is_for_zip, + runtime_details, + venv = None): + """Create a legacy bootstrap script that is written in Python.""" + runtime = runtime_details.effective_runtime + + if venv: + if venv.interpreter: + python_binary_path = runfiles_root_path(ctx, venv.interpreter.short_path) + else: + python_binary_path = "" + else: + python_binary_path = runtime_details.executable_interpreter_path + + python_binary_actual = venv.interpreter_actual_path if venv else "" + + # Guard against the following: + # * Runtime may be None on Windows due to the --python_path flag. + # * Runtime may not have 'supports_build_time_venv' if a really old version is autoloaded + # on bazel 7.6.x. + if runtime and getattr(runtime, "supports_build_time_venv", False): + resolve_python_binary_at_runtime = "0" + else: + resolve_python_binary_at_runtime = "1" + + subs = { + "%interpreter_args%": "\n".join(ctx.attr.interpreter_args), + "%is_zipfile%": "1" if is_for_zip else "0", + "%python_binary%": python_binary_path, + "%python_binary_actual%": python_binary_actual, + "%recreate_venv_at_runtime%": str(int(venv.recreate_venv_at_runtime)) if venv else "0", + "%resolve_python_binary_at_runtime%": resolve_python_binary_at_runtime, + "%target%": str(ctx.label), + "%venv_rel_site_packages%": venv.venv_site_packages if venv else "", + "%workspace_name%": ctx.workspace_name, + } + computed_subs = ctx.actions.template_dict() + if venv: + runtime_venv_symlinks = depset( + transitive = [venv.interpreter_symlinks, venv.lib_symlinks], + ) + computed_subs.add_joined( + "%runtime_venv_symlinks%", + runtime_venv_symlinks, + join_with = "\n", + map_each = _map_runtime_venv_symlink, + ) + + if stage2_bootstrap: + subs["%stage2_bootstrap%"] = runfiles_root_path(ctx, stage2_bootstrap.short_path) + template = runtime.bootstrap_template + subs["%shebang%"] = runtime.stub_shebang + elif not ctx.files.srcs: + fail("mandatory 'srcs' files have not been provided") + else: + if (ctx.configuration.coverage_enabled and + runtime and + runtime.coverage_tool): + coverage_tool_runfiles_path = runfiles_root_path(ctx, runtime.coverage_tool.short_path) + else: + coverage_tool_runfiles_path = "" + if runtime: + subs["%shebang%"] = runtime.stub_shebang + template = runtime.bootstrap_template + else: + subs["%shebang%"] = DEFAULT_STUB_SHEBANG + template = ctx.file._bootstrap_template + + subs["%coverage_tool%"] = coverage_tool_runfiles_path + subs["%import_all%"] = ("True" if read_possibly_native_flag(ctx, "python_import_all_repositories") else "False") + subs["%imports%"] = ":".join(imports.to_list()) + subs["%main%"] = runfiles_root_path(ctx, main_py.short_path) + + ctx.actions.expand_template( + template = template, + output = output, + substitutions = subs, + computed_substitutions = computed_subs, + is_executable = True, + ) + +def _map_runtime_venv_symlink(entry): + return entry.venv_path + "|" + entry.link_to_path + +def _create_zip_file(ctx, *, output, zip_main, runfiles): + """Create a Python zipapp (zip with __main__.py entry point).""" + workspace_name = ctx.workspace_name + legacy_external_runfiles = _py_builtins.get_legacy_external_runfiles(ctx) + + manifest = ctx.actions.args() + manifest.use_param_file("@%s", use_always = True) + manifest.set_param_file_format("multiline") + + manifest.add("__main__.py={}".format(zip_main.path)) + manifest.add("__init__.py=") + manifest.add( + "{}=".format(_get_zip_runfiles_path(_INIT_PY, workspace_name)), + ) + + def map_zip_empty_filenames(list_paths_cb): + return [ + # FIXME @aignas 2025-12-06: what kind of paths do we expect here? Will they + # ever start with `../` or `external`? + _get_zip_runfiles_path_legacy(path, workspace_name, legacy_external_runfiles) + "=" + for path in list_paths_cb().to_list() + ] + + manifest.add_all( + # NOTE: Accessing runfiles.empty_filenames implicitly flattens the runfiles. + # Smuggle a lambda in via a list to defer that flattening. + [lambda: runfiles.empty_filenames], + map_each = map_zip_empty_filenames, + allow_closure = True, + ) + + def map_zip_runfiles(file): + return ( + # NOTE: Use "+" for performance + _get_zip_runfiles_path_legacy(file.short_path, workspace_name, legacy_external_runfiles) + + "=" + file.path + ) + + manifest.add_all(runfiles.files, map_each = map_zip_runfiles, allow_closure = True) + + inputs = [zip_main] + zip_repo_mapping_manifest = maybe_create_repo_mapping( + ctx = ctx, + runfiles = runfiles, + ) + if zip_repo_mapping_manifest: + manifest.add("{}/_repo_mapping={}".format( + _ZIP_RUNFILES_DIRECTORY_NAME, + zip_repo_mapping_manifest.path, + )) + inputs.append(zip_repo_mapping_manifest) + + zip_cli_args = ctx.actions.args() + zip_cli_args.add("cC") + zip_cli_args.add(output) + + ctx.actions.run( + executable = ctx.executable._zipper, + arguments = [zip_cli_args, manifest], + inputs = depset(inputs, transitive = [runfiles.files]), + outputs = [output], + use_default_shell_env = True, + mnemonic = "PythonZipper", + progress_message = "Building Python zip: %{label}", + ) + +def _get_zip_runfiles_path(path, workspace_name = ""): + # NOTE @aignas 2025-12-06: This is to avoid the prefix checking in the very + # trivial case that is always happening once per this function call + + # NOTE: Use "+" for performance + if workspace_name: + # NOTE: Use "+" for performance + return _ZIP_RUNFILES_DIRECTORY_NAME + "/" + workspace_name + "/" + path + else: + return _ZIP_RUNFILES_DIRECTORY_NAME + "/" + path + +def _get_zip_runfiles_path_legacy(path, workspace_name, legacy_external_runfiles): + if legacy_external_runfiles and path.startswith(_EXTERNAL_PATH_PREFIX): + return _get_zip_runfiles_path(path.removeprefix(_EXTERNAL_PATH_PREFIX)) + elif path.startswith("../"): + # NOTE: External runfiles (artifacts in other repos) will have a leading + # path component of "../" so that they refer outside the main workspace + # directory and into the runfiles root. So we simplify it, e.g. + # "workspace/../foo/bar" to simply "foo/bar". + return _get_zip_runfiles_path(path[3:]) + else: + return _get_zip_runfiles_path(path, workspace_name) + +def _create_executable_zip_file( + ctx, + *, + output, + zip_file, + stage2_bootstrap, + runtime_details, + venv): + prelude = ctx.actions.declare_file( + "{}_zip_prelude.sh".format(output.basename), + sibling = output, + ) + if stage2_bootstrap: + _create_stage1_bootstrap( + ctx, + output = prelude, + stage2_bootstrap = stage2_bootstrap, + runtime_details = runtime_details, + is_for_zip = True, + venv = venv, + ) + else: + ctx.actions.write(prelude, "#!/usr/bin/env python3\n") + + args = ctx.actions.args() + args.add(prelude) + args.add(zip_file) + args.add(output) + actions_run( + ctx, + executable = ctx.attr._exe_zip_maker, + arguments = [args], + inputs = depset([prelude, zip_file]), + outputs = [output], + mnemonic = "PyBuildExecutableZip", + progress_message = "Build Python zip executable: %{label}", + ) + +def _get_cc_details_for_binary(ctx, extra_deps): + cc_info = collect_cc_info(ctx, extra_deps = extra_deps) + return create_cc_details_struct( + cc_info_for_propagating = cc_info, + cc_info_for_self_link = cc_info, + cc_info_with_extra_link_time_libraries = None, + extra_runfiles = ctx.runfiles(), + # Though the rules require the CcToolchain, it isn't actually used. + cc_toolchain = None, + feature_config = None, + ) + +def _get_interpreter_path(ctx, *, runtime, flag_interpreter_path): + if runtime: + if runtime.interpreter_path: + interpreter_path = runtime.interpreter_path + else: + interpreter_path = "{}/{}".format( + ctx.workspace_name, + runtime.interpreter.short_path, + ) + + # NOTE: External runfiles (artifacts in other repos) will have a + # leading path component of "../" so that they refer outside the + # main workspace directory and into the runfiles root. By + # normalizing, we simplify e.g. "workspace/../foo/bar" to simply + # "foo/bar" + interpreter_path = paths.normalize(interpreter_path) + + elif flag_interpreter_path: + interpreter_path = flag_interpreter_path + else: + fail("Unable to determine interpreter path") + + return interpreter_path + +def _get_native_deps_dso_name(ctx): + _ = ctx # @unused + fail("Building native deps DSO not supported.") + def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = []): """Base rule implementation for a Python executable. @@ -150,9 +1173,17 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = """ _validate_executable(ctx) - main_py = determine_main(ctx) + if not ctx.attr.main_module: + main_py = determine_main(ctx) + else: + main_py = None + + # Keep a reference to the main source file (before it may be replaced with a + # precompiled pyc below) so the test-main validation can statically analyze + # the original source. + main_py_source = main_py direct_sources = filter_to_py_srcs(ctx.files.srcs) - precompile_result = semantics.maybe_precompile(ctx, direct_sources) + precompile_result = maybe_precompile(ctx, direct_sources) required_py_files = precompile_result.keep_srcs required_pyc_files = [] @@ -176,20 +1207,16 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = default_outputs.add(precompile_result.keep_srcs) default_outputs.add(required_pyc_files) - imports = collect_imports(ctx, semantics) - - runtime_details = _get_runtime_details(ctx, semantics) - if ctx.configuration.coverage_enabled: - extra_deps = semantics.get_coverage_deps(ctx, runtime_details) - else: - extra_deps = [] + extra_deps = [] # The debugger dependency should be prevented by select() config elsewhere, # but just to be safe, also guard against adding it to the output here. if not _is_tool_config(ctx): - extra_deps.extend(semantics.get_debugger_deps(ctx, runtime_details)) + extra_deps.append(ctx.attr._debugger_flag) - cc_details = semantics.get_cc_details_for_binary(ctx, extra_deps = extra_deps) + imports = collect_imports(ctx, extra_deps = extra_deps) + runtime_details = _get_runtime_details(ctx) + cc_details = _get_cc_details_for_binary(ctx, extra_deps = extra_deps) native_deps_details = _get_native_deps_details( ctx, semantics = semantics, @@ -204,15 +1231,13 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = required_pyc_files = required_pyc_files, implicit_pyc_files = implicit_pyc_files, implicit_pyc_source_files = implicit_pyc_source_files, + runtime_runfiles = runtime_details.runfiles, extra_common_runfiles = [ - runtime_details.runfiles, cc_details.extra_runfiles, native_deps_details.runfiles, - semantics.get_extra_common_runfiles_for_binary(ctx), ], - semantics = semantics, ) - exec_result = semantics.create_executable( + exec_result = _create_executable( ctx, executable = executable, main_py = main_py, @@ -222,11 +1247,12 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = cc_details = cc_details, native_deps_details = native_deps_details, runfiles_details = runfiles_details, + extra_deps = extra_deps, ) - default_outputs.add(exec_result.extra_files_to_build) + default_outputs.add(exec_result.extra_default_outputs) extra_exec_runfiles = exec_result.extra_runfiles.merge( - ctx.runfiles(transitive_files = exec_result.extra_files_to_build), + ctx.runfiles(transitive_files = exec_result.extra_default_outputs), ) # Copy any existing fields in case of company patches. @@ -237,23 +1263,120 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = ) )) - return _create_providers( - ctx = ctx, + providers = [] + + _add_provider_default_info( + providers, + ctx, executable = executable, + default_outputs = default_outputs.build(), runfiles_details = runfiles_details, + ) + _add_provider_instrumented_files_info(providers, ctx) + _add_provider_run_environment_info(providers, ctx, inherited_environment) + _add_provider_py_executable_info( + providers, + app_runfiles = exec_result.app_runfiles, + build_data_file = runfiles_details.build_data_file, + interpreter_args = ctx.attr.interpreter_args, + interpreter_path = runtime_details.executable_interpreter_path, main_py = main_py, - imports = imports, + runfiles_without_exe = runfiles_details.runfiles_without_exe, + stage2_bootstrap = exec_result.stage2_bootstrap, + venv_app_symlinks = exec_result.venv_app_symlinks, + venv_interpreter_runfiles = exec_result.venv_interpreter_runfiles, + venv_interpreter_symlinks = exec_result.venv_interpreter_symlinks, + venv_python_exe = exec_result.venv_python_exe, + ) + _add_provider_py_runtime_info(providers, runtime_details) + _add_provider_py_cc_link_params_info(providers, cc_details.cc_info_for_propagating) + py_info = _add_provider_py_info( + providers, + ctx = ctx, + original_sources = direct_sources, required_py_files = required_py_files, required_pyc_files = required_pyc_files, implicit_pyc_files = implicit_pyc_files, implicit_pyc_source_files = implicit_pyc_source_files, - default_outputs = default_outputs.build(), - runtime_details = runtime_details, - cc_info = cc_details.cc_info_for_propagating, - inherited_environment = inherited_environment, - semantics = semantics, - output_groups = exec_result.output_groups, + imports = imports, + ) + output_groups = dict(exec_result.output_groups) + if is_test: + _maybe_add_test_main_validation(ctx, main_py_source, output_groups) + _add_provider_output_group_info(providers, py_info, output_groups) + + return providers + +def _maybe_add_test_main_validation(ctx, main_py, output_groups): + """Adds a validation action that checks the test main actually runs tests. + + This is a safeguard against the common pitfall of defining test classes or + functions but forgetting to invoke a test runner, which causes the test to + silently pass without running anything. See the + `//python/config_settings:validate_test_main` flag. + + Args: + ctx: Rule ctx. + main_py: File or None; the main entry point source file. None when the + target uses `main_module` (which can't be statically analyzed here). + output_groups: dict[str, depset[File]]; mutated in place to add the + `_validation` output group when a validation action is created. + """ + if not ValidateTestMainFlag.is_enabled(ctx): + return + + # `main_module` targets execute a module by name; there's no single source + # file to statically analyze, so the check doesn't apply. + if main_py == None: + return + + exec_tools_toolchain = ctx.toolchains[EXEC_TOOLS_TOOLCHAIN_TYPE] + if exec_tools_toolchain == None or exec_tools_toolchain.exec_tools.exec_interpreter == None: + fail( + "Validating py_test main modules requires the exec tools toolchain " + + "with an exec interpreter, but none was found. Either register one " + + "or set --@rules_python//python/config_settings:validate_test_main=disabled.", + ) + + exec_tools = exec_tools_toolchain.exec_tools + validator = ctx.attr._validate_test_main + program_info = validator[PyInterpreterProgramInfo] + interpreter = exec_tools.exec_interpreter[DefaultInfo].files_to_run + validator_files_to_run = validator[DefaultInfo].files_to_run + + validation_output = ctx.actions.declare_file(ctx.label.name + "_validate_test_main.txt") + + args = ctx.actions.args() + args.add_all(program_info.interpreter_args) + args.add(validator_files_to_run.executable) + args.add("--src", main_py) + args.add("--src_name", main_py.short_path) + args.add("--label", str(ctx.label)) + args.add("--output", validation_output) + + execution_requirements = {} + if testing.ExecutionInfo in validator: + execution_requirements = validator[testing.ExecutionInfo].requirements + + ctx.actions.run( + executable = interpreter, + arguments = [args], + inputs = [main_py], + outputs = [validation_output], + tools = [validator_files_to_run], + mnemonic = "PyValidateTestMain", + progress_message = "Validating py_test main %{label}", + env = program_info.env | { + "PYTHONNOUSERSITE": "1", + "PYTHONSAFEPATH": "1", + }, + execution_requirements = execution_requirements, + toolchain = EXEC_TOOLS_TOOLCHAIN_TYPE, ) + if "_validation" in output_groups: + output_groups["_validation"] = depset([validation_output], transitive = [output_groups["_validation"]]) + else: + output_groups["_validation"] = depset([validation_output]) def _get_build_info(ctx, cc_toolchain): build_info_files = py_internal.cc_toolchain_build_info_files(cc_toolchain) @@ -266,18 +1389,24 @@ def _get_build_info(ctx, cc_toolchain): return build_info_files.redacted_build_info_files.to_list() def _validate_executable(ctx): - if ctx.attr.python_version != "PY3": + if ctx.attr.python_version == "PY2": fail("It is not allowed to use Python 2") + if ctx.attr.main and ctx.attr.main_module: + fail(( + "Only one of main and main_module can be set, got: " + + "main={}, main_module={}" + ).format(ctx.attr.main, ctx.attr.main_module)) + def _declare_executable_file(ctx): - if target_platform_has_any_constraint(ctx, ctx.attr._windows_constraints): + if is_windows_platform(ctx): executable = ctx.actions.declare_file(ctx.label.name + ".exe") else: executable = ctx.actions.declare_file(ctx.label.name) return executable -def _get_runtime_details(ctx, semantics): +def _get_runtime_details(ctx): """Gets various information about the Python runtime to use. While most information comes from the toolchain, various legacy and @@ -285,7 +1414,6 @@ def _get_runtime_details(ctx, semantics): Args: ctx: Rule ctx - semantics: A `BinarySemantics` struct; see `create_binary_semantics_struct` Returns: A struct; see inline-field comments of the return value for details. @@ -293,26 +1421,13 @@ def _get_runtime_details(ctx, semantics): # Bazel has --python_path. This flag has a computed default of "python" when # its actual default is null (see - # BazelPythonConfiguration.java#getPythonPath). This flag is only used if - # toolchains are not enabled and `--python_top` isn't set. Note that Google - # used to have a variant of this named --python_binary, but it has since - # been removed. - # # TOOD(bazelbuild/bazel#7901): Remove this once --python_path flag is removed. - if IS_BAZEL: - flag_interpreter_path = ctx.fragments.bazel_py.python_path - toolchain_runtime, effective_runtime = _maybe_get_runtime_from_ctx(ctx) - if not effective_runtime: - # Clear these just in case - toolchain_runtime = None - effective_runtime = None + flag_interpreter_path = read_possibly_native_flag(ctx, "python_path") + if not flag_interpreter_path.startswith("python") and not paths.is_absolute(flag_interpreter_path): + fail("'python_path' must be an absolute path or a name to be resolved from the system PATH (e.g., 'python', 'python3').") - else: # Google code path - flag_interpreter_path = None - toolchain_runtime, effective_runtime = _maybe_get_runtime_from_ctx(ctx) - if not effective_runtime: - fail("Unable to find Python runtime") + effective_runtime = _maybe_get_runtime_from_ctx(ctx) if effective_runtime: direct = [] # List of files @@ -330,23 +1445,16 @@ def _get_runtime_details(ctx, semantics): else: runtime_files = depset() - executable_interpreter_path = semantics.get_interpreter_path( + executable_interpreter_path = _get_interpreter_path( ctx, runtime = effective_runtime, flag_interpreter_path = flag_interpreter_path, ) return struct( - # Optional PyRuntimeInfo: The runtime found from toolchain resolution. - # This may be None because, within Google, toolchain resolution isn't - # yet enabled. - toolchain_runtime = toolchain_runtime, - # Optional PyRuntimeInfo: The runtime that should be used. When - # toolchain resolution is enabled, this is the same as - # `toolchain_resolution`. Otherwise, this probably came from the - # `_python_top` attribute that the Google implementation still uses. - # This is separate from `toolchain_runtime` because toolchain_runtime - # is propagated as a provider, while non-toolchain runtimes are not. + # Optional PyRuntimeInfo: The runtime that should be used. + # If None, it's probably Windows using the legacy auto-detecting toolchain + # that acts as if no toolchain was found. effective_runtime = effective_runtime, # str; Path to the Python interpreter to use for running the executable # itself (not the bootstrap script). Either an absolute path (which @@ -363,43 +1471,30 @@ def _maybe_get_runtime_from_ctx(ctx): """Finds the PyRuntimeInfo from the toolchain or attribute, if available. Returns: - 2-tuple of toolchain_runtime, effective_runtime + A PyRuntimeInfo provider, or None. """ - if ctx.fragments.py.use_toolchains: - toolchain = ctx.toolchains[TOOLCHAIN_TYPE] - - if not hasattr(toolchain, "py3_runtime"): - fail("Python toolchain field 'py3_runtime' is missing") - if not toolchain.py3_runtime: - fail("Python toolchain missing py3_runtime") - py3_runtime = toolchain.py3_runtime - - # Hack around the fact that the autodetecting Python toolchain, which is - # automatically registered, does not yet support Windows. In this case, - # we want to return null so that _get_interpreter_path falls back on - # --python_path. See tools/python/toolchain.bzl. - # TODO(#7844): Remove this hack when the autodetecting toolchain has a - # Windows implementation. - if py3_runtime.interpreter_path == "/_magic_pyruntime_sentinel_do_not_use": - return None, None - - if py3_runtime.python_version != "PY3": - fail("Python toolchain py3_runtime must be python_version=PY3, got {}".format( - py3_runtime.python_version, - )) - toolchain_runtime = toolchain.py3_runtime - effective_runtime = toolchain_runtime - else: - toolchain_runtime = None - attr_target = getattr(ctx.attr, PY_RUNTIME_ATTR_NAME) - - # In Bazel, --python_top is null by default. - if attr_target and PyRuntimeInfo in attr_target: - effective_runtime = attr_target[PyRuntimeInfo] - else: - return None, None - - return toolchain_runtime, effective_runtime + toolchain = ctx.toolchains[TOOLCHAIN_TYPE] + + if not hasattr(toolchain, "py3_runtime"): + fail("Python toolchain field 'py3_runtime' is missing") + if not toolchain.py3_runtime: + fail("Python toolchain missing py3_runtime") + py3_runtime = toolchain.py3_runtime + + # Hack around the fact that the autodetecting Python toolchain, which is + # automatically registered, does not yet support Windows. In this case, + # we want to return null so that _get_interpreter_path falls back on + # --python_path. See tools/python/toolchain.bzl. + # TODO(#7844): Remove this hack when the autodetecting toolchain has a + # Windows implementation. + if py3_runtime.interpreter_path == "/_magic_pyruntime_sentinel_do_not_use": + return None + + if py3_runtime.python_version != "PY3": + fail("Python toolchain py3_runtime must be python_version=PY3, got {}".format( + py3_runtime.python_version, + )) + return py3_runtime def _get_base_runfiles_for_binary( ctx, @@ -410,8 +1505,8 @@ def _get_base_runfiles_for_binary( required_pyc_files, implicit_pyc_files, implicit_pyc_source_files, - extra_common_runfiles, - semantics): + runtime_runfiles, + extra_common_runfiles): """Returns the set of runfiles necessary prior to executable creation. NOTE: The term "common runfiles" refers to the runfiles that are common to @@ -431,9 +1526,9 @@ def _get_base_runfiles_for_binary( collection is enabled. implicit_pyc_source_files: `depset[File]` source files for implicit pyc files that are used when the implicit pyc files are not. + runtime_runfiles: runfiles for the python runtime. extra_common_runfiles: List of runfiles; additional runfiles that will be added to the common runfiles. - semantics: A `BinarySemantics` struct; see `create_binary_semantics_struct`. Returns: struct with attributes: @@ -441,102 +1536,103 @@ def _get_base_runfiles_for_binary( * data_runfiles: The data runfiles * runfiles_without_exe: The default runfiles, but without the executable or files specific to the original program/executable. - * build_data_file: A file with build stamp information if stamping is enabled, otherwise - None. + * build_data_file: A file with build stamp information if stamping is + enabled, otherwise None. + * app_runfiles: Runfiles for user-space dependencies (doesn't + include the runtime or build data files) """ - common_runfiles = builders.RunfilesBuilder() - common_runfiles.files.add(required_py_files) - common_runfiles.files.add(required_pyc_files) + app_runfiles = builders.RunfilesBuilder() + app_runfiles.files.add(required_py_files) + app_runfiles.files.add(required_pyc_files) pyc_collection_enabled = PycCollectionAttr.is_pyc_collection_enabled(ctx) if pyc_collection_enabled: - common_runfiles.files.add(implicit_pyc_files) + app_runfiles.files.add(implicit_pyc_files) else: - common_runfiles.files.add(implicit_pyc_source_files) + app_runfiles.files.add(implicit_pyc_source_files) for dep in (ctx.attr.deps + extra_deps): if not (PyInfo in dep or (BuiltinPyInfo != None and BuiltinPyInfo in dep)): continue info = dep[PyInfo] if PyInfo in dep else dep[BuiltinPyInfo] - common_runfiles.files.add(info.transitive_sources) + app_runfiles.files.add(info.transitive_sources) # Everything past this won't work with BuiltinPyInfo if not hasattr(info, "transitive_pyc_files"): continue - common_runfiles.files.add(info.transitive_pyc_files) + app_runfiles.files.add(info.transitive_pyc_files) if pyc_collection_enabled: - common_runfiles.files.add(info.transitive_implicit_pyc_files) + app_runfiles.files.add(info.transitive_implicit_pyc_files) else: - common_runfiles.files.add(info.transitive_implicit_pyc_source_files) + app_runfiles.files.add(info.transitive_implicit_pyc_source_files) - common_runfiles.runfiles.append(collect_runfiles(ctx)) + app_runfiles.runfiles.append(collect_runfiles(ctx)) if extra_deps: - common_runfiles.add_targets(extra_deps) - common_runfiles.add(extra_common_runfiles) - - common_runfiles = common_runfiles.build(ctx) - - if semantics.should_create_init_files(ctx): - common_runfiles = _py_builtins.merge_runfiles_with_generated_inits_empty_files_supplier( + app_runfiles.add_targets(extra_deps) + app_runfiles.add(extra_common_runfiles) + + app_runfiles = app_runfiles.build(ctx) + + if _should_create_init_files(ctx): + # buildifier: disable=print + print( + """ +====================================================================== +WARNING: Target {} is using implicit __init__.py creation. + This diabolic behavior is deprecated and will be disabled by default in a + future release. + See https://github.com/bazel-contrib/rules_python/issues/2945 + + Ensure all __init__.py files are explicitly created and + added to the srcs or deps of your targets. + + Disable implicit creation by setting: + legacy_create_init = 0 + on the target, or globally by setting: + --incompatible_default_to_explicit_init_py +====================================================================== + """.rstrip().format(ctx.label), + ) + app_runfiles = _py_builtins.merge_runfiles_with_generated_inits_empty_files_supplier( ctx = ctx, - runfiles = common_runfiles, + runfiles = app_runfiles, ) - # Don't include build_data.txt in the non-exe runfiles. The build data - # may contain program-specific content (e.g. target name). - runfiles_with_exe = common_runfiles.merge(ctx.runfiles([executable])) + runfiles_without_exe = builders.RunfilesBuilder() + runfiles_without_exe.add(app_runfiles) + runfiles_without_exe.add(runtime_runfiles) + build_data_file = _write_build_data(ctx) + runfiles_without_exe.add(build_data_file) - # Don't include build_data.txt in data runfiles. This allows binaries to - # contain other binaries while still using the same fixed location symlink - # for the build_data.txt file. Really, the fixed location symlink should be - # removed and another way found to locate the underlying build data file. - data_runfiles = runfiles_with_exe + runfiles_without_exe = runfiles_without_exe.build(ctx) - if is_stamping_enabled(ctx, semantics) and semantics.should_include_build_data(ctx): - build_data_file, build_data_runfiles = _create_runfiles_with_build_data( - ctx, - semantics.get_central_uncachable_version_file(ctx), - semantics.get_extra_write_build_data_env(ctx), - ) - default_runfiles = runfiles_with_exe.merge(build_data_runfiles) - else: - build_data_file = None - default_runfiles = runfiles_with_exe + runfiles_with_exe = runfiles_without_exe.merge(ctx.runfiles([executable])) + # There are three types of runfiles: + # 1. app: Deps added by a user. This is akin to the typical files that would + # be in a traditional venv. No Python runtime files or build data files. + # 2. without-exe: (1) + build data + python runtime + # 3. binary (default/data runfiles): (2) + main executable return struct( - runfiles_without_exe = common_runfiles, - default_runfiles = default_runfiles, + app_runfiles = app_runfiles, build_data_file = build_data_file, - data_runfiles = data_runfiles, + data_runfiles = runfiles_with_exe, + default_runfiles = runfiles_with_exe, + runfiles_without_exe = runfiles_without_exe, ) -def _create_runfiles_with_build_data( - ctx, - central_uncachable_version_file, - extra_write_build_data_env): - build_data_file = _write_build_data( - ctx, - central_uncachable_version_file, - extra_write_build_data_env, - ) - build_data_runfiles = ctx.runfiles(symlinks = { - BUILD_DATA_SYMLINK_PATH: build_data_file, - }) - return build_data_file, build_data_runfiles - -def _write_build_data(ctx, central_uncachable_version_file, extra_write_build_data_env): - # TODO: Remove this logic when a central file is always available - if not central_uncachable_version_file: - version_file = ctx.actions.declare_file(ctx.label.name + "-uncachable_version_file.txt") - _py_builtins.copy_without_caching( - ctx = ctx, - read_from = ctx.version_file, - write_to = version_file, - ) +def _write_build_data(ctx): + inputs = builders.DepsetBuilder() + if is_stamping_enabled(ctx): + # NOTE: ctx.info_file is undocumented; see + # https://github.com/bazelbuild/bazel/issues/9363 + info_file = ctx.info_file + version_file = ctx.files._uncachable_version_file[0] + inputs.add(info_file) + inputs.add(version_file) else: - version_file = central_uncachable_version_file - - direct_inputs = [ctx.info_file, version_file] + info_file = None + version_file = None # A "constant metadata" file is basically a special file that doesn't # support change detection logic and reports that it is unchanged. i.e., it @@ -568,23 +1664,41 @@ def _write_build_data(ctx, central_uncachable_version_file, extra_write_build_da root = ctx.bin_dir, ) + action_args = ctx.actions.args() + writer_file = ctx.files._build_data_writer[0] + if writer_file.path.endswith(".ps1"): + # powershell.exe is used for broader compatibility + # It is installed by default on most Windows versions + action_exe = "powershell.exe" + action_args.add_all([ + # Bypass execution policy is needed because, + # by default, Windows blocks ps1 scripts. + "-ExecutionPolicy", + "Bypass", + "-File", + writer_file, + ]) + inputs.add(writer_file) + else: + action_exe = ctx.attr._build_data_writer[DefaultInfo].files_to_run + ctx.actions.run( - executable = ctx.executable._build_data_gen, - env = dicts.add({ - # NOTE: ctx.info_file is undocumented; see - # https://github.com/bazelbuild/bazel/issues/9363 - "INFO_FILE": ctx.info_file.path, + executable = action_exe, + arguments = [action_args], + env = { + "INFO_FILE": info_file.path if info_file else "", "OUTPUT": build_data.path, - "PLATFORM": cc_helper.find_cpp_toolchain(ctx).toolchain_id, + # Include this so it's explicit, otherwise, one has to detect + # this by looking for the absense of info_file keys. + "STAMPED": "TRUE" if is_stamping_enabled(ctx) else "FALSE", "TARGET": str(ctx.label), - "VERSION_FILE": version_file.path, - }, extra_write_build_data_env), - inputs = depset( - direct = direct_inputs, - ), + "VERSION_FILE": version_file.path if version_file else "", + }, + inputs = inputs.build(), outputs = [build_data], mnemonic = "PyWriteBuildData", - progress_message = "Generating %{label} build_data.txt", + progress_message = "Reticulating %{label} build data", + toolchain = None, ) return build_data @@ -629,12 +1743,9 @@ def _get_native_deps_details(ctx, *, semantics, cc_details, is_test): feature_configuration = cc_feature_config.feature_configuration, cc_toolchain = cc_details.cc_toolchain, test_only_target = is_test, # private - stamp = 1 if is_stamping_enabled(ctx, semantics) else 0, + stamp = 1 if is_stamping_enabled(ctx) else 0, main_output = linked_lib, # private use_shareable_artifact_factory = True, # private - # NOTE: Only flags not captured by cc_info.linking_context need to - # be manually passed - user_link_flags = semantics.get_native_deps_user_link_flags(ctx), ) return struct( dso = dso, @@ -649,7 +1760,11 @@ def _create_shared_native_deps_dso( feature_configuration, requested_features, cc_toolchain): - linkstamps = py_internal.linking_context_linkstamps(cc_info.linking_context) + linkstamps = [ + py_internal.linkstamp_file(linkstamp) + for linker_input in cc_info.linking_context.linker_inputs.to_list() + for linkstamp in linker_input.linkstamps + ] partially_disabled_thin_lto = ( cc_common.is_enabled( @@ -673,10 +1788,7 @@ def _create_shared_native_deps_dso( for input in cc_info.linking_context.linker_inputs.to_list() for flag in input.user_link_flags ], - linkstamps = [ - py_internal.linkstamp_file(linkstamp) - for linkstamp in linkstamps.to_list() - ], + linkstamps = linkstamps, build_info_artifacts = _get_build_info(ctx, cc_toolchain) if linkstamps else [], features = requested_features, is_test_target_partially_disabled_thin_lto = is_test and partially_disabled_thin_lto, @@ -736,7 +1848,7 @@ def determine_main(ctx): """ if ctx.attr.main: proposed_main = ctx.attr.main.label.name - if not proposed_main.endswith(tuple(ALLOWED_MAIN_EXTENSIONS)): + if not proposed_main.endswith(".py"): fail("main must end in '.py'") else: if ctx.label.name.endswith(".py"): @@ -773,15 +1885,17 @@ def _path_endswith(path, endswith): # "ab/c.py".endswith("b/c.py") from incorrectly matching. return ("/" + path).endswith("/" + endswith) -def is_stamping_enabled(ctx, semantics): +def is_stamping_enabled(ctx): """Tells if stamping is enabled or not. Args: ctx: The rule ctx - semantics: a semantics struct (see create_semantics_struct). Returns: bool; True if stamping is enabled, False if not. """ + + # Always ignore stamping for exec config. This mitigates stamping + # invalidating build action caching. if _is_tool_config(ctx): return False @@ -791,7 +1905,9 @@ def is_stamping_enabled(ctx, semantics): elif stamp == 0: return False elif stamp == -1: - return semantics.get_stamp_flag(ctx) + # NOTE: ctx.configuration.stamp_binaries() exposes this, but that's + # a private API. To workaround, it'd been eposed via py_internal. + return py_internal.stamp_binaries(ctx) else: fail("Unsupported `stamp` value: {}".format(stamp)) @@ -801,81 +1917,120 @@ def _is_tool_config(ctx): # a more public API. Until that's available, py_internal to the rescue. return py_internal.is_tool_configuration(ctx) -def _create_providers( - *, - ctx, - executable, - main_py, - required_py_files, - required_pyc_files, - implicit_pyc_files, - implicit_pyc_source_files, - default_outputs, - runfiles_details, - imports, - cc_info, - inherited_environment, - runtime_details, - output_groups, - semantics): - """Creates the providers an executable should return. +def _add_provider_default_info(providers, ctx, *, executable, default_outputs, runfiles_details): + """Adds the DefaultInfo provider. Args: + providers: list of providers to append to. ctx: The rule ctx. executable: File; the target's executable file. - main_py: File; the main .py entry point. - required_py_files: `depset[File]` the direct, `.py` sources for the - target that **must** be included by downstream targets. This should - only be Python source files. It should not include pyc files. - required_pyc_files: `depset[File]` the direct `.pyc` files this target - produces. - implicit_pyc_files: `depset[File]` pyc files that are only used if pyc - collection is enabled. - implicit_pyc_source_files: `depset[File]` source files for implicit pyc - files that are used when the implicit pyc files are not. default_outputs: depset of Files; the files for DefaultInfo.files - runfiles_details: runfiles that will become the default and data runfiles. - imports: depset of strings; the import paths to propagate - cc_info: optional CcInfo; Linking information to propagate as - PyCcLinkParamsInfo. Note that only the linking information - is propagated, not the whole CcInfo. + runfiles_details: runfiles that will become the default and data runfiles. + """ + providers.append(DefaultInfo( + executable = executable, + files = default_outputs, + default_runfiles = _py_builtins.make_runfiles_respect_legacy_external_runfiles( + ctx, + runfiles_details.default_runfiles, + ), + data_runfiles = _py_builtins.make_runfiles_respect_legacy_external_runfiles( + ctx, + runfiles_details.data_runfiles, + ), + )) + +def _add_provider_instrumented_files_info(providers, ctx): + """Adds the InstrumentedFilesInfo provider. + + Args: + providers: list of providers to append to. + ctx: The rule ctx. + """ + providers.append(create_instrumented_files_info(ctx)) + +def _add_provider_run_environment_info(providers, ctx, inherited_environment): + """Adds the RunEnvironmentInfo provider. + + Args: + providers: list of providers to append to. + ctx: The rule ctx. inherited_environment: list of strings; Environment variable names that should be inherited from the environment the executuble is run within. - runtime_details: struct of runtime information; see _get_runtime_details() - output_groups: dict[str, depset[File]]; used to create OutputGroupInfo - semantics: BinarySemantics struct; see create_binary_semantics() + """ + expanded_env = {} + for key, value in ctx.attr.env.items(): + expanded_env[key] = _py_builtins.expand_location_and_make_variables( + ctx = ctx, + attribute_name = "env[{}]".format(key), + expression = value, + targets = ctx.attr.data, + ) + if "PYTHONBREAKPOINT" not in inherited_environment: + inherited_environment = inherited_environment + ["PYTHONBREAKPOINT"] + providers.append(RunEnvironmentInfo( + environment = expanded_env, + inherited_environment = inherited_environment, + )) - Returns: - A list of modern providers. +def _add_provider_py_executable_info( + providers, + *, + app_runfiles, + build_data_file, + interpreter_args, + interpreter_path, + main_py, + runfiles_without_exe, + stage2_bootstrap, + venv_app_symlinks, + venv_interpreter_runfiles, + venv_interpreter_symlinks, + venv_python_exe): + """Adds the PyExecutableInfo provider. + + Args: + providers: list of providers to append to. + app_runfiles: runfiles; the runfiles for the application (deps, etc). + build_data_file: File; a file with build stamp information. + interpreter_args: list of strings; arguments to pass to the interpreter. + interpreter_path: str; path to the Python interpreter. + main_py: File; the main .py entry point. + runfiles_without_exe: runfiles; the default runfiles, but without the executable. + stage2_bootstrap: File; the stage 2 bootstrap script. + venv_app_symlinks: depset[ExplicitSymlink]; symlinks to create for the + venv that are the application (deps, etc). + venv_interpreter_runfiles: runfiles; runfiles specific to the interpreter for the venv. + venv_interpreter_symlinks: depset[ExplicitSymlink]; interpreter-specific symlinks to create for the venv. + venv_python_exe: File; the python executable in the venv. """ - providers = [ - DefaultInfo( - executable = executable, - files = default_outputs, - default_runfiles = _py_builtins.make_runfiles_respect_legacy_external_runfiles( - ctx, - runfiles_details.default_runfiles, - ), - data_runfiles = _py_builtins.make_runfiles_respect_legacy_external_runfiles( - ctx, - runfiles_details.data_runfiles, - ), - ), - create_instrumented_files_info(ctx), - _create_run_environment_info(ctx, inherited_environment), - PyExecutableInfo( - main = main_py, - runfiles_without_exe = runfiles_details.runfiles_without_exe, - build_data_file = runfiles_details.build_data_file, - interpreter_path = runtime_details.executable_interpreter_path, - ), - ] + providers.append(PyExecutableInfo( + app_runfiles = app_runfiles, + build_data_file = build_data_file, + interpreter_args = interpreter_args, + interpreter_path = interpreter_path, + main = main_py, + runfiles_without_exe = runfiles_without_exe, + stage2_bootstrap = stage2_bootstrap, + venv_app_symlinks = venv_app_symlinks, + venv_interpreter_runfiles = venv_interpreter_runfiles, + venv_interpreter_symlinks = venv_interpreter_symlinks, + venv_python_exe = venv_python_exe, + )) + +def _add_provider_py_runtime_info(providers, runtime_details): + """Adds the PyRuntimeInfo provider. - # TODO(b/265840007): Make this non-conditional once Google enables - # --incompatible_use_python_toolchains. - if runtime_details.toolchain_runtime: - py_runtime_info = runtime_details.toolchain_runtime + Args: + providers: list of providers to append to. + runtime_details: struct of runtime information; see _get_runtime_details() + """ + + # TODO - The effective runtime can be None for Windows + auto detecting toolchain. + # This can be removed once that's fixed; see maybe_get_runtime_from_ctx(). + if runtime_details.effective_runtime: + py_runtime_info = runtime_details.effective_runtime providers.append(py_runtime_info) # Re-add the builtin PyRuntimeInfo for compatibility to make @@ -898,6 +2053,16 @@ def _create_providers( bootstrap_template = py_runtime_info.bootstrap_template, )) +def _add_provider_py_cc_link_params_info(providers, cc_info): + """Adds the PyCcLinkParamsInfo provider. + + Args: + providers: list of providers to append to. + cc_info: optional CcInfo; Linking information to propagate as + PyCcLinkParamsInfo. Note that only the linking information + is propagated, not the whole CcInfo. + """ + # TODO(b/163083591): Remove the PyCcLinkParamsInfo once binaries-in-deps # are cleaned up. if cc_info: @@ -905,76 +2070,160 @@ def _create_providers( PyCcLinkParamsInfo(cc_info = cc_info), ) - py_info, deps_transitive_sources, builtin_py_info = create_py_info( +def _add_provider_py_info( + providers, + *, + ctx, + original_sources, + required_py_files, + required_pyc_files, + implicit_pyc_files, + implicit_pyc_source_files, + imports): + """Adds the PyInfo provider. + + Args: + providers: list of providers to append to. + ctx: The rule ctx. + original_sources: `depset[File]` the direct `.py` sources for the + target that were the original input sources. + required_py_files: `depset[File]` the direct, `.py` sources for the + target that **must** be included by downstream targets. This should + only be Python source files. It should not include pyc files. + required_pyc_files: `depset[File]` the direct `.pyc` files this target + produces. + implicit_pyc_files: `depset[File]` pyc files that are only used if pyc + collection is enabled. + implicit_pyc_source_files: `depset[File]` source files for implicit pyc + files that are used when the implicit pyc files are not. + imports: depset of strings; the import paths to propagate + + Returns: + PyInfo. + """ + py_info, builtin_py_info = create_py_info( ctx, + original_sources = original_sources, required_py_files = required_py_files, required_pyc_files = required_pyc_files, implicit_pyc_files = implicit_pyc_files, implicit_pyc_source_files = implicit_pyc_source_files, imports = imports, ) - - # TODO(b/253059598): Remove support for extra actions; https://github.com/bazelbuild/bazel/issues/16455 - listeners_enabled = _py_builtins.are_action_listeners_enabled(ctx) - if listeners_enabled: - _py_builtins.add_py_extra_pseudo_action( - ctx = ctx, - dependency_transitive_python_sources = deps_transitive_sources, - ) - providers.append(py_info) if builtin_py_info: providers.append(builtin_py_info) + return py_info + +def _add_provider_output_group_info(providers, py_info, output_groups): + """Adds the OutputGroupInfo provider. + + Args: + providers: list of providers to append to. + py_info: PyInfo; the PyInfo provider. + output_groups: dict[str, depset[File]]; used to create OutputGroupInfo + """ providers.append(create_output_group_info(py_info.transitive_sources, output_groups)) - extra_providers = semantics.get_extra_providers( - ctx, - main_py = main_py, - runtime_details = runtime_details, - ) - providers.extend(extra_providers) - return providers +def _add_config_setting_defaults(kwargs): + config_settings = kwargs.get("config_settings", None) + if config_settings == None: + config_settings = {} + + # NOTE: This code runs in loading phase within the context of the caller. + # Label() must be used to resolve repo names within rules_python's + # context to avoid unknown repo name errors. + default = select({ + labels.PLATFORMS_OS_WINDOWS: { + labels.ENABLE_RUNFILES: "true", + }, + "//conditions:default": {}, + }) -def _create_run_environment_info(ctx, inherited_environment): - expanded_env = {} - for key, value in ctx.attr.env.items(): - expanded_env[key] = _py_builtins.expand_location_and_make_variables( - ctx = ctx, - attribute_name = "env[{}]".format(key), - expression = value, - targets = ctx.attr.data, - ) - return RunEnvironmentInfo( - environment = expanded_env, - inherited_environment = inherited_environment, + # Let user-provided settings have precedence + config_settings = default | config_settings + kwargs["config_settings"] = config_settings + +def common_executable_macro_kwargs_setup(kwargs): + convert_legacy_create_init_to_int(kwargs) + _add_config_setting_defaults(kwargs) + +def _transition_executable_impl(settings, attr): + settings = dict(settings) + apply_config_settings_attr(settings, attr) + + if attr.python_version and attr.python_version not in ("PY2", "PY3"): + settings[labels.PYTHON_VERSION] = attr.python_version + + if attr.stamp != -1: + settings["//command_line_option:stamp"] = str(attr.stamp) + + return settings + +def create_executable_rule(*, attrs, **kwargs): + return create_base_executable_rule( + attrs = attrs, + fragments = ["py", "bazel_py"], + **kwargs ) -def create_base_executable_rule(*, attrs, fragments = [], **kwargs): +def create_base_executable_rule(): """Create a function for defining for Python binary/test targets. - Args: - attrs: Rule attributes - fragments: List of str; extra config fragments that are required. - **kwargs: Additional args to pass onto `rule()` - Returns: A rule function """ - if "py" not in fragments: - # The list might be frozen, so use concatentation - fragments = fragments + ["py"] - kwargs.setdefault("provides", []).append(PyExecutableInfo) - kwargs["exec_groups"] = REQUIRED_EXEC_GROUPS | (kwargs.get("exec_groups") or {}) - return rule( - # TODO: add ability to remove attrs, i.e. for imports attr - attrs = dicts.add(EXECUTABLE_ATTRS, attrs), + return create_executable_rule_builder().build() + +_MaybeBuiltinPyInfo = [BuiltinPyInfo] if BuiltinPyInfo != None else [] + +# NOTE: Exported publicly +def create_executable_rule_builder(implementation, **kwargs): + """Create a rule builder for an executable Python program. + + :::{include} /_includes/volatile_api.md + ::: + + An executable rule is one that sets either `executable=True` or `test=True`, + and the output is something that can be run directly (e.g. `bazel run`, + `exec(...)` etc) + + :::{versionadded} 1.3.0 + ::: + + Returns: + {obj}`ruleb.Rule` with the necessary settings + for creating an executable Python rule. + """ + builder = ruleb.Rule( + implementation = implementation, + attrs = EXECUTABLE_ATTRS | (COVERAGE_ATTRS if kwargs.get("test") else {}), + exec_groups = dict(REQUIRED_EXEC_GROUP_BUILDERS), # Mutable copy + fragments = ["py", "bazel_py"], + provides = [PyExecutableInfo, PyInfo] + _MaybeBuiltinPyInfo, toolchains = [ - TOOLCHAIN_TYPE, - config_common.toolchain_type(EXEC_TOOLS_TOOLCHAIN_TYPE, mandatory = False), - ] + _CC_TOOLCHAINS, - fragments = fragments, + ruleb.ToolchainType(TOOLCHAIN_TYPE), + ruleb.ToolchainType(EXEC_TOOLS_TOOLCHAIN_TYPE, mandatory = False), + ruleb.ToolchainType(CC_TOOLCHAIN_TYPE, mandatory = False), + ] + ([ruleb.ToolchainType(LAUNCHER_MAKER_TOOLCHAIN_TYPE)] if rp_config.bazel_9_or_later else []), + cfg = dict( + implementation = _transition_executable_impl, + inputs = TRANSITION_LABELS + [ + labels.PYTHON_VERSION, + "//command_line_option:stamp", + "//command_line_option:build_runfile_links", + "//command_line_option:enable_runfiles", + ], + outputs = TRANSITION_LABELS + [ + labels.PYTHON_VERSION, + "//command_line_option:stamp", + "//command_line_option:build_runfile_links", + "//command_line_option:enable_runfiles", + ], + ), **kwargs ) + return builder def cc_configure_features( ctx, @@ -1011,7 +2260,3 @@ def cc_configure_features( feature_configuration = feature_configuration, requested_features = requested_features, ) - -only_exposed_for_google_internal_reason = struct( - create_runfiles_with_build_data = _create_runfiles_with_build_data, -) diff --git a/python/private/py_executable_bazel.bzl b/python/private/py_executable_bazel.bzl deleted file mode 100644 index 3778c192b4..0000000000 --- a/python/private/py_executable_bazel.bzl +++ /dev/null @@ -1,772 +0,0 @@ -# Copyright 2022 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -"""Implementation for Bazel Python executable.""" - -load("@bazel_skylib//lib:dicts.bzl", "dicts") -load("@bazel_skylib//lib:paths.bzl", "paths") -load(":attributes.bzl", "IMPORTS_ATTRS") -load( - ":common.bzl", - "create_binary_semantics_struct", - "create_cc_details_struct", - "create_executable_result_struct", - "target_platform_has_any_constraint", - "union_attrs", -) -load(":common_bazel.bzl", "collect_cc_info", "get_imports", "maybe_precompile") -load(":flags.bzl", "BootstrapImplFlag") -load( - ":py_executable.bzl", - "create_base_executable_rule", - "py_executable_base_impl", -) -load(":py_internal.bzl", "py_internal") -load(":py_runtime_info.bzl", "DEFAULT_STUB_SHEBANG") -load(":toolchain_types.bzl", "TARGET_TOOLCHAIN_TYPE") - -_py_builtins = py_internal -_EXTERNAL_PATH_PREFIX = "external" -_ZIP_RUNFILES_DIRECTORY_NAME = "runfiles" - -BAZEL_EXECUTABLE_ATTRS = union_attrs( - IMPORTS_ATTRS, - { - "legacy_create_init": attr.int( - default = -1, - values = [-1, 0, 1], - doc = """\ -Whether to implicitly create empty `__init__.py` files in the runfiles tree. -These are created in every directory containing Python source code or shared -libraries, and every parent directory of those directories, excluding the repo -root directory. The default, `-1` (auto), means true unless -`--incompatible_default_to_explicit_init_py` is used. If false, the user is -responsible for creating (possibly empty) `__init__.py` files and adding them to -the `srcs` of Python targets as required. - """, - ), - "_bootstrap_template": attr.label( - allow_single_file = True, - default = "@bazel_tools//tools/python:python_bootstrap_template.txt", - ), - "_launcher": attr.label( - cfg = "target", - # NOTE: This is an executable, but is only used for Windows. It - # can't have executable=True because the backing target is an - # empty target for other platforms. - default = "//tools/launcher:launcher", - ), - "_py_interpreter": attr.label( - # The configuration_field args are validated when called; - # we use the precense of py_internal to indicate this Bazel - # build has that fragment and name. - default = configuration_field( - fragment = "bazel_py", - name = "python_top", - ) if py_internal else None, - ), - # TODO: This appears to be vestigial. It's only added because - # GraphlessQueryTest.testLabelsOperator relies on it to test for - # query behavior of implicit dependencies. - "_py_toolchain_type": attr.label( - default = TARGET_TOOLCHAIN_TYPE, - ), - "_python_version_flag": attr.label( - default = "//python/config_settings:python_version", - ), - "_windows_launcher_maker": attr.label( - default = "@bazel_tools//tools/launcher:launcher_maker", - cfg = "exec", - executable = True, - ), - "_zipper": attr.label( - cfg = "exec", - executable = True, - default = "@bazel_tools//tools/zip:zipper", - ), - }, -) - -def create_executable_rule(*, attrs, **kwargs): - return create_base_executable_rule( - attrs = dicts.add(BAZEL_EXECUTABLE_ATTRS, attrs), - fragments = ["py", "bazel_py"], - **kwargs - ) - -def py_executable_bazel_impl(ctx, *, is_test, inherited_environment): - """Common code for executables for Bazel.""" - return py_executable_base_impl( - ctx = ctx, - semantics = create_binary_semantics_bazel(), - is_test = is_test, - inherited_environment = inherited_environment, - ) - -def create_binary_semantics_bazel(): - return create_binary_semantics_struct( - # keep-sorted start - create_executable = _create_executable, - get_cc_details_for_binary = _get_cc_details_for_binary, - get_central_uncachable_version_file = lambda ctx: None, - get_coverage_deps = _get_coverage_deps, - get_debugger_deps = _get_debugger_deps, - get_extra_common_runfiles_for_binary = lambda ctx: ctx.runfiles(), - get_extra_providers = _get_extra_providers, - get_extra_write_build_data_env = lambda ctx: {}, - get_imports = get_imports, - get_interpreter_path = _get_interpreter_path, - get_native_deps_dso_name = _get_native_deps_dso_name, - get_native_deps_user_link_flags = _get_native_deps_user_link_flags, - get_stamp_flag = _get_stamp_flag, - maybe_precompile = maybe_precompile, - should_build_native_deps_dso = lambda ctx: False, - should_create_init_files = _should_create_init_files, - should_include_build_data = lambda ctx: False, - # keep-sorted end - ) - -def _get_coverage_deps(ctx, runtime_details): - _ = ctx, runtime_details # @unused - return [] - -def _get_debugger_deps(ctx, runtime_details): - _ = ctx, runtime_details # @unused - return [] - -def _get_extra_providers(ctx, main_py, runtime_details): - _ = ctx, main_py, runtime_details # @unused - return [] - -def _get_stamp_flag(ctx): - # NOTE: Undocumented API; private to builtins - return ctx.configuration.stamp_binaries - -def _should_create_init_files(ctx): - if ctx.attr.legacy_create_init == -1: - return not ctx.fragments.py.default_to_explicit_init_py - else: - return bool(ctx.attr.legacy_create_init) - -def _create_executable( - ctx, - *, - executable, - main_py, - imports, - is_test, - runtime_details, - cc_details, - native_deps_details, - runfiles_details): - _ = is_test, cc_details, native_deps_details # @unused - - is_windows = target_platform_has_any_constraint(ctx, ctx.attr._windows_constraints) - - if is_windows: - if not executable.extension == "exe": - fail("Should not happen: somehow we are generating a non-.exe file on windows") - base_executable_name = executable.basename[0:-4] - else: - base_executable_name = executable.basename - - venv = None - - # The check for stage2_bootstrap_template is to support legacy - # BuiltinPyRuntimeInfo providers, which is likely to come from - # @bazel_tools//tools/python:autodetecting_toolchain, the toolchain used - # for workspace builds when no rules_python toolchain is configured. - if (BootstrapImplFlag.get_value(ctx) == BootstrapImplFlag.SCRIPT and - runtime_details.effective_runtime and - hasattr(runtime_details.effective_runtime, "stage2_bootstrap_template")): - venv = _create_venv( - ctx, - output_prefix = base_executable_name, - imports = imports, - runtime_details = runtime_details, - ) - - stage2_bootstrap = _create_stage2_bootstrap( - ctx, - output_prefix = base_executable_name, - output_sibling = executable, - main_py = main_py, - imports = imports, - runtime_details = runtime_details, - ) - extra_runfiles = ctx.runfiles([stage2_bootstrap] + venv.files_without_interpreter) - zip_main = _create_zip_main( - ctx, - stage2_bootstrap = stage2_bootstrap, - runtime_details = runtime_details, - venv = venv, - ) - else: - stage2_bootstrap = None - extra_runfiles = ctx.runfiles() - zip_main = ctx.actions.declare_file(base_executable_name + ".temp", sibling = executable) - _create_stage1_bootstrap( - ctx, - output = zip_main, - main_py = main_py, - imports = imports, - is_for_zip = True, - runtime_details = runtime_details, - ) - - zip_file = ctx.actions.declare_file(base_executable_name + ".zip", sibling = executable) - _create_zip_file( - ctx, - output = zip_file, - original_nonzip_executable = executable, - zip_main = zip_main, - runfiles = runfiles_details.default_runfiles.merge(extra_runfiles), - ) - - extra_files_to_build = [] - - # NOTE: --build_python_zip defaults to true on Windows - build_zip_enabled = ctx.fragments.py.build_python_zip - - # When --build_python_zip is enabled, then the zip file becomes - # one of the default outputs. - if build_zip_enabled: - extra_files_to_build.append(zip_file) - - # The logic here is a bit convoluted. Essentially, there are 3 types of - # executables produced: - # 1. (non-Windows) A bootstrap template based program. - # 2. (non-Windows) A self-executable zip file of a bootstrap template based program. - # 3. (Windows) A native Windows executable that finds and launches - # the actual underlying Bazel program (one of the above). Note that - # it implicitly assumes one of the above is located next to it, and - # that --build_python_zip defaults to true for Windows. - - should_create_executable_zip = False - bootstrap_output = None - if not is_windows: - if build_zip_enabled: - should_create_executable_zip = True - else: - bootstrap_output = executable - else: - _create_windows_exe_launcher( - ctx, - output = executable, - use_zip_file = build_zip_enabled, - python_binary_path = runtime_details.executable_interpreter_path, - ) - if not build_zip_enabled: - # On Windows, the main executable has an "exe" extension, so - # here we re-use the un-extensioned name for the bootstrap output. - bootstrap_output = ctx.actions.declare_file(base_executable_name) - - # The launcher looks for the non-zip executable next to - # itself, so add it to the default outputs. - extra_files_to_build.append(bootstrap_output) - - if should_create_executable_zip: - if bootstrap_output != None: - fail("Should not occur: bootstrap_output should not be used " + - "when creating an executable zip") - _create_executable_zip_file( - ctx, - output = executable, - zip_file = zip_file, - stage2_bootstrap = stage2_bootstrap, - runtime_details = runtime_details, - venv = venv, - ) - elif bootstrap_output: - _create_stage1_bootstrap( - ctx, - output = bootstrap_output, - stage2_bootstrap = stage2_bootstrap, - runtime_details = runtime_details, - is_for_zip = False, - imports = imports, - main_py = main_py, - venv = venv, - ) - else: - # Otherwise, this should be the Windows case of launcher + zip. - # Double check this just to make sure. - if not is_windows or not build_zip_enabled: - fail(("Should not occur: The non-executable-zip and " + - "non-bootstrap-template case should have windows and zip " + - "both true, but got " + - "is_windows={is_windows} " + - "build_zip_enabled={build_zip_enabled}").format( - is_windows = is_windows, - build_zip_enabled = build_zip_enabled, - )) - - # The interpreter is added this late in the process so that it isn't - # added to the zipped files. - if venv: - extra_runfiles = extra_runfiles.merge(ctx.runfiles([venv.interpreter])) - return create_executable_result_struct( - extra_files_to_build = depset(extra_files_to_build), - output_groups = {"python_zip_file": depset([zip_file])}, - extra_runfiles = extra_runfiles, - ) - -def _create_zip_main(ctx, *, stage2_bootstrap, runtime_details, venv): - python_binary = _runfiles_root_path(ctx, venv.interpreter.short_path) - python_binary_actual = venv.interpreter_actual_path - - # The location of this file doesn't really matter. It's added to - # the zip file as the top-level __main__.py file and not included - # elsewhere. - output = ctx.actions.declare_file(ctx.label.name + "_zip__main__.py") - ctx.actions.expand_template( - template = runtime_details.effective_runtime.zip_main_template, - output = output, - substitutions = { - "%python_binary%": python_binary, - "%python_binary_actual%": python_binary_actual, - "%stage2_bootstrap%": "{}/{}".format( - ctx.workspace_name, - stage2_bootstrap.short_path, - ), - "%workspace_name%": ctx.workspace_name, - }, - ) - return output - -def relative_path(from_, to): - """Compute a relative path from one path to another. - - Args: - from_: {type}`str` the starting directory. Note that it should be - a directory because relative-symlinks are relative to the - directory the symlink resides in. - to: {type}`str` the path that `from_` wants to point to - - Returns: - {type}`str` a relative path - """ - from_parts = from_.split("/") - to_parts = to.split("/") - - # Strip common leading parts from both paths - n = min(len(from_parts), len(to_parts)) - for _ in range(n): - if from_parts[0] == to_parts[0]: - from_parts.pop(0) - to_parts.pop(0) - else: - break - - # Impossible to compute a relative path without knowing what ".." is - if from_parts and from_parts[0] == "..": - fail("cannot compute relative path from '%s' to '%s'", from_, to) - - parts = ([".."] * len(from_parts)) + to_parts - return paths.join(*parts) - -# Create a venv the executable can use. -# For venv details and the venv startup process, see: -# * https://docs.python.org/3/library/venv.html -# * https://snarky.ca/how-virtual-environments-work/ -# * https://github.com/python/cpython/blob/main/Modules/getpath.py -# * https://github.com/python/cpython/blob/main/Lib/site.py -def _create_venv(ctx, output_prefix, imports, runtime_details): - venv = "_{}.venv".format(output_prefix.lstrip("_")) - - # The pyvenv.cfg file must be present to trigger the venv site hooks. - # Because it's paths are expected to be absolute paths, we can't reliably - # put much in it. See https://github.com/python/cpython/issues/83650 - pyvenv_cfg = ctx.actions.declare_file("{}/pyvenv.cfg".format(venv)) - ctx.actions.write(pyvenv_cfg, "") - - runtime = runtime_details.effective_runtime - if runtime.interpreter: - py_exe_basename = paths.basename(runtime.interpreter.short_path) - - # Even though ctx.actions.symlink() is used, using - # declare_symlink() is required to ensure that the resulting file - # in runfiles is always a symlink. An RBE implementation, for example, - # may choose to write what symlink() points to instead. - interpreter = ctx.actions.declare_symlink("{}/bin/{}".format(venv, py_exe_basename)) - - interpreter_actual_path = _runfiles_root_path(ctx, runtime.interpreter.short_path) - rel_path = relative_path( - # dirname is necessary because a relative symlink is relative to - # the directory the symlink resides within. - from_ = paths.dirname(_runfiles_root_path(ctx, interpreter.short_path)), - to = interpreter_actual_path, - ) - - ctx.actions.symlink(output = interpreter, target_path = rel_path) - else: - py_exe_basename = paths.basename(runtime.interpreter_path) - interpreter = ctx.actions.declare_symlink("{}/bin/{}".format(venv, py_exe_basename)) - ctx.actions.symlink(output = interpreter, target_path = runtime.interpreter_path) - interpreter_actual_path = runtime.interpreter_path - - if runtime.interpreter_version_info: - version = "{}.{}".format( - runtime.interpreter_version_info.major, - runtime.interpreter_version_info.minor, - ) - else: - version_flag = ctx.attr._python_version_flag[config_common.FeatureFlagInfo].value - version_flag_parts = version_flag.split(".")[0:2] - version = "{}.{}".format(*version_flag_parts) - - # See site.py logic: free-threaded builds append "t" to the venv lib dir name - if "t" in runtime.abi_flags: - version += "t" - - site_packages = "{}/lib/python{}/site-packages".format(venv, version) - pth = ctx.actions.declare_file("{}/bazel.pth".format(site_packages)) - ctx.actions.write(pth, "import _bazel_site_init\n") - - site_init = ctx.actions.declare_file("{}/_bazel_site_init.py".format(site_packages)) - computed_subs = ctx.actions.template_dict() - computed_subs.add_joined("%imports%", imports, join_with = ":", map_each = _map_each_identity) - ctx.actions.expand_template( - template = runtime.site_init_template, - output = site_init, - substitutions = { - "%import_all%": "True" if ctx.fragments.bazel_py.python_import_all_repositories else "False", - "%site_init_runfiles_path%": "{}/{}".format(ctx.workspace_name, site_init.short_path), - "%workspace_name%": ctx.workspace_name, - }, - computed_substitutions = computed_subs, - ) - - return struct( - interpreter = interpreter, - # Runfiles root relative path or absolute path - interpreter_actual_path = interpreter_actual_path, - files_without_interpreter = [pyvenv_cfg, pth, site_init], - ) - -def _map_each_identity(v): - return v - -def _create_stage2_bootstrap( - ctx, - *, - output_prefix, - output_sibling, - main_py, - imports, - runtime_details): - output = ctx.actions.declare_file( - # Prepend with underscore to prevent pytest from trying to - # process the bootstrap for files starting with `test_` - "_{}_stage2_bootstrap.py".format(output_prefix), - sibling = output_sibling, - ) - runtime = runtime_details.effective_runtime - if (ctx.configuration.coverage_enabled and - runtime and - runtime.coverage_tool): - coverage_tool_runfiles_path = "{}/{}".format( - ctx.workspace_name, - runtime.coverage_tool.short_path, - ) - else: - coverage_tool_runfiles_path = "" - - template = runtime.stage2_bootstrap_template - - ctx.actions.expand_template( - template = template, - output = output, - substitutions = { - "%coverage_tool%": coverage_tool_runfiles_path, - "%import_all%": "True" if ctx.fragments.bazel_py.python_import_all_repositories else "False", - "%imports%": ":".join(imports.to_list()), - "%main%": "{}/{}".format(ctx.workspace_name, main_py.short_path), - "%target%": str(ctx.label), - "%workspace_name%": ctx.workspace_name, - }, - is_executable = True, - ) - return output - -def _runfiles_root_path(ctx, short_path): - """Compute a runfiles-root relative path from `File.short_path` - - Args: - ctx: current target ctx - short_path: str, a main-repo relative path from `File.short_path` - - Returns: - {type}`str`, a runflies-root relative path - """ - - # The ../ comes from short_path is for files in other repos. - if short_path.startswith("../"): - return short_path[3:] - else: - return "{}/{}".format(ctx.workspace_name, short_path) - -def _create_stage1_bootstrap( - ctx, - *, - output, - main_py = None, - stage2_bootstrap = None, - imports = None, - is_for_zip, - runtime_details, - venv = None): - runtime = runtime_details.effective_runtime - - if venv: - python_binary_path = _runfiles_root_path(ctx, venv.interpreter.short_path) - else: - python_binary_path = runtime_details.executable_interpreter_path - - if is_for_zip and venv: - python_binary_actual = venv.interpreter_actual_path - else: - python_binary_actual = "" - - subs = { - "%is_zipfile%": "1" if is_for_zip else "0", - "%python_binary%": python_binary_path, - "%python_binary_actual%": python_binary_actual, - "%target%": str(ctx.label), - "%workspace_name%": ctx.workspace_name, - } - - if stage2_bootstrap: - subs["%stage2_bootstrap%"] = "{}/{}".format( - ctx.workspace_name, - stage2_bootstrap.short_path, - ) - template = runtime.bootstrap_template - subs["%shebang%"] = runtime.stub_shebang - else: - if (ctx.configuration.coverage_enabled and - runtime and - runtime.coverage_tool): - coverage_tool_runfiles_path = "{}/{}".format( - ctx.workspace_name, - runtime.coverage_tool.short_path, - ) - else: - coverage_tool_runfiles_path = "" - if runtime: - subs["%shebang%"] = runtime.stub_shebang - template = runtime.bootstrap_template - else: - subs["%shebang%"] = DEFAULT_STUB_SHEBANG - template = ctx.file._bootstrap_template - - subs["%coverage_tool%"] = coverage_tool_runfiles_path - subs["%import_all%"] = ("True" if ctx.fragments.bazel_py.python_import_all_repositories else "False") - subs["%imports%"] = ":".join(imports.to_list()) - subs["%main%"] = "{}/{}".format(ctx.workspace_name, main_py.short_path) - - ctx.actions.expand_template( - template = template, - output = output, - substitutions = subs, - ) - -def _create_windows_exe_launcher( - ctx, - *, - output, - python_binary_path, - use_zip_file): - launch_info = ctx.actions.args() - launch_info.use_param_file("%s", use_always = True) - launch_info.set_param_file_format("multiline") - launch_info.add("binary_type=Python") - launch_info.add(ctx.workspace_name, format = "workspace_name=%s") - launch_info.add( - "1" if py_internal.runfiles_enabled(ctx) else "0", - format = "symlink_runfiles_enabled=%s", - ) - launch_info.add(python_binary_path, format = "python_bin_path=%s") - launch_info.add("1" if use_zip_file else "0", format = "use_zip_file=%s") - - launcher = ctx.attr._launcher[DefaultInfo].files_to_run.executable - ctx.actions.run( - executable = ctx.executable._windows_launcher_maker, - arguments = [launcher.path, launch_info, output.path], - inputs = [launcher], - outputs = [output], - mnemonic = "PyBuildLauncher", - progress_message = "Creating launcher for %{label}", - # Needed to inherit PATH when using non-MSVC compilers like MinGW - use_default_shell_env = True, - ) - -def _create_zip_file(ctx, *, output, original_nonzip_executable, zip_main, runfiles): - """Create a Python zipapp (zip with __main__.py entry point).""" - workspace_name = ctx.workspace_name - legacy_external_runfiles = _py_builtins.get_legacy_external_runfiles(ctx) - - manifest = ctx.actions.args() - manifest.use_param_file("@%s", use_always = True) - manifest.set_param_file_format("multiline") - - manifest.add("__main__.py={}".format(zip_main.path)) - manifest.add("__init__.py=") - manifest.add( - "{}=".format( - _get_zip_runfiles_path("__init__.py", workspace_name, legacy_external_runfiles), - ), - ) - for path in runfiles.empty_filenames.to_list(): - manifest.add("{}=".format(_get_zip_runfiles_path(path, workspace_name, legacy_external_runfiles))) - - def map_zip_runfiles(file): - if file != original_nonzip_executable and file != output: - return "{}={}".format( - _get_zip_runfiles_path(file.short_path, workspace_name, legacy_external_runfiles), - file.path, - ) - else: - return None - - manifest.add_all(runfiles.files, map_each = map_zip_runfiles, allow_closure = True) - - inputs = [zip_main] - if _py_builtins.is_bzlmod_enabled(ctx): - zip_repo_mapping_manifest = ctx.actions.declare_file( - output.basename + ".repo_mapping", - sibling = output, - ) - _py_builtins.create_repo_mapping_manifest( - ctx = ctx, - runfiles = runfiles, - output = zip_repo_mapping_manifest, - ) - manifest.add("{}/_repo_mapping={}".format( - _ZIP_RUNFILES_DIRECTORY_NAME, - zip_repo_mapping_manifest.path, - )) - inputs.append(zip_repo_mapping_manifest) - - for artifact in runfiles.files.to_list(): - # Don't include the original executable because it isn't used by the - # zip file, so no need to build it for the action. - # Don't include the zipfile itself because it's an output. - if artifact != original_nonzip_executable and artifact != output: - inputs.append(artifact) - - zip_cli_args = ctx.actions.args() - zip_cli_args.add("cC") - zip_cli_args.add(output) - - ctx.actions.run( - executable = ctx.executable._zipper, - arguments = [zip_cli_args, manifest], - inputs = depset(inputs), - outputs = [output], - use_default_shell_env = True, - mnemonic = "PythonZipper", - progress_message = "Building Python zip: %{label}", - ) - -def _get_zip_runfiles_path(path, workspace_name, legacy_external_runfiles): - if legacy_external_runfiles and path.startswith(_EXTERNAL_PATH_PREFIX): - zip_runfiles_path = paths.relativize(path, _EXTERNAL_PATH_PREFIX) - else: - # NOTE: External runfiles (artifacts in other repos) will have a leading - # path component of "../" so that they refer outside the main workspace - # directory and into the runfiles root. By normalizing, we simplify e.g. - # "workspace/../foo/bar" to simply "foo/bar". - zip_runfiles_path = paths.normalize("{}/{}".format(workspace_name, path)) - return "{}/{}".format(_ZIP_RUNFILES_DIRECTORY_NAME, zip_runfiles_path) - -def _create_executable_zip_file( - ctx, - *, - output, - zip_file, - stage2_bootstrap, - runtime_details, - venv): - prelude = ctx.actions.declare_file( - "{}_zip_prelude.sh".format(output.basename), - sibling = output, - ) - if stage2_bootstrap: - _create_stage1_bootstrap( - ctx, - output = prelude, - stage2_bootstrap = stage2_bootstrap, - runtime_details = runtime_details, - is_for_zip = True, - venv = venv, - ) - else: - ctx.actions.write(prelude, "#!/usr/bin/env python3\n") - - ctx.actions.run_shell( - command = "cat {prelude} {zip} > {output}".format( - prelude = prelude.path, - zip = zip_file.path, - output = output.path, - ), - inputs = [prelude, zip_file], - outputs = [output], - use_default_shell_env = True, - mnemonic = "PyBuildExecutableZip", - progress_message = "Build Python zip executable: %{label}", - ) - -def _get_cc_details_for_binary(ctx, extra_deps): - cc_info = collect_cc_info(ctx, extra_deps = extra_deps) - return create_cc_details_struct( - cc_info_for_propagating = cc_info, - cc_info_for_self_link = cc_info, - cc_info_with_extra_link_time_libraries = None, - extra_runfiles = ctx.runfiles(), - # Though the rules require the CcToolchain, it isn't actually used. - cc_toolchain = None, - feature_config = None, - ) - -def _get_interpreter_path(ctx, *, runtime, flag_interpreter_path): - if runtime: - if runtime.interpreter_path: - interpreter_path = runtime.interpreter_path - else: - interpreter_path = "{}/{}".format( - ctx.workspace_name, - runtime.interpreter.short_path, - ) - - # NOTE: External runfiles (artifacts in other repos) will have a - # leading path component of "../" so that they refer outside the - # main workspace directory and into the runfiles root. By - # normalizing, we simplify e.g. "workspace/../foo/bar" to simply - # "foo/bar" - interpreter_path = paths.normalize(interpreter_path) - - elif flag_interpreter_path: - interpreter_path = flag_interpreter_path - else: - fail("Unable to determine interpreter path") - - return interpreter_path - -def _get_native_deps_dso_name(ctx): - _ = ctx # @unused - fail("Building native deps DSO not supported.") - -def _get_native_deps_user_link_flags(ctx): - _ = ctx # @unused - fail("Building native deps DSO not supported.") diff --git a/python/private/py_executable_info.bzl b/python/private/py_executable_info.bzl index deb119428d..7693fa950f 100644 --- a/python/private/py_executable_info.bzl +++ b/python/private/py_executable_info.bzl @@ -10,10 +10,36 @@ This provider is for executable-specific information (e.g. tests and binaries). ::: """, fields = { + "app_runfiles": """ +:type: runfiles + +The runfiles for the executable's "user" dependencies. These are things in e.g. +`deps` (or similar), but doesn't include "external" or "implicit" pieces, +e.g. the Python runtime itself. It's roughly akin to the files a traditional +venv would have installed into it. + +:::{seealso} +{obj}`PyRuntimeInfo` for the Python runtime files. The {obj}`py_binary` et al +rules provide it directly so that the runtime the binary original chose +can be accessed. +::: + +:::{versionadded} 1.9.0 +::: +""", "build_data_file": """ :type: None | File A symlink to build_data.txt if stamping is enabled, otherwise None. +""", + "interpreter_args": """ +:type: list[str] + +Args that should be passed to the interpreter before regular args +(e.g. `-X whatever`). + +:::{versionadded} 1.9.0 +::: """, "interpreter_path": """ :type: None | str @@ -28,6 +54,12 @@ should be within `runtime_files`) The user-level entry point file. Usually a `.py` file, but may also be `.pyc` file if precompiling is enabled. + +:::{seealso} + +The {obj}`stage2_bootstrap` attribute, which bootstraps an executable to run +the user main file. +::: """, "runfiles_without_exe": """ :type: runfiles @@ -35,6 +67,58 @@ file if precompiling is enabled. The runfiles the program needs, but without the original executable, files only added to support the original executable, or files specific to the original program. +""", + "stage2_bootstrap": """ +:type: File | None + +The Bazel-executable-level entry point to the program, which handles Bazel-specific +setup before running the file in {obj}`main`. May be None if a two-stage bootstrap +implementation isn't being used. + +:::{versionadded} 1.9.0 +::: +""", + "venv_app_symlinks": """ +:type: depset[ExplicitSymlink] | None + +Symlinks that are specific to the application within the venv (e.g. +dependencies). + +Only used with Windows for files that would have used `declare_symlink()` +to create relative symlinks. These may overlap with paths in runfiles; it's +up to the consumer to determine how to handle such overlaps. + +:::{versionadded} 2.1.0 +::: +""", + "venv_interpreter_runfiles": """ +:type: runfiles | None + +Runfiles that are specific to the interpreter within the venv. + +:::{versionadded} 2.0.0 +::: +""", + "venv_interpreter_symlinks": """ +:type: depset[ExplicitSymlink] | None + +Symlinks that are specific to the interpreter within the venv. + +Only used with Windows for files that would have used `declare_symlink()` +to create relative symlinks. These may overlap with paths in runfiles; it's +up to the consumer to determine how to handle such overlaps. + +:::{versionadded} 2.0.0 +::: +""", + "venv_python_exe": """ +:type: File | None + +The `bin/python3` file within the venv this binary uses. May be None if venv +mode is not enabled. + +:::{versionadded} 1.9.0 +::: """, }, ) diff --git a/python/private/py_info.bzl b/python/private/py_info.bzl index 4b2b8888c9..dac1ddeff3 100644 --- a/python/private/py_info.bzl +++ b/python/private/py_info.bzl @@ -13,10 +13,342 @@ # limitations under the License. """Implementation of PyInfo provider and PyInfo-specific utilities.""" -load("@rules_python_internal//:rules_python_config.bzl", "config") load(":builders.bzl", "builders") load(":reexports.bzl", "BuiltinPyInfo") -load(":util.bzl", "define_bazel_6_provider") + +def _VenvSymlinkKind_typedef(): + """An enum of types of venv directories. + + :::{field} BIN + :type: object + + Indicates to create paths under the directory that has binaries + within the venv. + ::: + + :::{field} LIB + :type: object + + Indicates to create paths under the venv's site-packages directory. + ::: + + :::{field} INCLUDE + :type: object + + Indicates to create paths under the venv's include directory. + ::: + + :::{field} DATA + :type: object + + Indicates to create paths under the venv's data directory. + ::: + """ + +# buildifier: disable=name-conventions +VenvSymlinkKind = struct( + TYPEDEF = _VenvSymlinkKind_typedef, + BIN = "BIN", + LIB = "LIB", + INCLUDE = "INCLUDE", + DATA = "DATA", +) + +def _VenvSymlinkEntry_init(**kwargs): + kwargs.setdefault("link_to_file", None) + return kwargs + +# A provider is used for memory efficiency. +# buildifier: disable=name-conventions +VenvSymlinkEntry, _ = provider( + doc = """ +An entry in `PyInfo.venv_symlinks` +""", + init = _VenvSymlinkEntry_init, + fields = { + "files": """ +:type: depset[File] + +Files under `link_to_path`. + +This is only used when multiple targets have overlapping `venv_path` paths. e.g. +if one adds files to `venv_path=a/` and another adds files to `venv_path=a/b/`. +""", + "kind": """ +:type: str + +One of the {obj}`VenvSymlinkKind` values. It represents which directory within +the venv to create the path under. +""", + "link_to_file": """ +:type: File | None + +A file that `venv_path` should point to. The file to link to should also be in +`files`. + +:::{versionadded} 1.7.0 +::: +""", + "link_to_path": """ +:type: str | None + +A runfiles-root relative path that `venv_path` will symlink to (if +`link_to_file` is `None`). If `None`, it means to not create it in the venv. +""", + "package": """ +:type: str | None + +Represents the PyPI package name that the code originates from. It is normalized according to the +PEP440 with all `-` replaced with `_`, i.e. the same as the package name in the hub repository that +it would come from. +""", + "venv_path": """ +:type: str + +A path relative to the `kind` directory within the venv. +""", + "version": """ +:type: str | None + +Represents the PyPI package version that the code originates from. It is normalized according to the +PEP440 standard. +""", + }, +) + +def _VenvSymlinkEntryBuilder_typedef(): + """Builder for VenvSymlinkEntry. + + To create an instance, use {obj}`PyInfoBuilder.add_venv_symlink()`. + + :::{field} files + :type: DepsetBuilder[File] + ::: + + :::{versionadded} 2.2.0 + ::: + """ + +def _VenvSymlinkEntryBuilder(): + # buildifier: disable=uninitialized + self = struct( + _state = { + "kind": None, + "link_to_file": None, + "link_to_path": None, + "package": None, + "venv_path": None, + "version": None, + }, + files = builders.DepsetBuilder(), + get_kind = lambda: _VenvSymlinkEntryBuilder_get_kind(self), + set_kind = lambda k: _VenvSymlinkEntryBuilder_set_kind(self, k), + get_link_to_file = lambda: _VenvSymlinkEntryBuilder_get_link_to_file(self), + set_link_to_file = lambda f: _VenvSymlinkEntryBuilder_set_link_to_file(self, f), + get_link_to_path = lambda: _VenvSymlinkEntryBuilder_get_link_to_path(self), + set_link_to_path = lambda p: _VenvSymlinkEntryBuilder_set_link_to_path(self, p), + get_package = lambda: _VenvSymlinkEntryBuilder_get_package(self), + set_package = lambda p: _VenvSymlinkEntryBuilder_set_package(self, p), + get_venv_path = lambda: _VenvSymlinkEntryBuilder_get_venv_path(self), + set_venv_path = lambda p: _VenvSymlinkEntryBuilder_set_venv_path(self, p), + get_version = lambda: _VenvSymlinkEntryBuilder_get_version(self), + set_version = lambda v: _VenvSymlinkEntryBuilder_set_version(self, v), + build = lambda: _VenvSymlinkEntryBuilder_build(self), + ) + return self + +def _VenvSymlinkEntryBuilder_get_kind(self): + """Get the kind of the symlink. + + Args: + self: implicitly added. + + Returns: + {type}`string` One of the {obj}`VenvSymlinkKind` values. + """ + return self._state["kind"] + +def _VenvSymlinkEntryBuilder_get_link_to_file(self): + """Get the file that the symlink points to. + + Args: + self: implicitly added. + + Returns: + {type}`File | None` + """ + return self._state["link_to_file"] + +def _VenvSymlinkEntryBuilder_get_link_to_path(self): + """Get the runfiles-root relative path that the symlink points to. + + Args: + self: implicitly added. + + Returns: + {type}`string | None` + """ + return self._state["link_to_path"] + +def _VenvSymlinkEntryBuilder_get_package(self): + """Get the PyPI package name that the code originates from. + + Args: + self: implicitly added. + + Returns: + {type}`string | None` + """ + return self._state["package"] + +def _VenvSymlinkEntryBuilder_get_venv_path(self): + """Get the path relative to the kind directory within the venv. + + Args: + self: implicitly added. + + Returns: + {type}`string` + """ + return self._state["venv_path"] + +def _VenvSymlinkEntryBuilder_get_version(self): + """Get the PyPI package version that the code originates from. + + Args: + self: implicitly added. + + Returns: + {type}`string | None` + """ + return self._state["version"] + +def _VenvSymlinkEntryBuilder_set_kind(self, kind): + """Set the kind of the symlink. + + Args: + self: implicitly added. + kind: {type}`string` One of the {obj}`VenvSymlinkKind` values. + + Returns: + {type}`VenvSymlinkEntryBuilder` self. + """ + _check_arg_type("kind", "string", kind) + self._state["kind"] = kind + return self + +def _VenvSymlinkEntryBuilder_set_link_to_file(self, link_to_file): + """Set the file that the symlink points to. + + Args: + self: implicitly added. + link_to_file: {type}`File | None` + + Returns: + {type}`VenvSymlinkEntryBuilder` self. + """ + if link_to_file != None: + _check_arg_type("link_to_file", "File", link_to_file) + self._state["link_to_file"] = link_to_file + return self + +def _VenvSymlinkEntryBuilder_set_link_to_path(self, link_to_path): + """Set the runfiles-root relative path that the symlink points to. + + Args: + self: implicitly added. + link_to_path: {type}`string | None` + + Returns: + {type}`VenvSymlinkEntryBuilder` self. + """ + if link_to_path != None: + _check_arg_type("link_to_path", "string", link_to_path) + self._state["link_to_path"] = link_to_path + return self + +def _VenvSymlinkEntryBuilder_set_package(self, package): + """Set the PyPI package name that the code originates from. + + Args: + self: implicitly added. + package: {type}`string | None` + + Returns: + {type}`VenvSymlinkEntryBuilder` self. + """ + if package != None: + _check_arg_type("package", "string", package) + self._state["package"] = package + return self + +def _VenvSymlinkEntryBuilder_set_venv_path(self, venv_path): + """Set the path relative to the kind directory within the venv. + + Args: + self: implicitly added. + venv_path: {type}`string` + + Returns: + {type}`VenvSymlinkEntryBuilder` self. + """ + _check_arg_type("venv_path", "string", venv_path) + self._state["venv_path"] = venv_path + return self + +def _VenvSymlinkEntryBuilder_set_version(self, version): + """Set the PyPI package version that the code originates from. + + Args: + self: implicitly added. + version: {type}`string | None` + + Returns: + {type}`VenvSymlinkEntryBuilder` self. + """ + if version != None: + _check_arg_type("version", "string", version) + self._state["version"] = version + return self + +def _VenvSymlinkEntryBuilder_build(self): + """Builds into a {obj}`VenvSymlinkEntry` object. + + Args: + self: implicitly added. + + Returns: + {type}`VenvSymlinkEntry` + """ + if not self._state["venv_path"]: + fail("venv_path must be set") + return VenvSymlinkEntry( + files = self.files.build(), + kind = self._state["kind"], + link_to_file = self._state["link_to_file"], + link_to_path = self._state["link_to_path"], + package = self._state["package"], + venv_path = self._state["venv_path"], + version = self._state["version"], + ) + +# buildifier: disable=name-conventions +VenvSymlinkEntryBuilder = struct( + TYPEDEF = _VenvSymlinkEntryBuilder_typedef, + build = _VenvSymlinkEntryBuilder_build, + get_kind = _VenvSymlinkEntryBuilder_get_kind, + set_kind = _VenvSymlinkEntryBuilder_set_kind, + get_link_to_file = _VenvSymlinkEntryBuilder_get_link_to_file, + set_link_to_file = _VenvSymlinkEntryBuilder_set_link_to_file, + get_link_to_path = _VenvSymlinkEntryBuilder_get_link_to_path, + set_link_to_path = _VenvSymlinkEntryBuilder_set_link_to_path, + get_package = _VenvSymlinkEntryBuilder_get_package, + set_package = _VenvSymlinkEntryBuilder_set_package, + get_venv_path = _VenvSymlinkEntryBuilder_get_venv_path, + set_venv_path = _VenvSymlinkEntryBuilder_set_venv_path, + get_version = _VenvSymlinkEntryBuilder_get_version, + set_version = _VenvSymlinkEntryBuilder_set_version, +) def _check_arg_type(name, required_type, value): """Check that a value is of an expected type.""" @@ -38,7 +370,12 @@ def _PyInfo_init( direct_pyc_files = depset(), transitive_pyc_files = depset(), transitive_implicit_pyc_files = depset(), - transitive_implicit_pyc_source_files = depset()): + transitive_implicit_pyc_source_files = depset(), + direct_original_sources = depset(), + transitive_original_sources = depset(), + direct_pyi_files = depset(), + transitive_pyi_files = depset(), + venv_symlinks = depset()): _check_arg_type("transitive_sources", "depset", transitive_sources) # Verify it's postorder compatible, but retain is original ordering. @@ -53,22 +390,49 @@ def _PyInfo_init( _check_arg_type("transitive_implicit_pyc_files", "depset", transitive_pyc_files) _check_arg_type("transitive_implicit_pyc_source_files", "depset", transitive_pyc_files) + + _check_arg_type("direct_original_sources", "depset", direct_original_sources) + _check_arg_type("transitive_original_sources", "depset", transitive_original_sources) + + _check_arg_type("direct_pyi_files", "depset", direct_pyi_files) + _check_arg_type("transitive_pyi_files", "depset", transitive_pyi_files) return { + "direct_original_sources": direct_original_sources, "direct_pyc_files": direct_pyc_files, + "direct_pyi_files": direct_pyi_files, "has_py2_only_sources": has_py2_only_sources, - "has_py3_only_sources": has_py2_only_sources, + "has_py3_only_sources": has_py3_only_sources, "imports": imports, "transitive_implicit_pyc_files": transitive_implicit_pyc_files, "transitive_implicit_pyc_source_files": transitive_implicit_pyc_source_files, + "transitive_original_sources": transitive_original_sources, "transitive_pyc_files": transitive_pyc_files, + "transitive_pyi_files": transitive_pyi_files, "transitive_sources": transitive_sources, "uses_shared_libraries": uses_shared_libraries, + "venv_symlinks": venv_symlinks, } -PyInfo, _unused_raw_py_info_ctor = define_bazel_6_provider( - doc = "Encapsulates information provided by the Python rules.", +PyInfo, _unused_raw_py_info_ctor = provider( + doc = """Encapsulates information provided by the Python rules. + +Instead of creating this object directly, use {obj}`PyInfoBuilder` and +the {obj}`PyCommonApi` utilities. +""", init = _PyInfo_init, fields = { + "direct_original_sources": """ +:type: depset[File] + +The `.py` source files (if any) that are considered directly provided by +the target. This field is intended so that static analysis tools can recover the +original Python source files, regardless of any build settings (e.g. +precompiling), so they can analyze source code. The values are typically the +`.py` files in the `srcs` attribute (or equivalent). + +::::{versionadded} 1.1.0 +:::: +""", "direct_pyc_files": """ :type: depset[File] @@ -78,6 +442,24 @@ by the target and **must be included**. These files usually come from, e.g., a library setting {attr}`precompile=enabled` to forcibly enable precompiling for itself. Downstream binaries are expected to always include these files, as the originating target expects them to exist. +""", + "direct_pyi_files": """ +:type: depset[File] + +Type definition files (usually `.pyi` files) for the Python modules provided by +this target. Usually they describe the source files listed in +`direct_original_sources`. This field is primarily for static analysis tools. + +These files are _usually_ build-time only and not included as part of a runnable +program. + +:::{note} +This may contain implementation-specific file types specific to a particular +type checker. +::: + +::::{versionadded} 1.1.0 +:::: """, "has_py2_only_sources": """ :type: bool @@ -116,6 +498,21 @@ then {obj}`transitive_implicit_pyc_files` should be included instead. ::::{versionadded} 0.37.0 :::: +""", + "transitive_original_sources": """ +:type: depset[File] + +The transitive set of `.py` source files (if any) that are considered the +original sources for this target and its transitive dependencies. This field is +intended so that static analysis tools can recover the original Python source +files, regardless of any build settings (e.g. precompiling), so they can analyze +source code. The values are typically the `.py` files in the `srcs` attribute +(or equivalent). + +This is superset of `direct_original_sources`. + +::::{versionadded} 1.1.0 +:::: """, "transitive_pyc_files": """ :type: depset[File] @@ -125,6 +522,25 @@ The transitive set of precompiled files that must be included. These files usually come from, e.g., a library setting {attr}`precompile=enabled` to forcibly enable precompiling for itself. Downstream binaries are expected to always include these files, as the originating target expects them to exist. +""", + "transitive_pyi_files": """ +:type: depset[File] + +The transitive set of type definition files (usually `.pyi` files) for the +Python modules for this target and its transitive dependencies. this target. +Usually they describe the source files listed in `transitive_original_sources`. +This field is primarily for static analysis tools. + +These files are _usually_ build-time only and not included as part of a runnable +program. + +:::{note} +This may contain implementation-specific file types specific to a particular +type checker. +::: + +::::{versionadded} 1.1.0 +:::: """, "transitive_sources": """\ :type: depset[File] @@ -150,22 +566,90 @@ Whether any of this target's transitive `deps` has a shared library file (such as a `.so` file). This field is currently unused in Bazel and may go away in the future. +""", + "venv_symlinks": """ +:type: depset[VenvSymlinkEntry] + +:::{include} /_includes/experimental_api.md +::: + +:::{versionadded} 1.5.0 +::: """, }, ) # The "effective" PyInfo is what the canonical //python:py_info.bzl%PyInfo symbol refers to -_EffectivePyInfo = PyInfo if (config.enable_pystar or BuiltinPyInfo == None) else BuiltinPyInfo +_EffectivePyInfo = PyInfo + +def _PyInfoBuilder_typedef(): + """Builder for PyInfo. + + To create an instance, use {obj}`py_common.get()` and call `PyInfoBuilder()` + + :::{field} direct_original_sources + :type: DepsetBuilder[File] + ::: + + :::{field} direct_pyc_files + :type: DepsetBuilder[File] + ::: + + :::{field} direct_pyi_files + :type: DepsetBuilder[File] + ::: + + :::{field} imports + :type: DepsetBuilder[str] + ::: + + :::{field} transitive_implicit_pyc_files + :type: DepsetBuilder[File] + ::: + + :::{field} transitive_implicit_pyc_source_files + :type: DepsetBuilder[File] + ::: + + :::{field} transitive_original_sources + :type: DepsetBuilder[File] + ::: + + :::{field} transitive_pyc_files + :type: DepsetBuilder[File] + ::: + + :::{field} transitive_pyi_files + :type: DepsetBuilder[File] + ::: + + :::{field} transitive_sources + :type: DepsetBuilder[File] + ::: + + :::{field} venv_symlinks + :type: DepsetBuilder[tuple[str | None, str]] + """ + +def _PyInfoBuilder_new(): + """Creates an instance. + + Returns: + {type}`PyInfoBuilder` + """ -def PyInfoBuilder(): # buildifier: disable=uninitialized self = struct( _has_py2_only_sources = [False], _has_py3_only_sources = [False], _uses_shared_libraries = [False], + _venv_symlink_builders = [], + add_venv_symlink = lambda *a, **k: _PyInfoBuilder_add_venv_symlink(self, *a, **k), build = lambda *a, **k: _PyInfoBuilder_build(self, *a, **k), build_builtin_py_info = lambda *a, **k: _PyInfoBuilder_build_builtin_py_info(self, *a, **k), + direct_original_sources = builders.DepsetBuilder(), direct_pyc_files = builders.DepsetBuilder(), + direct_pyi_files = builders.DepsetBuilder(), get_has_py2_only_sources = lambda *a, **k: _PyInfoBuilder_get_has_py2_only_sources(self, *a, **k), get_has_py3_only_sources = lambda *a, **k: _PyInfoBuilder_get_has_py3_only_sources(self, *a, **k), get_uses_shared_libraries = lambda *a, **k: _PyInfoBuilder_get_uses_shared_libraries(self, *a, **k), @@ -182,52 +666,178 @@ def PyInfoBuilder(): set_uses_shared_libraries = lambda *a, **k: _PyInfoBuilder_set_uses_shared_libraries(self, *a, **k), transitive_implicit_pyc_files = builders.DepsetBuilder(), transitive_implicit_pyc_source_files = builders.DepsetBuilder(), + transitive_original_sources = builders.DepsetBuilder(), transitive_pyc_files = builders.DepsetBuilder(), + transitive_pyi_files = builders.DepsetBuilder(), transitive_sources = builders.DepsetBuilder(), + venv_symlinks = builders.DepsetBuilder(), ) return self +def _PyInfoBuilder_add_venv_symlink(self): + """Create and return a new VenvSymlinkEntryBuilder. + + :::{versionadded} 2.2.0 + ::: + + Args: + self: implicitly added. + + Returns: + {type}`VenvSymlinkEntryBuilder` + """ + entry_builder = _VenvSymlinkEntryBuilder() + self._venv_symlink_builders.append(entry_builder) + return entry_builder + def _PyInfoBuilder_get_has_py3_only_sources(self): + """Get the `has_py3_only_sources` value. + + Args: + self: implicitly added. + + Returns: + {type}`bool` + """ return self._has_py3_only_sources[0] def _PyInfoBuilder_get_has_py2_only_sources(self): + """Get the `has_py2_only_sources` value. + + Args: + self: implicitly added. + + Returns: + {type}`bool` + """ return self._has_py2_only_sources[0] def _PyInfoBuilder_set_has_py2_only_sources(self, value): + """Sets `has_py2_only_sources` to `value`. + + Args: + self: implicitly added. + value: {type}`bool` The value to set. + + Returns: + {type}`PyInfoBuilder` self + """ self._has_py2_only_sources[0] = value return self def _PyInfoBuilder_set_has_py3_only_sources(self, value): + """Sets `has_py3_only_sources` to `value`. + + Args: + self: implicitly added. + value: {type}`bool` The value to set. + + Returns: + {type}`PyInfoBuilder` self + """ self._has_py3_only_sources[0] = value return self def _PyInfoBuilder_merge_has_py2_only_sources(self, value): + """Sets `has_py2_only_sources` based on current and incoming `value`. + + Args: + self: implicitly added. + value: {type}`bool` Another `has_py2_only_sources` value. It will + be merged into this builder's state. + + Returns: + {type}`PyInfoBuilder` self + """ self._has_py2_only_sources[0] = self._has_py2_only_sources[0] or value return self def _PyInfoBuilder_merge_has_py3_only_sources(self, value): + """Sets `has_py3_only_sources` based on current and incoming `value`. + + Args: + self: implicitly added. + value: {type}`bool` Another `has_py3_only_sources` value. It will + be merged into this builder's state. + + Returns: + {type}`PyInfoBuilder` self + """ self._has_py3_only_sources[0] = self._has_py3_only_sources[0] or value return self def _PyInfoBuilder_merge_uses_shared_libraries(self, value): + """Sets `uses_shared_libraries` based on current and incoming `value`. + + Args: + self: implicitly added. + value: {type}`bool` Another `uses_shared_libraries` value. It will + be merged into this builder's state. + + Returns: + {type}`PyInfoBuilder` self + """ self._uses_shared_libraries[0] = self._uses_shared_libraries[0] or value return self def _PyInfoBuilder_get_uses_shared_libraries(self): + """Get the `uses_shared_libraries` value. + + Args: + self: implicitly added. + + Returns: + {type}`bool` + """ return self._uses_shared_libraries[0] def _PyInfoBuilder_set_uses_shared_libraries(self, value): + """Sets `uses_shared_libraries` to `value`. + + Args: + self: implicitly added. + value: {type}`bool` The value to set. + + Returns: + {type}`PyInfoBuilder` self + """ self._uses_shared_libraries[0] = value return self def _PyInfoBuilder_merge(self, *infos, direct = []): + """Merge other PyInfos into this PyInfo. + + Args: + self: implicitly added. + *infos: {type}`PyInfo` objects to merge in, but only merge in their + information into this object's transitive fields. + direct: {type}`list[PyInfo]` objects to merge in, but also merge their + direct fields into this object's direct fields. + + Returns: + {type}`PyInfoBuilder` self + """ return self.merge_all(list(infos), direct = direct) def _PyInfoBuilder_merge_all(self, transitive, *, direct = []): + """Merge other PyInfos into this PyInfo. + + Args: + self: implicitly added. + transitive: {type}`list[PyInfo]` objects to merge in, but only merge in + their information into this object's transitive fields. + direct: {type}`list[PyInfo]` objects to merge in, but also merge their + direct fields into this object's direct fields. + + Returns: + {type}`PyInfoBuilder` self + """ for info in direct: # BuiltinPyInfo doesn't have this field if hasattr(info, "direct_pyc_files"): + self.direct_original_sources.add(info.direct_original_sources) self.direct_pyc_files.add(info.direct_pyc_files) + self.direct_pyi_files.add(info.direct_pyi_files) for info in direct + transitive: self.imports.add(info.imports) @@ -240,11 +850,25 @@ def _PyInfoBuilder_merge_all(self, transitive, *, direct = []): if hasattr(info, "transitive_pyc_files"): self.transitive_implicit_pyc_files.add(info.transitive_implicit_pyc_files) self.transitive_implicit_pyc_source_files.add(info.transitive_implicit_pyc_source_files) + self.transitive_original_sources.add(info.transitive_original_sources) self.transitive_pyc_files.add(info.transitive_pyc_files) + self.transitive_pyi_files.add(info.transitive_pyi_files) + self.venv_symlinks.add(info.venv_symlinks) return self def _PyInfoBuilder_merge_target(self, target): + """Merge a target's Python information in this object. + + Args: + self: implicitly added. + target: {type}`Target` targets that provide PyInfo, or other relevant + providers, will be merged into this object. If a target doesn't provide + any relevant providers, it is ignored. + + Returns: + {type}`PyInfoBuilder` self. + """ if PyInfo in target: self.merge(target[PyInfo]) elif BuiltinPyInfo != None and BuiltinPyInfo in target: @@ -252,20 +876,35 @@ def _PyInfoBuilder_merge_target(self, target): return self def _PyInfoBuilder_merge_targets(self, targets): + """Merge multiple targets into this object. + + Args: + self: implicitly added. + targets: {type}`list[Target]` + targets that provide PyInfo, or other relevant + providers, will be merged into this object. If a target doesn't provide + any relevant providers, it is ignored. + + Returns: + {type}`PyInfoBuilder` self. + """ for t in targets: self.merge_target(t) return self def _PyInfoBuilder_build(self): - if config.enable_pystar: - kwargs = dict( - direct_pyc_files = self.direct_pyc_files.build(), - transitive_pyc_files = self.transitive_pyc_files.build(), - transitive_implicit_pyc_files = self.transitive_implicit_pyc_files.build(), - transitive_implicit_pyc_source_files = self.transitive_implicit_pyc_source_files.build(), - ) - else: - kwargs = {} + """Builds into a {obj}`PyInfo` object. + + Args: + self: implicitly added. + + Returns: + {type}`PyInfo` + """ + venv_symlinks = depset( + direct = [b.build() for b in self._venv_symlink_builders], + transitive = [self.venv_symlinks.build()], + ) return _EffectivePyInfo( has_py2_only_sources = self._has_py2_only_sources[0], @@ -273,10 +912,27 @@ def _PyInfoBuilder_build(self): imports = self.imports.build(), transitive_sources = self.transitive_sources.build(), uses_shared_libraries = self._uses_shared_libraries[0], - **kwargs + direct_original_sources = self.direct_original_sources.build(), + direct_pyc_files = self.direct_pyc_files.build(), + direct_pyi_files = self.direct_pyi_files.build(), + transitive_implicit_pyc_files = self.transitive_implicit_pyc_files.build(), + transitive_implicit_pyc_source_files = self.transitive_implicit_pyc_source_files.build(), + transitive_original_sources = self.transitive_original_sources.build(), + transitive_pyc_files = self.transitive_pyc_files.build(), + transitive_pyi_files = self.transitive_pyi_files.build(), + venv_symlinks = venv_symlinks, ) def _PyInfoBuilder_build_builtin_py_info(self): + """Builds into a Bazel-builtin PyInfo object, if available. + + Args: + self: implicitly added. + + Returns: + {type}`BuiltinPyInfo | None` None is returned if Bazel's + builtin PyInfo object is disabled. + """ if BuiltinPyInfo == None: return None @@ -287,3 +943,26 @@ def _PyInfoBuilder_build_builtin_py_info(self): transitive_sources = self.transitive_sources.build(), uses_shared_libraries = self._uses_shared_libraries[0], ) + +# Provided for documentation purposes +# buildifier: disable=name-conventions +PyInfoBuilder = struct( + TYPEDEF = _PyInfoBuilder_typedef, + new = _PyInfoBuilder_new, + add_venv_symlink = _PyInfoBuilder_add_venv_symlink, + build = _PyInfoBuilder_build, + build_builtin_py_info = _PyInfoBuilder_build_builtin_py_info, + get_has_py2_only_sources = _PyInfoBuilder_get_has_py2_only_sources, + get_has_py3_only_sources = _PyInfoBuilder_get_has_py3_only_sources, + get_uses_shared_libraries = _PyInfoBuilder_get_uses_shared_libraries, + merge = _PyInfoBuilder_merge, + merge_all = _PyInfoBuilder_merge_all, + merge_has_py2_only_sources = _PyInfoBuilder_merge_has_py2_only_sources, + merge_has_py3_only_sources = _PyInfoBuilder_merge_has_py3_only_sources, + merge_target = _PyInfoBuilder_merge_target, + merge_targets = _PyInfoBuilder_merge_targets, + merge_uses_shared_libraries = _PyInfoBuilder_merge_uses_shared_libraries, + set_has_py2_only_sources = _PyInfoBuilder_set_has_py2_only_sources, + set_has_py3_only_sources = _PyInfoBuilder_set_has_py3_only_sources, + set_uses_shared_libraries = _PyInfoBuilder_set_uses_shared_libraries, +) diff --git a/python/private/py_internal.bzl b/python/private/py_internal.bzl index 429637253f..000501d26c 100644 --- a/python/private/py_internal.bzl +++ b/python/private/py_internal.bzl @@ -18,9 +18,10 @@ Re-exports the restricted-use py_internal helper under its original name. These may change at any time and are closely coupled to the rule implementation. """ -# The py_internal global is only available in Bazel 7+, so loading of it -# must go through a repo rule with Bazel version detection logic. -load("@rules_python_internal//:py_internal.bzl", "py_internal_impl") +# The native `py_internal` object is only visible to Starlark files under +# `tools/build_defs/python`. To access it from `//python/private`, we use an +# indirection through `//tools/build_defs/python/private/py_internal_renamed.bzl`, +# which re-exports it under a different name. +load("//tools/build_defs/python/private:py_internal_renamed.bzl", "py_internal_renamed") # buildifier: disable=bzl-visibility -# NOTE: This is None prior to Bazel 7, as set by @rules_python_internal -py_internal = py_internal_impl +py_internal = py_internal_renamed diff --git a/python/private/py_interpreter_program.bzl b/python/private/py_interpreter_program.bzl index cd62a7190d..1f1abb1bfe 100644 --- a/python/private/py_interpreter_program.bzl +++ b/python/private/py_interpreter_program.bzl @@ -15,6 +15,7 @@ """Internal only bootstrap level binary-like rule.""" load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") +load("//python/private:sentinel_impl.bzl", "SentinelInfo") PyInterpreterProgramInfo = provider( doc = "Information about how to run a program with an external interpreter.", @@ -28,14 +29,20 @@ PyInterpreterProgramInfo = provider( def _py_interpreter_program_impl(ctx): # Bazel requires the executable file to be an output created by this target. - executable = ctx.actions.declare_file(ctx.label.name) + # To avoid colliding with the source file (e.g. target=foo, main=foo.py), + # we append an underscore to the name, but keep the extension so that + # the original extension is preserved. + extension = ctx.file.main.extension + executable_name = "{}_.{}".format(ctx.label.name, extension) + executable = ctx.actions.declare_file(executable_name) ctx.actions.symlink(output = executable, target_file = ctx.file.main) execution_requirements = {} - execution_requirements.update([ - value.split("=", 1) - for value in ctx.attr.execution_requirements[BuildSettingInfo].value - if value.strip() - ]) + if BuildSettingInfo in ctx.attr.execution_requirements: + execution_requirements.update([ + value.split("=", 1) + for value in ctx.attr.execution_requirements[BuildSettingInfo].value + if value.strip() + ]) return [ DefaultInfo( @@ -85,8 +92,9 @@ ctx.actions.run( doc = "Environment variables that should set prior to running.", ), "execution_requirements": attr.label( + default = "//python:none", doc = "Execution requirements to set when running it as an action", - providers = [BuildSettingInfo], + providers = [[BuildSettingInfo], [SentinelInfo]], ), "interpreter_args": attr.string_list( doc = "Args that should be passed to the interpreter.", diff --git a/python/private/py_library.bzl b/python/private/py_library.bzl index 6a65038e8a..3d8ba4aa71 100644 --- a/python/private/py_library.bzl +++ b/python/private/py_library.bzl @@ -14,66 +14,122 @@ """Common code for implementing py_library rules.""" load("@bazel_skylib//lib:dicts.bzl", "dicts") +load("@bazel_skylib//lib:paths.bzl", "paths") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") +load(":attr_builders.bzl", "attrb") load( ":attributes.bzl", "COMMON_ATTRS", "IMPORTS_ATTRS", "PY_SRCS_ATTRS", "PrecompileAttr", - "REQUIRED_EXEC_GROUPS", - "SRCS_VERSION_ALL_VALUES", - "create_srcs_attr", - "create_srcs_version_attr", + "REQUIRED_EXEC_GROUP_BUILDERS", ) load(":builders.bzl", "builders") load( ":common.bzl", + "collect_cc_info", "collect_imports", "collect_runfiles", "create_instrumented_files_info", "create_output_group_info", "create_py_info", "filter_to_py_srcs", - "union_attrs", ) -load(":flags.bzl", "AddSrcsToRunfilesFlag", "PrecompileFlag") +load(":common_labels.bzl", "labels") +load(":flags.bzl", "AddSrcsToRunfilesFlag", "PrecompileFlag", "VenvsSitePackages") +load(":normalize_name.bzl", "normalize_name") +load(":precompile.bzl", "maybe_precompile") load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") -load(":py_internal.bzl", "py_internal") +load(":py_info.bzl", "PyInfo") +load(":reexports.bzl", "BuiltinPyInfo") +load(":rule_builders.bzl", "ruleb") load( ":toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", TOOLCHAIN_TYPE = "TARGET_TOOLCHAIN_TYPE", ) +load(":venv_runfiles.bzl", "get_venv_symlinks") +load(":version.bzl", "version") -_py_builtins = py_internal - -LIBRARY_ATTRS = union_attrs( +LIBRARY_ATTRS = dicts.add( COMMON_ATTRS, PY_SRCS_ATTRS, IMPORTS_ATTRS, - create_srcs_version_attr(values = SRCS_VERSION_ALL_VALUES), - create_srcs_attr(mandatory = False), { - "_add_srcs_to_runfiles_flag": attr.label( - default = "//python/config_settings:add_srcs_to_runfiles", + "experimental_venvs_site_packages": lambda: attrb.Label( + doc = """ +**INTERNAL ATTRIBUTE. SHOULD ONLY BE SET BY rules_python-INTERNAL CODE.** + +:::{include} /_includes/experimental_api.md +::: + +A flag that decides whether the library should treat its sources as a +site-packages layout. + +When the flag is `yes`, then the `srcs` files are treated as a site-packages +layout that is relative to the `imports` attribute. The `imports` attribute +can have only a single element. It is a repo-relative runfiles path. + +For example, in the `my/pkg/BUILD.bazel` file, given +`srcs=["site-packages/foo/bar.py"]`, specifying +`imports=["my/pkg/site-packages"]` means `foo/bar.py` is the file path +under the binary's venv site-packages directory that should be made available (i.e. +`import foo.bar` will work). + +`__init__.py` files are treated specially to provide basic support for [implicit +namespace packages]( +https://packaging.python.org/en/latest/guides/packaging-namespace-packages/#native-namespace-packages). +However, the *content* of the files cannot be taken into account, merely their +presence or absence. Stated another way: [pkgutil-style namespace packages]( +https://packaging.python.org/en/latest/guides/packaging-namespace-packages/#pkgutil-style-namespace-packages) +won't be understood as namespace packages; they'll be seen as regular packages. This will +likely lead to conflicts with other targets that contribute to the namespace. + +:::{seealso} +This attributes populates {obj}`PyInfo.venv_symlinks`. +::: + +:::{versionadded} 1.4.0 +::: +:::{versionchanged} 1.5.0 +The topological order has been removed and if 2 different versions of the same PyPI +package are observed, the behaviour has no guarantees except that it is deterministic +and that only one package version will be included. +::: +""", + ), + "namespace_package_files": lambda: attrb.LabelList( + allow_empty = True, + allow_files = True, + doc = """ +Files whose directories are namespace packages. + +When {obj}`--venvs_site_packages=yes` is set, this helps inform which directories should be +treated as namespace packages and expect files from other targets to be contributed. +This allows optimizing the generation of symlinks to be cheaper at analysis time. +:::{versionadded} 1.8.0 +::: +""", + ), + "_add_srcs_to_runfiles_flag": lambda: attrb.Label( + default = labels.ADD_SRCS_TO_RUNFILES, ), }, ) -def py_library_impl(ctx, *, semantics): +def py_library_impl(ctx): """Abstract implementation of py_library rule. Args: ctx: The rule ctx - semantics: A `LibrarySemantics` struct; see `create_library_semantics_struct` Returns: A list of modern providers to propagate. """ direct_sources = filter_to_py_srcs(ctx.files.srcs) - precompile_result = semantics.maybe_precompile(ctx, direct_sources) + precompile_result = maybe_precompile(ctx, direct_sources) required_py_files = precompile_result.keep_srcs required_pyc_files = [] @@ -99,24 +155,23 @@ def py_library_impl(ctx, *, semantics): runfiles.add(collect_runfiles(ctx)) runfiles = runfiles.build(ctx) - cc_info = semantics.get_cc_info_for_library(ctx) - py_info, deps_transitive_sources, builtins_py_info = create_py_info( + imports = [] + venv_symlinks = [] + + imports, venv_symlinks = _get_imports_and_venv_symlinks(ctx) + + cc_info = collect_cc_info(ctx) + py_info, builtins_py_info = create_py_info( ctx, + original_sources = direct_sources, required_py_files = required_py_files, required_pyc_files = required_pyc_files, implicit_pyc_files = implicit_pyc_files, implicit_pyc_source_files = implicit_pyc_source_files, - imports = collect_imports(ctx, semantics), + imports = imports, + venv_symlinks = venv_symlinks, ) - # TODO(b/253059598): Remove support for extra actions; https://github.com/bazelbuild/bazel/issues/16455 - listeners_enabled = _py_builtins.are_action_listeners_enabled(ctx) - if listeners_enabled: - _py_builtins.add_py_extra_pseudo_action( - ctx = ctx, - dependency_transitive_python_sources = deps_transitive_sources, - ) - providers = [ DefaultInfo(files = default_outputs, runfiles = runfiles), py_info, @@ -144,29 +199,102 @@ Source files are no longer added to the runfiles directly. ::: """ -def create_py_library_rule(*, attrs = {}, **kwargs): - """Creates a py_library rule. +def _get_package_and_version(ctx): + """Return package name and version - Args: - attrs: dict of rule attributes. - **kwargs: Additional kwargs to pass onto the rule() call. - Returns: - A rule object + If the package comes from PyPI then it will have a `.dist-info` as part of `data`, which + allows us to get the name of the package and its version. """ + dist_info_metadata = None + for d in ctx.files.data: + # work on case insensitive FSes + if d.basename.lower() != "metadata": + continue + + if d.dirname.endswith(".dist-info"): + dist_info_metadata = d + + if not dist_info_metadata: + return None, None + + # in order to be able to have replacements in the venv, we have to add a + # third value into the venv_symlinks, which would be the normalized + # package name. This allows us to ensure that we can replace the `dist-info` + # directories by checking if the package key is there. + dist_info_dir = paths.basename(dist_info_metadata.dirname) + package, _, _suffix = dist_info_dir.rpartition(".dist-info") + package, _, version_str = package.rpartition("-") + return ( + normalize_name(package), # will have no dashes + version.normalize(version_str), # will have no dashes either + ) + +def _get_imports_and_venv_symlinks(ctx): + imports = depset() + venv_symlinks = [] + if VenvsSitePackages.is_enabled(ctx): + package, version_str = _get_package_and_version(ctx) + + # NOTE: Already a list, but buildifier thinks its a depset and + # adds to_list() calls later. + imports = list(ctx.attr.imports) + if len(imports) == 0: + fail("When venvs_site_packages is enabled, exactly one `imports` " + + "value must be specified, got 0") + elif len(imports) > 1: + fail("When venvs_site_packages is enabled, exactly one `imports` " + + "value must be specified, got {}".format(imports)) - # Within Google, the doc attribute is overridden - kwargs.setdefault("doc", _DEFAULT_PY_LIBRARY_DOC) + site_packages_root = paths.normalize(paths.join( + ctx.label.package, + imports[0], + )) - # TODO: b/253818097 - fragments=py is only necessary so that - # RequiredConfigFragmentsTest passes - fragments = kwargs.pop("fragments", None) or [] - kwargs["exec_groups"] = REQUIRED_EXEC_GROUPS | (kwargs.get("exec_groups") or {}) - return rule( - attrs = dicts.add(LIBRARY_ATTRS, attrs), + # Prevent escaping out of the repo root. + if site_packages_root.startswith("../") or site_packages_root == "..": + fail(("Invalid `imports` value '{}': resolves to '{}' which is " + + "above the repo root").format( + imports[0], + site_packages_root, + )) + venv_symlinks = get_venv_symlinks( + ctx, + ctx.files.srcs + ctx.files.data + ctx.files.pyi_srcs, + package, + version_str, + site_packages_root = site_packages_root, + namespace_package_files = ctx.files.namespace_package_files, + ) + else: + imports = collect_imports(ctx) + return imports, venv_symlinks + +_MaybeBuiltinPyInfo = [BuiltinPyInfo] if BuiltinPyInfo != None else [] + +# NOTE: Exported publicaly +def create_py_library_rule_builder(): + """Create a rule builder for a py_library. + + :::{include} /_includes/volatile_api.md + ::: + + :::{versionadded} 1.3.0 + ::: + + Returns: + {obj}`ruleb.Rule` with the necessary settings + for creating a `py_library` rule. + """ + builder = ruleb.Rule( + implementation = py_library_impl, + doc = _DEFAULT_PY_LIBRARY_DOC, + exec_groups = dict(REQUIRED_EXEC_GROUP_BUILDERS), + attrs = LIBRARY_ATTRS, + fragments = ["py"], + provides = [PyCcLinkParamsInfo, PyInfo] + _MaybeBuiltinPyInfo, toolchains = [ - config_common.toolchain_type(TOOLCHAIN_TYPE, mandatory = False), - config_common.toolchain_type(EXEC_TOOLS_TOOLCHAIN_TYPE, mandatory = False), + ruleb.ToolchainType(TOOLCHAIN_TYPE, mandatory = False), + ruleb.ToolchainType(EXEC_TOOLS_TOOLCHAIN_TYPE, mandatory = False), ], - fragments = fragments + ["py"], - **kwargs ) + return builder diff --git a/python/private/py_library_rule.bzl b/python/private/py_library_rule.bzl index ed64716122..ac256bccc1 100644 --- a/python/private/py_library_rule.bzl +++ b/python/private/py_library_rule.bzl @@ -13,20 +13,6 @@ # limitations under the License. """Implementation of py_library rule.""" -load(":common.bzl", "create_library_semantics_struct") -load(":common_bazel.bzl", "collect_cc_info", "get_imports", "maybe_precompile") -load(":py_library.bzl", "create_py_library_rule", "py_library_impl") +load(":py_library.bzl", "create_py_library_rule_builder") -def _py_library_impl_with_semantics(ctx): - return py_library_impl( - ctx, - semantics = create_library_semantics_struct( - get_imports = get_imports, - maybe_precompile = maybe_precompile, - get_cc_info_for_library = collect_cc_info, - ), - ) - -py_library = create_py_library_rule( - implementation = _py_library_impl_with_semantics, -) +py_library = create_py_library_rule_builder().build() diff --git a/python/private/py_package.bzl b/python/private/py_package.bzl index fd8bc2724c..d23276a53b 100644 --- a/python/private/py_package.bzl +++ b/python/private/py_package.bzl @@ -34,17 +34,15 @@ def _path_inside_wheel(input_file): def _py_package_impl(ctx): inputs = builders.DepsetBuilder() - py_info = PyInfoBuilder() + py_info = PyInfoBuilder.new() for dep in ctx.attr.deps: inputs.add(dep[DefaultInfo].data_runfiles.files) inputs.add(dep[DefaultInfo].default_runfiles.files) py_info.merge_target(dep) py_info = py_info.build() inputs.add(py_info.transitive_sources) - - # Remove conditional once Bazel 6 support dropped. - if hasattr(py_info, "transitive_pyc_files"): - inputs.add(py_info.transitive_pyc_files) + inputs.add(py_info.transitive_pyc_files) + inputs.add(py_info.transitive_pyi_files) inputs = inputs.build() diff --git a/python/private/py_repositories.bzl b/python/private/py_repositories.bzl index 46ca903df4..3489613a6c 100644 --- a/python/private/py_repositories.bzl +++ b/python/private/py_repositories.bzl @@ -24,40 +24,65 @@ load(":pythons_hub.bzl", "hub_repo") def http_archive(**kwargs): maybe(_http_archive, **kwargs) -def py_repositories(): +def py_repositories(transition_settings = []): """Runtime dependencies that users must install. This function should be loaded and called in the user's `WORKSPACE`. With `bzlmod` enabled, this function is not needed since `MODULE.bazel` handles transitive deps. + + Args: + transition_settings: A list of labels that terminal rules transition on + by default. """ + + # NOTE: The @rules_python_internal repo is special cased by Bazel: it + # has autoloading disabled. This allows the rules to load from it + # without triggering recursion. maybe( internal_config_repo, name = "rules_python_internal", + transition_settings = transition_settings, ) maybe( hub_repo, name = "pythons_hub", minor_mapping = MINOR_MAPPING, default_python_version = "", - toolchain_prefixes = [], - toolchain_python_versions = [], - toolchain_set_python_version_constraints = [], - toolchain_user_repository_names = [], python_versions = sorted(TOOL_VERSIONS.keys()), + toolchain_names = [], + toolchain_repo_names = {}, + toolchain_target_compatible_with_map = {}, + toolchain_target_settings_map = {}, + toolchain_platform_keys = {}, + toolchain_python_versions = {}, + toolchain_set_python_version_constraints = {}, + host_compatible_repo_names = [], ) http_archive( name = "bazel_skylib", - sha256 = "d00f1389ee20b60018e92644e0948e16e350a7707219e7a390fb0a99b6ec9262", + sha256 = "6e78f0e57de26801f6f564fa7c4a48dc8b36873e416257a92bbb0937eeac8446", urls = [ - "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.7.0/bazel-skylib-1.7.0.tar.gz", - "https://github.com/bazelbuild/bazel-skylib/releases/download/1.7.0/bazel-skylib-1.7.0.tar.gz", + "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.8.2/bazel-skylib-1.8.2.tar.gz", + "https://github.com/bazelbuild/bazel-skylib/releases/download/1.8.2/bazel-skylib-1.8.2.tar.gz", ], ) http_archive( name = "rules_cc", - sha256 = "4b12149a041ddfb8306a8fd0e904e39d673552ce82e4296e96fac9cbf0780e59", - strip_prefix = "rules_cc-0.1.0", - urls = ["https://github.com/bazelbuild/rules_cc/releases/download/0.1.0/rules_cc-0.1.0.tar.gz"], + sha256 = "b8b918a85f9144c01f6cfe0f45e4f2838c7413961a8ff23bc0c6cdf8bb07a3b6", + strip_prefix = "rules_cc-0.1.5", + urls = ["https://github.com/bazelbuild/rules_cc/releases/download/0.1.5/rules_cc-0.1.5.tar.gz"], + ) + http_archive( + name = "toml.bzl", + sha256 = "63633c762bdb4d836add5a9a81deeeae93c4b2edbd62ac032733020b5652b90a", + strip_prefix = "toml.bzl-0.4.0", + url = "https://github.com/jvolkman/toml.bzl/releases/download/v0.4.0/toml.bzl-v0.4.0.tar.gz", + ) + http_archive( + name = "package_metadata", + sha256 = "8f27dc7393e3f3bdc793bdc4ba36d67a63c22cc9d38cc65d3204654974ea4563", + strip_prefix = "supply-chain-0.0.7/metadata", + url = "https://github.com/bazel-contrib/supply-chain/releases/download/v0.0.7/supply-chain-v0.0.7.tar.gz", ) # Needed by rules_cc, triggered by @rules_java_prebuilt in Bazel by using @rules_cc//cc:defs.bzl diff --git a/python/private/py_runtime_info.bzl b/python/private/py_runtime_info.bzl index 34be0db69b..daf01e6c7e 100644 --- a/python/private/py_runtime_info.bzl +++ b/python/private/py_runtime_info.bzl @@ -13,12 +13,8 @@ # limitations under the License. """Providers for Python rules.""" -load(":util.bzl", "define_bazel_6_provider") - DEFAULT_STUB_SHEBANG = "#!/usr/bin/env python3" -DEFAULT_BOOTSTRAP_TEMPLATE = Label("//python/private:bootstrap_template") - _PYTHON_VERSION_VALUES = ["PY2", "PY3"] def _optional_int(value): @@ -59,6 +55,7 @@ def _PyRuntimeInfo_init( interpreter_path = None, interpreter = None, files = None, + interpreter_files_to_run = None, coverage_tool = None, coverage_files = None, pyc_tag = None, @@ -69,13 +66,24 @@ def _PyRuntimeInfo_init( stage2_bootstrap_template = None, zip_main_template = None, abi_flags = "", - site_init_template = None): + site_init_template = None, + supports_build_time_venv = True, + venv_bin_files = None): if (interpreter_path and interpreter) or (not interpreter_path and not interpreter): fail("exactly one of interpreter or interpreter_path must be specified") if interpreter_path and files != None: fail("cannot specify 'files' if 'interpreter_path' is given") + if interpreter_path and interpreter_files_to_run: + fail("cannot specify 'interpreter_files_to_run' if 'interpreter_path' is given") + + if interpreter_files_to_run: + if not interpreter_files_to_run.executable: + fail("'interpreter_files_to_run' must have an executable") + if interpreter_files_to_run.executable != interpreter: + fail("'interpreter_files_to_run.executable' must match 'interpreter'") + if (coverage_tool and not coverage_files) or (not coverage_tool and coverage_files): fail( "coverage_tool and coverage_files must both be set or neither must be set, " + @@ -114,6 +122,7 @@ def _PyRuntimeInfo_init( "files": files, "implementation_name": implementation_name, "interpreter": interpreter, + "interpreter_files_to_run": interpreter_files_to_run, "interpreter_path": interpreter_path, "interpreter_version_info": interpreter_version_info_struct_from_dict(interpreter_version_info), "pyc_tag": pyc_tag, @@ -121,10 +130,12 @@ def _PyRuntimeInfo_init( "site_init_template": site_init_template, "stage2_bootstrap_template": stage2_bootstrap_template, "stub_shebang": stub_shebang, + "supports_build_time_venv": supports_build_time_venv, + "venv_bin_files": venv_bin_files, "zip_main_template": zip_main_template, } -PyRuntimeInfo, _unused_raw_py_runtime_info_ctor = define_bazel_6_provider( +PyRuntimeInfo, _unused_raw_py_runtime_info_ctor = provider( doc = """Contains information about a Python runtime, as returned by the `py_runtime` rule. @@ -147,7 +158,7 @@ the same conventions as the standard CPython interpreter. The runtime's ABI flags, i.e. `sys.abiflags`. -:::{versionadded} 0.41.0 +:::{versionadded} 1.0.0 ::: """, "bootstrap_template": """ @@ -239,6 +250,19 @@ The Python implementation name (`sys.implementation.name`) If this is an in-build runtime, this field is a `File` representing the interpreter. Otherwise, this is `None`. Note that an in-build runtime can use either a prebuilt, checked-in interpreter or an interpreter built from source. +""", + "interpreter_files_to_run": """ +:type: None | FilesToRunProvider + +The `FilesToRunProvider` for the interpreter target when this runtime was +created from an executable target. This includes the interpreter executable and +the runfiles metadata needed to use it as an action tool. Rules that execute the +interpreter in an action should use this field so Bazel can stage the +interpreter together with its runfiles. This is `None` for platform runtimes +using `interpreter_path` and for file-only interpreter targets. + +:::{versionadded} 2.1.0 +::: """, "interpreter_path": """ :type: str | None @@ -281,7 +305,7 @@ are (only) `"PY2"` and `"PY3"`. The template to use for the binary-specific site-init hook run by the interpreter at startup. -:::{versionadded} 0.41.0 +:::{versionadded} 1.0.0 ::: """, "stage2_bootstrap_template": """ @@ -314,6 +338,36 @@ The following substitutions are made during template expansion: "Shebang" expression prepended to the bootstrapping Python stub script used when executing {obj}`py_binary` targets. Does not apply to Windows. +""", + "supports_build_time_venv": """ +:type: bool + +True if this toolchain supports the build-time created virtual environment. +False if not or unknown. If build-time venv creation isn't supported, then binaries may +fallback to non-venv solutions or creating a venv at runtime. + +In order to use the build-time created virtual environment, a toolchain needs +to meet two criteria: +1. Specifying the underlying executable (e.g. `/usr/bin/python3`, as reported by + `sys._base_executable`) for the venv executable (`$venv/bin/python3`, as reported + by `sys.executable`). This typically requires relative symlinking the venv + path to the underlying path at build time, or using the `PYTHONEXECUTABLE` + environment variable (Python 3.11+) at runtime. +2. Having the build-time created site-packages directory + (`/lib/python{version}/site-packages`) recognized by the runtime + interpreter. This typically requires the Python version to be known at + build-time and match at runtime. + +:::{versionadded} 1.5.0 +::: +""", + "venv_bin_files": """ +:type: list[File] + +Files that should be added to the venv's `bin/` (or platform-specific equivalent) +directory (using the file's basename). + +:::{versionadded} 2.0.0 """, "zip_main_template": """ :type: File diff --git a/python/private/py_runtime_pair_rule.bzl b/python/private/py_runtime_pair_rule.bzl index b3b7a4e5f8..c6c4c34d19 100644 --- a/python/private/py_runtime_pair_rule.bzl +++ b/python/private/py_runtime_pair_rule.bzl @@ -16,8 +16,8 @@ load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load("//python:py_runtime_info.bzl", "PyRuntimeInfo") +load(":common_labels.bzl", "labels") load(":reexports.bzl", "BuiltinPyRuntimeInfo") -load(":util.bzl", "IS_BAZEL_7_OR_HIGHER") def _py_runtime_pair_impl(ctx): if ctx.attr.py2_runtime != None: @@ -36,10 +36,9 @@ def _py_runtime_pair_impl(ctx): else: py3_runtime = None - # TODO: Uncomment this after --incompatible_python_disable_py2 defaults to true - # if _is_py2_disabled(ctx) and py2_runtime != None: - # fail("Using Python 2 is not supported and disabled; see " + - # "https://github.com/bazelbuild/bazel/issues/15684") + if py2_runtime != None: + fail("Using Python 2 is not supported and disabled; see " + + "https://github.com/bazelbuild/bazel/issues/15684") extra_kwargs = {} if ctx.attr._visible_for_testing[BuildSettingInfo].value: @@ -56,20 +55,11 @@ def _get_py_runtime_info(target): # py_binary (implemented in Java) performs a type check on the provider # value to verify it is an instance of the Java-implemented PyRuntimeInfo # class. - if (IS_BAZEL_7_OR_HIGHER and PyRuntimeInfo in target) or BuiltinPyRuntimeInfo == None: + if (PyRuntimeInfo in target) or BuiltinPyRuntimeInfo == None: return target[PyRuntimeInfo] else: return target[BuiltinPyRuntimeInfo] -# buildifier: disable=unused-variable -def _is_py2_disabled(ctx): - # Because this file isn't bundled with Bazel, so we have to conditionally - # check for this flag. - # TODO: Remove this once all supported Balze versions have this flag. - if not hasattr(ctx.fragments.py, "disable_py"): - return False - return ctx.fragments.py.disable_py2 - _MaybeBuiltinPyRuntimeInfo = [[BuiltinPyRuntimeInfo]] if BuiltinPyRuntimeInfo != None else [] py_runtime_pair = rule( @@ -94,7 +84,7 @@ The runtime to use for Python 3 targets. Must have `python_version` set to """, ), "_visible_for_testing": attr.label( - default = "//python/private:visible_for_testing", + default = labels.VISIBLE_FOR_TESTING, ), }, fragments = ["py"], diff --git a/python/private/py_runtime_rule.bzl b/python/private/py_runtime_rule.bzl index 5ce8161cf0..4f450c4c9b 100644 --- a/python/private/py_runtime_rule.bzl +++ b/python/private/py_runtime_rule.bzl @@ -16,15 +16,50 @@ load("@bazel_skylib//lib:dicts.bzl", "dicts") load("@bazel_skylib//lib:paths.bzl", "paths") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") -load(":attributes.bzl", "NATIVE_RULES_ALLOWLIST_ATTRS") +load(":common_labels.bzl", "labels") load(":flags.bzl", "FreeThreadedFlag") load(":py_internal.bzl", "py_internal") -load(":py_runtime_info.bzl", "DEFAULT_BOOTSTRAP_TEMPLATE", "DEFAULT_STUB_SHEBANG", "PyRuntimeInfo") +load(":py_runtime_info.bzl", "DEFAULT_STUB_SHEBANG", "PyRuntimeInfo") load(":reexports.bzl", "BuiltinPyRuntimeInfo") -load(":util.bzl", "IS_BAZEL_7_OR_HIGHER") +load(":version.bzl", "version") _py_builtins = py_internal +def coverage_tool_missing_message(*, coverage_enabled, coverage_tool, label): + """Build the warning for a selected runtime that cannot produce coverage. + + Kept separate from the rule implementation so the decision is unit testable + without having to capture analysis-phase output. + + Args: + coverage_enabled: {type}`bool` whether the build is collecting coverage. + coverage_tool: {type}`File | None` the runtime's coverage entry point. + label: {type}`Label` the `py_runtime` being analyzed. + + Returns: + {type}`str | None` the message to print, or `None` when no warning is + warranted. + """ + if not coverage_enabled or coverage_tool: + return None + + return """ +====================================================================== +WARNING: Python runtime {label} has no coverage_tool. + `bazel coverage` will produce empty lcov data for py_test targets that + resolve to this runtime. + + For rules_python's hermetic toolchains, enable the bundled coverage.py: + python.toolchain(configure_coverage_tool = True) # bzlmod + python_register_toolchains(register_coverage_tool = True) # WORKSPACE + A bundled wheel must exist for this interpreter's version and platform; + python/private/coverage_deps.bzl lists what ships with rules_python. + + Otherwise, set py_runtime.coverage_tool directly. See + https://rules-python.readthedocs.io/en/latest/coverage.html +====================================================================== +""".format(label = label) + def _py_runtime_impl(ctx): interpreter_path = ctx.attr.interpreter_path or None # Convert empty string to None interpreter = ctx.attr.interpreter @@ -39,6 +74,7 @@ def _py_runtime_impl(ctx): runfiles = ctx.runfiles() hermetic = bool(interpreter) + interpreter_files_to_run = None if not hermetic: if runtime_files: fail("if 'interpreter_path' is given then 'files' must be empty") @@ -46,9 +82,35 @@ def _py_runtime_impl(ctx): fail("interpreter_path must be an absolute path") else: interpreter_di = interpreter[DefaultInfo] + interpreter_file = None + + if _is_singleton_depset(interpreter_di.files): + interpreter_file = interpreter_di.files.to_list()[0] - if interpreter_di.files_to_run and interpreter_di.files_to_run.executable: + is_executable_source_file = ( + interpreter_file and + interpreter_file.is_source and + interpreter_di.files_to_run and + interpreter_di.files_to_run.executable == interpreter_file + ) + + if is_executable_source_file: + # Source files Bazel treats as executable, e.g. direct file labels + # and filegroups: preserve historical runtime-file expansion, but + # do not expose a FilesToRunProvider. + interpreter = interpreter_file + runfiles = runfiles.merge(interpreter_di.default_runfiles) + + runtime_files = depset(transitive = [ + interpreter_di.files, + interpreter_di.default_runfiles.files, + runtime_files, + ]) + elif interpreter_di.files_to_run and interpreter_di.files_to_run.executable: + # Executable rule target: use the executable and preserve the full + # FilesToRunProvider so action consumers can stage its runfiles. interpreter = interpreter_di.files_to_run.executable + interpreter_files_to_run = interpreter_di.files_to_run runfiles = runfiles.merge(interpreter_di.default_runfiles) runtime_files = depset(transitive = [ @@ -56,8 +118,10 @@ def _py_runtime_impl(ctx): interpreter_di.default_runfiles.files, runtime_files, ]) - elif _is_singleton_depset(interpreter_di.files): - interpreter = interpreter_di.files.to_list()[0] + elif interpreter_file: + # Non-executable rule with exactly one output: preserve the + # historical file-only interpreter behavior. + interpreter = interpreter_file else: fail("interpreter must be an executable target or must produce exactly one file.") @@ -79,6 +143,19 @@ def _py_runtime_impl(ctx): coverage_tool = None coverage_files = None + # Reported here rather than where the toolchains are registered: this rule is + # analyzed once per configuration, and only for the runtime that toolchain + # resolution actually selected, so the empty-lcov outcome is real rather than + # hypothetical. See https://github.com/bazel-contrib/rules_python/issues/3950. + coverage_warning = coverage_tool_missing_message( + coverage_enabled = ctx.configuration.coverage_enabled, + coverage_tool = coverage_tool, + label = ctx.label, + ) + if coverage_warning: + # buildifier: disable=print + print(coverage_warning) + python_version = ctx.attr.python_version interpreter_version_info = ctx.attr.interpreter_version_info @@ -87,10 +164,9 @@ def _py_runtime_impl(ctx): if python_version_flag: interpreter_version_info = _interpreter_version_info_from_version_str(python_version_flag) - # TODO: Uncomment this after --incompatible_python_disable_py2 defaults to true - # if ctx.fragments.py.disable_py2 and python_version == "PY2": - # fail("Using Python 2 is not supported and disabled; see " + - # "https://github.com/bazelbuild/bazel/issues/15684") + if python_version == "PY2": + fail("Using Python 2 is not supported and disabled; see " + + "https://github.com/bazelbuild/bazel/issues/15684") pyc_tag = ctx.attr.pyc_tag if not pyc_tag and (ctx.attr.implementation_name and @@ -125,16 +201,16 @@ def _py_runtime_impl(ctx): py_runtime_info_kwargs.update(dict( implementation_name = ctx.attr.implementation_name, interpreter_version_info = interpreter_version_info, + interpreter_files_to_run = interpreter_files_to_run, pyc_tag = pyc_tag, stage2_bootstrap_template = ctx.file.stage2_bootstrap_template, zip_main_template = ctx.file.zip_main_template, abi_flags = abi_flags, site_init_template = ctx.file.site_init_template, + supports_build_time_venv = ctx.attr.supports_build_time_venv, + venv_bin_files = ctx.files.venv_bin_files, )) - if not IS_BAZEL_7_OR_HIGHER: - builtin_py_runtime_info_kwargs.pop("bootstrap_template") - providers = [ PyRuntimeInfo(**py_runtime_info_kwargs), DefaultInfo( @@ -188,19 +264,20 @@ py_runtime( ``` """, fragments = ["py"], - attrs = dicts.add(NATIVE_RULES_ALLOWLIST_ATTRS, { - "abi_flags": attr.string( - default = "", - doc = """ + attrs = dicts.add( + { + "abi_flags": attr.string( + default = "", + doc = """ The runtime's ABI flags, i.e. `sys.abiflags`. If not set, then it will be set based on flags. """, - ), - "bootstrap_template": attr.label( - allow_single_file = True, - default = DEFAULT_BOOTSTRAP_TEMPLATE, - doc = """ + ), + "bootstrap_template": attr.label( + allow_single_file = True, + default = Label("//python/private:bootstrap_template"), + doc = """ The bootstrap script template file to use. Should have %python_binary%, %workspace_name%, %main%, and %imports%. @@ -218,10 +295,10 @@ itself. See @bazel_tools//tools/python:python_bootstrap_template.txt for more variables. """, - ), - "coverage_tool": attr.label( - allow_files = False, - doc = """ + ), + "coverage_tool": attr.label( + allow_files = False, + doc = """ This is a target to use for collecting code coverage information from {rule}`py_binary` and {rule}`py_test` targets. @@ -235,25 +312,25 @@ The entry point for the tool must be loadable by a Python interpreter (e.g. a of [`coverage.py`](https://coverage.readthedocs.io), at least including the `run` and `lcov` subcommands. """, - ), - "files": attr.label_list( - allow_files = True, - doc = """ + ), + "files": attr.label_list( + allow_files = True, + doc = """ For an in-build runtime, this is the set of files comprising this runtime. These files will be added to the runfiles of Python binaries that use this runtime. For a platform runtime this attribute must not be set. """, - ), - "implementation_name": attr.string( - doc = "The Python implementation name (`sys.implementation.name`)", - default = "cpython", - ), - "interpreter": attr.label( - # We set `allow_files = True` to allow specifying executable - # targets from rules that have more than one default output, - # e.g. sh_binary. - allow_files = True, - doc = """ + ), + "implementation_name": attr.string( + doc = "The Python implementation name (`sys.implementation.name`)", + default = "cpython", + ), + "interpreter": attr.label( + # We set `allow_files = True` to allow specifying executable + # targets from rules that have more than one default output, + # e.g. sh_binary. + allow_files = True, + doc = """ For an in-build runtime, this is the target to invoke as the interpreter. It can be either of: @@ -267,18 +344,18 @@ can be either of: NOTE: the runfiles of the target may not yet be properly respected/propagated to consumers of the toolchain/interpreter, see - bazelbuild/rules_python/issues/1612 + bazel-contrib/rules_python/issues/1612 For a platform runtime (i.e. `interpreter_path` being set) this attribute must not be set. """, - ), - "interpreter_path": attr.string(doc = """ + ), + "interpreter_path": attr.string(doc = """ For a platform runtime, this is the absolute path of a Python interpreter on the target platform. For an in-build runtime this attribute must not be set. """), - "interpreter_version_info": attr.string_dict( - doc = """ + "interpreter_version_info": attr.string_dict( + doc = """ Version information about the interpreter this runtime provides. If not specified, uses {obj}`--python_version` @@ -295,20 +372,20 @@ values are strings, most are converted to ints. The supported keys are: {obj}`--python_version` determines the default value. ::: """, - mandatory = False, - ), - "pyc_tag": attr.string( - doc = """ + mandatory = False, + ), + "pyc_tag": attr.string( + doc = """ Optional string; the tag portion of a pyc filename, e.g. the `cpython-39` infix of `foo.cpython-39.pyc`. See PEP 3147. If not specified, it will be computed from `implementation_name` and `interpreter_version_info`. If no pyc_tag is available, then only source-less pyc generation will function correctly. """, - ), - "python_version": attr.string( - default = "PY3", - values = ["PY2", "PY3"], - doc = """ + ), + "python_version": attr.string( + default = "PY3", + values = ["PY2", "PY3"], + doc = """ Whether this runtime is for Python major version 2 or 3. Valid values are `"PY2"` and `"PY3"`. @@ -316,45 +393,57 @@ The default value is controlled by the `--incompatible_py3_is_default` flag. However, in the future this attribute will be mandatory and have no default value. """, - ), - "site_init_template": attr.label( - allow_single_file = True, - default = "//python/private:site_init_template", - doc = """ + ), + "site_init_template": attr.label( + allow_single_file = True, + default = "//python/private:site_init_template", + doc = """ The template to use for the binary-specific site-init hook run by the interpreter at startup. :::{versionadded} 0.41.0 ::: """, - ), - "stage2_bootstrap_template": attr.label( - default = "//python/private:stage2_bootstrap_template", - allow_single_file = True, - doc = """ + ), + "stage2_bootstrap_template": attr.label( + default = "//python/private:stage2_bootstrap_template", + allow_single_file = True, + doc = """ The template to use when two stage bootstrapping is enabled :::{seealso} {obj}`PyRuntimeInfo.stage2_bootstrap_template` and {obj}`--bootstrap_impl` ::: """, - ), - "stub_shebang": attr.string( - default = DEFAULT_STUB_SHEBANG, - doc = """ + ), + "stub_shebang": attr.string( + default = DEFAULT_STUB_SHEBANG, + doc = """ "Shebang" expression prepended to the bootstrapping Python stub script used when executing {rule}`py_binary` targets. -See https://github.com/bazelbuild/bazel/issues/8685 for +See [bazelbuild/bazel#8685](https://github.com/bazelbuild/bazel/issues/8685) for motivation. Does not apply to Windows. """, - ), - "zip_main_template": attr.label( - default = "//python/private:zip_main_template", - allow_single_file = True, - doc = """ + ), + "supports_build_time_venv": attr.bool( + doc = """ +Whether this runtime supports virtualenvs created at build time. + +See {obj}`PyRuntimeInfo.supports_build_time_venv` for docs. + +:::{versionadded} 1.5.0 +::: +""", + default = True, + ), + "venv_bin_files": attr.label_list(allow_files = True), + "zip_main_template": attr.label( + default = "//python/private/zipapp:zip_main_template", + allow_single_file = True, + doc = """ The template to use for a zip's top-level `__main__.py` file. This becomes the entry point executed when `python foo.zip` is run. @@ -363,29 +452,26 @@ This becomes the entry point executed when `python foo.zip` is run. The {obj}`PyRuntimeInfo.zip_main_template` field. ::: """, - ), - "_py_freethreaded_flag": attr.label( - default = "//python/config_settings:py_freethreaded", - ), - "_python_version_flag": attr.label( - default = "//python/config_settings:python_version", - ), - }), + ), + "_py_freethreaded_flag": attr.label( + default = labels.PY_FREETHREADED, + ), + "_python_version_flag": attr.label( + default = labels.PYTHON_VERSION, + ), + }, + ), ) def _is_singleton_depset(files): - # Bazel 6 doesn't have this helper to optimize detecting singleton depsets. - if _py_builtins: - return _py_builtins.is_singleton_depset(files) - else: - return len(files.to_list()) == 1 + return _py_builtins.is_singleton_depset(files) def _interpreter_version_info_from_version_str(version_str): - parts = version_str.split(".") + v = version.parse(version_str) version_info = {} + parts = list(v.release) for key in ("major", "minor", "micro"): if not parts: break version_info[key] = parts.pop(0) - return version_info diff --git a/python/private/py_test_macro.bzl b/python/private/py_test_macro.bzl index 1f9330f8e5..bc58f859f8 100644 --- a/python/private/py_test_macro.bzl +++ b/python/private/py_test_macro.bzl @@ -13,9 +13,12 @@ # limitations under the License. """Implementation of macro-half of py_test rule.""" -load(":common_bazel.bzl", "convert_legacy_create_init_to_int") +load(":py_executable.bzl", "common_executable_macro_kwargs_setup") load(":py_test_rule.bzl", py_test_rule = "py_test") def py_test(**kwargs): - convert_legacy_create_init_to_int(kwargs) - py_test_rule(**kwargs) + py_test_macro(py_test_rule, **kwargs) + +def py_test_macro(py_rule, **kwargs): + common_executable_macro_kwargs_setup(kwargs) + py_rule(**kwargs) diff --git a/python/private/py_test_main_validator.py b/python/private/py_test_main_validator.py new file mode 100644 index 0000000000..e3849c5f57 --- /dev/null +++ b/python/private/py_test_main_validator.py @@ -0,0 +1,224 @@ +"""Static check that a py_test main module actually runs something. + +A common ``py_test`` pitfall is to define test classes or functions but forget +to add any code that actually executes them (for example, assuming that +``py_test`` automatically invokes ``unittest`` or ``pytest``). When that +happens, running the test does nothing and the target silently passes. + +This validator parses the main module with :mod:`ast` and fails if the +module body is "inert", i.e. every top-level statement is one that does not +run anything (definitions, imports, assignments, docstrings, ``pass``). A +single active statement -- a bare call, a loop, or the conventional +``if __name__ == "__main__":`` guard -- is enough to consider the module able +to run tests. Top-level ``if`` and ``try`` blocks are inspected recursively, so +common guards like ``if TYPE_CHECKING:`` or ``try: import foo except +ImportError:`` (whose branches are themselves inert) don't bypass the check. + +As an exception, a module that defines no classes or functions at all (for +example, one that only imports other modules) is always allowed: it isn't the +"defined some tests but forgot to run them" case this check targets, and it +may legitimately rely on import side effects. +""" + +import argparse +import ast +import sys + +# Statement node types that never run any code on their own, regardless of +# their contents. A module whose top-level body consists solely of these (and +# inert assignments/expressions/guards, see below) is considered inert. +_INERT_NODE_TYPES = [ + ast.FunctionDef, + ast.AsyncFunctionDef, + ast.ClassDef, + ast.Import, + ast.ImportFrom, + ast.Global, + ast.Pass, +] + +# `ast.TypeAlias` (PEP 695, e.g. `type Alias = int`) only exists on Python +# 3.12+. Add it dynamically so the validator still imports on older versions. +if hasattr(ast, "TypeAlias"): + _INERT_NODE_TYPES.append(ast.TypeAlias) + +_INERT_NODE_TYPES = tuple(_INERT_NODE_TYPES) + +# `ast.TryStar` (PEP 654, `try/except*`) only exists on Python 3.11+. +_TRY_NODE_TYPES = (ast.Try, ast.TryStar) if hasattr(ast, "TryStar") else (ast.Try,) + +# Expression node types whose evaluation runs code (and thus may run tests). +# `await`/`yield` can't appear at the module top level, but are included for +# completeness when walking nested expressions. +_ACTIVE_EXPR_NODE_TYPES = (ast.Call, ast.Await, ast.Yield, ast.YieldFrom) + + +def _expression_runs_code(value) -> bool: + """Returns True if evaluating the expression would invoke/await/yield. + + Note this walks the expression as written; it does not try to model what + actually executes (e.g. a call inside a `lambda` body won't run until the + lambda is called). Erring toward "runs code" keeps the safeguard from + flagging valid tests. + """ + if value is None: + return False + return any(isinstance(child, _ACTIVE_EXPR_NODE_TYPES) for child in ast.walk(value)) + + +def _all_inert(nodes) -> bool: + return all(_is_inert_statement(node) for node in nodes) + + +def _is_inert_statement(node: ast.stmt) -> bool: + """Returns True if the top-level statement does not run any test code.""" + if isinstance(node, _INERT_NODE_TYPES): + return True + + # Assignments are inert unless their value runs code, e.g. + # `exit_code = unittest.main()` actually runs the tests. + if isinstance(node, (ast.Assign, ast.AnnAssign, ast.AugAssign)): + return not _expression_runs_code(node.value) + + # Bare expression statements: docstrings and other no-op expressions are + # inert; a call/await/yield (e.g. `unittest.main()`) runs code. + # Assert statements are inert unless their condition or message runs code. + if isinstance(node, ast.Assert): + return not (_expression_runs_code(node.test) or _expression_runs_code(node.msg)) + + if isinstance(node, ast.Expr): + return not _expression_runs_code(node.value) + + # An `if` whose condition and branches are entirely inert, e.g. the very + # common `if TYPE_CHECKING:` guard around typing-only imports. A call in the + # condition (e.g. `if feature_enabled():`) or a runner call in a branch body + # makes it active. + if isinstance(node, ast.If): + return ( + not _expression_runs_code(node.test) + and _all_inert(node.body) + and _all_inert(node.orelse) + ) + + # A `try` (or `try/except*`) whose body, handlers, else, and finally are all + # inert, e.g. the common `try: import foo except ImportError: ...` optional + # import pattern. + if isinstance(node, _TRY_NODE_TYPES): + if not ( + _all_inert(node.body) + and _all_inert(node.orelse) + and _all_inert(node.finalbody) + ): + return False + return all(_all_inert(handler.body) for handler in node.handlers) + + return False + + +def module_runs_something(tree: ast.Module) -> bool: + """Returns True if the module body has at least one active statement.""" + return any(not _is_inert_statement(node) for node in tree.body) + + +def _defines_test_code(tree: ast.Module) -> bool: + """Returns True if the module defines any top-level class or function.""" + return any( + isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)) + for node in tree.body + ) + + +def module_runs_tests(tree: ast.Module) -> bool: + """Returns True if the module appears able to run tests (or isn't checked). + + The check targets the specific pitfall of defining test classes/functions + but never running them. A module that defines no classes or functions at + all (for example, one that only imports other modules) is not subject to + the check and is always allowed, since it isn't the "defined but never run" + case and may legitimately rely on import side effects. + """ + if module_runs_something(tree): + return True + return not _defines_test_code(tree) + + +def _format_error(label: str, src_name: str) -> str: + target = label or src_name + return ( + "py_test target {target} will not run any tests.\n" + "\n" + "The main module '{src_name}' only contains inert top-level statements " + "(class/function definitions, imports, assignments). Running it does " + "nothing, so the test silently passes without executing any test " + "code.\n" + "\n" + "py_test runs the main module directly; it does not automatically " + "invoke a test runner such as unittest or pytest. Add code that runs " + "your tests, for example:\n" + "\n" + ' if __name__ == "__main__":\n' + " unittest.main()\n" + "\n" + "or use a main module that invokes a runner (e.g. pytest.main()).\n" + "\n" + "This check can be disabled by setting " + "--@rules_python//python/config_settings:validate_test_main=disabled." + ).format(target=target, src_name=src_name) + + +def main(args) -> int: + parser = argparse.ArgumentParser() + parser.add_argument( + "--src", + required=True, + help="Path to the main .py source file to analyze.", + ) + parser.add_argument( + "--src_name", + default="", + help="Human-friendly name of the source file, used in error messages.", + ) + parser.add_argument( + "--label", + default="", + help="The py_test target label, used in error messages.", + ) + parser.add_argument( + "--output", + required=True, + help="Path to the validation marker file to write on success.", + ) + options = parser.parse_args(args) + + src_name = options.src_name or options.src + + with open(options.src, "rb") as f: + source = f.read() + + try: + tree = ast.parse(source, filename=src_name) + except SyntaxError as e: + # A syntax error is surfaced by other actions (compilation/execution). + # The validator can't analyze the file, so don't fail here; treat it as + # passing to avoid duplicate or confusing errors. + sys.stderr.write( + "WARNING: py_test main validator could not parse {}: {}\n".format( + src_name, e + ) + ) + with open(options.output, "w") as out: + out.write("") + return 0 + + if not module_runs_tests(tree): + sys.stderr.write(_format_error(options.label, src_name) + "\n") + return 1 + + # Validation actions must produce their declared outputs on success. + with open(options.output, "w") as out: + out.write("") + return 0 + + +if __name__ == "__main__": + sys.exit(main(sys.argv[1:])) diff --git a/python/private/py_test_rule.bzl b/python/private/py_test_rule.bzl index 64d5f21f81..5848682deb 100644 --- a/python/private/py_test_rule.bzl +++ b/python/private/py_test_rule.bzl @@ -13,34 +13,16 @@ # limitations under the License. """Implementation of py_test rule.""" -load("@bazel_skylib//lib:dicts.bzl", "dicts") load(":attributes.bzl", "AGNOSTIC_TEST_ATTRS") load(":common.bzl", "maybe_add_test_execution_info") load( - ":py_executable_bazel.bzl", - "create_executable_rule", - "py_executable_bazel_impl", + ":py_executable.bzl", + "create_executable_rule_builder", + "py_executable_impl", ) -_BAZEL_PY_TEST_ATTRS = { - # This *might* be a magic attribute to help C++ coverage work. There's no - # docs about this; see TestActionBuilder.java - "_collect_cc_coverage": attr.label( - default = "@bazel_tools//tools/test:collect_cc_coverage", - executable = True, - cfg = "exec", - ), - # This *might* be a magic attribute to help C++ coverage work. There's no - # docs about this; see TestActionBuilder.java - "_lcov_merger": attr.label( - default = configuration_field(fragment = "coverage", name = "output_generator"), - cfg = "exec", - executable = True, - ), -} - def _py_test_impl(ctx): - providers = py_executable_bazel_impl( + providers = py_executable_impl( ctx = ctx, is_test = True, inherited_environment = ctx.attr.env_inherit, @@ -48,8 +30,25 @@ def _py_test_impl(ctx): maybe_add_test_execution_info(providers, ctx) return providers -py_test = create_executable_rule( - implementation = _py_test_impl, - attrs = dicts.add(AGNOSTIC_TEST_ATTRS, _BAZEL_PY_TEST_ATTRS), - test = True, -) +# NOTE: Exported publicaly +def create_py_test_rule_builder(): + """Create a rule builder for a py_test. + + :::{include} /_includes/volatile_api.md + ::: + + :::{versionadded} 1.3.0 + ::: + + Returns: + {obj}`ruleb.Rule` with the necessary settings + for creating a `py_test` rule. + """ + builder = create_executable_rule_builder( + implementation = _py_test_impl, + test = True, + ) + builder.attrs.update(AGNOSTIC_TEST_ATTRS) + return builder + +py_test = create_py_test_rule_builder().build() diff --git a/python/private/py_toolchain_suite.bzl b/python/private/py_toolchain_suite.bzl index a69be376b4..fa73d5daa3 100644 --- a/python/private/py_toolchain_suite.bzl +++ b/python/private/py_toolchain_suite.bzl @@ -15,6 +15,7 @@ """Create the toolchain defs in a BUILD.bazel file.""" load("@bazel_skylib//lib:selects.bzl", "selects") +load("@platforms//host:constraints.bzl", "HOST_CONSTRAINTS") load(":text_util.bzl", "render") load( ":toolchain_types.bzl", @@ -33,15 +34,20 @@ def py_toolchain_suite( python_version, set_python_version_constraint, flag_values, + target_settings = [], target_compatible_with = []): """For internal use only. Args: prefix: Prefix for toolchain target names. - user_repository_name: The name of the user repository. + user_repository_name: The name of the repository with the toolchain + implementation (it's assumed to have particular target names within + it). Does not include the leading "@". python_version: The full (X.Y.Z) version of the interpreter. set_python_version_constraint: True or False as a string. - flag_values: Extra flag values to match for this toolchain. + flag_values: Extra flag values to match for this toolchain. These + are prepended to target_settings. + target_settings: Extra target_settings to match for this toolchain. target_compatible_with: list constraints the toolchains are compatible with. """ @@ -81,7 +87,7 @@ def py_toolchain_suite( match_any = match_any, visibility = ["//visibility:private"], ) - target_settings = [name] + target_settings = [name] + target_settings else: fail(("Invalid set_python_version_constraint value: got {} {}, wanted " + "either the string 'True' or the string 'False'; " + @@ -95,9 +101,15 @@ def py_toolchain_suite( runtime_repo_name = user_repository_name, target_settings = target_settings, target_compatible_with = target_compatible_with, + exec_compatible_with = [], ) -def _internal_toolchain_suite(prefix, runtime_repo_name, target_compatible_with, target_settings): +def _internal_toolchain_suite( + prefix, + runtime_repo_name, + target_compatible_with, + target_settings, + exec_compatible_with): native.toolchain( name = "{prefix}_toolchain".format(prefix = prefix), toolchain = "@{runtime_repo_name}//:python_runtimes".format( @@ -106,6 +118,7 @@ def _internal_toolchain_suite(prefix, runtime_repo_name, target_compatible_with, toolchain_type = TARGET_TOOLCHAIN_TYPE, target_settings = target_settings, target_compatible_with = target_compatible_with, + exec_compatible_with = exec_compatible_with, ) native.toolchain( @@ -116,6 +129,7 @@ def _internal_toolchain_suite(prefix, runtime_repo_name, target_compatible_with, toolchain_type = PY_CC_TOOLCHAIN_TYPE, target_settings = target_settings, target_compatible_with = target_compatible_with, + exec_compatible_with = exec_compatible_with, ) native.toolchain( @@ -142,7 +156,13 @@ def _internal_toolchain_suite(prefix, runtime_repo_name, target_compatible_with, # call in python/repositories.bzl. Bzlmod doesn't need anything; it will # register `:all`. -def define_local_toolchain_suites(name, version_aware_repo_names, version_unaware_repo_names): +def define_local_toolchain_suites( + name, + version_aware_repo_names, + version_unaware_repo_names, + repo_exec_compatible_with, + repo_target_compatible_with, + repo_target_settings): """Define toolchains for `local_runtime_repo` backed toolchains. This generates `toolchain` targets that can be registered using `:all`. The @@ -156,24 +176,60 @@ def define_local_toolchain_suites(name, version_aware_repo_names, version_unawar version-aware toolchains defined. version_unaware_repo_names: `list[str]` of the repo names that will have version-unaware toolchains defined. + repo_target_settings: {type}`dict[str, list[str]]` mapping of repo names + to string labels that are added to the `target_settings` for the + respective repo's toolchain. + repo_target_compatible_with: {type}`dict[str, list[str]]` mapping of repo names + to string labels that are added to the `target_compatible_with` for + the respective repo's toolchain. + repo_exec_compatible_with: {type}`dict[str, list[str]]` mapping of repo names + to string labels that are added to the `exec_compatible_with` for + the respective repo's toolchain. """ + i = 0 for i, repo in enumerate(version_aware_repo_names, start = i): - prefix = render.left_pad_zero(i, 4) + target_settings = ["@{}//:is_matching_python_version".format(repo)] + + if repo_target_settings.get(repo): + selects.config_setting_group( + name = "_{}_user_guard".format(repo), + match_all = repo_target_settings.get(repo, []) + target_settings, + ) + target_settings = ["_{}_user_guard".format(repo)] _internal_toolchain_suite( - prefix = prefix, + prefix = render.left_pad_zero(i, 4), runtime_repo_name = repo, - target_compatible_with = ["@{}//:os".format(repo)], - target_settings = ["@{}//:is_matching_python_version".format(repo)], + target_compatible_with = _get_local_toolchain_target_compatible_with( + repo, + repo_target_compatible_with, + ), + target_settings = target_settings, + exec_compatible_with = repo_exec_compatible_with.get(repo, []), ) # The version unaware entries must go last because they will match any Python # version. for i, repo in enumerate(version_unaware_repo_names, start = i + 1): - prefix = render.left_pad_zero(i, 4) _internal_toolchain_suite( - prefix = prefix, + prefix = render.left_pad_zero(i, 4) + "_default", runtime_repo_name = repo, - target_settings = [], - target_compatible_with = ["@{}//:os".format(repo)], + target_compatible_with = _get_local_toolchain_target_compatible_with( + repo, + repo_target_compatible_with, + ), + # We don't call _get_local_toolchain_target_settings because that + # will add the version matching condition by default. + target_settings = repo_target_settings.get(repo, []), + exec_compatible_with = repo_exec_compatible_with.get(repo, []), ) + +def _get_local_toolchain_target_compatible_with(repo, repo_target_compatible_with): + if repo in repo_target_compatible_with: + target_compatible_with = repo_target_compatible_with[repo] + if "HOST_CONSTRAINTS" in target_compatible_with: + target_compatible_with.remove("HOST_CONSTRAINTS") + target_compatible_with.extend(HOST_CONSTRAINTS) + else: + target_compatible_with = ["@{}//:os".format(repo)] + return target_compatible_with diff --git a/python/private/py_wheel.bzl b/python/private/py_wheel.bzl index 6d047ad680..c28aefff48 100644 --- a/python/private/py_wheel.bzl +++ b/python/private/py_wheel.bzl @@ -14,9 +14,13 @@ "Implementation of py_wheel rule" +load(":attributes.bzl", "CONFIG_SETTINGS_ATTR", "apply_config_settings_attr") +load(":py_info.bzl", "PyInfo") load(":py_package.bzl", "py_package_lib") -load(":py_wheel_normalize_pep440.bzl", "normalize_pep440") -load(":stamp.bzl", "is_stamping_enabled") +load(":rule_builders.bzl", "ruleb") +load(":stamp_impl.bzl", "is_stamping_enabled") +load(":transition_labels.bzl", "TRANSITION_LABELS") +load(":version.bzl", "version") PyWheelInfo = provider( doc = "Information about a wheel produced by `py_wheel`", @@ -50,7 +54,7 @@ Workspace status keys are expanded using `{NAME}` format, for example: - `distribution = "package.{CLASSIFIER}"` - `distribution = "{DISTRIBUTION}"` -For the available keys, see https://bazel.build/docs/user-manual#workspace-status +For the available keys, see """, ), "platform": attr.string( @@ -166,6 +170,23 @@ entry_points, e.g. `{'console_scripts': ['main = examples.wheel.main:main']}`. } _other_attrs = { + "add_path_prefix": attr.string( + default = "", + doc = """\ +Path prefix to prepend to files added to the generated package. +This prefix will be prepended **after** the paths are first stripped of the prefixes +specified in `strip_path_prefixes`. + +For example: ++ `"foo/" will prepend to `"bar/baz/file.py"` as `"foo/bar/baz/file.py"` ++ `"foo_" will prepend to `"bar/baz/file.py"` as `"foo_bar/baz/file.py"` ++ `stripping ["bar/"] and adding "foo/" will change `"bar/baz/file.py"` to `"foo/baz/file.py"` + +:::{versionadded} 2.0.0 +The {attr}`add_path_prefix` attribute was added. +::: +""", + ), "author": attr.string( doc = "A string specifying the author of the package.", default = "", @@ -175,17 +196,44 @@ _other_attrs = { default = "", ), "classifiers": attr.string_list( - doc = "A list of strings describing the categories for the package. For valid classifiers see https://pypi.org/classifiers", + doc = "A list of strings describing the categories for the package. For valid classifiers see ", ), "data_files": attr.label_keyed_string_dict( - doc = ("Any file that is not normally installed inside site-packages goes into the .data directory, named " + - "as the .dist-info directory but with the .data/ extension. Allowed paths: {prefixes}".format(prefixes = ALLOWED_DATA_FILE_PREFIX)), + doc = (""" +Mapping of data files to go into the wheel. + +The keys are targets of files to include, and the values are the `.data`-relative +path to use. + +Any file that is not normally installed inside site-packages goes into the .data +directory, named as the .dist-info directory but with the .data/ extension. If +the destination of a file or group of files ends in a `/`, the destination is a +folder and files are placed with their existing basenames under that folder. + +For example: + +``` +":file1.txt": "data/file1.txt", # Destination: .data/data/file1.txt +":file1.txt": "data/", # Destination: .data/data/file1.txt +":file1.txt": "data/special.txt", # Destination: .data/data/special.txt + +filegroup(name = "files", srcs = [":file1.txt", ":file2.txt"]) +":files": "data/", # Destinations: .data/data/file1.txt, .data/data/file2.txt +``` + +Allowed paths: {prefixes} + +:::{{versionchanged}} 2.0.0 +Values can end in slash (`/`) to indicate that all files of the target should +be moved under that directory. +::: +""".format(prefixes = ALLOWED_DATA_FILE_PREFIX)), allow_files = True, ), "description_content_type": attr.string( doc = ("The type of contents in description_file. " + "If not provided, the type will be inferred from the extension of description_file. " + - "Also see https://packaging.python.org/en/latest/specifications/core-metadata/#description-content-type"), + "Also see "), ), "description_file": attr.label( doc = "A file containing text describing the package.", @@ -216,7 +264,15 @@ _other_attrs = { ), "strip_path_prefixes": attr.string_list( default = [], - doc = "path prefixes to strip from files added to the generated package", + doc = """\ +Path prefixes to strip from files added to the generated package. +Prefixes are checked **in order** and only the **first match** will be used. + +For example: ++ `["foo", "foo/bar/baz"]` will strip `"foo/bar/baz/file.py"` to `"bar/baz/file.py"` ++ `["foo/bar/baz", "foo"]` will strip `"foo/bar/baz/file.py"` to `"file.py"` and + `"foo/file2.py"` to `"file2.py"` +""", ), "summary": attr.string( doc = "A one-line summary of what the distribution does", @@ -276,7 +332,7 @@ def _escape_filename_distribution_name(name): def _escape_filename_segment(segment): """Escape a segment of the wheel filename. - See https://www.python.org/dev/peps/pep-0427/#escaping-and-unicode + See {pep}`PEP 427` """ # TODO: this is wrong, isalnum replaces non-ascii letters, while we should @@ -305,11 +361,11 @@ def _input_file_to_arg(input_file): def _py_wheel_impl(ctx): abi = _replace_make_variables(ctx.attr.abi, ctx) python_tag = _replace_make_variables(ctx.attr.python_tag, ctx) - version = _replace_make_variables(ctx.attr.version, ctx) + version_str = _replace_make_variables(ctx.attr.version, ctx) filename_segments = [ _escape_filename_distribution_name(ctx.attr.distribution), - normalize_pep440(version), + version.normalize(version_str), _escape_filename_segment(python_tag), _escape_filename_segment(abi), _escape_filename_segment(ctx.attr.platform), @@ -319,31 +375,38 @@ def _py_wheel_impl(ctx): name_file = ctx.actions.declare_file(ctx.label.name + ".name") + direct_pyi_files = [] + for dep in ctx.attr.deps: + if PyInfo in dep: + direct_pyi_files.extend(dep[PyInfo].direct_pyi_files.to_list()) + inputs_to_package = depset( - direct = ctx.files.deps, + direct = ctx.files.deps + direct_pyi_files, ) # Inputs to this rule which are not to be packaged. # Currently this is only the description file (if used). other_inputs = [] - # Wrap the inputs into a file to reduce command line length. + # Wrap the inputs into a file to reduce command line length, deferring + # depset expansion to execution time via Args.add_all with map_each. packageinputfile = ctx.actions.declare_file(ctx.attr.name + "_target_wrapped_inputs.txt") - content = "" - for input_file in inputs_to_package.to_list(): - content += _input_file_to_arg(input_file) + "\n" - ctx.actions.write(output = packageinputfile, content = content) + package_args = ctx.actions.args() + package_args.set_param_file_format("multiline") + package_args.add_all(inputs_to_package, map_each = _input_file_to_arg) + ctx.actions.write(output = packageinputfile, content = package_args) other_inputs.append(packageinputfile) args = ctx.actions.args() args.add("--name", ctx.attr.distribution) - args.add("--version", version) + args.add("--version", version_str) args.add("--python_tag", python_tag) args.add("--abi", abi) args.add("--platform", ctx.attr.platform) args.add("--out", outfile) args.add("--name_file", name_file) args.add_all(ctx.attr.strip_path_prefixes, format_each = "--strip_path_prefix=%s") + args.add("--path_prefix", ctx.attr.add_path_prefix) # Pass workspace status files if stamping is enabled if is_stamping_enabled(ctx.attr): @@ -474,7 +537,7 @@ def _py_wheel_impl(ctx): args.add("--no_compress") for target, filename in ctx.attr.extra_distinfo_files.items(): - target_files = target.files.to_list() + target_files = target[DefaultInfo].files.to_list() if len(target_files) != 1: fail( "Multi-file target listed in extra_distinfo_files %s", @@ -487,10 +550,10 @@ def _py_wheel_impl(ctx): ) for target, filename in ctx.attr.data_files.items(): - target_files = target.files.to_list() - if len(target_files) != 1: + target_files = target[DefaultInfo].files.to_list() + if len(target_files) != 1 and not filename.endswith("/"): fail( - "Multi-file target listed in data_files %s", + "Multi-file target listed in data_files %s, this is only supported when specifying a folder path (i.e. a path ending in '/')", filename, ) @@ -502,11 +565,15 @@ def _py_wheel_impl(ctx): filename, ), ) - other_inputs.extend(target_files) - args.add( - "--data_files", - filename + ";" + target_files[0].path, - ) + + for file in target_files: + final_filename = filename + file.basename if filename.endswith("/") else filename + + other_inputs.extend(target_files) + args.add( + "--data_files", + final_filename + ";" + file.path, + ) ctx.actions.run( mnemonic = "PyWheel", @@ -514,6 +581,9 @@ def _py_wheel_impl(ctx): outputs = [outfile, name_file], arguments = [args], executable = ctx.executable._wheelmaker, + # The default shell env is used to better support toolchains that look + # up python at runtime using PATH. + use_default_shell_env = True, progress_message = "Building wheel {}".format(ctx.label), ) return [ @@ -560,10 +630,15 @@ tries to locate `.runfiles` directory which is not packaged in the wheel. _requirement_attrs, _entrypoint_attrs, _other_attrs, + CONFIG_SETTINGS_ATTR, ), ) -py_wheel = rule( +def _transition_wheel_impl(settings, attr): + """Transition for py_wheel.""" + return apply_config_settings_attr(dict(settings), attr) + +py_wheel = ruleb.Rule( implementation = py_wheel_lib.implementation, doc = """\ Internal rule used by the [py_wheel macro](#py_wheel). @@ -573,4 +648,9 @@ For example, a `bazel query` for a user's `py_wheel` macro expands to `py_wheel` in the way they expect. """, attrs = py_wheel_lib.attrs, -) + cfg = transition( + implementation = _transition_wheel_impl, + inputs = TRANSITION_LABELS, + outputs = TRANSITION_LABELS, + ), +).build() diff --git a/python/private/py_wheel_normalize_pep440.bzl b/python/private/py_wheel_normalize_pep440.bzl deleted file mode 100644 index 9566348987..0000000000 --- a/python/private/py_wheel_normalize_pep440.bzl +++ /dev/null @@ -1,519 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"Implementation of PEP440 version string normalization" - -def mkmethod(self, method): - """Bind a struct as the first arg to a function. - - This is loosely equivalent to creating a bound method of a class. - """ - return lambda *args, **kwargs: method(self, *args, **kwargs) - -def _isdigit(token): - return token.isdigit() - -def _isalnum(token): - return token.isalnum() - -def _lower(token): - # PEP 440: Case sensitivity - return token.lower() - -def _is(reference): - """Predicate testing a token for equality with `reference`.""" - return lambda token: token == reference - -def _is_not(reference): - """Predicate testing a token for inequality with `reference`.""" - return lambda token: token != reference - -def _in(reference): - """Predicate testing if a token is in the list `reference`.""" - return lambda token: token in reference - -def _ctx(start): - return {"norm": "", "start": start} - -def _open_context(self): - """Open an new parsing ctx. - - If the current parsing step succeeds, call self.accept(). - If the current parsing step fails, call self.discard() to - go back to how it was before we opened a new ctx. - - Args: - self: The normalizer. - """ - self.contexts.append(_ctx(_context(self)["start"])) - return self.contexts[-1] - -def _accept(self): - """Close the current ctx successfully and merge the results.""" - finished = self.contexts.pop() - self.contexts[-1]["norm"] += finished["norm"] - self.contexts[-1]["start"] = finished["start"] - return True - -def _context(self): - return self.contexts[-1] - -def _discard(self): - self.contexts.pop() - return False - -def _new(input): - """Create a new normalizer""" - self = struct( - input = input, - contexts = [_ctx(0)], - ) - - public = struct( - # methods: keep sorted - accept = mkmethod(self, _accept), - context = mkmethod(self, _context), - discard = mkmethod(self, _discard), - open_context = mkmethod(self, _open_context), - - # attributes: keep sorted - input = self.input, - ) - return public - -def accept(parser, predicate, value): - """If `predicate` matches the next token, accept the token. - - Accepting the token means adding it (according to `value`) to - the running results maintained in ctx["norm"] and - advancing the cursor in ctx["start"] to the next token in - `version`. - - Args: - parser: The normalizer. - predicate: function taking a token and returning a boolean - saying if we want to accept the token. - value: the string to add if there's a match, or, if `value` - is a function, the function to apply to the current token - to get the string to add. - - Returns: - whether a token was accepted. - """ - - ctx = parser.context() - - if ctx["start"] >= len(parser.input): - return False - - token = parser.input[ctx["start"]] - - if predicate(token): - if type(value) in ["function", "builtin_function_or_method"]: - value = value(token) - - ctx["norm"] += value - ctx["start"] += 1 - return True - - return False - -def accept_placeholder(parser): - """Accept a Bazel placeholder. - - Placeholders aren't actually part of PEP 440, but are used for - stamping purposes. A placeholder might be - ``{BUILD_TIMESTAMP}``, for instance. We'll accept these as - they are, assuming they will expand to something that makes - sense where they appear. Before the stamping has happened, a - resulting wheel file name containing a placeholder will not - actually be valid. - - Args: - parser: The normalizer. - - Returns: - whether a placeholder was accepted. - """ - ctx = parser.open_context() - - if not accept(parser, _is("{"), str): - return parser.discard() - - start = ctx["start"] - for _ in range(start, len(parser.input) + 1): - if not accept(parser, _is_not("}"), str): - break - - if not accept(parser, _is("}"), str): - return parser.discard() - - return parser.accept() - -def accept_digits(parser): - """Accept multiple digits (or placeholders). - - Args: - parser: The normalizer. - - Returns: - whether some digits (or placeholders) were accepted. - """ - - ctx = parser.open_context() - start = ctx["start"] - - for i in range(start, len(parser.input) + 1): - if not accept(parser, _isdigit, str) and not accept_placeholder(parser): - if i - start >= 1: - if ctx["norm"].isdigit(): - # PEP 440: Integer Normalization - ctx["norm"] = str(int(ctx["norm"])) - return parser.accept() - break - - return parser.discard() - -def accept_string(parser, string, replacement): - """Accept a `string` in the input. Output `replacement`. - - Args: - parser: The normalizer. - string: The string to search for in the parser input. - replacement: The normalized string to use if the string was found. - - Returns: - whether the string was accepted. - """ - ctx = parser.open_context() - - for character in string.elems(): - if not accept(parser, _in([character, character.upper()]), ""): - return parser.discard() - - ctx["norm"] = replacement - - return parser.accept() - -def accept_alnum(parser): - """Accept an alphanumeric sequence. - - Args: - parser: The normalizer. - - Returns: - whether an alphanumeric sequence was accepted. - """ - - ctx = parser.open_context() - start = ctx["start"] - - for i in range(start, len(parser.input) + 1): - if not accept(parser, _isalnum, _lower) and not accept_placeholder(parser): - if i - start >= 1: - return parser.accept() - break - - return parser.discard() - -def accept_dot_number(parser): - """Accept a dot followed by digits. - - Args: - parser: The normalizer. - - Returns: - whether a dot+digits pair was accepted. - """ - parser.open_context() - - if accept(parser, _is("."), ".") and accept_digits(parser): - return parser.accept() - else: - return parser.discard() - -def accept_dot_number_sequence(parser): - """Accept a sequence of dot+digits. - - Args: - parser: The normalizer. - - Returns: - whether a sequence of dot+digits pairs was accepted. - """ - ctx = parser.context() - start = ctx["start"] - i = start - - for i in range(start, len(parser.input) + 1): - if not accept_dot_number(parser): - break - return i - start >= 1 - -def accept_separator_alnum(parser): - """Accept a separator followed by an alphanumeric string. - - Args: - parser: The normalizer. - - Returns: - whether a separator and an alphanumeric string were accepted. - """ - parser.open_context() - - # PEP 440: Local version segments - if ( - accept(parser, _in([".", "-", "_"]), ".") and - (accept_digits(parser) or accept_alnum(parser)) - ): - return parser.accept() - - return parser.discard() - -def accept_separator_alnum_sequence(parser): - """Accept a sequence of separator+alphanumeric. - - Args: - parser: The normalizer. - - Returns: - whether a sequence of separator+alphanumerics was accepted. - """ - ctx = parser.context() - start = ctx["start"] - i = start - - for i in range(start, len(parser.input) + 1): - if not accept_separator_alnum(parser): - break - - return i - start >= 1 - -def accept_epoch(parser): - """PEP 440: Version epochs. - - Args: - parser: The normalizer. - - Returns: - whether a PEP 440 epoch identifier was accepted. - """ - ctx = parser.open_context() - if accept_digits(parser) and accept(parser, _is("!"), "!"): - if ctx["norm"] == "0!": - ctx["norm"] = "" - return parser.accept() - else: - return parser.discard() - -def accept_release(parser): - """Accept the release segment, numbers separated by dots. - - Args: - parser: The normalizer. - - Returns: - whether a release segment was accepted. - """ - parser.open_context() - - if not accept_digits(parser): - return parser.discard() - - accept_dot_number_sequence(parser) - return parser.accept() - -def accept_pre_l(parser): - """PEP 440: Pre-release spelling. - - Args: - parser: The normalizer. - - Returns: - whether a prerelease keyword was accepted. - """ - parser.open_context() - - if ( - accept_string(parser, "alpha", "a") or - accept_string(parser, "a", "a") or - accept_string(parser, "beta", "b") or - accept_string(parser, "b", "b") or - accept_string(parser, "c", "rc") or - accept_string(parser, "preview", "rc") or - accept_string(parser, "pre", "rc") or - accept_string(parser, "rc", "rc") - ): - return parser.accept() - else: - return parser.discard() - -def accept_prerelease(parser): - """PEP 440: Pre-releases. - - Args: - parser: The normalizer. - - Returns: - whether a prerelease identifier was accepted. - """ - ctx = parser.open_context() - - # PEP 440: Pre-release separators - accept(parser, _in(["-", "_", "."]), "") - - if not accept_pre_l(parser): - return parser.discard() - - accept(parser, _in(["-", "_", "."]), "") - - if not accept_digits(parser): - # PEP 440: Implicit pre-release number - ctx["norm"] += "0" - - return parser.accept() - -def accept_implicit_postrelease(parser): - """PEP 440: Implicit post releases. - - Args: - parser: The normalizer. - - Returns: - whether an implicit postrelease identifier was accepted. - """ - ctx = parser.open_context() - - if accept(parser, _is("-"), "") and accept_digits(parser): - ctx["norm"] = ".post" + ctx["norm"] - return parser.accept() - - return parser.discard() - -def accept_explicit_postrelease(parser): - """PEP 440: Post-releases. - - Args: - parser: The normalizer. - - Returns: - whether an explicit postrelease identifier was accepted. - """ - ctx = parser.open_context() - - # PEP 440: Post release separators - if not accept(parser, _in(["-", "_", "."]), "."): - ctx["norm"] += "." - - # PEP 440: Post release spelling - if ( - accept_string(parser, "post", "post") or - accept_string(parser, "rev", "post") or - accept_string(parser, "r", "post") - ): - accept(parser, _in(["-", "_", "."]), "") - - if not accept_digits(parser): - # PEP 440: Implicit post release number - ctx["norm"] += "0" - - return parser.accept() - - return parser.discard() - -def accept_postrelease(parser): - """PEP 440: Post-releases. - - Args: - parser: The normalizer. - - Returns: - whether a postrelease identifier was accepted. - """ - parser.open_context() - - if accept_implicit_postrelease(parser) or accept_explicit_postrelease(parser): - return parser.accept() - - return parser.discard() - -def accept_devrelease(parser): - """PEP 440: Developmental releases. - - Args: - parser: The normalizer. - - Returns: - whether a developmental release identifier was accepted. - """ - ctx = parser.open_context() - - # PEP 440: Development release separators - if not accept(parser, _in(["-", "_", "."]), "."): - ctx["norm"] += "." - - if accept_string(parser, "dev", "dev"): - accept(parser, _in(["-", "_", "."]), "") - - if not accept_digits(parser): - # PEP 440: Implicit development release number - ctx["norm"] += "0" - - return parser.accept() - - return parser.discard() - -def accept_local(parser): - """PEP 440: Local version identifiers. - - Args: - parser: The normalizer. - - Returns: - whether a local version identifier was accepted. - """ - parser.open_context() - - if accept(parser, _is("+"), "+") and accept_alnum(parser): - accept_separator_alnum_sequence(parser) - return parser.accept() - - return parser.discard() - -def normalize_pep440(version): - """Escape the version component of a filename. - - See https://packaging.python.org/en/latest/specifications/binary-distribution-format/#escaping-and-unicode - and https://peps.python.org/pep-0440/ - - Args: - version: version string to be normalized according to PEP 440. - - Returns: - string containing the normalized version. - """ - parser = _new(version.strip()) # PEP 440: Leading and Trailing Whitespace - accept(parser, _is("v"), "") # PEP 440: Preceding v character - accept_epoch(parser) - accept_release(parser) - accept_prerelease(parser) - accept_postrelease(parser) - accept_devrelease(parser) - accept_local(parser) - if parser.input[parser.context()["start"]:]: - fail( - "Failed to parse PEP 440 version identifier '%s'." % parser.input, - "Parse error at '%s'" % parser.input[parser.context()["start"]:], - ) - return parser.context()["norm"] diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 2cc073a832..bb84ff9280 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -18,6 +18,34 @@ package(default_visibility = ["//:__subpackages__"]) licenses(["notice"]) +exports_files( + srcs = ["namespace_pkg_tmpl.py"], + visibility = ["//visibility:public"], +) + +alias( + name = "venv_entry_point_template", + actual = select({ + "@platforms//os:windows": "venv_entry_point_template.bat", + "//conditions:default": "venv_entry_point_template.sh", + }), + visibility = ["//visibility:public"], +) + +alias( + name = "venv_shebang_rewriter", + actual = select({ + "@platforms//os:windows": "venv_shebang_rewriter.ps1", + "//conditions:default": "venv_shebang_rewriter.sh", + }), + visibility = ["//visibility:public"], +) + +exports_files( + srcs = ["deps.bzl"], + visibility = ["//tools/private/update_deps:__pkg__"], +) + filegroup( name = "distribution", srcs = glob( @@ -46,289 +74,521 @@ filegroup( visibility = ["//tools/private/update_deps:__pkg__"], ) -# Keep sorted by library name and keep the files named by the main symbol they export - bzl_library( - name = "attrs_bzl", - srcs = ["attrs.bzl"], + name = "config_settings", + srcs = ["config_settings.bzl"], + deps = ["@bazel_skylib//lib:selects"], ) bzl_library( - name = "extension_bzl", - srcs = ["extension.bzl"], - deps = [ - ":attrs_bzl", - ":evaluate_markers_bzl", - ":hub_repository_bzl", - ":parse_requirements_bzl", - ":parse_whl_name_bzl", - ":pip_repository_attrs_bzl", - ":simpleapi_download_bzl", - ":whl_config_setting_bzl", - ":whl_library_bzl", - ":whl_repo_name_bzl", - "//python/private:full_version_bzl", - "//python/private:normalize_name_bzl", - "//python/private:semver_bzl", - "//python/private:version_label_bzl", - "@bazel_features//:features", - "@pythons_hub//:interpreters_bzl", - ], + name = "deps", + srcs = ["deps.bzl"], + deps = ["//python/private:bazel_tools"], ) bzl_library( - name = "config_settings_bzl", - srcs = ["config_settings.bzl"], - deps = ["flags_bzl"], + name = "env_marker_setting", + srcs = ["env_marker_setting.bzl"], + deps = [ + ":env_marker_info", + ":pep508_env", + ":pep508_evaluate", + "//python/private:common_labels", + "//python/private:toolchain_types", + "@bazel_skylib//rules:common_settings", + ], ) bzl_library( - name = "deps_bzl", - srcs = ["deps.bzl"], + name = "evaluate_markers", + srcs = ["evaluate_markers.bzl"], deps = [ - "//python/private:bazel_tools_bzl", - "//python/private:glob_excludes_bzl", + ":pep508_evaluate", + ":pep508_requirement", ], ) +# keep bzl_library( - name = "evaluate_markers_bzl", - srcs = ["evaluate_markers.bzl"], + name = "extension", + srcs = ["extension.bzl"], deps = [ - ":pypi_repo_utils_bzl", + ":hub_builder", + ":hub_repository", + ":parse_whl_name", + ":pep508_env", + ":pip_repository_attrs", + ":platform", + ":pypi_cache", + ":simpleapi_download", + ":unified_hub_repo", + ":whl_library", + "//python/private:auth", + "//python/private:normalize_name", + "//python/private:pyproject_utils", + "//python/private:repo_utils", + "@pythons_hub//:interpreters", + "@pythons_hub//:versions", + "@rules_python_internal//:rules_python_config", + "@toml.bzl//:toml", ], ) bzl_library( - name = "flags_bzl", + name = "flags", srcs = ["flags.bzl"], deps = [ - "//python/private:enum_bzl", + ":env_marker_info", + ":pep508_env", + "//python/private:common_labels", "@bazel_skylib//rules:common_settings", ], ) bzl_library( - name = "generate_whl_library_build_bazel_bzl", - srcs = ["generate_whl_library_build_bazel.bzl"], + name = "generate_group_library_build_bazel", + srcs = ["generate_group_library_build_bazel.bzl"], deps = [ - "//python/private:text_util_bzl", + ":labels", + "//python/private:normalize_name", + "//python/private:text_util", ], ) bzl_library( - name = "generate_group_library_build_bazel_bzl", - srcs = ["generate_group_library_build_bazel.bzl"], - deps = [ - ":labels_bzl", - "//python/private:normalize_name_bzl", - ], + name = "generate_whl_library_build_bazel", + srcs = ["generate_whl_library_build_bazel.bzl"], + deps = ["//python/private:text_util"], ) bzl_library( - name = "group_library_bzl", - srcs = ["group_library.bzl"], + name = "hub_builder", + srcs = ["hub_builder.bzl"], deps = [ - ":generate_group_library_build_bazel_bzl", + ":attrs", + ":hash", + ":parse_requirements", + ":pep508_env", + ":pep508_evaluate", + ":python_tag", + ":requirements_files_by_platform", + ":whl_config_setting", + ":whl_repo_name", + "//python/private:envsubst", + "//python/private:full_version", + "//python/private:normalize_name", + "//python/private:repo_utils", + "//python/private:text_util", + "//python/private:version", + "//python/private:version_label", ], ) bzl_library( - name = "hub_repository_bzl", + name = "hub_repository", srcs = ["hub_repository.bzl"], - visibility = ["//:__subpackages__"], deps = [ - ":render_pkg_aliases_bzl", - "//python/private:text_util_bzl", + ":render_pkg_aliases", + ":whl_config_setting", + "//python/private:text_util", ], ) bzl_library( - name = "index_sources_bzl", - srcs = ["index_sources.bzl"], -) - -bzl_library( - name = "labels_bzl", - srcs = ["labels.bzl"], + name = "missing_package", + srcs = ["missing_package.bzl"], + deps = [ + "//python/private:py_info", + "//python/private:reexports", + ], ) bzl_library( - name = "multi_pip_parse_bzl", + name = "multi_pip_parse", srcs = ["multi_pip_parse.bzl"], deps = [ - ":pip_repository_bzl", - "//python/private:text_util_bzl", + ":pip_repository", + "//python/private:text_util", ], ) bzl_library( - name = "package_annotation_bzl", - srcs = ["package_annotation.bzl"], + name = "namespace_pkgs", + srcs = ["namespace_pkgs.bzl"], + deps = ["@bazel_skylib//rules:copy_file"], ) bzl_library( - name = "parse_requirements_bzl", + name = "parse_requirements", srcs = ["parse_requirements.bzl"], deps = [ - ":index_sources_bzl", - ":parse_requirements_txt_bzl", - ":pypi_repo_utils_bzl", - ":requirements_files_by_platform_bzl", - ":whl_target_platforms_bzl", - "//python/private:normalize_name_bzl", - "//python/private:repo_utils_bzl", + ":argparse", + ":hash", + ":index_sources", + ":parse_requirements_txt", + ":pep508_evaluate", + ":pep508_requirement", + ":select_whl", + "//python/private:normalize_name", + "//python/private:repo_utils", + "//python/uv/private:uv_lock_to_requirements", ], ) bzl_library( - name = "parse_requirements_txt_bzl", - srcs = ["parse_requirements_txt.bzl"], + name = "parse_simpleapi_html", + srcs = ["parse_simpleapi_html.bzl"], + deps = [ + ":hash", + ":version_from_filename", + "//python/private:normalize_name", + ], ) bzl_library( - name = "parse_simpleapi_html_bzl", - srcs = ["parse_simpleapi_html.bzl"], + name = "patch_whl", + srcs = ["patch_whl.bzl"], + deps = [ + ":parse_whl_name", + "//python/private:repo_utils", + "@rules_python_internal//:rules_python_config", + ], ) bzl_library( - name = "parse_whl_name_bzl", - srcs = ["parse_whl_name.bzl"], + name = "pep508_deps", + srcs = ["pep508_deps.bzl"], + deps = [ + ":pep508_env", + ":pep508_evaluate", + ":pep508_requirement", + "//python/private:normalize_name", + ], ) bzl_library( - name = "patch_whl_bzl", - srcs = ["patch_whl.bzl"], + name = "pep508_env", + srcs = ["pep508_env.bzl"], deps = [ - ":parse_whl_name_bzl", - "//python/private:repo_utils_bzl", + "//python/private:normalize_name", + "//python/private:version", ], ) bzl_library( - name = "pip_bzl", - srcs = ["pip.bzl"], + name = "pep508_evaluate", + srcs = ["pep508_evaluate.bzl"], deps = [ - ":extension_bzl", + "//python/private:enum", + "//python/private:version", ], ) bzl_library( - name = "pip_compile_bzl", + name = "pep508_requirement", + srcs = ["pep508_requirement.bzl"], + deps = ["//python/private:normalize_name"], +) + +bzl_library( + name = "pip", + srcs = ["pip.bzl"], + deps = [":extension"], +) + +bzl_library( + name = "pip_compile", srcs = ["pip_compile.bzl"], deps = [ - ":deps_bzl", - "//python:defs_bzl", + "//python:py_binary", + "//python:py_test", ], ) bzl_library( - name = "pip_repository_bzl", + name = "pip_repository", srcs = ["pip_repository.bzl"], deps = [ - ":attrs_bzl", - ":evaluate_markers_bzl", - ":parse_requirements_bzl", - ":pip_repository_attrs_bzl", - ":render_pkg_aliases_bzl", - ":whl_config_setting_bzl", - "//python/private:normalize_name_bzl", - "//python/private:repo_utils_bzl", - "//python/private:text_util_bzl", + ":parse_requirements", + ":pep508_env", + ":pip_repository_attrs", + ":pypi_repo_utils", + ":render_pkg_aliases", + ":requirements_files_by_platform", + "//python/private:normalize_name", + "//python/private:repo_utils", + "//python/private:text_util", "@bazel_skylib//lib:sets", ], ) bzl_library( - name = "pip_repository_attrs_bzl", + name = "pip_repository_attrs", srcs = ["pip_repository_attrs.bzl"], + deps = [":attrs"], +) + +bzl_library( + name = "pkg_aliases", + srcs = ["pkg_aliases.bzl"], + deps = [ + ":labels", + "//python/private:common_labels", + "//python/private:text_util", + "@bazel_skylib//lib:selects", + ], +) + +bzl_library( + name = "pypi_cache", + srcs = ["pypi_cache.bzl"], + deps = [":version_from_filename"], ) bzl_library( - name = "pypi_repo_utils_bzl", + name = "pypi_repo_utils", srcs = ["pypi_repo_utils.bzl"], deps = [ - "//python/private:repo_utils_bzl", + "//python/private:repo_utils", + "//python/private:util", "@bazel_skylib//lib:types", ], ) bzl_library( - name = "render_pkg_aliases_bzl", + name = "python_tag", + srcs = ["python_tag.bzl"], + deps = ["//python/private:version"], +) + +bzl_library( + name = "render_pkg_aliases", srcs = ["render_pkg_aliases.bzl"], deps = [ - ":generate_group_library_build_bazel_bzl", - ":parse_whl_name_bzl", - ":whl_config_setting_bzl", - ":whl_target_platforms_bzl", - "//python/private:normalize_name_bzl", - "//python/private:text_util_bzl", + ":generate_group_library_build_bazel", + "//python/private:normalize_name", + "//python/private:text_util", ], ) bzl_library( - name = "requirements_files_by_platform_bzl", + name = "requirements_files_by_platform", srcs = ["requirements_files_by_platform.bzl"], deps = [ - ":whl_target_platforms_bzl", + ":argparse", + ":whl_target_platforms", ], ) bzl_library( - name = "simpleapi_download_bzl", + name = "select_whl", + srcs = ["select_whl.bzl"], + deps = [ + ":parse_whl_name", + ":python_tag", + "//python/private:version", + ], +) + +bzl_library( + name = "simpleapi_download", srcs = ["simpleapi_download.bzl"], deps = [ - ":parse_simpleapi_html_bzl", - "//python/private:auth_bzl", - "//python/private:normalize_name_bzl", - "//python/private:text_util_bzl", - "@bazel_features//:features", + ":parse_simpleapi_html", + ":urllib", + "//python/private:auth", + "//python/private:envsubst", + "//python/private:normalize_name", ], ) bzl_library( - name = "whl_config_setting_bzl", - srcs = ["whl_config_setting.bzl"], + name = "unified_hub_repo", + srcs = ["unified_hub_repo.bzl"], + deps = [ + "//python/private:normalize_name", + "//python/private:text_util", + ], ) bzl_library( - name = "whl_library_alias_bzl", - srcs = ["whl_library_alias.bzl"], + name = "unified_hub_setup", + srcs = ["unified_hub_setup.bzl"], deps = [ - ":render_pkg_aliases_bzl", - "//python/private:full_version_bzl", + "@rules_python//python/private/pypi:labels", + "@rules_python//python/private/pypi:missing_package", ], ) bzl_library( - name = "whl_library_bzl", + name = "venv_entry_point", + srcs = ["venv_entry_point.bzl"], + deps = [ + "//python/private:attributes", + "//python/private:common", + "//python/private:rule_builders", + ], +) + +bzl_library( + name = "venv_rewrite_shebang", + srcs = ["venv_rewrite_shebang.bzl"], + deps = [ + "//python/private:attributes", + "//python/private:common", + "//python/private:py_info", + "//python/private:rule_builders", + ], +) + +bzl_library( + name = "whl_config_repo", + srcs = ["whl_config_repo.bzl"], + deps = [ + ":generate_group_library_build_bazel", + "//python/private:text_util", + ], +) + +bzl_library( + name = "whl_extract", + srcs = ["whl_extract.bzl"], + deps = [ + ":whl_metadata", + "//python/private:repo_utils", + "@rules_python_internal//:rules_python_config", + ], +) + +bzl_library( + name = "whl_library", srcs = ["whl_library.bzl"], deps = [ - ":attrs_bzl", - ":deps_bzl", - ":generate_whl_library_build_bazel_bzl", - ":parse_whl_name_bzl", - ":patch_whl_bzl", - ":pypi_repo_utils_bzl", - ":whl_target_platforms_bzl", - "//python/private:auth_bzl", - "//python/private:envsubst_bzl", - "//python/private:is_standalone_interpreter_bzl", - "//python/private:repo_utils_bzl", + ":attrs", + ":deps", + ":generate_whl_library_build_bazel", + ":patch_whl", + ":pep508_requirement", + ":pypi_repo_utils", + ":urllib", + ":whl_extract", + ":whl_metadata", + "//python/private:auth", + "//python/private:envsubst", + "//python/private:is_standalone_interpreter", + "//python/private:normalize_name", + "//python/private:repo_utils", + ], +) + +bzl_library( + name = "whl_library_alias", + srcs = ["whl_library_alias.bzl"], + deps = [ + ":render_pkg_aliases", + "//python/private:full_version", + ], +) + +bzl_library( + name = "whl_library_targets", + srcs = ["whl_library_targets.bzl"], + deps = [ + ":env_marker_setting", + ":labels", + ":namespace_pkgs", + ":pep508_deps", + ":venv_entry_point", + ":venv_rewrite_shebang", + "//python:py_binary", + "//python:py_library", + "//python/private:normalize_name", + "@bazel_skylib//rules:copy_file", ], ) bzl_library( - name = "whl_repo_name_bzl", + name = "whl_repo_name", srcs = ["whl_repo_name.bzl"], deps = [ - ":parse_whl_name_bzl", - "//python/private:normalize_name_bzl", + ":parse_whl_name", + "//python/private:normalize_name", ], ) bzl_library( - name = "whl_target_platforms_bzl", + name = "urllib", + srcs = ["urllib.bzl"], + deps = ["//python/private:envsubst"], +) + +bzl_library( + name = "index_sources", + srcs = ["index_sources.bzl"], + deps = [":hash"], +) + +bzl_library( + name = "argparse", + srcs = ["argparse.bzl"], +) + +bzl_library( + name = "attrs", + srcs = ["attrs.bzl"], +) + +bzl_library( + name = "env_marker_info", + srcs = ["env_marker_info.bzl"], +) + +bzl_library( + name = "hash", + srcs = ["hash.bzl"], +) + +bzl_library( + name = "labels", + srcs = ["labels.bzl"], +) + +bzl_library( + name = "package_annotation", + srcs = ["package_annotation.bzl"], +) + +bzl_library( + name = "parse_requirements_txt", + srcs = ["parse_requirements_txt.bzl"], +) + +bzl_library( + name = "parse_whl_name", + srcs = ["parse_whl_name.bzl"], +) + +bzl_library( + name = "platform", + srcs = ["platform.bzl"], +) + +bzl_library( + name = "version_from_filename", + srcs = ["version_from_filename.bzl"], +) + +bzl_library( + name = "whl_config_setting", + srcs = ["whl_config_setting.bzl"], +) + +bzl_library( + name = "whl_metadata", + srcs = ["whl_metadata.bzl"], +) + +bzl_library( + name = "whl_target_platforms", srcs = ["whl_target_platforms.bzl"], - deps = [":parse_whl_name_bzl"], ) diff --git a/python/private/pypi/argparse.bzl b/python/private/pypi/argparse.bzl new file mode 100644 index 0000000000..8964411a3e --- /dev/null +++ b/python/private/pypi/argparse.bzl @@ -0,0 +1,40 @@ +"""A small set of utilities for parsing pip args.""" + +def _get_pip_args(args, *arg_names, value = None, repeated = False): + set_next = False + if repeated: + value = [] + (value or []) + + for arg in (args or []): + if arg in arg_names: + set_next = True + continue + + val = None + if set_next: + set_next = False + val = arg + else: + for arg_name in arg_names: + start = "{}=".format(arg_name) + + if arg.startswith(start): + val = arg[len(start):] + break + + if val == None: + continue + + if repeated: + if val not in value: + value.append(val) + else: + value = val + + return value + +argparse = struct( + index_url = lambda args, default: _get_pip_args(args, "-i", "--index-url", value = default), + extra_index_url = lambda args, default: _get_pip_args(args, "--extra-index-url", value = default, repeated = True), + platform = lambda args, default: _get_pip_args(args, "--platform", value = default, repeated = True), +) diff --git a/python/private/pypi/attrs.bzl b/python/private/pypi/attrs.bzl index c9b7ea66a9..271c17e35c 100644 --- a/python/private/pypi/attrs.bzl +++ b/python/private/pypi/attrs.bzl @@ -15,6 +15,15 @@ "common attributes for whl_library and pip_repository" ATTRS = { + "add_libdir_to_library_search_path": attr.bool( + default = False, + doc = """ +If true, add the lib dir of the bundled interpreter to the library search path via `LDFLAGS`. + +:::{versionadded} 1.3.0 +::: +""", + ), "download_only": attr.bool( doc = """ Whether to use "pip download" instead of "pip wheel". Disables building wheels from source, but allows use of @@ -55,102 +64,6 @@ here do not cause packages to be re-fetched. Don't fetch different things based on the value of these variables. """, ), - "experimental_requirement_cycles": attr.string_list_dict( - default = {}, - doc = """\ -A mapping of dependency cycle names to a list of requirements which form that cycle. - -Requirements which form cycles will be installed together and taken as -dependencies together in order to ensure that the cycle is always satisified. - -Example: - `sphinx` depends on `sphinxcontrib-serializinghtml` - When listing both as requirements, ala - - ``` - py_binary( - name = "doctool", - ... - deps = [ - "@pypi//sphinx:pkg", - "@pypi//sphinxcontrib_serializinghtml", - ] - ) - ``` - - Will produce a Bazel error such as - - ``` - ERROR: .../external/pypi_sphinxcontrib_serializinghtml/BUILD.bazel:44:6: in alias rule @pypi_sphinxcontrib_serializinghtml//:pkg: cycle in dependency graph: - //:doctool (...) - @pypi//sphinxcontrib_serializinghtml:pkg (...) - .-> @pypi_sphinxcontrib_serializinghtml//:pkg (...) - | @pypi_sphinxcontrib_serializinghtml//:_pkg (...) - | @pypi_sphinx//:pkg (...) - | @pypi_sphinx//:_pkg (...) - `-- @pypi_sphinxcontrib_serializinghtml//:pkg (...) - ``` - - Which we can resolve by configuring these two requirements to be installed together as a cycle - - ``` - pip_parse( - ... - experimental_requirement_cycles = { - "sphinx": [ - "sphinx", - "sphinxcontrib-serializinghtml", - ] - }, - ) - ``` - -Warning: - If a dependency participates in multiple cycles, all of those cycles must be - collapsed down to one. For instance `a <-> b` and `a <-> c` cannot be listed - as two separate cycles. -""", - ), - "experimental_target_platforms": attr.string_list( - default = [], - doc = """\ -A list of platforms that we will generate the conditional dependency graph for -cross platform wheels by parsing the wheel metadata. This will generate the -correct dependencies for packages like `sphinx` or `pylint`, which include -`colorama` when installed and used on Windows platforms. - -An empty list means falling back to the legacy behaviour where the host -platform is the target platform. - -WARNING: It may not work as expected in cases where the python interpreter -implementation that is being used at runtime is different between different platforms. -This has been tested for CPython only. - -For specific target platforms use values of the form `_` where `` -is one of `linux`, `osx`, `windows` and arch is one of `x86_64`, `x86_32`, -`aarch64`, `s390x` and `ppc64le`. - -You can also target a specific Python version by using `cp3__`. -If multiple python versions are specified as target platforms, then select statements -of the `lib` and `whl` targets will include usage of version aware toolchain config -settings like `@rules_python//python/config_settings:is_python_3.y`. - -Special values: `host` (for generating deps for the host platform only) and -`_*` values. For example, `cp39_*`, `linux_*`, `cp39_linux_*`. - -NOTE: this is not for cross-compiling Python wheels but rather for parsing the `whl` METADATA correctly. -""", - ), - "extra_hub_aliases": attr.string_list_dict( - doc = """\ -Extra aliases to make for specific wheels in the hub repo. This is useful when -paired with the {attr}`whl_modifications`. - -:::{versionadded} 0.38.0 -::: -""", - mandatory = False, - ), "extra_pip_args": attr.string_list( doc = """Extra arguments to pass on to pip. Must not contain spaces. @@ -198,7 +111,7 @@ If True, suppress printing stdout and stderr output to the terminal. If you would like to get more diagnostic output, set {envvar}`RULES_PYTHON_REPO_DEBUG=1 ` or -{envvar}`RULES_PYTHON_REPO_DEBUG_VERBOSITY= ` +{envvar}`RULES_PYTHON_REPO_DEBUG_VERBOSITY=INFO|DEBUG|TRACE ` """, ), # 600 is documented as default here: https://docs.bazel.build/versions/master/skylark/lib/repository_ctx.html#execute @@ -221,7 +134,7 @@ def use_isolated(ctx, attr): use_isolated = attr.isolated # The environment variable will take precedence over the attribute - isolated_env = ctx.os.environ.get("RULES_PYTHON_PIP_ISOLATED", None) + isolated_env = ctx.getenv("RULES_PYTHON_PIP_ISOLATED", None) if isolated_env != None: if isolated_env.lower() in ("0", "false"): use_isolated = False diff --git a/python/private/pypi/config.bzl.tmpl b/python/private/pypi/config.bzl.tmpl new file mode 100644 index 0000000000..1037153cac --- /dev/null +++ b/python/private/pypi/config.bzl.tmpl @@ -0,0 +1,7 @@ +"""Extra configuration values that are exposed from the hub repository for spoke repositories to access. + +NOTE: This is internal `rules_python` API and if you would like to depend on it, please raise an issue +with your usecase. This may change in between rules_python versions without any notice. +""" + +packages = %%PACKAGES%% diff --git a/python/private/pypi/config_settings.bzl b/python/private/pypi/config_settings.bzl index 6f927f2a4c..dcb6779d5b 100644 --- a/python/private/pypi/config_settings.bzl +++ b/python/private/pypi/config_settings.bzl @@ -13,326 +13,98 @@ # limitations under the License. """ -This module is used to construct the config settings for selecting which distribution is used in the pip hub repository. - -Bazel's selects work by selecting the most-specialized configuration setting -that matches the target platform. We can leverage this fact to ensure that the -most specialized wheels are used by default with the users being able to -configure string_flag values to select the less specialized ones. - -The list of specialization of the dists goes like follows: -* sdist -* py*-none-any.whl -* py*-abi3-any.whl -* py*-cpxy-any.whl -* cp*-none-any.whl -* cp*-abi3-any.whl -* cp*-cpxy-plat.whl -* py*-none-plat.whl -* py*-abi3-plat.whl -* py*-cpxy-plat.whl -* cp*-none-plat.whl -* cp*-abi3-plat.whl -* cp*-cpxy-plat.whl - -Note, that here the specialization of musl vs manylinux wheels is the same in -order to ensure that the matching fails if the user requests for `musl` and we don't have it or vice versa. +The {obj}`config_settings` macro is used to create the config setting targets +that can be used in the {obj}`pkg_aliases` macro for selecting the compatible +repositories. + +The config settings are of the form `:is_cp3`. Suffix is a +normalized user provided value for the platform name. The goal of this macro is to +ensure that we can incorporate the user provided `config_setting` targets to create +our composite config_setting targets. """ -load("//python/private:flags.bzl", "LibcFlag") -load(":flags.bzl", "INTERNAL_FLAGS", "UniversalWhlFlag") - -FLAGS = struct( - **{ - f: str(Label("//python/config_settings:" + f)) - for f in [ - "python_version", - "pip_whl_glibc_version", - "pip_whl_muslc_version", - "pip_whl_osx_arch", - "pip_whl_osx_version", - "py_linux_libc", - "is_pip_whl_no", - "is_pip_whl_only", - "is_pip_whl_auto", - ] - } -) - -# Here we create extra string flags that are just to work with the select -# selecting the most specialized match. We don't allow the user to change -# them. -_flags = struct( - **{ - f: str(Label("//python/config_settings:_internal_pip_" + f)) - for f in INTERNAL_FLAGS - } -) +load("@bazel_skylib//lib:selects.bzl", "selects") def config_settings( *, python_versions = [], - glibc_versions = [], - muslc_versions = [], - osx_versions = [], - target_platforms = [], name = None, - visibility = None, - native = native): + platform_config_settings = {}, + **kwargs): """Generate all of the pip config settings. Args: name (str): Currently unused. python_versions (list[str]): The list of python versions to configure config settings for. - glibc_versions (list[str]): The list of glibc version of the wheels to - configure config settings for. - muslc_versions (list[str]): The list of musl version of the wheels to - configure config settings for. - osx_versions (list[str]): The list of OSX OS versions to configure - config settings for. - target_platforms (list[str]): The list of "{os}_{cpu}" for deriving - constraint values for each condition. - visibility (list[str], optional): The visibility to be passed to the - exposed labels. All other labels will be private. - native (struct): The struct containing alias and config_setting rules - to use for creating the objects. Can be overridden for unit tests - reasons. + platform_config_settings: {type}`dict[str, list[str]]` the constraint + values to use instead of the default ones. Key are platform names + (a human-friendly platform string). Values are lists of + `constraint_value` label strings. + **kwargs: Other args passed to the underlying implementations, such as + {obj}`native`. """ - - glibc_versions = [""] + glibc_versions - muslc_versions = [""] + muslc_versions - osx_versions = [""] + osx_versions - target_platforms = [("", "")] + [ - t.split("_", 1) - for t in target_platforms - ] - - for python_version in [""] + python_versions: - is_python = "is_python_{}".format(python_version or "version_unset") - - # The aliases defined in @rules_python//python/config_settings may not - # have config settings for the versions we need, so define our own - # config settings instead. - native.config_setting( - name = is_python, - flag_values = { - Label("//python/config_settings:python_version_major_minor"): python_version, - }, - visibility = visibility, - ) - - for os, cpu in target_platforms: - constraint_values = [] - suffix = "" - if os: - constraint_values.append("@platforms//os:" + os) - suffix += "_" + os - if cpu: - constraint_values.append("@platforms//cpu:" + cpu) - suffix += "_" + cpu - - _dist_config_settings( - suffix = suffix, - plat_flag_values = _plat_flag_values( - os = os, - cpu = cpu, - osx_versions = osx_versions, - glibc_versions = glibc_versions, - muslc_versions = muslc_versions, - ), - constraint_values = constraint_values, - python_version = python_version, - is_python = is_python, - visibility = visibility, - native = native, - ) - -def _dist_config_settings(*, suffix, plat_flag_values, **kwargs): - if kwargs.get("constraint_values"): - # Add python version + platform config settings - _dist_config_setting( - name = suffix.strip("_"), - **kwargs - ) - - flag_values = {_flags.dist: ""} - - # First create an sdist, we will be building upon the flag values, which - # will ensure that each sdist config setting is the least specialized of - # all. However, we need at least one flag value to cover the case where we - # have `sdist` for any platform, hence we have a non-empty `flag_values` - # here. - _dist_config_setting( - name = "sdist{}".format(suffix), - flag_values = flag_values, - is_pip_whl = FLAGS.is_pip_whl_no, - **kwargs - ) - - for name, f in [ - ("py_none", _flags.whl_py2_py3), - ("py3_none", _flags.whl_py3), - ("py3_abi3", _flags.whl_py3_abi3), - ("cp3x_none", _flags.whl_pycp3x), - ("cp3x_abi3", _flags.whl_pycp3x_abi3), - ("cp3x_cp", _flags.whl_pycp3x_abicp), - ]: - if f in flag_values: - # This should never happen as all of the different whls should have - # unique flag values. - fail("BUG: the flag {} is attempted to be added twice to the list".format(f)) - else: - flag_values[f] = "" - - _dist_config_setting( - name = "{}_any{}".format(name, suffix), - flag_values = flag_values, - is_pip_whl = FLAGS.is_pip_whl_only, - **kwargs - ) - - generic_flag_values = flag_values - - for (suffix, flag_values) in plat_flag_values: - flag_values = flag_values | generic_flag_values - - for name, f in [ - ("py_none", _flags.whl_plat), - ("py3_none", _flags.whl_plat_py3), - ("py3_abi3", _flags.whl_plat_py3_abi3), - ("cp3x_none", _flags.whl_plat_pycp3x), - ("cp3x_abi3", _flags.whl_plat_pycp3x_abi3), - ("cp3x_cp", _flags.whl_plat_pycp3x_abicp), - ]: - if f in flag_values: - # This should never happen as all of the different whls should have - # unique flag values. - fail("BUG: the flag {} is attempted to be added twice to the list".format(f)) - else: - flag_values[f] = "" + target_platforms = { + "": [], + } | platform_config_settings + + for platform_name, config_settings in target_platforms.items(): + suffix = "_{}".format(platform_name) if platform_name else "" + + # We parse the target settings and if there is a "platforms//os" or + # "platforms//cpu" value in here, we also add it into the constraint_values + # + # this is to ensure that we can still pass all of the unit tests for config + # setting specialization. + # + # TODO @aignas 2025-07-23: is this the right way? Maybe we should drop these + # and remove the tests? + constraint_values = [] + for setting in config_settings: + setting_label = Label(setting) + if setting_label.repo_name == "platforms" and setting_label.package in ["os", "cpu"]: + constraint_values.append(setting) + + for python_version in python_versions: + cpv = "cp" + python_version.replace(".", "") + prefix = "is_{}".format(cpv) _dist_config_setting( - name = "{}_{}".format(name, suffix), - flag_values = flag_values, - is_pip_whl = FLAGS.is_pip_whl_only, + name = prefix + suffix, + flag_values = { + Label("//python/config_settings:python_version_major_minor"): python_version, + }, + config_settings = config_settings, + constraint_values = constraint_values, **kwargs ) -def _to_version_string(version, sep = "."): - if not version: - return "" - - return "{}{}{}".format(version[0], sep, version[1]) - -def _plat_flag_values(os, cpu, osx_versions, glibc_versions, muslc_versions): - ret = [] - if os == "": - return [] - elif os == "windows": - ret.append(("{}_{}".format(os, cpu), {})) - elif os == "osx": - for cpu_, arch in { - cpu: UniversalWhlFlag.ARCH, - cpu + "_universal2": UniversalWhlFlag.UNIVERSAL, - }.items(): - for osx_version in osx_versions: - flags = { - FLAGS.pip_whl_osx_version: _to_version_string(osx_version), - } - if arch == UniversalWhlFlag.ARCH: - flags[FLAGS.pip_whl_osx_arch] = arch - - if not osx_version: - suffix = "{}_{}".format(os, cpu_) - else: - suffix = "{}_{}_{}".format(os, _to_version_string(osx_version, "_"), cpu_) - - ret.append((suffix, flags)) - - elif os == "linux": - for os_prefix, linux_libc in { - os: LibcFlag.GLIBC, - "many" + os: LibcFlag.GLIBC, - "musl" + os: LibcFlag.MUSL, - }.items(): - if linux_libc == LibcFlag.GLIBC: - libc_versions = glibc_versions - libc_flag = FLAGS.pip_whl_glibc_version - elif linux_libc == LibcFlag.MUSL: - libc_versions = muslc_versions - libc_flag = FLAGS.pip_whl_muslc_version - else: - fail("Unsupported libc type: {}".format(linux_libc)) - - for libc_version in libc_versions: - if libc_version and os_prefix == os: - continue - elif libc_version: - suffix = "{}_{}_{}".format(os_prefix, _to_version_string(libc_version, "_"), cpu) - else: - suffix = "{}_{}".format(os_prefix, cpu) - - ret.append(( - suffix, - { - FLAGS.py_linux_libc: linux_libc, - libc_flag: _to_version_string(libc_version), - }, - )) - else: - fail("Unsupported os: {}".format(os)) - - return ret - -def _dist_config_setting(*, name, is_python, python_version, is_pip_whl = None, native = native, **kwargs): - """A macro to create a target that matches is_pip_whl_auto and one more value. +def _dist_config_setting(*, name, selects = selects, native = native, config_settings = None, **kwargs): + """A macro to create a target for matching Python binary and source distributions. Args: name: The name of the public target. - is_pip_whl: The config setting to match in addition to - `is_pip_whl_auto` when evaluating the config setting. - is_python: The python version config_setting to match. - python_version: The python version name. + config_settings: {type}`list[str | Label]` the list of target settings that must + be matched before we try to evaluate the config_setting that we may create in + this function. + selects (struct): The struct containing config_setting_group function + to use for creating config setting groups. Can be overridden for unit tests + reasons. native (struct): The struct containing alias and config_setting rules to use for creating the objects. Can be overridden for unit tests reasons. **kwargs: The kwargs passed to the config_setting rule. Visibility of the main alias target is also taken from the kwargs. """ - _name = "_is_" + name - - visibility = kwargs.get("visibility") - native.alias( - name = "is_cp{}_{}".format(python_version, name) if python_version else "is_{}".format(name), - actual = select({ - # First match by the python version - is_python: _name, - "//conditions:default": is_python, - }), - visibility = visibility, - ) - if python_version: - # Reuse the config_setting targets that we use with the default - # `python_version` setting. - return - - if not is_pip_whl: - native.config_setting(name = _name, **kwargs) - return - - config_setting_name = _name + "_setting" - native.config_setting(name = config_setting_name, **kwargs) - - # Next match by the `pip_whl` flag value and then match by the flags that - # are intrinsic to the distribution. - native.alias( + # first define the config setting that has all of the constraint values + _name = "_" + name + native.config_setting( name = _name, - actual = select({ - "//conditions:default": FLAGS.is_pip_whl_auto, - FLAGS.is_pip_whl_auto: config_setting_name, - is_pip_whl: config_setting_name, - }), - visibility = visibility, + **kwargs + ) + selects.config_setting_group( + name = name, + match_all = config_settings + [_name], + visibility = kwargs.get("visibility"), ) diff --git a/python/private/pypi/dependency_resolver/dependency_resolver.py b/python/private/pypi/dependency_resolver/dependency_resolver.py index 0ff9b2fb7c..34f2fb1e5a 100644 --- a/python/private/pypi/dependency_resolver/dependency_resolver.py +++ b/python/private/pypi/dependency_resolver/dependency_resolver.py @@ -15,14 +15,17 @@ "Set defaults for the pip-compile command to run it under Bazel" import atexit +import functools import os import shutil import sys from pathlib import Path -from typing import Optional, Tuple +from typing import List, Optional, Tuple import click import piptools.writer as piptools_writer +from pip._internal.exceptions import DistributionNotFound +from pip._vendor.resolvelib.resolvers import ResolutionImpossible from piptools.scripts.compile import cli from python.runfiles import runfiles @@ -82,7 +85,7 @@ def _locate(bazel_runfiles, file): @click.command(context_settings={"ignore_unknown_options": True}) @click.option("--src", "srcs", multiple=True, required=True) @click.argument("requirements_txt") -@click.argument("update_target_label") +@click.argument("target_label_prefix") @click.option("--requirements-linux") @click.option("--requirements-darwin") @click.option("--requirements-windows") @@ -90,7 +93,7 @@ def _locate(bazel_runfiles, file): def main( srcs: Tuple[str, ...], requirements_txt: str, - update_target_label: str, + target_label_prefix: str, requirements_linux: Optional[str], requirements_darwin: Optional[str], requirements_windows: Optional[str], @@ -148,13 +151,23 @@ def main( requirements_out = os.path.join( os.environ["TEST_TMPDIR"], os.path.basename(requirements_file) + ".out" ) + # Why this uses shutil.copyfileobj: + # # Those two files won't necessarily be on the same filesystem, so we can't use os.replace # or shutil.copyfile, as they will fail with OSError: [Errno 18] Invalid cross-device link. - shutil.copy(resolved_requirements_file, requirements_out) - - update_command = os.getenv("CUSTOM_COMPILE_COMMAND") or "bazel run %s" % ( - update_target_label, + # + # Further, shutil.copy preserves the source file's mode, and so if + # our source file is read-only (the default under Perforce Helix), + # this scratch file will also be read-only, defeating its purpose. + with open(resolved_requirements_file, "rb") as fsrc, open( + requirements_out, "wb" + ) as fdst: + shutil.copyfileobj(fsrc, fdst) + + update_command = ( + os.getenv("CUSTOM_COMPILE_COMMAND") or f"bazel run {target_label_prefix}.update" ) + test_command = f"bazel test {target_label_prefix}.test" os.environ["CUSTOM_COMPILE_COMMAND"] = update_command os.environ["PIP_CONFIG_FILE"] = os.getenv("PIP_CONFIG_FILE") or os.devnull @@ -168,63 +181,91 @@ def main( ) argv.extend(extra_args) + _run_pip_compile = functools.partial( + run_pip_compile, + argv, + srcs_relative=srcs_relative, + ) + if UPDATE: print("Updating " + requirements_file_relative) + + # Make sure the output file for pip_compile exists. It won't if we are on Windows and --enable_runfiles is not set. + if not os.path.exists(requirements_file_relative): + os.makedirs(os.path.dirname(requirements_file_relative), exist_ok=True) + shutil.copy(resolved_requirements_file, requirements_file_relative) + if "BUILD_WORKSPACE_DIRECTORY" in os.environ: workspace = os.environ["BUILD_WORKSPACE_DIRECTORY"] requirements_file_tree = os.path.join(workspace, requirements_file_relative) + absolute_output_file = Path(requirements_file_relative).absolute() # In most cases, requirements_file will be a symlink to the real file in the source tree. # If symlinks are not enabled (e.g. on Windows), then requirements_file will be a copy, # and we should copy the updated requirements back to the source tree. - if not os.path.samefile(resolved_requirements_file, requirements_file_tree): + if not absolute_output_file.samefile(requirements_file_tree): atexit.register( - lambda: shutil.copy( - resolved_requirements_file, requirements_file_tree - ) + lambda: shutil.copy(absolute_output_file, requirements_file_tree) ) - cli(argv) + _run_pip_compile(verbose_command=f"{update_command} -- --verbose") requirements_file_relative_path = Path(requirements_file_relative) content = requirements_file_relative_path.read_text() content = content.replace(absolute_path_prefix, "") requirements_file_relative_path.write_text(content) else: - # cli will exit(0) on success - try: - print("Checking " + requirements_file) - cli(argv) - print("cli() should exit", file=sys.stderr) + print("Checking " + requirements_file) + sys.stdout.flush() + _run_pip_compile(verbose_command=f"{test_command} --test_arg=--verbose") + golden = open(_locate(bazel_runfiles, requirements_file)).readlines() + out = open(requirements_out).readlines() + out = [line.replace(absolute_path_prefix, "") for line in out] + if golden != out: + import difflib + + print("".join(difflib.unified_diff(golden, out)), file=sys.stderr) + print( + f"Lock file out of date. Run '{update_command}' to update.", + file=sys.stderr, + ) + sys.exit(1) + + +def run_pip_compile( + args: List[str], + *, + srcs_relative: List[str], + verbose_command: str, +) -> None: + try: + cli(args, standalone_mode=False) + except DistributionNotFound as e: + if isinstance(e.__cause__, ResolutionImpossible): + # pip logs an informative error to stderr already + # just render the error and exit + print(e) + sys.exit(1) + else: + raise + except SystemExit as e: + if e.code == 0: + return # shouldn't happen, but just in case + elif e.code == 2: + print( + "pip-compile exited with code 2. This means that pip-compile found " + "incompatible requirements or could not find a version that matches " + f"the install requirement in one of {srcs_relative}.\n" + "Try re-running with verbose:\n" + f" {verbose_command}", + file=sys.stderr, + ) + sys.exit(1) + else: + print( + f"pip-compile unexpectedly exited with code {e.code}.\n" + "Try re-running with verbose:\n" + f" {verbose_command}", + file=sys.stderr, + ) sys.exit(1) - except SystemExit as e: - if e.code == 2: - print( - "pip-compile exited with code 2. This means that pip-compile found " - "incompatible requirements or could not find a version that matches " - f"the install requirement in one of {srcs_relative}.", - file=sys.stderr, - ) - sys.exit(1) - elif e.code == 0: - golden = open(_locate(bazel_runfiles, requirements_file)).readlines() - out = open(requirements_out).readlines() - out = [line.replace(absolute_path_prefix, "") for line in out] - if golden != out: - import difflib - - print("".join(difflib.unified_diff(golden, out)), file=sys.stderr) - print( - "Lock file out of date. Run '" - + update_command - + "' to update.", - file=sys.stderr, - ) - sys.exit(1) - sys.exit(0) - else: - print( - f"pip-compile unexpectedly exited with code {e.code}.", - file=sys.stderr, - ) - sys.exit(1) if __name__ == "__main__": diff --git a/python/private/pypi/deps.bzl b/python/private/pypi/deps.bzl index 8949ed4abe..5d0507cb98 100644 --- a/python/private/pypi/deps.bzl +++ b/python/private/pypi/deps.bzl @@ -21,13 +21,13 @@ _RULE_DEPS = [ # START: maintained by 'bazel run //tools/private/update_deps:update_pip_deps' ( "pypi__build", - "https://files.pythonhosted.org/packages/e2/03/f3c8ba0a6b6e30d7d18c40faab90807c9bb5e9a1e3b2fe2008af624a9c97/build-1.2.1-py3-none-any.whl", - "75e10f767a433d9a86e50d83f418e83efc18ede923ee5ff7df93b6cb0306c5d4", + "https://files.pythonhosted.org/packages/c5/0d/84a4380f930db0010168e0aa7b7a8fed9ba1835a8fbb1472bc6d0201d529/build-1.4.0-py3-none-any.whl", + "6a07c1b8eb6f2b311b96fcbdbce5dab5fe637ffda0fd83c9cac622e927501596", ), ( "pypi__click", - "https://files.pythonhosted.org/packages/00/2e/d53fa4befbf2cfa713304affc7ca780ce4fc1fd8710527771b58311a3229/click-8.1.7-py3-none-any.whl", - "ae74fb96c20a0277a1d615f1e4d73c8414f5a98db8b799a7931d1582f3390c28", + "https://files.pythonhosted.org/packages/98/78/01c019cdb5d6498122777c1a43056ebb3ebfeef2076d9d026bfe15583b2b/click-8.3.1-py3-none-any.whl", + "981153a64e25f12d547d3426c367a4857371575ee7ad18df2a6183ab0545b2a6", ), ( "pypi__colorama", @@ -36,8 +36,8 @@ _RULE_DEPS = [ ), ( "pypi__importlib_metadata", - "https://files.pythonhosted.org/packages/2d/0a/679461c511447ffaf176567d5c496d1de27cbe34a87df6677d7171b2fbd4/importlib_metadata-7.1.0-py3-none-any.whl", - "30962b96c0c223483ed6cc7280e7f0199feb01a0e40cfae4d4450fc6fab1f570", + "https://files.pythonhosted.org/packages/fa/5e/f8e9a1d23b9c20a551a8a02ea3637b4642e22c2626e3a13a9a29cdea99eb/importlib_metadata-8.7.1-py3-none-any.whl", + "5a1f80bf1daa489495071efbb095d75a634cf28a8bc299581244063b53176151", ), ( "pypi__installer", @@ -46,13 +46,13 @@ _RULE_DEPS = [ ), ( "pypi__more_itertools", - "https://files.pythonhosted.org/packages/50/e2/8e10e465ee3987bb7c9ab69efb91d867d93959095f4807db102d07995d94/more_itertools-10.2.0-py3-none-any.whl", - "686b06abe565edfab151cb8fd385a05651e1fdf8f0a14191e4439283421f8684", + "https://files.pythonhosted.org/packages/a4/8e/469e5a4a2f5855992e425f3cb33804cc07bf18d48f2db061aec61ce50270/more_itertools-10.8.0-py3-none-any.whl", + "52d4362373dcf7c52546bc4af9a86ee7c4579df9a8dc268be0a2f949d376cc9b", ), ( "pypi__packaging", - "https://files.pythonhosted.org/packages/49/df/1fceb2f8900f8639e278b056416d49134fb8d84c5942ffaa01ad34782422/packaging-24.0-py3-none-any.whl", - "2ddfb553fdf02fb784c234c7ba6ccc288296ceabec964ad2eae3777778130bc5", + "https://files.pythonhosted.org/packages/b7/b9/c538f279a4e237a006a2c98387d081e9eb060d203d8ed34467cc0f0b9b53/packaging-26.0-py3-none-any.whl", + "b36f1fef9334a5588b4166f8bcd26a14e521f2b55e6b9de3aaa80d3ff7a37529", ), ( "pypi__pep517", @@ -61,38 +61,38 @@ _RULE_DEPS = [ ), ( "pypi__pip", - "https://files.pythonhosted.org/packages/8a/6a/19e9fe04fca059ccf770861c7d5721ab4c2aebc539889e97c7977528a53b/pip-24.0-py3-none-any.whl", - "ba0d021a166865d2265246961bec0152ff124de910c5cc39f1156ce3fa7c69dc", + "https://files.pythonhosted.org/packages/de/f0/c81e05b613866b76d2d1066490adf1a3dbc4ee9d9c839961c3fc8a6997af/pip-26.0.1-py3-none-any.whl", + "bdb1b08f4274833d62c1aa29e20907365a2ceb950410df15fc9521bad440122b", ), ( "pypi__pip_tools", - "https://files.pythonhosted.org/packages/0d/dc/38f4ce065e92c66f058ea7a368a9c5de4e702272b479c0992059f7693941/pip_tools-7.4.1-py3-none-any.whl", - "4c690e5fbae2f21e87843e89c26191f0d9454f362d8acdbd695716493ec8b3a9", + "https://files.pythonhosted.org/packages/6e/74/59906d876c6cb1137f42a137164f2fe683b06283cde84bfcf7f5dd43970b/pip_tools-7.5.3-py3-none-any.whl", + "3aac0c473240ae90db7213c033401f345b05197293ccbdd2704e52e7a783785e", ), ( "pypi__pyproject_hooks", - "https://files.pythonhosted.org/packages/ae/f3/431b9d5fe7d14af7a32340792ef43b8a714e7726f1d7b69cc4e8e7a3f1d7/pyproject_hooks-1.1.0-py3-none-any.whl", - "7ceeefe9aec63a1064c18d939bdc3adf2d8aa1988a510afec15151578b232aa2", + "https://files.pythonhosted.org/packages/bd/24/12818598c362d7f300f18e74db45963dbcb85150324092410c8b49405e42/pyproject_hooks-1.2.0-py3-none-any.whl", + "9e5c6bfa8dcc30091c74b0cf803c81fdd29d94f01992a7707bc97babb1141913", ), ( "pypi__setuptools", - "https://files.pythonhosted.org/packages/de/88/70c5767a0e43eb4451c2200f07d042a4bcd7639276003a9c54a68cfcc1f8/setuptools-70.0.0-py3-none-any.whl", - "54faa7f2e8d2d11bcd2c07bed282eef1046b5c080d1c32add737d7b5817b1ad4", + "https://files.pythonhosted.org/packages/e1/c6/76dc613121b793286a3f91621d7b75a2b493e0390ddca50f11993eadf192/setuptools-82.0.0-py3-none-any.whl", + "70b18734b607bd1da571d097d236cfcfacaf01de45717d59e6e04b96877532e0", ), ( "pypi__tomli", - "https://files.pythonhosted.org/packages/97/75/10a9ebee3fd790d20926a90a2547f0bf78f371b2f13aa822c759680ca7b9/tomli-2.0.1-py3-none-any.whl", - "939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc", + "https://files.pythonhosted.org/packages/23/d1/136eb2cb77520a31e1f64cbae9d33ec6df0d78bdf4160398e86eec8a8754/tomli-2.4.0-py3-none-any.whl", + "1f776e7d669ebceb01dee46484485f43a4048746235e683bcdffacdf1fb4785a", ), ( "pypi__wheel", - "https://files.pythonhosted.org/packages/7d/cd/d7460c9a869b16c3dd4e1e403cce337df165368c71d6af229a74699622ce/wheel-0.43.0-py3-none-any.whl", - "55c570405f142630c6b9f72fe09d9b67cf1477fcf543ae5b8dcb1f5b7377da81", + "https://files.pythonhosted.org/packages/87/22/b76d483683216dde3d67cba61fb2444be8d5be289bf628c13fc0fd90e5f9/wheel-0.46.3-py3-none-any.whl", + "4b399d56c9d9338230118d705d9737a2a468ccca63d5e813e2a4fc7815d8bc4d", ), ( "pypi__zipp", - "https://files.pythonhosted.org/packages/da/55/a03fd7240714916507e1fcf7ae355bd9d9ed2e6db492595f1a67f61681be/zipp-3.18.2-py3-none-any.whl", - "dce197b859eb796242b0622af1b8beb0a722d52aa2f57133ead08edd5bf5374e", + "https://files.pythonhosted.org/packages/2e/54/647ade08bf0db230bfea292f893923872fd20be6ac6f53b2b936ba839d75/zipp-3.23.0-py3-none-any.whl", + "071652d6115ed432f5ce1d34c336c0adfd6a884660d1e9712a256d3d3bd4b14e", ), # END: maintained by 'bazel run //tools/private/update_deps:update_pip_deps' ] @@ -100,8 +100,7 @@ _RULE_DEPS = [ _GENERIC_WHEEL = """\ package(default_visibility = ["//visibility:public"]) -load("@rules_python//python:defs.bzl", "py_library") -load("@rules_python//python/private:glob_excludes.bzl", "glob_excludes") +load("@rules_python//python:py_library.bzl", "py_library") py_library( name = "lib", @@ -115,7 +114,7 @@ py_library( "**/*.dist-info/RECORD", "BUILD", "WORKSPACE", - ] + glob_excludes.version_dependent_exclusions()), + ]), # This makes this directory a top-level in the python import # search path for anything that depends on this. imports = ["."], @@ -124,6 +123,13 @@ py_library( # Collate all the repository names so they can be easily consumed all_repo_names = [name for (name, _, _) in _RULE_DEPS] +record_files = { + name: Label("@{}//:{}.dist-info/RECORD".format( + name, + url.rpartition("/")[-1].partition("-py3-none")[0], + )) + for (name, url, _) in _RULE_DEPS +} def pypi_deps(): """ diff --git a/python/private/pypi/env_marker_info.bzl b/python/private/pypi/env_marker_info.bzl new file mode 100644 index 0000000000..37eefb2a0f --- /dev/null +++ b/python/private/pypi/env_marker_info.bzl @@ -0,0 +1,26 @@ +"""Provider for implementing environment marker values.""" + +EnvMarkerInfo = provider( + doc = """ +The values to use during environment marker evaluation. + +:::{seealso} +The {obj}`--//python/config_settings:pip_env_marker_config` flag. +::: + +:::{versionadded} 1.5.0 +""", + fields = { + "env": """ +:type: dict[str, str] + +The values to use for environment markers when evaluating an expression. + +The keys and values should be compatible with the [PyPA dependency specifiers +specification](https://packaging.python.org/en/latest/specifications/dependency-specifiers/). + +Missing values will be set to the specification's defaults or computed using +available toolchain information. +""", + }, +) diff --git a/python/private/pypi/env_marker_setting.bzl b/python/private/pypi/env_marker_setting.bzl new file mode 100644 index 0000000000..71c6b410ed --- /dev/null +++ b/python/private/pypi/env_marker_setting.bzl @@ -0,0 +1,141 @@ +"""Implement a flag for matching the dependency specifiers at analysis time.""" + +load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") +load("//python/private:common_labels.bzl", "labels") +load("//python/private:toolchain_types.bzl", "TARGET_TOOLCHAIN_TYPE") +load(":env_marker_info.bzl", "EnvMarkerInfo") +load(":pep508_env.bzl", "create_env", "set_missing_env_defaults") +load(":pep508_evaluate.bzl", "evaluate") + +# Use capitals to hint its not an actual boolean type. +_ENV_MARKER_TRUE = "TRUE" +_ENV_MARKER_FALSE = "FALSE" + +def env_marker_setting(*, name, expression, **kwargs): + """Creates an env_marker setting. + + Generated targets: + + * `is_{name}_true`: config_setting that matches when the expression is true. + * `{name}`: env marker target that evalutes the expression. + + Args: + name: {type}`str` target name + expression: {type}`str` the environment marker string to evaluate + **kwargs: {type}`dict` additional common kwargs. + """ + native.config_setting( + name = "is_{}_true".format(name), + flag_values = { + ":{}".format(name): _ENV_MARKER_TRUE, + }, + **kwargs + ) + _env_marker_setting( + name = name, + expression = expression, + **kwargs + ) + +def _env_marker_setting_impl(ctx): + env = create_env() + env.update( + ctx.attr._env_marker_config_flag[EnvMarkerInfo].env, + ) + + runtime = ctx.toolchains[TARGET_TOOLCHAIN_TYPE].py3_runtime + + if "python_version" not in env: + if runtime.interpreter_version_info: + version_info = runtime.interpreter_version_info + env["python_version"] = "{major}.{minor}".format( + major = version_info.major, + minor = version_info.minor, + ) + full_version = _format_full_version(version_info) + env["python_full_version"] = full_version + env["implementation_version"] = full_version + else: + env["python_version"] = _get_flag(ctx.attr._python_version_major_minor_flag) + full_version = _get_flag(ctx.attr._python_full_version_flag) + env["python_full_version"] = full_version + env["implementation_version"] = full_version + + if "implementation_name" not in env and runtime.implementation_name: + env["implementation_name"] = runtime.implementation_name + + set_missing_env_defaults(env) + if evaluate(ctx.attr.expression, env = env): + value = _ENV_MARKER_TRUE + else: + value = _ENV_MARKER_FALSE + return [config_common.FeatureFlagInfo(value = value)] + +_env_marker_setting = rule( + doc = """ +Evaluates an environment marker expression using target configuration info. + +See +https://packaging.python.org/en/latest/specifications/dependency-specifiers +for the specification of behavior. +""", + implementation = _env_marker_setting_impl, + attrs = { + "expression": attr.string( + mandatory = True, + doc = "Environment marker expression to evaluate.", + ), + "_env_marker_config_flag": attr.label( + default = labels.PIP_ENV_MARKER_CONFIG, + providers = [EnvMarkerInfo], + ), + "_python_full_version_flag": attr.label( + default = labels.PYTHON_VERSION, + providers = [config_common.FeatureFlagInfo], + ), + "_python_version_major_minor_flag": attr.label( + default = labels.PYTHON_VERSION_MAJOR_MINOR, + providers = [config_common.FeatureFlagInfo], + ), + }, + provides = [config_common.FeatureFlagInfo], + toolchains = [ + TARGET_TOOLCHAIN_TYPE, + ], +) + +def _format_full_version(info): + """Format the full python interpreter version. + + Adapted from spec code at: + https://packaging.python.org/en/latest/specifications/dependency-specifiers/#environment-markers + + Args: + info: The provider from the Python runtime. + + Returns: + a {type}`str` with the version + """ + kind = info.releaselevel + if kind == "final": + kind = "" + serial = "" + else: + kind = kind[0] if kind else "" + serial = str(info.serial) if info.serial else "" + + return "{major}.{minor}.{micro}{kind}{serial}".format( + v = info, + major = info.major, + minor = info.minor, + micro = info.micro, + kind = kind, + serial = serial, + ) + +def _get_flag(t): + if config_common.FeatureFlagInfo in t: + return t[config_common.FeatureFlagInfo].value + if BuildSettingInfo in t: + return t[BuildSettingInfo].value + fail("Should not occur: {} does not have necessary providers") diff --git a/python/private/pypi/evaluate_markers.bzl b/python/private/pypi/evaluate_markers.bzl index c805fd7a59..34076290db 100644 --- a/python/private/pypi/evaluate_markers.bzl +++ b/python/private/pypi/evaluate_markers.bzl @@ -14,64 +14,29 @@ """A simple function that evaluates markers using a python interpreter.""" -load(":pypi_repo_utils.bzl", "pypi_repo_utils") +load(":pep508_evaluate.bzl", "evaluate") +load(":pep508_requirement.bzl", "requirement") -# Used as a default value in a rule to ensure we fetch the dependencies. -SRCS = [ - # When the version, or any of the files in `packaging` package changes, - # this file will change as well. - Label("@pypi__packaging//:packaging-24.0.dist-info/RECORD"), - Label("//python/private/pypi/requirements_parser:resolve_target_platforms.py"), - Label("//python/private/pypi/whl_installer:platform.py"), -] - -def evaluate_markers(mrctx, *, requirements, python_interpreter, python_interpreter_target, srcs, logger = None): +def evaluate_markers(*, requirements, platforms): """Return the list of supported platforms per requirements line. Args: - mrctx: repository_ctx or module_ctx. - requirements: list[str] of the requirement file lines to evaluate. - python_interpreter: str, path to the python_interpreter to use to - evaluate the env markers in the given requirements files. It will - be only called if the requirements files have env markers. This - should be something that is in your PATH or an absolute path. - python_interpreter_target: Label, same as python_interpreter, but in a - label format. - srcs: list[Label], the value of SRCS passed from the `rctx` or `mctx` to this function. - logger: repo_utils.logger or None, a simple struct to log diagnostic - messages. Defaults to None. + requirements: {type}`dict[str, list[str]]` of the requirement file lines to evaluate. + platforms: {type}`dict[str, dict[str, str]]` The environments that we for each requirement + file to evaluate. The keys between the platforms and requirements should be shared. Returns: dict of string lists with target platforms """ - if not requirements: - return {} - - in_file = mrctx.path("requirements_with_markers.in.json") - out_file = mrctx.path("requirements_with_markers.out.json") - mrctx.file(in_file, json.encode(requirements)) - - pypi_repo_utils.execute_checked( - mrctx, - op = "ResolveRequirementEnvMarkers({})".format(in_file), - arguments = [ - pypi_repo_utils.resolve_python_interpreter( - mrctx, - python_interpreter = python_interpreter, - python_interpreter_target = python_interpreter_target, - ), - "-m", - "python.private.pypi.requirements_parser.resolve_target_platforms", - in_file, - out_file, - ], - srcs = srcs, - environment = { - "PYTHONPATH": [ - Label("@pypi__packaging//:BUILD.bazel"), - Label("//:BUILD.bazel"), - ], - }, - logger = logger, - ) - return json.decode(mrctx.read(out_file)) + ret = {} + for req_string, platform_strings in requirements.items(): + req = requirement(req_string) + for platform_str in platform_strings: + plat = platforms.get(platform_str) + if not plat: + fail("Please define platform: '{}'".format(platform_str)) + + if evaluate(req.marker, env = plat.env): + ret.setdefault(req_string, []).append(platform_str) + + return ret diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index edfd5809f4..ed1fa28664 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -14,34 +14,30 @@ "pip module extension for use with bzlmod" -load("@bazel_features//:features.bzl", "bazel_features") load("@pythons_hub//:interpreters.bzl", "INTERPRETER_LABELS") +load("@pythons_hub//:versions.bzl", "MINOR_MAPPING") +load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") +load("@toml.bzl", "toml") load("//python/private:auth.bzl", "AUTH_ATTRS") load("//python/private:normalize_name.bzl", "normalize_name") +load("//python/private:pyproject_utils.bzl", "read_pyproject", "version_from_requires_python") load("//python/private:repo_utils.bzl", "repo_utils") -load("//python/private:semver.bzl", "semver") -load("//python/private:version_label.bzl", "version_label") -load(":attrs.bzl", "use_isolated") -load(":evaluate_markers.bzl", "evaluate_markers", EVALUATE_MARKERS_SRCS = "SRCS") +load(":hub_builder.bzl", "hub_builder") load(":hub_repository.bzl", "hub_repository", "whl_config_settings_to_json") -load(":parse_requirements.bzl", "parse_requirements") load(":parse_whl_name.bzl", "parse_whl_name") +load(":pep508_env.bzl", "env") load(":pip_repository_attrs.bzl", "ATTRS") -load(":requirements_files_by_platform.bzl", "requirements_files_by_platform") +load(":platform.bzl", _plat = "platform") +load(":pypi_cache.bzl", "pypi_cache") load(":simpleapi_download.bzl", "simpleapi_download") -load(":whl_config_setting.bzl", "whl_config_setting") +load(":unified_hub_repo.bzl", "unified_hub_repo") load(":whl_library.bzl", "whl_library") -load(":whl_repo_name.bzl", "pypi_repo_name", "whl_repo_name") - -def _major_minor_version(version): - version = semver(version) - return "{}.{}".format(version.major, version.minor) def _whl_mods_impl(whl_mods_dict): """Implementation of the pip.whl_mods tag class. This creates the JSON files used to modify the creation of different wheels. -""" + """ for hub_name, whl_maps in whl_mods_dict.items(): whl_mods = {} @@ -62,305 +58,235 @@ def _whl_mods_impl(whl_mods_dict): whl_mods = whl_mods, ) -def _create_whl_repos( - module_ctx, - *, - pip_attr, - whl_overrides, - simpleapi_cache, - evaluate_markers = evaluate_markers, - available_interpreters = INTERPRETER_LABELS, - simpleapi_download = simpleapi_download): - """create all of the whl repositories +def default_platforms(): + """Return the built-in default platform definitions. - Args: - module_ctx: {type}`module_ctx`. - pip_attr: {type}`struct` - the struct that comes from the tag class iteration. - whl_overrides: {type}`dict[str, struct]` - per-wheel overrides. - simpleapi_cache: {type}`dict` - an opaque dictionary used for caching the results from calling - SimpleAPI evaluating all of the tag class invocations {bzl:obj}`pip.parse`. - evaluate_markers: the function to use to evaluate markers. - simpleapi_download: Used for testing overrides - available_interpreters: {type}`dict[str, Label]` The dictionary of available - interpreters that have been registered using the `python` bzlmod extension. - The keys are in the form `python_{snake_case_version}_host`. This is to be - used during the `repository_rule` and must be always compatible with the host. - - Returns a {type}`struct` with the following attributes: - whl_map: {type}`dict[str, list[struct]]` the output is keyed by the - normalized package name and the values are the instances of the - {bzl:obj}`whl_config_setting` return values. - exposed_packages: {type}`dict[str, Any]` this is just a way to - represent a set of string values. - whl_libraries: {type}`dict[str, dict[str, Any]]` the keys are the - aparent repository names for the hub repo and the values are the - arguments that will be passed to {bzl:obj}`whl_library` repository - rule. - is_reproducible: {type}`bool` set to True if does not make calls to the - internet to evaluate the requirements files. + These provide the platform metadata needed for pip wheel resolution + (whl_abi_tags, whl_platform_tags, config_settings, etc.) across all + common OS/arch combinations. They are always used as the starting point + for build_config; root modules can override individual platforms via + pip.default tags. + + Returns: + A dict of platform name to platform config dicts. """ - logger = repo_utils.logger(module_ctx, "pypi:create_whl_repos") - python_interpreter_target = pip_attr.python_interpreter_target - is_reproducible = True + # NOTE @aignas 2025-07-06: we define these platforms to keep backwards compatibility. Whilst we + # stabilize the API this list may be updated with a mention in the CHANGELOG. + + platforms = {} + + # Linux platforms + for cpu in ["x86_64", "aarch64"]: + for freethreaded in ["", "_freethreaded"]: + platform_name = "linux_{}{}".format(cpu, freethreaded) + platforms[platform_name] = { + "arch_name": cpu, + "config_settings": [ + "@platforms//cpu:{}".format(cpu), + "@platforms//os:linux", + "//python/config_settings:_is_py_freethreaded_{}".format( + "yes" if freethreaded else "no", + ), + ], + "env": {"platform_version": "0"}, + "marker": "python_version >= '3.13'" if freethreaded else "", + "name": platform_name, + "os_name": "linux", + "whl_abi_tags": ["cp{major}{minor}t"] if freethreaded else [ + "abi3", + "cp{major}{minor}", + ], + "whl_platform_tags": [ + "linux_{}".format(cpu), + "manylinux_*_{}".format(cpu), + ], + } - # containers to aggregate outputs from this function - whl_map = {} - exposed_packages = {} - extra_aliases = { - whl_name: {alias: True for alias in aliases} - for whl_name, aliases in pip_attr.extra_hub_aliases.items() - } - whl_libraries = {} + # macOS platforms + for cpu, platform_tag_cpus in { + "aarch64": ["universal2", "arm64"], + "x86_64": ["universal2", "x86_64"], + }.items(): + for freethreaded in ["", "_freethreaded"]: + platform_name = "osx_{}{}".format(cpu, freethreaded) + platforms[platform_name] = { + "arch_name": cpu, + "config_settings": [ + "@platforms//cpu:{}".format(cpu), + "@platforms//os:osx", + "//python/config_settings:_is_py_freethreaded_{}".format( + "yes" if freethreaded else "no", + ), + ], + "env": {"platform_version": "14.0"}, + "marker": "python_version >= '3.13'" if freethreaded else "", + "name": platform_name, + "os_name": "osx", + "whl_abi_tags": ["cp{major}{minor}t"] if freethreaded else [ + "abi3", + "cp{major}{minor}", + ], + "whl_platform_tags": [ + "macosx_*_{}".format(suffix) + for suffix in platform_tag_cpus + ], + } - # if we do not have the python_interpreter set in the attributes - # we programmatically find it. - hub_name = pip_attr.hub_name - if python_interpreter_target == None and not pip_attr.python_interpreter: - python_name = "python_{}_host".format( - pip_attr.python_version.replace(".", "_"), - ) - if python_name not in available_interpreters: - fail(( - "Unable to find interpreter for pip hub '{hub_name}' for " + - "python_version={version}: Make sure a corresponding " + - '`python.toolchain(python_version="{version}")` call exists.' + - "Expected to find {python_name} among registered versions:\n {labels}" - ).format( - hub_name = hub_name, - version = pip_attr.python_version, - python_name = python_name, - labels = " \n".join(available_interpreters), - )) - python_interpreter_target = available_interpreters[python_name] + # Windows x86_64 platforms + for freethreaded in ["", "_freethreaded"]: + platform_name = "windows_x86_64{}".format(freethreaded) + platforms[platform_name] = { + "arch_name": "x86_64", + "config_settings": [ + "@platforms//cpu:x86_64", + "@platforms//os:windows", + "//python/config_settings:_is_py_freethreaded_{}".format( + "yes" if freethreaded else "no", + ), + ], + "env": {"platform_version": "0"}, + "marker": "python_version >= '3.13'" if freethreaded else "", + "name": platform_name, + "os_name": "windows", + "whl_abi_tags": ["cp{major}{minor}t"] if freethreaded else [ + "abi3", + "cp{major}{minor}", + ], + "whl_platform_tags": ["win_amd64"], + } - pip_name = "{}_{}".format( - hub_name, - version_label(pip_attr.python_version), - ) - major_minor = _major_minor_version(pip_attr.python_version) + # Windows aarch64 platforms + for freethreaded in ["", "_freethreaded"]: + platform_name = "windows_aarch64{}".format(freethreaded) + platforms[platform_name] = { + "arch_name": "aarch64", + "config_settings": [ + "@platforms//cpu:aarch64", + "@platforms//os:windows", + "//python/config_settings:_is_py_freethreaded_{}".format( + "yes" if freethreaded else "no", + ), + ], + "env": {"platform_version": "0"}, + "marker": "python_version >= '3.13'" if freethreaded else "python_version >= '3.11'", + "name": platform_name, + "os_name": "windows", + "whl_abi_tags": ["cp{major}{minor}t"] if freethreaded else [ + "abi3", + "cp{major}{minor}", + ], + "whl_platform_tags": ["win_arm64"], + } - whl_modifications = {} - if pip_attr.whl_modifications != None: - for mod, whl_name in pip_attr.whl_modifications.items(): - whl_modifications[normalize_name(whl_name)] = mod + return platforms - if pip_attr.experimental_requirement_cycles: - requirement_cycles = { - name: [normalize_name(whl_name) for whl_name in whls] - for name, whls in pip_attr.experimental_requirement_cycles.items() - } +def _configure(config, *, override = False, **kwargs): + """Set the value in the config if the value is provided""" + env = kwargs.get("env") + if env: + for key in env: + if key not in _SUPPORTED_PEP508_KEYS: + fail("Unsupported key in the PEP508 environment: {}".format(key)) - whl_group_mapping = { - whl_name: group_name - for group_name, group_whls in requirement_cycles.items() - for whl_name in group_whls - } - else: - whl_group_mapping = {} - requirement_cycles = {} - - # Create a new wheel library for each of the different whls - - get_index_urls = None - if pip_attr.experimental_index_url: - get_index_urls = lambda ctx, distributions: simpleapi_download( - ctx, - attr = struct( - index_url = pip_attr.experimental_index_url, - extra_index_urls = pip_attr.experimental_extra_index_urls or [], - index_url_overrides = pip_attr.experimental_index_url_overrides or {}, - sources = distributions, - envsubst = pip_attr.envsubst, - # Auth related info - netrc = pip_attr.netrc, - auth_patterns = pip_attr.auth_patterns, - ), - cache = simpleapi_cache, - parallel_download = pip_attr.parallel_download, - ) + for key, value in kwargs.items(): + if value and (override or key not in config): + config[key] = value - requirements_by_platform = parse_requirements( +def build_config( + *, module_ctx, - requirements_by_platform = requirements_files_by_platform( - requirements_by_platform = pip_attr.requirements_by_platform, - requirements_linux = pip_attr.requirements_linux, - requirements_lock = pip_attr.requirements_lock, - requirements_osx = pip_attr.requirements_darwin, - requirements_windows = pip_attr.requirements_windows, - extra_pip_args = pip_attr.extra_pip_args, - python_version = major_minor, - logger = logger, - ), - extra_pip_args = pip_attr.extra_pip_args, - get_index_urls = get_index_urls, - # NOTE @aignas 2024-08-02: , we will execute any interpreter that we find either - # in the PATH or if specified as a label. We will configure the env - # markers when evaluating the requirement lines based on the output - # from the `requirements_files_by_platform` which should have something - # similar to: - # { - # "//:requirements.txt": ["cp311_linux_x86_64", ...] - # } - # - # We know the target python versions that we need to evaluate the - # markers for and thus we don't need to use multiple python interpreter - # instances to perform this manipulation. This function should be executed - # only once by the underlying code to minimize the overhead needed to - # spin up a Python interpreter. - evaluate_markers = lambda module_ctx, requirements: evaluate_markers( - module_ctx, - requirements = requirements, - python_interpreter = pip_attr.python_interpreter, - python_interpreter_target = python_interpreter_target, - srcs = pip_attr._evaluate_markers_srcs, - logger = logger, - ), - logger = logger, - ) + enable_pipstar_extract): + """Parse 'configure' and 'default' extension tags - for whl_name, requirements in requirements_by_platform.items(): - whl_name = normalize_name(whl_name) - - group_name = whl_group_mapping.get(whl_name) - group_deps = requirement_cycles.get(group_name, []) + Args: + module_ctx: {type}`module_ctx` module context. + enable_pipstar_extract: {type}`bool | None` a flag to also not pass Python + interpreter to `whl_library` when possible. - # Construct args separately so that the lock file can be smaller and does not include unused - # attrs. - whl_library_args = dict( - repo = pip_name, - dep_template = "@{}//{{name}}:{{target}}".format(hub_name), - ) - maybe_args = dict( - # The following values are safe to omit if they have false like values - annotation = whl_modifications.get(whl_name), - download_only = pip_attr.download_only, - enable_implicit_namespace_pkgs = pip_attr.enable_implicit_namespace_pkgs, - environment = pip_attr.environment, - envsubst = pip_attr.envsubst, - experimental_target_platforms = pip_attr.experimental_target_platforms, - group_deps = group_deps, - group_name = group_name, - pip_data_exclude = pip_attr.pip_data_exclude, - python_interpreter = pip_attr.python_interpreter, - python_interpreter_target = python_interpreter_target, - whl_patches = { - p: json.encode(args) - for p, args in whl_overrides.get(whl_name, {}).items() - }, - ) - whl_library_args.update({k: v for k, v in maybe_args.items() if v}) - maybe_args_with_default = dict( - # The following values have defaults next to them - isolated = (use_isolated(module_ctx, pip_attr), True), - quiet = (pip_attr.quiet, True), - timeout = (pip_attr.timeout, 600), - ) - whl_library_args.update({ - k: v - for k, (v, default) in maybe_args_with_default.items() - if v != default - }) - - is_exposed = False - if get_index_urls: - # TODO @aignas 2024-05-26: move to a separate function - found_something = False - for requirement in requirements: - is_exposed = is_exposed or requirement.is_exposed - dists = requirement.whls - if not pip_attr.download_only and requirement.sdist: - dists = dists + [requirement.sdist] - - for distribution in dists: - found_something = True - is_reproducible = False - - args = dict(whl_library_args) - if pip_attr.netrc: - args["netrc"] = pip_attr.netrc - if pip_attr.auth_patterns: - args["auth_patterns"] = pip_attr.auth_patterns - - if not distribution.filename.endswith(".whl"): - # pip is not used to download wheels and the python - # `whl_library` helpers are only extracting things, however - # for sdists, they will be built by `pip`, so we still - # need to pass the extra args there. - args["extra_pip_args"] = requirement.extra_pip_args - - # This is no-op because pip is not used to download the wheel. - args.pop("download_only", None) - - repo_name = whl_repo_name(pip_name, distribution.filename, distribution.sha256) - args["requirement"] = requirement.srcs.requirement - args["urls"] = [distribution.url] - args["sha256"] = distribution.sha256 - args["filename"] = distribution.filename - args["experimental_target_platforms"] = requirement.target_platforms - - # Pure python wheels or sdists may need to have a platform here - target_platforms = None - if distribution.filename.endswith("-any.whl") or not distribution.filename.endswith(".whl"): - if len(requirements) > 1: - target_platforms = requirement.target_platforms - - whl_libraries[repo_name] = args - - whl_map.setdefault(whl_name, {})[whl_config_setting( - version = major_minor, - filename = distribution.filename, - target_platforms = target_platforms, - )] = repo_name - - if found_something: - if is_exposed: - exposed_packages[whl_name] = None - continue + Returns: + A struct with the configuration. + """ + default_hub = None + defaults = { + "platforms": default_platforms(), + "python_version": None, + } + for mod in module_ctx.modules: + if not (mod.is_root or mod.name == "rules_python"): + continue + + for tag in mod.tags.default: + if tag.default_hub: + if mod.is_root: + if default_hub: + fail("Duplicate pip.default tag: only one explicit default PyPI hub is allowed.") + default_hub = tag.default_hub + pyproject_toml = tag.pyproject_toml + if pyproject_toml: + pyproject = read_pyproject(module_ctx, pyproject_toml) + if pyproject.requires_python: + defaults["python_version"] = version_from_requires_python(pyproject.requires_python) + + platform = tag.platform + if platform: + specific_config = defaults["platforms"].setdefault(platform, {}) + _configure( + specific_config, + arch_name = tag.arch_name, + config_settings = tag.config_settings, + env = tag.env, + os_name = tag.os_name, + marker = tag.marker, + name = platform.replace("-", "_").lower(), + whl_abi_tags = tag.whl_abi_tags, + whl_platform_tags = tag.whl_platform_tags, + override = mod.is_root, + ) - is_exposed = False - for requirement in requirements: - is_exposed = is_exposed or requirement.is_exposed - if get_index_urls: - logger.warn(lambda: "falling back to pip for installing the right file for {}".format(requirement.requirement_line)) - - args = dict(whl_library_args) # make a copy - args["requirement"] = requirement.requirement_line - if requirement.extra_pip_args: - args["extra_pip_args"] = requirement.extra_pip_args - - if pip_attr.download_only: - args.setdefault("experimental_target_platforms", requirement.target_platforms) - - target_platforms = requirement.target_platforms if len(requirements) > 1 else [] - repo_name = pypi_repo_name( - pip_name, - whl_name, - *target_platforms + if platform and not (tag.arch_name or tag.config_settings or tag.env or tag.os_name or tag.whl_abi_tags or tag.whl_platform_tags or tag.marker): + defaults["platforms"].pop(platform) + + _configure( + defaults, + override = mod.is_root, + index_url = tag.index_url, + # extra values that we just add + auth_patterns = tag.auth_patterns, + netrc = tag.netrc, + # TODO @aignas 2025-05-19: add more attr groups: + # * for index/downloader config. This includes all of those attributes for + # overrides, etc. Index overrides per platform could be also used here. ) - whl_libraries[repo_name] = args - whl_map.setdefault(whl_name, {})[whl_config_setting( - version = major_minor, - target_platforms = target_platforms or None, - )] = repo_name - - if is_exposed: - exposed_packages[whl_name] = None return struct( - is_reproducible = is_reproducible, - whl_map = whl_map, - exposed_packages = exposed_packages, - extra_aliases = extra_aliases, - whl_libraries = whl_libraries, + auth_patterns = defaults.get("auth_patterns", {}), + default_hub = default_hub, + index_url = defaults.get("index_url", "https://pypi.org/simple").rstrip("/"), + netrc = defaults.get("netrc", None), + python_version = defaults.get("python_version", None), + platforms = { + name: _plat(**values) + for name, values in defaults["platforms"].items() + }, + enable_pipstar_extract = enable_pipstar_extract, + toml_decode = toml.decode, ) -def parse_modules(module_ctx, _fail = fail, **kwargs): +def parse_modules( + module_ctx, + _fail = fail, + simpleapi_download = simpleapi_download, + enable_pipstar_extract = False, + **kwargs): """Implementation of parsing the tag classes for the extension and return a struct for registering repositories. Args: module_ctx: {type}`module_ctx` module context. + simpleapi_download: Used for testing overrides + enable_pipstar_extract: {type}`bool` a flag to enable dropping Python dependency for + extracting wheels. _fail: {type}`function` the failure function, mainly for testing. - **kwargs: Extra arguments passed to the layers below. + **kwargs: Extra arguments passed to the hub_builder. Returns: A struct with the following attributes: @@ -396,12 +322,17 @@ You cannot use both the additive_build_content and additive_build_content_file a srcs_exclude_glob = whl_mod.srcs_exclude_glob, ) + config = build_config(module_ctx = module_ctx, enable_pipstar_extract = enable_pipstar_extract) + + # TODO @aignas 2025-06-03: Merge override API with the builder? _overriden_whl_set = {} whl_overrides = {} for module in module_ctx.modules: for attr in module.tags.override: if not module.is_root: - fail("overrides are only supported in root modules") + # Overrides are only supported in root modules. Silently + # ignore the override: + continue if not attr.file.endswith(".whl"): fail("Only whl overrides are supported at this time") @@ -426,28 +357,68 @@ You cannot use both the additive_build_content and additive_build_content_file a # Used to track all the different pip hubs and the spoke pip Python # versions. + # dict[str repo, HubBuilder] + # See `hub_builder.bzl%hub_builder()` for `HubBuilder` pip_hub_map = {} - simpleapi_cache = {} - - # Keeps track of all the hub's whl repos across the different versions. - # dict[hub, dict[whl, dict[version, str pip]]] - # Where hub, whl, and pip are the repo names - hub_whl_map = {} - hub_group_map = {} - exposed_packages = {} - extra_aliases = {} - whl_libraries = {} + simpleapi_cache = pypi_cache(mctx = module_ctx) - is_reproducible = True + is_pypi_hub_reserved = module_ctx.getenv("RULES_PYTHON_PYPI_HUB_RESERVED", "0") == "1" + renamed_default_hub = None for mod in module_ctx.modules: for pip_attr in mod.tags.parse: + python_version = pip_attr.python_version + if not python_version and pip_attr.pyproject_toml: + pyproject = read_pyproject(module_ctx, pip_attr.pyproject_toml) + if pyproject.requires_python: + python_version = version_from_requires_python(pyproject.requires_python) + python_version = python_version or config.python_version + if not python_version: + _fail("pip.parse() requires one of `python_version`, `pyproject_toml`, or `pip.default(pyproject_toml=...)` to be set") + hub_name = pip_attr.hub_name + if hub_name == "pypi": + if is_pypi_hub_reserved: + renamed_name = mod.name + "_pypi" + print( + ( + "WARNING: The PyPI hub name 'pypi' is reserved " + + "(module '{}'). The hub was renamed to '{}'. " + + "Please rename your hub." + ).format( + mod.name, + renamed_name, + ), + ) # buildifier: disable=print + hub_name = renamed_name + if not renamed_default_hub: + renamed_default_hub = hub_name + else: + print( + ( + "WARNING: The PyPI hub name 'pypi' is reserved " + + "(module '{}'). Please rename your hub, otherwise " + + "a future release will rename it to '{}_pypi'." + ).format( + mod.name, + mod.name, + ), + ) # buildifier: disable=print + if hub_name not in pip_hub_map: - pip_hub_map[pip_attr.hub_name] = struct( + builder = hub_builder( + name = hub_name, module_name = mod.name, - python_versions = [pip_attr.python_version], + config = config, + whl_overrides = whl_overrides, + simpleapi_download_fn = simpleapi_download, + simpleapi_cache = simpleapi_cache, + # TODO @aignas 2025-09-06: do not use kwargs + minor_mapping = kwargs.get("minor_mapping", MINOR_MAPPING), + available_interpreters = kwargs.get("available_interpreters", INTERPRETER_LABELS), + logger = repo_utils.logger(module_ctx, "pypi:hub:" + hub_name, mod = mod), ) + pip_hub_map[hub_name] = builder elif pip_hub_map[hub_name].module_name != mod.name: # We cannot have two hubs with the same name in different # modules. @@ -462,77 +433,108 @@ You cannot use both the additive_build_content and additive_build_content_file a second_module = mod.name, )) - elif pip_attr.python_version in pip_hub_map[hub_name].python_versions: - fail(( - "Duplicate pip python version '{version}' for hub " + - "'{hub}' in module '{module}': the Python versions " + - "used for a hub must be unique" - ).format( - hub = hub_name, - module = mod.name, - version = pip_attr.python_version, - )) else: - pip_hub_map[pip_attr.hub_name].python_versions.append(pip_attr.python_version) + builder = pip_hub_map[hub_name] - out = _create_whl_repos( + builder.pip_parse( module_ctx, pip_attr = pip_attr, - simpleapi_cache = simpleapi_cache, - whl_overrides = whl_overrides, - **kwargs + python_version = python_version, ) - hub_whl_map.setdefault(hub_name, {}) - for key, settings in out.whl_map.items(): - for setting, repo in settings.items(): - hub_whl_map[hub_name].setdefault(key, {}).setdefault(repo, []).append(setting) - extra_aliases.setdefault(hub_name, {}) - for whl_name, aliases in out.extra_aliases.items(): - extra_aliases[hub_name].setdefault(whl_name, {}).update(aliases) - exposed_packages.setdefault(hub_name, {}).update(out.exposed_packages) - whl_libraries.update(out.whl_libraries) - is_reproducible = is_reproducible and out.is_reproducible - - # TODO @aignas 2024-04-05: how do we support different requirement - # cycles for different abis/oses? For now we will need the users to - # assume the same groups across all versions/platforms until we start - # using an alternative cycle resolution strategy. - hub_group_map[hub_name] = pip_attr.experimental_requirement_cycles + + # dict[str package, dict[str, None] extra_targets] + declared_deps = {} + for mod in module_ctx.modules: + for dep_attr in mod.tags.dep: + name = normalize_name(dep_attr.name) + targets = declared_deps.setdefault(name, {}) + for target in dep_attr.extra_targets: + targets[target] = None + + # Keeps track of all the hub's whl repos across the different versions. + # dict[hub, dict[whl, dict[version, str pip]]] + # Where hub, whl, and pip are the repo names + hub_whl_map = {} + hub_group_map = {} + exposed_packages = {} + extra_aliases = {} + whl_libraries = {} + for hub in pip_hub_map.values(): + out = hub.build() + + for whl_name, lib in out.whl_libraries.items(): + if whl_name in whl_libraries: + fail("'{}' already in created".format(whl_name)) + else: + whl_libraries[whl_name] = lib + + exposed_packages[hub.name] = out.exposed_packages + extra_aliases[hub.name] = out.extra_aliases + hub_group_map[hub.name] = out.group_map + hub_whl_map[hub.name] = out.whl_map return struct( - # We sort so that the lock-file remains the same no matter the order of how the - # args are manipulated in the code going before. - whl_mods = dict(sorted(whl_mods.items())), - hub_whl_map = { - hub_name: { - whl_name: dict(settings) - for whl_name, settings in sorted(whl_map.items()) - } - for hub_name, whl_map in sorted(hub_whl_map.items()) - }, - hub_group_map = { - hub_name: { - key: sorted(values) - for key, values in sorted(group_map.items()) - } - for hub_name, group_map in sorted(hub_group_map.items()) - }, - exposed_packages = { - k: sorted(v) - for k, v in sorted(exposed_packages.items()) - }, - extra_aliases = { + config = config, + declared_deps = declared_deps, + default_hub = config.default_hub or renamed_default_hub, + exposed_packages = exposed_packages, + extra_aliases = extra_aliases, + facts = simpleapi_cache.get_facts(), + hub_group_map = hub_group_map, + hub_whl_map = hub_whl_map, + whl_libraries = whl_libraries, + whl_mods = whl_mods, + platform_config_settings = { hub_name: { - whl_name: sorted(aliases) - for whl_name, aliases in extra_whl_aliases.items() + platform_name: sorted([str(Label(cv)) for cv in p.config_settings]) + for platform_name, p in config.platforms.items() } - for hub_name, extra_whl_aliases in extra_aliases.items() + for hub_name in hub_whl_map }, - whl_libraries = { - k: dict(sorted(args.items())) - for k, args in sorted(whl_libraries.items()) - }, - is_reproducible = is_reproducible, + ) + +def _create_unified_hub_repo(mods): + if "pypi" in mods.hub_whl_map: + return + + hubs = sorted(mods.hub_whl_map.keys()) + if mods.default_hub and mods.default_hub not in hubs: + fail("default_hub '%s' is not a defined PyPI hub. Available hubs: %s" % (mods.default_hub, ", ".join(hubs))) + + packages = {} + extra_aliases = {} + + for hub_name in hubs: + for pkg_name in mods.exposed_packages.get(hub_name, []): + norm_pkg = normalize_name(pkg_name) + if norm_pkg not in packages: + packages[norm_pkg] = [] + if hub_name not in packages[norm_pkg]: + packages[norm_pkg].append(hub_name) + + extra = mods.extra_aliases.get(hub_name, {}).get(norm_pkg, []) + for alias_name in extra: + qual_alias = "%s:%s" % (norm_pkg, alias_name) + if qual_alias not in extra_aliases: + extra_aliases[qual_alias] = [] + if hub_name not in extra_aliases[qual_alias]: + extra_aliases[qual_alias].append(hub_name) + + for norm_pkg, extra_targets in mods.declared_deps.items(): + if norm_pkg not in packages: + packages[norm_pkg] = [] + + for target_name in extra_targets: + qual_alias = "%s:%s" % (norm_pkg, target_name) + if qual_alias not in extra_aliases: + extra_aliases[qual_alias] = [] + + unified_hub_repo( + name = "pypi", + default_hub = mods.default_hub or (hubs[0] if hubs else ""), + extra_aliases = extra_aliases, + hubs = hubs, + packages = packages, ) def _pip_impl(module_ctx): @@ -601,7 +603,10 @@ def _pip_impl(module_ctx): module_ctx: module contents """ - mods = parse_modules(module_ctx) + mods = parse_modules( + module_ctx, + enable_pipstar_extract = rp_config.bazel_8_or_later, + ) # Build all of the wheel modifications if the tag class is called. _whl_mods_impl(mods.whl_mods) @@ -619,19 +624,204 @@ def _pip_impl(module_ctx): for key, values in whl_map.items() }, packages = mods.exposed_packages.get(hub_name, []), + platform_config_settings = mods.platform_config_settings.get(hub_name, {}), groups = mods.hub_group_map.get(hub_name), ) - if bazel_features.external_deps.extension_metadata_has_reproducible: - # If we are not using the `experimental_index_url feature, the extension is fully - # deterministic and we don't need to create a lock entry for it. - # - # In order to be able to dogfood the `experimental_index_url` feature before it gets - # stabilized, we have created the `_pip_non_reproducible` function, that will result - # in extra entries in the lock file. - return module_ctx.extension_metadata(reproducible = mods.is_reproducible) + _create_unified_hub_repo(mods) + + # The code is smart to not return facts if we don't support the mechanism for that. + # Hence we should not pass it to the metadata + if mods.facts: + return module_ctx.extension_metadata( + reproducible = True, + facts = mods.facts, + ) else: - return None + return module_ctx.extension_metadata(reproducible = True) + +_default_attrs = { + "arch_name": attr.string( + doc = """\ +The CPU architecture name to be used. +You can use any cpu name from the `@platforms//cpu:` package. + +:::{note} +Either this or {attr}`env` `platform_machine` key should be specified. +::: +""", + ), + "config_settings": attr.label_list( + doc = """\ +The list of labels to `config_setting` targets that need to be matched for the platform to be +selected. Mandatory if platform is specified. +""", + ), + "default_hub": attr.string( + doc = "The name of the concrete PyPI hub to use by default when {flag}`--venv=auto`.", + ), + "env": attr.string_dict( + doc = """\ +The values to use for environment markers when evaluating an expression. + +The keys and values should be compatible with the [PyPA dependency specifiers +specification](https://packaging.python.org/en/latest/specifications/dependency-specifiers/). + +Missing values will be set to the specification's defaults or computed using +available toolchain information. + +Supported keys: +* `implementation_name`, defaults to `cpython`. +* `os_name`, defaults to a value inferred from the {attr}`os_name`. +* `platform_machine`, defaults to a value inferred from the {attr}`arch_name`. +* `platform_release`, defaults to an empty value. +* `platform_system`, defaults to a value inferred from the {attr}`os_name`. +* `platform_version`, defaults to `0`. +* `sys_platform`, defaults to a value inferred from the {attr}`os_name`. +""", + ), + "index_url": attr.string( + doc = """\ +The index URL to use as a default when downloading packages from PyPI. This is used if nothing is +specified via `--index-url` or `--extra-index-url` parameters in the `requirements.txt` file or via +the {attr}`pip.parse.extra_pip_args`. + +This value is going to be subject to `envsubst` substitutions if necessary, look at the +{attr}`pip.parse.envsubst` documentation for more information.. + +The indexes must support Simple API as described here: + + +Index metadata will be used to get the hash digest values for packages even +if the `--hash` values are not present in the requirements.txt lock file. + +Defaults to `https://pypi.org/simple`. + +:::{versionadded} 2.0.0 +This has been added as a replacement for +{obj}`pip.parse.experimental_index_url` and +{obj}`pip.parse.experimental_extra_index_urls`. +::: +""", + ), + "marker": attr.string( + doc = """\ +An environment marker expression that is used to enable/disable platforms for specific python +versions, operating systems or CPU architectures. + +If specified, the expression is evaluated during the `bzlmod` extension evaluation phase and if it +evaluates to `True`, then the platform will be used to construct the hub repositories, otherwise, it +will be skipped. + +This is especially useful for setting up freethreaded platform variants only for particular Python +versions for which the interpreter builds are available. However, this could be also used for other +things, such as setting up platforms for different `libc` variants. +""", + ), + # The values for PEP508 env marker evaluation during the lock file parsing + "os_name": attr.string( + doc = """\ +The OS name to be used. +You can use any OS name from the `@platforms//os:` package. + +:::{note} +Either this or the appropriate `env` keys should be specified. +::: +""", + ), + "platform": attr.string( + doc = """\ +A platform identifier which will be used as the unique identifier within the extension evaluation. +If you are defining custom platforms in your project and don't want things to clash, use extension +[isolation] feature. + +[isolation]: https://bazel.build/rules/lib/globals/module#use_extension.isolate +""", + ), + "pyproject_toml": attr.label( + mandatory = False, + doc = """\ +Label pointing to pyproject.toml file to read the default Python version from. +When specified, reads the `requires-python` field from pyproject.toml and uses +it as the default python_version for all `pip.parse()` calls that don't +explicitly specify one. + +:::{note} +The version must be specified as `==X.Y.Z` (exact version with full semver). +This is designed to work with dependency management tools like Renovate. +::: + +:::{versionadded} 2.3.0 +::: +""", + ), + "whl_abi_tags": attr.string_list( + doc = """\ +A list of ABIs to select wheels for. The values can be either strings or include template +parameters like `{major}` and `{minor}` which will be replaced with python version parts. e.g. +`cp{major}{minor}` will result in `cp313` given the full python version is `3.13.5`. +Will always include `"none"` even if it is not specified. + +:::{note} +We select a single wheel and the last match will take precedence. +::: + +:::{seealso} +See official [docs](https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#abi-tag) for more information. +::: +""", + ), + "whl_platform_tags": attr.string_list( + doc = """\ +A list of `platform_tag` matchers so that we can select the best wheel based on the user +preference. +Will always include `"any"` even if it is not specified. + +The items in this list can contain a single `*` character that is equivalent to matching the +lowest available version component in the platform_tag. If the wheel platform tag does not +have a version component, e.g. `linux_x86_64` or `win_amd64`, then `*` will act as a regular character. + +:::{note} +Normally, the `*` in the matcher means that we will target the lowest platform version that we can +and will give preference to whls built targeting the older versions of the platform. If you +specify the version, then we will use the MVS (Minimal Version Selection) algorithm to select the +compatible wheel. As such, you need to keep in mind how to configure the target platforms to +select a particular wheel of your preference. + +We select a single wheel and the last match will take precedence, if the platform_tag that we +match has a version component (e.g. `android_x_arch`, then the version `x` will be used in the +MVS matching algorithm). + +Common patterns: +* To select any versioned wheel for an ``, ``, use `_*_`, e.g. + `manylinux_2_17_x86_64`. +* To exclude versions up to `X.Y` - **submit a PR supporting this feature**. +* To exclude versions above `X.Y`, provide the full platform tag specifier, e.g. + `musllinux_1_2_x86_64`, which will ensure that no wheels with `musllinux_1_3_x86_64` or higher + are selected. +::: + +:::{seealso} +See official [docs](https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#platform-tag) for more information. +::: +:::{versionchanged} 1.6.3 +The matching of versioned platforms have been switched to MVS (Minimal Version Selection) +algorithm for easier evaluation logic and fewer surprises. The legacy platform tags are +supported from this version without extra handling from the user. +::: +""", + ), +} | AUTH_ATTRS + +_SUPPORTED_PEP508_KEYS = [ + "implementation_name", + "os_name", + "platform_machine", + "platform_release", + "platform_system", + "platform_version", + "sys_platform", +] def _pip_parse_ext_attrs(**kwargs): """Get the attributes for the pip extension. @@ -646,33 +836,23 @@ def _pip_parse_ext_attrs(**kwargs): attrs = dict({ "experimental_extra_index_urls": attr.string_list( doc = """\ -The extra index URLs to use for downloading wheels using bazel downloader. -Each value is going to be subject to `envsubst` substitutions if necessary. - -The indexes must support Simple API as described here: -https://packaging.python.org/en/latest/specifications/simple-repository-api/ +May be removed in future releases. -This is equivalent to `--extra-index-urls` `pip` option. +:::{versionchanged} 2.0.0 +This is deprecated, please use {obj}`pip.default.index_url` or pass the `--index-url` parameter via the +lock-file or {obj}`pip.parse.extra_pip_args`. +::: """, default = [], ), "experimental_index_url": attr.string( default = kwargs.get("experimental_index_url", ""), doc = """\ -The index URL to use for downloading wheels using bazel downloader. This value is going -to be subject to `envsubst` substitutions if necessary. - -The indexes must support Simple API as described here: -https://packaging.python.org/en/latest/specifications/simple-repository-api/ +May be removed in future releases. -In the future this could be defaulted to `https://pypi.org` when this feature becomes -stable. - -This is equivalent to `--index-url` `pip` option. - -:::{versionchanged} 0.37.0 -If {attr}`download_only` is set, then `sdist` archives will be discarded and `pip.parse` will -operate in wheel-only mode. +:::{versionchanged} 2.0.0 +This is deprecated, please use {obj}`pip.default.index_url` or pass the `--index-url` parameter via the +lock-file or {obj}`pip.parse.extra_pip_args`. ::: """, ), @@ -687,10 +867,10 @@ index URL. This design pattern has been chosen in order to be fully deterministic about which packages come from which source. We want to avoid issues similar to what happened in -https://pytorch.org/blog/compromised-nightly-dependency/. +. The indexes must support Simple API as described here: -https://packaging.python.org/en/latest/specifications/simple-repository-api/ + """, ), "hub_name": attr.string( @@ -698,6 +878,11 @@ https://packaging.python.org/en/latest/specifications/simple-repository-api/ doc = """ The name of the repo pip dependencies will be accessible from. +The hub name `"pypi"` is reserved for the automatically generated +[Unified @pypi Hub](unified-pypi-hub) repository. Please choose a different name +for your concrete hubs. See [Unified @pypi Hub](unified-pypi-hub) for how to +handle collisions. + This name must be unique between modules; unless your module is guaranteed to always be the root module, it's highly recommended to include your module name in the hub name. Repo mapping, `use_repo(..., pip="my_modules_pip_deps")`, can @@ -713,6 +898,12 @@ is not required. Each hub is a separate resolution of pip dependencies. This means if different programs need different versions of some library, separate hubs can be created, and each program can use its respective hub's targets. Targets from different hubs should not be used together. + +:::{versionchanged} 2.2.0 +Using the hub name `"pypi"` is deprecated and is changed to +`{module_name}_pypi` depending on the +{envvar}`RULES_PYTHON_PYPI_HUB_RESERVED` environment variable. +::: """, ), "parallel_download": attr.bool( @@ -732,8 +923,23 @@ find in case extra indexes are specified. """, default = True, ), + "pyproject_toml": attr.label( + mandatory = False, + doc = """\ +Label pointing to a pyproject.toml file to read the Python version from. +When specified, the `requires-python` field is used as the `python_version` +for this `pip.parse()` call, unless `python_version` is set explicitly. + +:::{note} +The version must be specified as `==X.Y.Z` (exact version with full semver). +::: + +:::{versionadded} 2.3.0 +::: +""", + ), "python_version": attr.string( - mandatory = True, + mandatory = False, doc = """ The Python version the dependencies are targetting, in Major.Minor format (e.g., "3.11") or patch level granularity (e.g. "3.11.1"). @@ -741,6 +947,56 @@ The Python version the dependencies are targetting, in Major.Minor format If an interpreter isn't explicitly provided (using `python_interpreter` or `python_interpreter_target`), then the version specified here must have a corresponding `python.toolchain()` configured. + +:::{seealso} +The {obj}`pyproject_toml` attribute for getting the version from a project file. +::: + +:::{versionchanged} 2.3.0 +No longer mandatory if the {obj}`pyproject_toml` attribute or +{obj}`pip.default.pyproject_toml` is specified. +::: +""", + ), + "simpleapi_skip": attr.string_list( + doc = """\ +The list of packages to skip fetching metadata for from SimpleAPI index. You should +normally not need this attribute, but in case you do, please report this as a bug +to `rules_python` and use this attribute until the bug is fixed. + +EXPERIMENTAL: this may be removed without notice. + +:::{versionadded} 1.4.0 +::: +""", + ), + "target_platforms": attr.string_list( + default = [], + doc = """\ +The list of platforms for which we would evaluate the requirements files. If you need to be able to +only evaluate for a particular platform (e.g. "linux_x86_64"), then put it in here. + +If you want `freethreaded` variant, then you can use `_freethreaded` suffix as `rules_python` is +defining target platforms for these variants in its `MODULE.bazel` file. The identifiers for this +function in general are the same as used in the {obj}`pip.default.platform` attribute. + +If you only care for the host platform and do not have a usecase to cross-build, then you can put in +a string `"{os}_{arch}"` as the value here. You could also use `"{os}_{arch}_freethreaded"` as well. + +:::{include} /_includes/experimental_api.md +::: + +:::{versionadded} 1.8.0 +::: +""", + ), + "uv_lock": attr.label( + doc = """ +(label, optional): A label pointing to the uv.lock file. If provided, +the uv.lock file will be used as the primary source for package metadata. + +:::{versionadded} 2.2.0 +::: """, ), "whl_modifications": attr.label_keyed_string_dict( @@ -748,13 +1004,6 @@ a corresponding `python.toolchain()` configured. doc = """\ A dict of labels to wheel names that is typically generated by the whl_modifications. The labels are JSON config files describing the modifications. -""", - ), - "_evaluate_markers_srcs": attr.label_list( - default = EVALUATE_MARKERS_SRCS, - doc = """\ -The list of labels to use as SRCS for the marker evaluation code. This ensures that the -code will be re-evaluated when any of files in the default changes. """, ), }, **ATTRS) @@ -847,7 +1096,33 @@ Apply any overrides (e.g. patches) to a given Python distribution defined by other tags in this extension.""", ) +_dep_tag = tag_class( + attrs = { + "extra_targets": attr.string_list( + doc = """\ +A list of extra target names in the package that are expected to be available. +See {obj}`pip.parse.extra_hub_aliases`. +""", + default = [], + ), + "name": attr.string( + doc = "The name of a pypi package. Note that the name is normalized.", + mandatory = True, + ), + }, + doc = """\ +Declare an abstract PyPI dependency to ensure its target structure exists in the unified hub. + +This is useful for targets or rules that need to depend on a package (e.g., `@pypi//numpy`) +but do not want to force a specific version or concrete requirements lock file on their +consumers. The concrete version and implementation must be provided by downstreams calling +`pip.parse`. If they are not, the target will still be defined, but it will result in an +execution-phase error when built. +""", +) + pypi = module_extension( + environ = ["RULES_PYTHON_PYPI_HUB_RESERVED"], doc = """\ This extension is used to make dependencies from pip available. @@ -861,12 +1136,42 @@ can be made to configure different Python versions, and will be grouped by the `hub_name` argument. This allows the same logical name, e.g. `@pip//numpy` to automatically resolve to different, Python version-specific, libraries. +A unified `@pypi` proxy repository is always generated (unless a hub is +explicitly named "pypi") to route dependencies dynamically. See +[Unified @pypi Hub](unified-pypi-hub) for details. + +Environment Variables: +- `RULES_PYTHON_PYPI_HUB_RESERVED`: Enable fallback renaming for reserved hub name collisions. + See the {envvar}`RULES_PYTHON_PYPI_HUB_RESERVED` documentation for details. + pip.whl_mods: This tag class is used to help create JSON files to describe modifications to the BUILD files for wheels. """, implementation = _pip_impl, tag_classes = { + "default": tag_class( + attrs = _default_attrs, + doc = """\ +This tag class allows for more customization of how the configuration for the hub repositories is built. + +It can also be used to designate the default hub for the automatically +generated [Unified @pypi Hub](unified-pypi-hub) using the `default_hub` +attribute. + + +:::{seealso} +The [environment markers][environment_markers] specification for the explanation of the +terms used in this extension. + +[environment_markers]: https://packaging.python.org/en/latest/specifications/dependency-specifiers/#environment-markers +::: + +:::{versionadded} 1.6.0 +::: +""", + ), + "dep": _dep_tag, "override": _override_tag, "parse": tag_class( attrs = _pip_parse_ext_attrs(), @@ -875,6 +1180,9 @@ This tag class is used to create a pip hub and all of the spokes that are part o This tag class reuses most of the attributes found in {bzl:obj}`pip_parse`. The exception is it does not use the arg 'repo_prefix'. We set the repository prefix for the user and the alias arg is always True in bzlmod. + +You can use the automatically generated [Unified @pypi Hub](unified-pypi-hub) +repository to route package dependencies dynamically at build time. """, ), "whl_mods": tag_class( diff --git a/python/private/pypi/flags.bzl b/python/private/pypi/flags.bzl index 11727b5853..36cb5373ff 100644 --- a/python/private/pypi/flags.bzl +++ b/python/private/pypi/flags.bzl @@ -18,53 +18,79 @@ NOTE: The transitive loads of this should be kept minimal. This avoids loading unnecessary files when all that are needed are flag definitions. """ -load("@bazel_skylib//rules:common_settings.bzl", "string_flag") -load("//python/private:enum.bzl", "enum") - -# Determines if we should use whls for third party -# -# buildifier: disable=name-conventions -UseWhlFlag = enum( - # Automatically decide the effective value based on environment, target - # platform and the presence of distributions for a particular package. - AUTO = "auto", - # Do not use `sdist` and fail if there are no available whls suitable for the target platform. - ONLY = "only", - # Do not use whl distributions and instead build the whls from `sdist`. - NO = "no", +load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") +load("//python/private:common_labels.bzl", "labels") +load(":env_marker_info.bzl", "EnvMarkerInfo") +load( + ":pep508_env.bzl", + "create_env", + "os_name_select_map", + "platform_machine_select_map", + "platform_system_select_map", + "sys_platform_select_map", ) -# Determines whether universal wheels should be preferred over arch platform specific ones. -# -# buildifier: disable=name-conventions -UniversalWhlFlag = enum( - # Prefer platform-specific wheels over universal wheels. - ARCH = "arch", - # Prefer universal wheels over platform-specific wheels. - UNIVERSAL = "universal", -) +def define_pypi_internal_flags(name): + """define internal PyPI flags used in PyPI hub repository by pkg_aliases. -INTERNAL_FLAGS = [ - "dist", - "whl_plat", - "whl_plat_py3", - "whl_plat_py3_abi3", - "whl_plat_pycp3x", - "whl_plat_pycp3x_abi3", - "whl_plat_pycp3x_abicp", - "whl_py2_py3", - "whl_py3", - "whl_py3_abi3", - "whl_pycp3x", - "whl_pycp3x_abi3", - "whl_pycp3x_abicp", -] + Args: + name: not used + """ + _default_env_marker_config( + name = "_pip_env_marker_default_config", + ) -def define_pypi_internal_flags(name): - for flag in INTERNAL_FLAGS: - string_flag( - name = "_internal_pip_" + flag, - build_setting_default = "", - values = [""], - visibility = ["//visibility:public"], - ) +def _default_env_marker_config(**kwargs): + _env_marker_config( + os_name = select(os_name_select_map), + sys_platform = select(sys_platform_select_map), + platform_machine = select(platform_machine_select_map), + platform_system = select(platform_system_select_map), + platform_release = select({ + "@platforms//os:osx": "USE_OSX_VERSION_FLAG", + "//conditions:default": "", + }), + **kwargs + ) + +def _env_marker_config_impl(ctx): + env = create_env() + env["os_name"] = ctx.attr.os_name + env["sys_platform"] = ctx.attr.sys_platform + env["platform_machine"] = ctx.attr.platform_machine + + # NOTE: Platform release for Android will be Android version: + # https://peps.python.org/pep-0738/#platform + # Similar for iOS: + # https://peps.python.org/pep-0730/#platform + platform_release = ctx.attr.platform_release + if platform_release == "USE_OSX_VERSION_FLAG": + platform_release = _get_flag(ctx.attr._pip_whl_osx_version_flag) + env["platform_release"] = platform_release + env["platform_system"] = ctx.attr.platform_system + + # NOTE: We intentionally do not call set_missing_env_defaults() here because + # `env_marker_setting()` computes missing values using the toolchain. + return [EnvMarkerInfo(env = env)] + +_env_marker_config = rule( + implementation = _env_marker_config_impl, + attrs = { + "os_name": attr.string(), + "platform_machine": attr.string(), + "platform_release": attr.string(), + "platform_system": attr.string(), + "sys_platform": attr.string(), + "_pip_whl_osx_version_flag": attr.label( + default = labels.PIP_WHL_OSX_VERSION, + providers = [[BuildSettingInfo], [config_common.FeatureFlagInfo]], + ), + }, +) + +def _get_flag(t): + if config_common.FeatureFlagInfo in t: + return t[config_common.FeatureFlagInfo].value + if BuildSettingInfo in t: + return t[BuildSettingInfo].value + fail("Should not occur: {} does not have necessary providers") diff --git a/python/private/pypi/generate_group_library_build_bazel.bzl b/python/private/pypi/generate_group_library_build_bazel.bzl index 54da066b42..571cfd6b3f 100644 --- a/python/private/pypi/generate_group_library_build_bazel.bzl +++ b/python/private/pypi/generate_group_library_build_bazel.bzl @@ -25,7 +25,7 @@ load( ) _PRELUDE = """\ -load("@rules_python//python:defs.bzl", "py_library") +load("@rules_python//python:py_library.bzl", "py_library") """ _GROUP_TEMPLATE = """\ diff --git a/python/private/pypi/generate_whl_library_build_bazel.bzl b/python/private/pypi/generate_whl_library_build_bazel.bzl index 8050cd22ad..ceee319cd7 100644 --- a/python/private/pypi/generate_whl_library_build_bazel.bzl +++ b/python/private/pypi/generate_whl_library_build_bazel.bzl @@ -21,10 +21,12 @@ _RENDER = { "copy_files": render.dict, "data": render.list, "data_exclude": render.list, - "dependencies": render.list, - "dependencies_by_platform": lambda x: render.dict(x, value_repr = render.list), - "entry_points": render.dict, + "entry_points": render.dict_dict, + "extras": render.list, "group_deps": render.list, + "include": str, + "repo": lambda maybe_label: repr(str(maybe_label)), + "requires_dist": render.list, "srcs_exclude": render.list, "tags": render.list, } @@ -33,23 +35,39 @@ _RENDER = { # this repository can be publicly visible without the need for # export_files _TEMPLATE = """\ -load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets") +{loads} package(default_visibility = ["//visibility:public"]) -whl_library_targets( +{fn}( {kwargs} ) """ +_PURL = """\ +package_metadata( + name = "package_metadata", + purl = {purl}, + visibility = ["//:__subpackages__"], +) +""" + def generate_whl_library_build_bazel( *, + metadata_version, annotation = None, + config_load, + purl = None, + requires_dist = [], **kwargs): """Generate a BUILD file for an unzipped Wheel Args: + metadata_version: The version to use for tag generation. annotation: The annotation for the build file. + config_load: {type}`str` The location from where to load the config. + purl: The purl. + requires_dist: {type}`list[str]` The list of dependencies from the METADATA file. **kwargs: Extra args serialized to be passed to the {obj}`whl_library_targets`. @@ -57,6 +75,35 @@ def generate_whl_library_build_bazel( A complete BUILD file as a string """ + loads = [ + """load("@package_metadata//rules:package_metadata.bzl", "package_metadata")""", + ] + + if kwargs.get("repo"): + fn = "whl_library_deps_targets" + else: + fn = "whl_library_targets" + + tags = [ + "pypi_name={}".format(kwargs.get("metadata_name")), + "pypi_version={}".format(metadata_version), + ] + kwargs["tags"] = tags + + if not requires_dist: + # no deps, we can leave the extra loads out + pass + elif config_load: + loads.append("""load("{}", "{}")""".format(config_load, "packages")) + kwargs["include"] = "packages" + kwargs["requires_dist"] = requires_dist + else: + kwargs["requires_dist"] = requires_dist + + loads.extend([ + """load("@rules_python//python/private/pypi:whl_library_targets.bzl", "{}")""".format(fn), + ]) + additional_content = [] if annotation: kwargs["data"] = annotation.data @@ -70,12 +117,14 @@ def generate_whl_library_build_bazel( contents = "\n".join( [ _TEMPLATE.format( + loads = "\n".join(loads), + fn = fn, kwargs = render.indent("\n".join([ "{} = {},".format(k, _RENDER.get(k, repr)(v)) for k, v in sorted(kwargs.items()) ])), ), - ] + additional_content, + ] + ([_PURL.format(purl = repr(purl))] if purl else []) + additional_content, ) # NOTE: Ensure that we terminate with a new line diff --git a/python/private/pypi/group_library.bzl b/python/private/pypi/group_library.bzl deleted file mode 100644 index ff800e2f18..0000000000 --- a/python/private/pypi/group_library.bzl +++ /dev/null @@ -1,40 +0,0 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""group_library implementation for WORKSPACE setups.""" - -load(":generate_group_library_build_bazel.bzl", "generate_group_library_build_bazel") - -def _group_library_impl(rctx): - build_file_contents = generate_group_library_build_bazel( - repo_prefix = rctx.attr.repo_prefix, - groups = rctx.attr.groups, - ) - rctx.file("BUILD.bazel", build_file_contents) - -group_library = repository_rule( - attrs = { - "groups": attr.string_list_dict( - doc = "A mapping of group names to requirements within that group.", - ), - "repo_prefix": attr.string( - doc = "Prefix used for the whl_library created components of each group", - ), - }, - implementation = _group_library_impl, - doc = """ -Create a package containing only wrapper py_library and whl_library rules for implementing dependency groups. -This is an implementation detail of dependency groups and should not be used alone. - """, -) diff --git a/python/private/pypi/hash.bzl b/python/private/pypi/hash.bzl new file mode 100644 index 0000000000..bcdfaecf00 --- /dev/null +++ b/python/private/pypi/hash.bzl @@ -0,0 +1,189 @@ +# Copyright 2026 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Helpers for working with distribution hash digests. + +Both the requirement `--hash=:` option (PEP 508 tooling) and the +Simple API `#=` URL fragments (PEP 503) or `hashes` dicts +(PEP 691) may use any algorithm from `hashlib.algorithms_guaranteed`, even +though `sha256` is the most common one. + +Internally a single digest is always represented as the canonical +`:` string (the same shape as the pip `--hash` values and +the uv.lock `hash` values) and only converted to the Subresource Integrity +format at the `ctx.download(integrity = ...)` boundary. +""" + +# The names from `hashlib.algorithms_guaranteed`, which PEP 691 uses as the +# set of valid hash names and PEP 503 strongly recommends fragments to come +# from. +_ALGOS = [ + "blake2b", + "blake2s", + "md5", + "sha1", + "sha224", + "sha256", + "sha384", + "sha3_224", + "sha3_256", + "sha3_384", + "sha3_512", + "sha512", + "shake_128", + "shake_256", +] + +# The algorithms supported by the Subresource Integrity format understood by +# `ctx.download(integrity = ...)`, strongest first. +_SRI_ALGOS = ["sha512", "sha384", "sha256"] + +# The order used to pick the digest identifying an artifact when several are +# available: `sha256` first so that repo names stay stable for the common +# case, then the remaining algorithms that the bazel downloader can verify. +_PREFERRED_ALGOS = ["sha256", "sha512", "sha384"] + +_HEX = "0123456789abcdef" +_B64 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/" + +def _hex_to_b64(hex_digest): + if len(hex_digest) % 2: + return None + + data = [] + for i in range(0, len(hex_digest), 2): + hi = _HEX.find(hex_digest[i]) + lo = _HEX.find(hex_digest[i + 1]) + if hi < 0 or lo < 0: + return None + data.append(hi * 16 + lo) + + out = [] + for i in range(0, len(data) // 3 * 3, 3): + n = (data[i] << 16) | (data[i + 1] << 8) | data[i + 2] + out.append(_B64[n >> 18]) + out.append(_B64[(n >> 12) & 63]) + out.append(_B64[(n >> 6) & 63]) + out.append(_B64[n & 63]) + + rem = len(data) % 3 + if rem == 1: + n = data[-1] << 16 + out.append(_B64[n >> 18]) + out.append(_B64[(n >> 12) & 63]) + out.append("==") + elif rem == 2: + n = (data[-2] << 16) | (data[-1] << 8) + out.append(_B64[n >> 18]) + out.append(_B64[(n >> 12) & 63]) + out.append(_B64[(n >> 6) & 63]) + out.append("=") + + return "".join(out) + +def _digest(algo, hex_digest): + """Construct a canonical `:` digest string. + + Args: + algo: {type}`str` the hash algorithm name, e.g. `sha256`. + hex_digest: {type}`str` the hex encoded digest. + + Returns: + {type}`str` the canonical digest string or an empty string if the + algorithm is not a known `hashlib` algorithm or the digest is empty. + """ + algo = algo.lower() + if not hex_digest or algo not in _ALGOS: + return "" + + return "{}:{}".format(algo, hex_digest) + +def _hex_to_sri(algo, hex_digest): + """Convert a hex digest to a Subresource Integrity value. + + Args: + algo: {type}`str` the hash algorithm name, e.g. `sha512`. + hex_digest: {type}`str` the hex encoded digest. + + Returns: + {type}`str` the SRI value (e.g. `sha512-...`) that can be passed to + `ctx.download(integrity = ...)` or an empty string if the algorithm + cannot be expressed as SRI or the digest is not valid hex. + """ + if algo not in _SRI_ALGOS or not hex_digest: + return "" + + b64 = _hex_to_b64(hex_digest.lower()) + if b64 == None: + return "" + + return "{}-{}".format(algo, b64) + +def _integrity(digest): + """Get the SRI value for a canonical `:` string. + + Args: + digest: {type}`str` the canonical digest string. + + Returns: + {type}`str` the SRI value (e.g. `sha256-...`) that can be passed to + `ctx.download(integrity = ...)` or an empty string if the algorithm + cannot be expressed as SRI. Fails if the algorithm is SRI supported + but the digest is not valid hex, because silently returning an empty + string would disable the download verification for an artifact that + should be verifiable. + """ + algo, _, hex_digest = digest.partition(":") + if algo not in _SRI_ALGOS or not hex_digest: + return "" + + sri = _hex_to_sri(algo, hex_digest) + if not sri: + fail("Invalid {} digest: {}".format(algo, hex_digest)) + + return sri + +def _preferred_digest(digests): + """Pick the digest that identifies an artifact, e.g. for repo naming. + + Args: + digests: {type}`list[str]` canonical `:` strings. + + Returns: + {type}`str` the canonical digest string of the most preferred + algorithm present or an empty string if there are no digests. + """ + by_algo = {} + for digest in digests: + if not digest: + continue + algo, _, _ = digest.partition(":") + by_algo.setdefault(algo, digest) + + if not by_algo: + return "" + + for algo in _PREFERRED_ALGOS: + if algo in by_algo: + return by_algo[algo] + + return by_algo[sorted(by_algo.keys())[0]] + +hash = struct( + ALGOS = _ALGOS, + digest = _digest, + hex_to_sri = _hex_to_sri, + integrity = _integrity, + preferred_digest = _preferred_digest, +) diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl new file mode 100644 index 0000000000..596cfb2cbe --- /dev/null +++ b/python/private/pypi/hub_builder.bzl @@ -0,0 +1,706 @@ +"""A hub repository builder for incrementally building the hub configuration.""" + +load("//python/private:envsubst.bzl", "envsubst") +load("//python/private:full_version.bzl", "full_version") +load("//python/private:normalize_name.bzl", "normalize_name") +load("//python/private:repo_utils.bzl", "repo_utils") +load("//python/private:text_util.bzl", "render") +load("//python/private:version.bzl", "version") +load("//python/private:version_label.bzl", "version_label") +load(":attrs.bzl", "use_isolated") +load(":hash.bzl", "hash") +load(":parse_requirements.bzl", "parse_requirements") +load(":pep508_env.bzl", "env") +load(":pep508_evaluate.bzl", "evaluate") +load(":python_tag.bzl", "python_tag") +load(":requirements_files_by_platform.bzl", "requirements_files_by_platform") +load(":whl_config_setting.bzl", "whl_config_setting") +load(":whl_repo_name.bzl", "pypi_repo_name", "whl_repo_name") + +def _major_minor_version(version_str): + ver = version.parse(version_str) + return "{}.{}".format(ver.release[0], ver.release[1]) + +def hub_builder( + *, + name, + module_name, + config, + whl_overrides, + minor_mapping, + available_interpreters, + simpleapi_download_fn, + logger, + simpleapi_cache): + """Return a hub builder instance + + Args: + name: {type}`str`, the name of the hub. + module_name: {type}`str`, the module name that has created the hub. + config: The platform configuration. + whl_overrides: {type}`dict[str, struct]` - per-wheel overrides. + minor_mapping: {type}`dict[str, str]` the mapping between minor and full versions. + available_interpreters: {type}`dict[str, Label]` The dictionary of available + interpreters that have been registered using the `python` bzlmod extension. + The keys are in the form `python_{snake_case_version}_host`. This is to be + used during the `repository_rule` and must be always compatible with the host. + simpleapi_download_fn: the function used to download from SimpleAPI. + simpleapi_cache: the cache for the download results. + logger: the logger for this builder. + """ + + # buildifier: disable=uninitialized + self = struct( + name = name, + module_name = module_name, + + # public methods, keep sorted and to minimum + build = lambda: _build(self), + pip_parse = lambda *a, **k: _pip_parse(self, *a, **k), + + # Build output related + + # The set of package names to expose. Dict acting as set + # dict[str package, None] + _exposed_packages = {}, # modified by _add_exposed_packages + # Map of the per-package aliases. + # The nested dict is a dict acting as a set. + # dict[str whl_name, dict[str alias_name, bool]] + _extra_aliases = {}, # modified by _add_extra_aliases + # dict[str group_name, list[str]] + _group_map = {}, # modified by _add_group_map + # Mapping of whl_library repo names and their kwargs. + # dict[str repo_name, dict[str, object] kwargs] + _whl_libraries = {}, # modified by _add_whl_library + # Map of repos and their config settings, and repo the config + # setting originated from. + # dict[str whl_name, dict[str config_setting, str repo_name]] + _whl_map = {}, # modified by _add_whl_library + + # Internal + + # dict[str python_version, dict[str platform, PlatformInfo]] + # where `PlatformInfo` is from `_platforms()` + _platforms = {}, + # Supplemental index of `_group_map` + # dict[str whl_name, str group_name] + _group_name_by_whl = {}, + # Functions to download according to the config + # dict[str python_version, callable] + _get_index_urls = {}, + # Tells whether to use the downloader for a package. + # dict[str python_version, dict[str package_name, bool use_downloader]] + _use_downloader = {}, + _simpleapi_cache = simpleapi_cache, + + # Instance constants passed in by callers + _config = config, + _whl_overrides = whl_overrides, + _logger = logger, + _minor_mapping = minor_mapping, + _available_interpreters = available_interpreters, + _simpleapi_download_fn = simpleapi_download_fn, + ) + + # buildifier: enable=uninitialized + return self + +### PUBLIC methods + +def _build(self): + ret = struct( + whl_map = {}, + group_map = {}, + extra_aliases = {}, + exposed_packages = [], + whl_libraries = {}, + ) + if self._logger.failed(): + return ret + + whl_map = {} + for key, settings in self._whl_map.items(): + for setting, repo in settings.items(): + whl_map.setdefault(key, {}).setdefault(repo, []).append(setting) + + return struct( + # The config settings for matching repo spokes. + # dict[str repo_name, dict[str repo_name, list[str]]] + whl_map = whl_map, + # Maps a wheel to a list of groups + # dict[str group_name, list[str]] + group_map = self._group_map, + # The per-package aliases for the hub to create. + # dict[str package, list[str]] + extra_aliases = { + whl: sorted(aliases) + for whl, aliases in self._extra_aliases.items() + }, + # The list of exposed packages in the hub. + # list[str] + exposed_packages = sorted(self._exposed_packages), + + # Mapping of whl_library repo names and their kwargs. + # dict[str repo_name, dict[str, object] kwargs] + whl_libraries = self._whl_libraries, + ) + +def _pip_parse(self, module_ctx, pip_attr, python_version = None): + python_version = python_version or pip_attr.python_version + if python_version in self._platforms: + fail(( + "Duplicate pip python version '{version}' for hub " + + "'{hub}' in module '{module}': the Python versions " + + "used for a hub must be unique" + ).format( + hub = self.name, + module = self.module_name, + version = python_version, + )) + + full_python_version = full_version( + version = python_version, + minor_mapping = self._minor_mapping, + fail_on_err = False, + ) + if not full_python_version: + # NOTE @aignas 2025-11-18: If the python version is not present in our + # minor_mapping, then we will not register any packages and then the + # select in the hub repository will fail, which will prompt the user to + # configure the toolchain correctly and move forward. + self._logger.info(lambda: ( + "Ignoring pip python version '{version}' for hub " + + "'{hub}' in module '{module}' because there is no registered " + + "toolchain for it." + ).format( + hub = self.name, + module = self.module_name, + version = python_version, + )) + return + + _set_get_index_urls(self, module_ctx, pip_attr) + self._platforms[python_version] = _platforms( + module_ctx, + python_version = full_python_version, + config = self._config, + target_platforms = pip_attr.target_platforms or ["{os}_{arch}"], + ) + _add_group_map(self, pip_attr.experimental_requirement_cycles) + _add_extra_aliases(self, pip_attr.extra_hub_aliases) + _create_whl_repos( + self, + module_ctx, + pip_attr = pip_attr, + python_version = python_version, + enable_pipstar_extract = bool(self._config.enable_pipstar_extract or self._get_index_urls.get(python_version)), + ) + +### end of PUBLIC methods +### setters for build outputs + +def _add_exposed_packages(self, exposed_packages): + """Add packages that are exposed. + + Args: + self: implicitly added + exposed_packages: {type}`dict[str package, None]` a dict acting as + a set. The set of packages that should be exposed. + """ + if self._exposed_packages: + intersection = {} + for pkg in exposed_packages: + if pkg not in self._exposed_packages: + continue + intersection[pkg] = None + self._exposed_packages.clear() + exposed_packages = intersection + + self._exposed_packages.update(exposed_packages) + +def _add_group_map(self, group_map): + """Adds a group mapping for cycle breaking. + + Args: + self: implicitly added. + group_map: {type}`dict[str name, list[str]]` + """ + + # TODO @aignas 2024-04-05: how do we support different requirement + # cycles for different abis/oses? For now we will need the users to + # assume the same groups across all versions/platforms until we start + # using an alternative cycle resolution strategy. + group_map = { + name: [normalize_name(whl_name) for whl_name in whls] + for name, whls in group_map.items() + } + self._group_map.clear() + self._group_name_by_whl.clear() + + self._group_map.update(group_map) + self._group_name_by_whl.update({ + whl_name: group_name + for group_name, group_whls in self._group_map.items() + for whl_name in group_whls + }) + +def _add_extra_aliases(self, extra_hub_aliases): + """Adds per-package aliases to the hub. + + Args: + self: Implicitly added + extra_hub_aliases: {type}`dict[str package, list[str]]` Alias target + names to add to a package's hub BUILD file. + """ + for whl_name, aliases in extra_hub_aliases.items(): + self._extra_aliases.setdefault(whl_name, {}).update( + {alias: True for alias in aliases}, + ) + +def _diff_dict(first, second): + """A simple utility to shallow compare dictionaries. + + Args: + first: The first dictionary to compare. + second: The second dictionary to compare. + + Returns: + A dictionary containing the differences, with keys "common", "different", + "extra", and "missing", or None if the dictionaries are identical. + """ + missing = {} + extra = { + key: value + for key, value in second.items() + if key not in first + } + common = {} + different = {} + + for key, value in first.items(): + if key not in second: + missing[key] = value + elif value == second[key]: + common[key] = value + else: + different[key] = (value, second[key]) + + if missing or extra or different: + return { + "common": common, + "different": different, + "extra": extra, + "missing": missing, + } + else: + return None + +def _add_whl_library(self, *, python_version, whl, repo): + """Add a whl_library and kwargs to call it with for the hub. + + Args: + self: implicitly added + python_version: {type}`str` the python version to assume + whl: struct from `_whl_library_args()` + repo: struct from `_whl_repo` + """ + if repo == None: + # NOTE @aignas 2025-07-07: we guard against an edge-case where there + # are more platforms defined than there are wheels for and users + # disallow building from sdist. + return + + # TODO @aignas 2025-06-29: we should not need the version in the repo_name if + # we are using pipstar and we are downloading the wheel using the downloader + # + # However, for that we should first have a different way to reference closures with + # extras. For example, if some package depends on `foo[extra]` and another depends on + # `foo`, we should have 2 py_library targets. + repo_name = "{}_{}_{}".format(self.name, version_label(python_version), repo.repo_name) + + if repo_name in self._whl_libraries: + diff = _diff_dict(self._whl_libraries[repo_name], repo.args) + if diff: + self._logger.fail(lambda: ( + "Attempting to create a duplicate library {repo_name} for {whl_name} with different arguments. Already existing declaration has:\n".format( + repo_name = repo_name, + whl_name = whl.name, + ) + "\n".join([ + " {}: {}".format(key, render.indent(render.dict(value)).lstrip()) + for key, value in diff.items() + if value + ]) + )) + return + self._whl_libraries[repo_name] = repo.args + + mapping = self._whl_map.setdefault(whl.name, {}) + if repo.config_setting in mapping and mapping[repo.config_setting] != repo_name: + fail( + "attempting to override an existing repo '{}' for config setting '{}' with a new repo '{}'".format( + mapping[repo.config_setting], + repo.config_setting, + repo_name, + ), + ) + else: + mapping[repo.config_setting] = repo_name + +### end of setters, below we have various functions to implement the public methods + +def _set_get_index_urls(self, mctx, pip_attr): + # Resolve the index URL through envsubst so the ``$VAR`` / ``${VAR:-default}`` + # form is honored when deciding whether the experimental index-url mode is + # active. Without this, an unsubstituted template like ``$RULES_PYTHON_PIP_INDEX_URL`` + # is treated as truthy and the mode is forced on, even when the env var + # would expand to the empty string. + default_index_url = envsubst( + pip_attr.experimental_index_url, + pip_attr.envsubst, + mctx.getenv, + ) or self._config.index_url + default_extra_index_urls = pip_attr.experimental_extra_index_urls or [] + + if not default_index_url: + # parallel_download is set to True by default, so we are not checking/validating it + # here + return False + + python_version = pip_attr.python_version + self._use_downloader.setdefault(python_version, {}).update({ + normalize_name(s): False + for s in pip_attr.simpleapi_skip + }) + self._get_index_urls[python_version] = lambda ctx, distributions, *, index_url = None, extra_index_urls = None: self._simpleapi_download_fn( + ctx, + attr = struct( + index_url = (index_url or default_index_url).rstrip("/"), + extra_index_urls = [ + x.rstrip("/") + for x in (extra_index_urls or default_extra_index_urls) + ], + index_url_overrides = pip_attr.experimental_index_url_overrides or {}, + sources = { + d: versions + for d, versions in distributions.items() + if _use_downloader(self, python_version, d) + }, + envsubst = pip_attr.envsubst, + # Auth related info + netrc = self._config.netrc or pip_attr.netrc, + auth_patterns = self._config.auth_patterns or pip_attr.auth_patterns, + ), + cache = self._simpleapi_cache, + parallel_download = pip_attr.parallel_download, + ) + return True + +def _detect_interpreter(self, pip_attr, python_version): + python_interpreter_target = pip_attr.python_interpreter_target + if python_interpreter_target == None and not pip_attr.python_interpreter: + python_name = "python_{}_host".format( + python_version.replace(".", "_"), + ) + if python_name not in self._available_interpreters: + fail(( + "Unable to find interpreter for pip hub '{hub_name}' for " + + "python_version={version}: Make sure a corresponding " + + '`python.toolchain(python_version="{version}")` call exists.' + + "Expected to find {python_name} among registered versions:\n {labels}" + ).format( + hub_name = self.name, + version = python_version, + python_name = python_name, + labels = " \n".join(self._available_interpreters), + )) + python_interpreter_target = self._available_interpreters[python_name] + + return struct( + target = python_interpreter_target, + path = pip_attr.python_interpreter, + ) + +def _platforms(module_ctx, *, python_version, config, target_platforms): + platforms = {} + python_version = version.parse( + python_version, + strict = True, + ) + + target_platforms = sorted({ + p.format( + os = repo_utils.get_platforms_os_name(module_ctx), + arch = repo_utils.get_platforms_cpu_name(module_ctx), + ): None + for p in target_platforms + }) + + for platform, values in config.platforms.items(): + if target_platforms and platform not in target_platforms: + continue + + # TODO @aignas 2025-07-07: this is probably doing the parsing of the version too + # many times. + abi = "{}{}{}.{}".format( + python_tag(values.env["implementation_name"]), + python_version.release[0], + python_version.release[1], + python_version.release[2], + ) + key = "{}_{}".format(abi, platform) + + env_ = env( + env = values.env, + os = values.os_name, + arch = values.arch_name, + python_version = python_version.string, + ) + + if values.marker and not evaluate(values.marker, env = env_): + continue + + platforms[key] = struct( + env = env_, + triple = "{}_{}_{}".format(abi, values.os_name, values.arch_name), + whl_abi_tags = [ + v.format( + major = python_version.release[0], + minor = python_version.release[1], + ) + for v in values.whl_abi_tags + ], + whl_platform_tags = values.whl_platform_tags, + ) + return platforms + +def _create_whl_repos( + self, + module_ctx, + *, + pip_attr, + python_version, + enable_pipstar_extract = False): + """create all of the whl repositories + + Args: + self: the builder. + module_ctx: {type}`module_ctx`. + pip_attr: {type}`struct` - the struct that comes from the tag class iteration. + python_version: {type}`str` - the resolved python version for this pip.parse call. + enable_pipstar_extract: {type}`bool` - enable the pipstar extraction or not. + """ + logger = self._logger + platforms = self._platforms[python_version] + requirements_by_platform = parse_requirements( + module_ctx, + requirements_by_platform = requirements_files_by_platform( + requirements_by_platform = pip_attr.requirements_by_platform, + requirements_linux = pip_attr.requirements_linux, + requirements_lock = pip_attr.requirements_lock, + requirements_osx = pip_attr.requirements_darwin, + requirements_windows = pip_attr.requirements_windows, + uv_lock = pip_attr.uv_lock, + extra_pip_args = pip_attr.extra_pip_args, + platforms = sorted(platforms), # here we only need keys + python_version = full_version( + version = python_version, + minor_mapping = self._minor_mapping, + ), + logger = logger, + ), + uv_lock = pip_attr.uv_lock, + platforms = platforms, + extra_pip_args = pip_attr.extra_pip_args, + get_index_urls = self._get_index_urls.get(pip_attr.python_version), + toml_decode = getattr(self._config, "toml_decode", None), + logger = logger, + ) + + _add_exposed_packages(self, { + whl.name: None + for whl in requirements_by_platform + if whl.is_exposed + }) + + whl_modifications = {} + if pip_attr.whl_modifications != None: + for mod, whl_name in pip_attr.whl_modifications.items(): + whl_modifications[normalize_name(whl_name)] = mod + + common_args = _common_args( + self, + module_ctx, + pip_attr = pip_attr, + ) + + interpreter = _detect_interpreter(self, pip_attr, python_version) + + for whl in requirements_by_platform: + whl_library_args = common_args | _whl_library_args( + self, + whl = whl, + whl_modifications = whl_modifications, + ) + for src in whl.srcs: + repo = _whl_repo( + src = src, + index_url = whl.index_url, + whl_library_args = whl_library_args, + download_only = pip_attr.download_only, + netrc = self._config.netrc or pip_attr.netrc, + use_downloader = src.url and _use_downloader(self, python_version, whl.name), + auth_patterns = self._config.auth_patterns or pip_attr.auth_patterns, + python_version = _major_minor_version(python_version), + is_multiple_versions = whl.is_multiple_versions, + interpreter = interpreter, + enable_pipstar_extract = enable_pipstar_extract, + ) + _add_whl_library( + self, + python_version = python_version, + whl = whl, + repo = repo, + ) + +def _common_args(self, module_ctx, *, pip_attr): + # Construct args separately so that the lock file can be smaller and does not include unused + # attrs. + whl_library_args = dict( + dep_template = "@{}//{{name}}:{{target}}".format(self.name), + config_load = "@{}//:config.bzl".format(self.name), + ) + maybe_args = dict( + # The following values are safe to omit if they have false like values + add_libdir_to_library_search_path = pip_attr.add_libdir_to_library_search_path, + download_only = pip_attr.download_only, + enable_implicit_namespace_pkgs = pip_attr.enable_implicit_namespace_pkgs, + environment = pip_attr.environment, + envsubst = pip_attr.envsubst, + pip_data_exclude = pip_attr.pip_data_exclude, + ) + + whl_library_args.update({k: v for k, v in maybe_args.items() if v}) + maybe_args_with_default = dict( + # The following values have defaults next to them + isolated = (use_isolated(module_ctx, pip_attr), True), + quiet = (pip_attr.quiet, True), + timeout = (pip_attr.timeout, 600), + ) + whl_library_args.update({ + k: v + for k, (v, default) in maybe_args_with_default.items() + if v != default + }) + return whl_library_args + +def _whl_library_args(self, *, whl, whl_modifications): + group_name = self._group_name_by_whl.get(whl.name) + group_deps = self._group_map.get(group_name, []) + + # Construct args separately so that the lock file can be smaller and does not include unused + # attrs. + whl_library_args = dict( + dep_template = "@{}//{{name}}:{{target}}".format(self.name), + ) + maybe_args = dict( + # The following values are safe to omit if they have false like values + annotation = whl_modifications.get(whl.name), + group_deps = group_deps, + group_name = group_name, + whl_patches = { + p: json.encode(args) + for p, args in self._whl_overrides.get(whl.name, {}).items() + }, + ) + + whl_library_args.update({k: v for k, v in maybe_args.items() if v}) + return whl_library_args + +def _whl_repo( + *, + src, + whl_library_args, + index_url, + is_multiple_versions, + download_only, + netrc, + auth_patterns, + python_version, + use_downloader, + interpreter, + enable_pipstar_extract = False): + args = dict(whl_library_args) + args["requirement"] = src.requirement_line + is_whl = src.filename.endswith(".whl") + + if src.extra_pip_args and not is_whl: + # pip is not used to download wheels and the python + # `whl_library` helpers are only extracting things, however + # for sdists, they will be built by `pip`, so we still + # need to pass the extra args there, so only pop this for whls + args["extra_pip_args"] = src.extra_pip_args + + if not (enable_pipstar_extract and is_whl): + if interpreter.path: + args["python_interpreter"] = interpreter.path + if interpreter.target: + args["python_interpreter_target"] = interpreter.target + + if not src.url or (not is_whl and download_only): + if download_only and use_downloader: + # If the user did not allow using sdists and we are using the downloader + # and we are not using simpleapi_skip for this + return None + else: + # Fallback to a pip-installed wheel + target_platforms = src.target_platforms if is_multiple_versions else [] + return struct( + repo_name = pypi_repo_name( + normalize_name(src.distribution), + *target_platforms + ), + args = args, + config_setting = whl_config_setting( + version = python_version, + target_platforms = target_platforms or None, + ), + ) + + # This is no-op because pip is not used to download the wheel. + args.pop("download_only", None) + + if netrc: + args["netrc"] = netrc + if auth_patterns: + args["auth_patterns"] = auth_patterns + if index_url: + args["index_url"] = index_url + + args["urls"] = [src.url] + + # The digest is always passed in the SRI format that the bazel downloader + # supports. It may be empty if the digest uses a hash algorithm that + # cannot be expressed as SRI (e.g. `md5`), in which case `whl_library` + # will download without verification and warn. + args["integrity"] = hash.integrity(src.digest) + args["filename"] = src.filename + + # TODO @aignas 2025-11-02: once we have pipstar enabled we can add extra + # targets to each hub for each extra combination and solve this more cleanly as opposed to + # duplicating whl_library repositories. + target_platforms = src.target_platforms if is_multiple_versions else [] + + return struct( + repo_name = whl_repo_name(src.filename, src.digest, *target_platforms), + args = args, + config_setting = whl_config_setting( + version = python_version, + target_platforms = src.target_platforms, + ), + ) + +def _use_downloader(self, python_version, whl_name): + return self._use_downloader.get(python_version, {}).get( + normalize_name(whl_name), + self._get_index_urls.get(python_version) != None, + ) diff --git a/python/private/pypi/hub_repository.bzl b/python/private/pypi/hub_repository.bzl index 48245b4106..f915aa1c77 100644 --- a/python/private/pypi/hub_repository.bzl +++ b/python/private/pypi/hub_repository.bzl @@ -34,6 +34,7 @@ def _impl(rctx): }, extra_hub_aliases = rctx.attr.extra_hub_aliases, requirement_cycles = rctx.attr.groups, + platform_config_settings = rctx.attr.platform_config_settings, ) for path, contents in aliases.items(): rctx.file(path, contents) @@ -45,7 +46,14 @@ def _impl(rctx): macro_tmpl = "@@{name}//{{}}:{{}}".format(name = rctx.attr.name) rctx.file("BUILD.bazel", _BUILD_FILE_CONTENTS) - rctx.template("requirements.bzl", rctx.attr._template, substitutions = { + rctx.template( + "config.bzl", + rctx.attr._config_template, + substitutions = { + "%%PACKAGES%%": render.dict(rctx.attr.whl_map, value_repr = lambda x: "None"), + }, + ) + rctx.template("requirements.bzl", rctx.attr._requirements_bzl_template, substitutions = { "%%ALL_DATA_REQUIREMENTS%%": render.list([ macro_tmpl.format(p, "data") for p in bzl_packages @@ -76,6 +84,10 @@ hub_repository = repository_rule( The list of packages that will be exposed via all_*requirements macros. Defaults to whl_map keys. """, ), + "platform_config_settings": attr.string_list_dict( + doc = "The constraint values for each platform name. The values are string canonical string Label representations", + mandatory = False, + ), "repo_name": attr.string( mandatory = True, doc = "The apparent name of the repo. This is needed because in bzlmod, the name attribute becomes the canonical name.", @@ -87,7 +99,10 @@ The wheel map where values are json.encoded strings of the whl_map constructed in the pip.parse tag class. """, ), - "_template": attr.label( + "_config_template": attr.label( + default = ":config.bzl.tmpl", + ), + "_requirements_bzl_template": attr.label( default = ":requirements.bzl.tmpl.bzlmod", ), }, @@ -127,10 +142,6 @@ def whl_config_settings_to_json(repo_mapping): def _whl_config_setting_dict(a): ret = {} - if a.config_setting: - ret["config_setting"] = a.config_setting - if a.filename: - ret["filename"] = a.filename if a.target_platforms: ret["target_platforms"] = a.target_platforms if a.version: diff --git a/python/private/pypi/index_sources.bzl b/python/private/pypi/index_sources.bzl index 21660141db..4353f5051a 100644 --- a/python/private/pypi/index_sources.bzl +++ b/python/private/pypi/index_sources.bzl @@ -16,6 +16,25 @@ A file that houses private functions used in the `bzlmod` extension with the same name. """ +load(":hash.bzl", "hash") + +# Just list them here and me super conservative +_KNOWN_EXTS = [ + # Note, the following source in pip has more extensions + # https://github.com/pypa/pip/blob/3c5a189141a965f21a473e46c3107e689eb9f79f/src/pip/_vendor/distlib/locators.py#L90 + # + # ".tar.bz2", + # ".tar", + # ".tgz", + # ".tbz", + # + # But we support only the following, used in 'packaging' + # https://github.com/pypa/pip/blob/3c5a189141a965f21a473e46c3107e689eb9f79f/src/pip/_vendor/packaging/utils.py#L137 + ".whl", + ".tar.gz", + ".zip", +] + def index_sources(line): """Get PyPI sources from a requirements.txt line. @@ -26,28 +45,78 @@ def index_sources(line): line(str): The requirements.txt entry. Returns: - A struct with shas attribute containing a list of shas to download from pypi_index. + A struct with hashes attribute containing: + * `hashes` - list[str]; `:` hashes of the artifacts + to download from pypi_index. Note that any hash algorithm from + {obj}`hash.ALGOS` is accepted, not only `sha256`. + * `version` - str; version of the package. + * `marker` - str; the marker expression, as per PEP508 spec. + * `requirement` - str; a requirement line without the marker. This can + be given to `pip` to install a package. + * `url` - str; URL if the requirement specifies a direct URL, empty string otherwise. """ + line = line.replace("\\", " ") head, _, maybe_hashes = line.partition(";") _, _, version = head.partition("==") version = version.partition(" ")[0].strip() - if "@" in head: - shas = [] - else: - maybe_hashes = maybe_hashes or line - shas = [ - sha.strip() - for sha in maybe_hashes.split("--hash=sha256:")[1:] - ] + marker, _, _ = maybe_hashes.partition("--hash=") + maybe_hashes = maybe_hashes or line + hashes = [] + for h in maybe_hashes.split("--hash=")[1:]: + algo, _, hex_digest = h.strip().partition(" ")[0].partition(":") + digest = hash.digest(algo, hex_digest) + if digest: + hashes.append(digest) + marker = marker.strip() if head == line: - head = line.partition("--hash=")[0].strip() + requirement = line.partition("--hash=")[0].strip() else: - head = head + ";" + maybe_hashes.partition("--hash=")[0].strip() + requirement = head.strip() + + requirement_line = "{} {}".format( + requirement, + " ".join(["--hash={}".format(h) for h in hashes]), + ).strip() + + url = "" + filename = "" + if "@" in head: + maybe_requirement, _, url_and_rest = requirement.partition("@") + url = url_and_rest.strip().partition(" ")[0].strip() + + url, _, fragment = url.partition("#") + algo, _, hex_digest = fragment.partition("=") + digest = hash.digest(algo, hex_digest) + if digest: + hashes.append(digest) + elif fragment: + # Not a hash fragment (e.g. `#egg=`), keep it as part of the URL. + url = "{}#{}".format(url, fragment) + _, _, filename = url.rpartition("/") + + # Replace URL encoded characters and luckily there is only one case + filename = filename.replace("%2B", "+") + is_known_ext = False + for ext in _KNOWN_EXTS: + if filename.endswith(ext): + is_known_ext = True + break + + requirement = requirement_line + if filename.endswith(".whl"): + requirement = maybe_requirement.strip() + elif not is_known_ext: + # could not detect filename from the URL + filename = "" return struct( - requirement = line if not shas else head, + requirement = requirement, + requirement_line = requirement_line, version = version, - shas = sorted(shas), + hashes = sorted(hashes), + marker = marker, + url = url, + filename = filename, ) diff --git a/python/private/pypi/labels.bzl b/python/private/pypi/labels.bzl index 73df07b2d2..c77167b593 100644 --- a/python/private/pypi/labels.bzl +++ b/python/private/pypi/labels.bzl @@ -14,11 +14,13 @@ """Constants used by parts of pip_repository for naming libraries and wheels.""" +EXTRACTED_WHEEL_FILES = "extracted_whl_files" WHEEL_FILE_PUBLIC_LABEL = "whl" WHEEL_FILE_IMPL_LABEL = "_whl" +WHEEL_FILE = "whl_file" PY_LIBRARY_PUBLIC_LABEL = "pkg" PY_LIBRARY_IMPL_LABEL = "_pkg" +PY_SRCS_LABEL = "srcs" DATA_LABEL = "data" DIST_INFO_LABEL = "dist_info" -WHEEL_ENTRY_POINT_PREFIX = "rules_python_wheel_entry_point" NODEPS_LABEL = "no_deps" diff --git a/python/private/pypi/missing_package.bzl b/python/private/pypi/missing_package.bzl new file mode 100644 index 0000000000..46823e813b --- /dev/null +++ b/python/private/pypi/missing_package.bzl @@ -0,0 +1,43 @@ +"""Rule for generating an execution-phase action failure when a PyPI package is missing.""" + +load("//python/private:py_info.bzl", "PyInfo") +load("//python/private:reexports.bzl", "BuiltinPyInfo") + +def _missing_package_error_impl(ctx): + out = ctx.actions.declare_file(ctx.label.name + ".error") + + if ctx.attr.hub_name: + hub_clause = ' when building under PyPI hub "{hub}". Try adding it to the requirements of this hub (e.g. requirements_lock or requirements_by_platform in pip.parse)'.format( + hub = ctx.attr.hub_name, + ) + else: + hub_clause = ' because no default PyPI hub was configured. Try designating a default hub via pip.default(default_hub = "...") or select a hub using --@rules_python//python/config_settings:venv' + + # Register an action that fails when Bazel attempts to stage/build this file + ctx.actions.run_shell( + outputs = [out], + command = "echo 'ERROR: PyPI package \"{pkg}\" is not available{hub_clause}.' >&2 && exit 1".format( + pkg = ctx.attr.package_name, + hub_clause = hub_clause, + ), + ) + + maybe_builtin = [BuiltinPyInfo(transitive_sources = depset([out]))] if BuiltinPyInfo != None else [] + + return [ + DefaultInfo( + files = depset([out]), + data_runfiles = ctx.runfiles([out]), + ), + PyInfo( + transitive_sources = depset([out]), + ), + ] + maybe_builtin + +missing_package_error = rule( + implementation = _missing_package_error_impl, + attrs = { + "hub_name": attr.string(mandatory = True), + "package_name": attr.string(mandatory = True), + }, +) diff --git a/python/private/pypi/multi_pip_parse.bzl b/python/private/pypi/multi_pip_parse.bzl index 6e824f674c..60496c2eca 100644 --- a/python/private/pypi/multi_pip_parse.bzl +++ b/python/private/pypi/multi_pip_parse.bzl @@ -18,7 +18,7 @@ load("//python/private:text_util.bzl", "render") load(":pip_repository.bzl", pip_parse = "pip_repository") def _multi_pip_parse_impl(rctx): - rules_python = rctx.attr._rules_python_workspace.workspace_name + rules_python = rctx.attr._rules_python_workspace.repo_name load_statements = [] install_deps_calls = [] process_requirements_calls = [] @@ -69,7 +69,7 @@ def _process_requirements(pkg_labels, python_version, repo_prefix): wheel_name = Label(pkg_label).package if not wheel_name: # We are dealing with the cases where we don't have aliases. - workspace_name = Label(pkg_label).workspace_name + workspace_name = Label(pkg_label).repo_name wheel_name = workspace_name[len(repo_prefix):] _wheel_names.append(wheel_name) diff --git a/python/private/pypi/namespace_pkg_tmpl.py b/python/private/pypi/namespace_pkg_tmpl.py new file mode 100644 index 0000000000..80da1dfb2a --- /dev/null +++ b/python/private/pypi/namespace_pkg_tmpl.py @@ -0,0 +1,2 @@ +# __path__ manipulation added by bazel-contrib/rules_python to support namespace pkgs. +__path__ = __import__("pkgutil").extend_path(__path__, __name__) # noqa: F821 diff --git a/python/private/pypi/namespace_pkgs.bzl b/python/private/pypi/namespace_pkgs.bzl new file mode 100644 index 0000000000..be6244efc7 --- /dev/null +++ b/python/private/pypi/namespace_pkgs.bzl @@ -0,0 +1,90 @@ +"""Utilities to get where we should write namespace pkg paths.""" + +load("@bazel_skylib//rules:copy_file.bzl", "copy_file") + +_ext = struct( + py = ".py", + pyd = ".pyd", + so = ".so", + pyc = ".pyc", +) + +_TEMPLATE = Label("//python/private/pypi:namespace_pkg_tmpl.py") + +def _add_all(dirname, dirs): + dir_path = "." + for dir_name in dirname.split("/"): + dir_path = "{}/{}".format(dir_path, dir_name) + dirs[dir_path[2:]] = None + +def get_files(*, srcs, ignored_dirnames = [], root = None): + """Get the list of filenames to write the namespace pkg files. + + Args: + srcs: {type}`src` a list of files to be passed to {bzl:obj}`py_library` + as `srcs` and `data`. This is usually a result of a {obj}`glob`. + ignored_dirnames: {type}`str` a list of patterns to ignore. + root: {type}`str` the prefix to use as the root. + + Returns: + {type}`src` a list of paths to write the namespace pkg `__init__.py` file. + """ + dirs = {} + ignored = {i: None for i in ignored_dirnames} + + if root: + _add_all(root, ignored) + + for file in srcs: + dirname, _, filename = file.rpartition("/") + + if filename == "__init__.py": + ignored[dirname] = None + dirname, _, _ = dirname.rpartition("/") + elif filename.endswith(_ext.py): + pass + elif filename.endswith(_ext.pyc): + pass + elif filename.endswith(_ext.pyd): + pass + elif filename.endswith(_ext.so): + pass + else: + continue + + if dirname in dirs or not dirname: + continue + + _add_all(dirname, dirs) + + return sorted([d for d in dirs if d not in ignored]) + +def create_inits(*, srcs, ignored_dirnames = [], root = None, copy_file = copy_file, **kwargs): + """Create init files and return the list to be included `py_library` srcs. + + Args: + srcs: {type}`src` a list of files to be passed to {bzl:obj}`py_library` + as `srcs` and `data`. This is usually a result of a {obj}`glob`. + ignored_dirnames: {type}`str` a list of patterns to ignore. + root: {type}`str` the prefix to use as the root. + copy_file: the `copy_file` rule to copy files in build context. + **kwargs: passed to {obj}`copy_file`. + + Returns: + {type}`list[str]` to be included as part of `py_library`. + """ + ret = [] + for i, out in enumerate(get_files(srcs = srcs, ignored_dirnames = ignored_dirnames, root = root)): + src = "{}/__init__.py".format(out) + ret.append(src) + + copy_file( + # For the target name, use a number instead of trying to convert an output + # path into a valid label. + name = "_cp_{}_namespace".format(i), + src = _TEMPLATE, + out = src, + **kwargs + ) + + return ret diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index 133ed18db8..4874c7ac79 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -28,96 +28,319 @@ behavior. load("//python/private:normalize_name.bzl", "normalize_name") load("//python/private:repo_utils.bzl", "repo_utils") +load("//python/uv/private:uv_lock_to_requirements.bzl", "uv_lock_extras_map") # buildifier: disable=bzl-visibility +load(":argparse.bzl", "argparse") +load(":hash.bzl", "hash") load(":index_sources.bzl", "index_sources") load(":parse_requirements_txt.bzl", "parse_requirements_txt") -load(":whl_target_platforms.bzl", "select_whls") - -def _extract_version(entry): - """Extract the version part from the requirement string. - - - Args: - entry: {type}`str` The requirement string. - """ - version_start = entry.find("==") - if version_start != -1: - # Extract everything after '==' until the next space or end of the string - version, _, _ = entry[version_start + 2:].partition(" ") - return version - return None +load(":pep508_evaluate.bzl", "evaluate") +load(":pep508_requirement.bzl", "requirement") +load(":select_whl.bzl", "select_whl") def parse_requirements( ctx, *, requirements_by_platform = {}, extra_pip_args = [], + platforms, get_index_urls = None, - evaluate_markers = None, - logger = None): + extract_url_srcs = True, + uv_lock = None, + toml_decode = None, + logger): """Get the requirements with platforms that the requirements apply to. Args: ctx: A context that has .read function that would read contents from a label. + platforms: The target platform descriptions. Cannot be empty and needs to have + at least the host platform and the definitions. requirements_by_platform (label_keyed_string_dict): a way to have different package versions (or different packages) for different os, arch combinations. extra_pip_args (string list): Extra pip arguments to perform extra validations and to be joined with args found in files. - get_index_urls: Callable[[ctx, list[str]], dict], a callable to get all + get_index_urls: Callable[[ctx, dict[str, list[str]]], dict], a callable to get all of the distribution URLs from a PyPI index. Accepts ctx and distribution names to query. - evaluate_markers: A function to use to evaluate the requirements. - Accepts the ctx and a dict where keys are requirement lines to - evaluate against the platforms stored as values in the input dict. - Returns the same dict, but with values being platforms that are - compatible with the requirements line. - logger: repo_utils.logger or None, a simple struct to log diagnostic messages. + extract_url_srcs: A boolean to enable extracting URLs from requirement + lines to enable using bazel downloader. + uv_lock: {type}`str | None` an optional label/file path to the uv.lock + file. The ctx.read function will be used to read the contents. + If provided, the function will use the uv.lock file as the primary + source for package metadata and perform a consistency check against + requirements files if both are provided. + toml_decode: {type}`callable | None` A function to decode TOML + content (e.g. `toml.decode`). Required when `uv_lock` is provided. + logger: repo_utils.logger, a simple struct to log diagnostic messages. Returns: - A tuple where the first element a dict of dicts where the first key is - the normalized distribution name (with underscores) and the second key - is the requirement_line, then value and the keys are structs with the - following attributes: - * distribution: The non-normalized distribution name. - * srcs: The Simple API downloadable source list. - * requirement_line: The original requirement line. - * target_platforms: The list of target platforms that this package is for. - * is_exposed: A boolean if the package should be exposed via the hub + {type}`list[struct]` where each struct contains the following attributes: + * `name`: {type}`str` The normalized distribution name. + * `is_exposed`: {type}`bool` `True` if the package should be exposed via the hub repository. + * `is_multiple_versions`: {type}`bool` `True` if multiple versions have been + specified for this package. + * `index_url`: {type}`str` The index URL used to download the package. + * `srcs`: {type}`list[struct]` A list of per-distribution source entries, each + containing: `distribution`, `extra_pip_args`, `requirement_line`, + `target_platforms`, `filename`, `digest`, `url`, `yanked`. The `digest` + is a `:` string or empty if the artifact digest is unknown. + """ + if uv_lock and toml_decode: + uv_lock = toml_decode(ctx.read(uv_lock)) + return _parse_requirements_with_uv_lock( + ctx = ctx, + requirements_by_platform = requirements_by_platform, + extra_pip_args = extra_pip_args, + platforms = platforms, + get_index_urls = get_index_urls, + extract_url_srcs = extract_url_srcs, + uv_lock = uv_lock, + logger = logger, + ) + + return _parse_requirements_from_req_files( + ctx = ctx, + requirements_by_platform = requirements_by_platform, + extra_pip_args = extra_pip_args, + platforms = platforms, + get_index_urls = get_index_urls, + extract_url_srcs = extract_url_srcs, + logger = logger, + ) + +def _get_all_platforms(requirements_by_platform): + """Get the set of all platform names from requirements_by_platform.""" + all_platforms = {} + for plats in requirements_by_platform.values(): + for p in plats: + all_platforms[p] = None + return sorted(all_platforms) + +def _parse_requirements_with_uv_lock( + ctx, + *, + requirements_by_platform, + extra_pip_args, + platforms, + get_index_urls, + extract_url_srcs, + uv_lock, + logger): + """Parse requirements using uv.lock as the primary source.""" + if uv_lock: + return _parse_uv_lock_json( + uv_lock = uv_lock, + all_platforms = _get_all_platforms(requirements_by_platform) if requirements_by_platform else sorted(platforms.keys()), + platforms = platforms, + extra_pip_args = extra_pip_args, + logger = logger, + ) + + return _parse_requirements_from_req_files( + ctx = ctx, + requirements_by_platform = requirements_by_platform, + extra_pip_args = extra_pip_args, + platforms = platforms, + get_index_urls = get_index_urls, + extract_url_srcs = extract_url_srcs, + logger = logger, + ) + +def _parse_uv_lock_json(uv_lock, all_platforms, logger, extra_pip_args = None, platforms = {}): + """Parse uv.lock JSON and build the same return structs as parse_requirements. + + Args: + uv_lock: {type}`dict` The decoded uv.lock contents. + all_platforms: {type}`list[str]` The list of all platform names. + logger: {type}`struct` A logger for diagnostic messages. + extra_pip_args: {type}`list[str] | None` Extra pip arguments to pass through. + platforms: {type}`dict[str, struct]` A dict of platform name to platform info + (containing `.env` with PEP 508 marker environment). + + Returns: + {type}`list[struct]` The same format as {func}`parse_requirements`. + """ + extras_map = uv_lock_extras_map(uv_lock) + + uv_packages = {} + + if not platforms: + fail("BUG: platforms must be configured") + + for pkg in uv_lock["package"]: + name = pkg["name"] + version = pkg["version"] + norm_name = normalize_name(name) + entry = uv_packages.setdefault(norm_name, { + "distribution": name, + "resolved_srcs": [], + "versions": {}, + }) + entry["versions"][version] = None + source = pkg.get("source") or {} + registry = (source.get("registry") or "").rstrip("/") + if registry.startswith("http://") or registry.startswith("https://"): + index_url = "{}/{}".format(registry, norm_name.replace("_", "-")) + else: + index_url = "" + entry["index_url"] = index_url + + pkg_extras = sorted(extras_map.get(name, [])) + extra_str = "[{}]".format(",".join(pkg_extras)) if pkg_extras else "" + + markers = pkg.get("resolution-markers", []) + if markers: + marker_expr = " or ".join(markers) + pkg_platforms = [ + p + for p in all_platforms + if evaluate(marker_expr, env = platforms[p].env) + ] + else: + pkg_platforms = list(all_platforms) + + # Prepare candidates + candidates = [] + for wheel in pkg.get("wheels", []): + url = wheel["url"] + _, _, filename = url.rpartition("/") + candidates.append(struct( + filename = filename, + url = url, + digest = _parse_uv_lock_hash(wheel.get("hash", "")), + kind = "wheel", + )) + + sdist_struct = None + sdist = pkg.get("sdist", None) + if sdist: + url = sdist["url"] + _, _, filename = url.rpartition("/") + sdist_struct = struct( + filename = filename, + url = url, + digest = _parse_uv_lock_hash(sdist.get("hash", "")), + kind = "sdist", + ) + + git_struct = None + if pkg.get("source", {}).get("git"): + url = pkg["source"]["git"] + _, _, filename = url.rpartition("/") + git_struct = struct( + filename = filename, + url = url, + digest = "", + kind = "git", + source = pkg["source"], + ) + + plat_to_src = {} + for p in pkg_platforms: + platform = platforms.get(p) + if not platform: + continue + + best_wheel = None + if candidates: + best_wheel = select_whl( + whls = candidates, + python_version = platform.env.get("python_full_version", "3"), + whl_platform_tags = platform.whl_platform_tags, + whl_abi_tags = platform.whl_abi_tags, + implementation_name = platform.env.get("implementation_name", "cpython"), + limit = 1, + logger = logger, + ) + + if best_wheel: + plat_to_src[p] = best_wheel + elif sdist_struct: + plat_to_src[p] = sdist_struct + elif git_struct: + plat_to_src[p] = git_struct + + # Group platforms by resolved source + src_to_plats = {} + for p, src in plat_to_src.items(): + key = src.filename + src.digest + src_to_plats.setdefault(key, struct(src = src, plats = [])).plats.append(p) + + # Build resolved_srcs + for key, val in src_to_plats.items(): + src = val.src + plats = sorted(val.plats) + requirement_line = "{name}{extras}=={version}".format( + name = name, + extras = extra_str, + version = version, + ) + entry["resolved_srcs"].append(struct( + distribution = name, + extra_pip_args = extra_pip_args or [], + requirement_line = requirement_line, + target_platforms = plats, + filename = src.filename, + digest = src.digest, + url = src.url, + yanked = None, + )) + + ret = [] + for norm_name, info in sorted(uv_packages.items()): + versions = sorted(info["versions"].keys()) + item = struct( + name = norm_name, + # Only expose packages that resolved to at least one source. uv + # workspace/root members (e.g. `source = { virtual = "." }` or + # editable installs) resolve to no wheel/sdist, so exposing them + # would add a dangling entry to the hub's `all_requirements` / + # `all_whl_requirements` and create an alias to a subpackage that + # doesn't exist. This mirrors the requirements path, which also + # gates `is_exposed`. + is_exposed = bool(info["resolved_srcs"]), + is_multiple_versions = len(versions) > 1, + index_url = info["index_url"], + srcs = info["resolved_srcs"], + ) + ret.append(item) + + logger.debug(lambda: "Parsed {} packages from uv.lock".format(len(ret))) + return ret + +def _parse_uv_lock_hash(hash_str): + """Parse a uv.lock `hash` value of the form `:`. - The second element is extra_pip_args should be passed to `whl_library`. + The algorithm is not necessarily `sha256` because uv records the strongest + digest that the index offers. """ - evaluate_markers = evaluate_markers or (lambda *_: {}) + algo, _, hex_digest = hash_str.partition(":") + return hash.digest(algo, hex_digest) + +def _parse_requirements_from_req_files( + ctx, + *, + requirements_by_platform, + extra_pip_args, + platforms, + get_index_urls, + extract_url_srcs, + logger): + """Parse requirements from requirements.txt files (existing behavior).""" options = {} requirements = {} + all_files_parsed = {} + index_url = None + extra_index_urls = [] for file, plats in requirements_by_platform.items(): - if logger: - logger.debug(lambda: "Using {} for {}".format(file, plats)) + logger.trace(lambda: "Using {} for {}".format(file, plats)) contents = ctx.read(file) - # Parse the requirements file directly in starlark to get the information - # needed for the whl_library declarations later. parse_result = parse_requirements_txt(contents) - # Replicate a surprising behavior that WORKSPACE builds allowed: - # Defining a repo with the same name multiple times, but only the last - # definition is respected. - # The requirement lines might have duplicate names because lines for extras - # are returned as just the base package name. e.g., `foo[bar]` results - # in an entry like `("foo", "foo[bar] == 1.0 ...")`. - requirements_dict = { - (normalize_name(entry[0]), _extract_version(entry[1])): entry - for entry in sorted( - parse_result.requirements, - # Get the longest match and fallback to original WORKSPACE sorting, - # which should get us the entry with most extras. - # - # FIXME @aignas 2024-05-13: The correct behaviour might be to get an - # entry with all aggregated extras, but it is unclear if we - # should do this now. - key = lambda x: (len(x[1].partition("==")[0]), x), - ) - }.values() + if file not in all_files_parsed: + all_files_parsed[file] = parse_result.requirements tokenized_options = [] for opt in parse_result.options: @@ -125,103 +348,192 @@ def parse_requirements( tokenized_options.append(p) pip_args = tokenized_options + extra_pip_args + + # Parse the index URL from the requirement files once per file + index_url = argparse.index_url(pip_args, index_url) + extra_index_urls = argparse.extra_index_url(pip_args, []) + if argparse.platform(pip_args, []): + # No use of downloader if the user specifies "--platform" pip arg. This means that + # they intend to use pip to download the wheels + # + # TODO @aignas 2026-04-11: consider removing this line in the next major release + # (3.0). + get_index_urls = None + + # Pre-parse requirements once per file to avoid redundant parsing in loops + parsed_reqs = [(entry, requirement(entry[1])) for entry in parse_result.requirements] + for plat in plats: - requirements[plat] = requirements_dict + plat_env = platforms.get(plat) + + requirements[plat] = [ + entry + for entry, req in parsed_reqs + if not req.marker or (plat_env and evaluate(req.marker, env = plat_env.env)) + ] + options[plat] = pip_args + index_url = argparse.index_url(pip_args, index_url) + extra_index_urls = argparse.extra_index_url(pip_args, []) + platform = argparse.platform(pip_args, []) + if platform: + get_index_urls = None + + reqs_by_name = {} requirements_by_platform = {} - reqs_with_env_markers = {} - for target_platform, reqs_ in requirements.items(): - extra_pip_args = options[target_platform] + for plat, parse_results in requirements.items(): + requirements_dict = {} + for entry in sorted( + parse_results, + key = lambda x: (len(x[1].partition("==")[0]), x), + ): + req_line = entry[1] + req = requirement(req_line) + + requirements_dict[req.name] = entry - for distribution, requirement_line in reqs_: - for_whl = requirements_by_platform.setdefault( + extra_pip_args_for_plat = options[plat] + + for distribution, requirement_line in requirements_dict.values(): + for_whl = reqs_by_name.setdefault( normalize_name(distribution), {}, ) - if ";" in requirement_line: - reqs_with_env_markers.setdefault(requirement_line, []).append(target_platform) - for_req = for_whl.setdefault( - (requirement_line, ",".join(extra_pip_args)), + (requirement_line, ",".join(extra_pip_args_for_plat)), struct( distribution = distribution, srcs = index_sources(requirement_line), requirement_line = requirement_line, target_platforms = [], - extra_pip_args = extra_pip_args, + extra_pip_args = extra_pip_args_for_plat, ), ) - for_req.target_platforms.append(target_platform) - - # This may call to Python, so execute it early (before calling to the - # internet below) and ensure that we call it only once. - # - # NOTE @aignas 2024-07-13: in the future, if this is something that we want - # to do, we could use Python to parse the requirement lines and infer the - # URL of the files to download things from. This should be important for - # VCS package references. - env_marker_target_platforms = evaluate_markers(ctx, reqs_with_env_markers) - if logger: - logger.debug(lambda: "Evaluated env markers from:\n{}\n\nTo:\n{}".format( - reqs_with_env_markers, - env_marker_target_platforms, - )) + for_req.target_platforms.append(plat) index_urls = {} if get_index_urls: + distributions = {} + for entries in all_files_parsed.values(): + for entry in entries: + name, req_line = entry + srcs = index_sources(req_line) + if srcs.url: + continue + versions = distributions.setdefault(normalize_name(name), {}) + versions[srcs.version] = None + + distributions = {k: sorted(v.keys()) for k, v in distributions.items()} + index_urls = get_index_urls( ctx, - # Use list({}) as a way to have a set - list({ - req.distribution: None - for reqs in requirements_by_platform.values() - for req in reqs.values() - }), + distributions, + index_url = index_url, + extra_index_urls = extra_index_urls, ) - ret = {} - for whl_name, reqs in sorted(requirements_by_platform.items()): + ret = [] + for name, reqs in sorted(reqs_by_name.items()): requirement_target_platforms = {} for r in reqs.values(): - target_platforms = env_marker_target_platforms.get(r.requirement_line, r.target_platforms) - for p in target_platforms: + for p in r.target_platforms: requirement_target_platforms[p] = None - is_exposed = len(requirement_target_platforms) == len(requirements) - if not is_exposed and logger: - logger.debug(lambda: "Package '{}' will not be exposed because it is only present on a subset of platforms: {} out of {}".format( - whl_name, + pkg_sources = index_urls.get(name) + package_srcs = _package_srcs( + name = name, + reqs = reqs, + pkg_sources = pkg_sources, + platforms = platforms, + extract_url_srcs = extract_url_srcs, + logger = logger, + ) + + item = struct( + name = normalize_name(name), + is_exposed = len(requirement_target_platforms) == len(requirements), + is_multiple_versions = len(reqs.values()) > 1, + index_url = pkg_sources.index_url if pkg_sources else "", + srcs = package_srcs, + ) + ret.append(item) + if not item.is_exposed and logger: + logger.trace(lambda: "Package '{}' will not be exposed because it is only present on a subset of platforms: {} out of {}".format( + name, sorted(requirement_target_platforms), sorted(requirements), )) - for r in sorted(reqs.values(), key = lambda r: r.requirement_line): - whls, sdist = _add_dists( + logger.debug(lambda: "Will configure whl repos: {}".format([w.name for w in ret])) + + return ret + +def _package_srcs( + *, + name, + reqs, + pkg_sources, + platforms, + logger, + extract_url_srcs): + """A function to return sources for a particular package.""" + srcs = {} + for r in sorted(reqs.values(), key = lambda r: r.requirement_line): + extra_pip_args = tuple(r.extra_pip_args) + + for target_platform in r.target_platforms: + if platforms and target_platform not in platforms: + fail("The target platform '{}' could not be found in {}".format( + target_platform, + platforms.keys(), + )) + + dist, can_fallback = _add_dists( requirement = r, - index_urls = index_urls.get(whl_name), + target_platform = platforms.get(target_platform), + index_urls = pkg_sources, logger = logger, ) - - target_platforms = env_marker_target_platforms.get(r.requirement_line, r.target_platforms) - ret.setdefault(whl_name, []).append( + logger.debug(lambda: "The whl dist is: {}".format(dist.filename if dist else dist)) + + if extract_url_srcs and dist: + req_line = r.srcs.requirement + elif can_fallback or (not extract_url_srcs and dist): + dist = struct( + url = "", + filename = "", + digest = "", + yanked = None, + ) + req_line = r.srcs.requirement_line + else: + continue + + key = ( + dist.filename, + req_line, + extra_pip_args, + ) + entry = srcs.setdefault( + key, struct( - distribution = r.distribution, - srcs = r.srcs, - requirement_line = r.requirement_line, - target_platforms = sorted(target_platforms), + distribution = name, extra_pip_args = r.extra_pip_args, - whls = whls, - sdist = sdist, - is_exposed = is_exposed, + requirement_line = req_line, + target_platforms = [], + filename = dist.filename, + digest = dist.digest, + url = dist.url, + yanked = dist.yanked, ), ) - if logger: - logger.debug(lambda: "Will configure whl repos: {}".format(ret.keys())) + if target_platform not in entry.target_platforms: + entry.target_platforms.append(target_platform) - return ret + return srcs.values() def select_requirement(requirements, *, platform): """A simple function to get a requirement for a particular platform. @@ -273,7 +585,7 @@ def host_platform(ctx): repo_utils.get_platforms_cpu_name(ctx), ) -def _add_dists(*, requirement, index_urls, logger = None): +def _add_dists(*, requirement, index_urls, target_platform, logger = None): """Populate dists based on the information from the PyPI index. This function will modify the given requirements_by_platform data structure. @@ -281,38 +593,68 @@ def _add_dists(*, requirement, index_urls, logger = None): Args: requirement: The result of parse_requirements function. index_urls: The result of simpleapi_download. + target_platform: The target_platform information. logger: A logger for printing diagnostic info. + + Returns: + (dist, can_fallback_to_pip): a struct with distribution details and how to fetch + it and a boolean flag to tell the other layers if we should add an entry to + fallback for pip if there are no supported whls found - if there is an sdist, we + can attempt the fallback, otherwise better to not, because the pip command will + fail and the error message will be confusing. What is more that would lead to + breakage of the bazel query. """ + + if requirement.srcs.url: + if not requirement.srcs.filename: + logger.debug(lambda: "Could not detect the filename from the URL, falling back to pip: {}".format( + requirement.srcs.url, + )) + return None, True + + # Handle direct URLs in requirements + dist = struct( + url = requirement.srcs.url, + filename = requirement.srcs.filename, + digest = hash.preferred_digest(requirement.srcs.hashes), + yanked = None, + ) + + return dist, False + if not index_urls: - return [], None + return None, True whls = [] sdist = None - # TODO @aignas 2024-05-22: it is in theory possible to add all - # requirements by version instead of by sha256. This may be useful - # for some projects. - for sha256 in requirement.srcs.shas: + # First try to find distributions by the hashes if provided + hashes_to_use = requirement.srcs.hashes + if not hashes_to_use: + version = requirement.srcs.version + hashes_to_use = index_urls.hashes_by_version.get(version, []) + logger.warn(lambda: "requirement file has been generated without hashes, will use all hashes for the given version {} that could find on the index:\n {}".format(version, hashes_to_use)) + + for digest in hashes_to_use: # For now if the artifact is marked as yanked we just ignore it. # # See https://packaging.python.org/en/latest/specifications/simple-repository-api/#adding-yank-support-to-the-simple-api - maybe_whl = index_urls.whls.get(sha256) - if maybe_whl and not maybe_whl.yanked: + maybe_whl = index_urls.whls.get(digest) + if maybe_whl and maybe_whl.yanked == None: whls.append(maybe_whl) continue - maybe_sdist = index_urls.sdists.get(sha256) - if maybe_sdist and not maybe_sdist.yanked: + maybe_sdist = index_urls.sdists.get(digest) + if maybe_sdist and maybe_sdist.yanked == None: sdist = maybe_sdist continue - if logger: - logger.warn(lambda: "Could not find a whl or an sdist with sha256={}".format(sha256)) + logger.warn(lambda: "Could not find a whl or an sdist with hash {}; note that the index may be advertising digests calculated with a different hash algorithm".format(digest)) yanked = {} for dist in whls + [sdist]: - if dist and dist.yanked: + if dist and dist.yanked != None: yanked.setdefault(dist.yanked, []).append(dist.filename) if yanked: logger.warn(lambda: "\n".join([ @@ -322,7 +664,23 @@ def _add_dists(*, requirement, index_urls, logger = None): for reason, dists in yanked.items() ])) - # Filter out the wheels that are incompatible with the target_platforms. - whls = select_whls(whls = whls, want_platforms = requirement.target_platforms, logger = logger) - - return whls, sdist + if not target_platform: + # The pipstar platforms are undefined here, so we cannot do any matching + return sdist, True + + if not whls and not sdist: + # If there are no suitable wheels to handle for now allow fallback to pip, it + # may be a little bit more helpful when debugging? Most likely something is + # going a bit wrong here, should we raise an error because the digests have + # most likely mismatched? We are already printing a warning above. + return None, True + + # Select a single wheel that can work on the target_platform + return select_whl( + whls = whls, + python_version = target_platform.env["python_full_version"], + implementation_name = target_platform.env["implementation_name"], + whl_abi_tags = target_platform.whl_abi_tags, + whl_platform_tags = target_platform.whl_platform_tags, + logger = logger, + ) or sdist, sdist != None diff --git a/python/private/pypi/parse_simpleapi_html.bzl b/python/private/pypi/parse_simpleapi_html.bzl index b4e7dd8330..e919c8ca55 100644 --- a/python/private/pypi/parse_simpleapi_html.bzl +++ b/python/private/pypi/parse_simpleapi_html.bzl @@ -16,126 +16,218 @@ Parse SimpleAPI HTML in Starlark. """ -def parse_simpleapi_html(*, url, content): - """Get the package URLs for given shas by parsing the Simple API HTML. +load("//python/private:normalize_name.bzl", "normalize_name") +load(":hash.bzl", "hash") +load(":version_from_filename.bzl", "version_from_filename") + +def parse_simpleapi_html(*, content, parse_index = False): + """Get the package URLs for given digests by parsing the Simple API HTML. Args: - url(str): The URL that the HTML content can be downloaded from. - content(str): The Simple API HTML content. + content: {type}`str` The Simple API HTML content. + parse_index: {type}`bool` whether to parse the content as the index page of the PyPI index, + e.g. the `https://pypi.org/simple/`. This only has the URLs for the individual package. Returns: - A list of structs with: - * filename: The filename of the artifact. - * url: The URL to download the artifact. - * sha256: The sha256 of the artifact. - * metadata_sha256: The whl METADATA sha256 if we can download it. If this is - present, then the 'metadata_url' is also present. Defaults to "". - * metadata_url: The URL for the METADATA if we can download it. Defaults to "". + If it is the index page, return the map of package to URL it can be queried from. + Otherwise, a struct with `whls` and `sdists` dicts keyed by the `:` + value advertised in the URL fragment (PEP 503, most commonly `sha256`, but any + algorithm from {obj}`hash.ALGOS` is accepted) and `hashes_by_version` mapping each + version to the digests of its artifacts. The dict values are structs with: + * filename: {type}`str` The filename of the artifact. + * version: {type}`str` The version of the artifact. + * url: {type}`str` The URL to download the artifact. + * digest: {type}`str` The `:` value of the artifact, may be empty + if the index does not advertise any digest. + * metadata_sha256: {type}`str` The whl METADATA sha256 if we can download it. If this is + present, then the 'metadata_url' is also present. Defaults to "". + * metadata_url: {type}`str` The URL for the METADATA if we can download it. Defaults to "". + * yanked: {type}`str | None` the yank reason if the package is yanked. If it is not yanked, + then it will be `None`. An empty string yank reason means that the package is yanked but + the reason is not provided. """ sdists = {} whls = {} - lines = content.split("= (2, 0): # We don't expect to have version 2.0 here, but have this check in place just in case. # https://packaging.python.org/en/latest/specifications/simple-repository-api/#versioning-pypi-s-simple-api fail("Unsupported API version: {}".format(api_version)) - # Each line follows the following pattern - # filename
- for line in lines[1:]: - dist_url, _, tail = line.partition("#sha256=") - sha256, _, tail = tail.partition("\"") - - # See https://packaging.python.org/en/latest/specifications/simple-repository-api/#adding-yank-support-to-the-simple-api - yanked = "data-yanked" in line - - head, _, _ = tail.rpartition("") - maybe_metadata, _, filename = head.rpartition(">") - + packages = {} + + # 2. Iterate using find() to avoid huge list allocations from .split(" tag first, then find the end of the opening + # tag using rfind. This correctly handles attributes that + # contain > characters, e.g. data-requires-python=">=3.6". + end_tag = content.find("", start_tag) + if end_tag == -1: + break + tag_end = content.rfind(">", start_tag, end_tag) + if tag_end == -1 or tag_end <= start_tag: + cursor = end_tag + 4 + continue + + # Extract only the necessary slices + filename = content[tag_end + 1:end_tag].strip() + attr_part = content[start_tag + 3:tag_end] + + # Update cursor for next iteration + cursor = end_tag + 4 + + attrs = _parse_attrs(attr_part) + href = attrs.get("href", "") + if not href: + continue + + if parse_index: + pkg_name = filename + packages[normalize_name(pkg_name)] = href + continue + + # 3. Efficient Attribute Parsing + # PEP 503 says the URL fragment SHOULD be `#=` where the + # hash name may be any algorithm from `hashlib`, not only `sha256`. + dist_url, _, fragment = href.partition("#") + algo, _, hex_digest = fragment.partition("=") + digest = hash.digest(algo, hex_digest) + if not digest and fragment: + # Not a hash fragment, keep it as part of the URL. + dist_url = "{}#{}".format(dist_url, fragment) + + # Handle Yanked status + yanked = None + if "data-yanked" in attrs: + yanked = _unescape_pypi_html(attrs["data-yanked"]) + + version = version_from_filename(filename) + hashes_by_version.setdefault(version, []).append(digest) + + # 4. Optimized Metadata Check (PEP 714) metadata_sha256 = "" metadata_url = "" - for metadata_marker in ["data-core-metadata", "data-dist-info-metadata"]: - metadata_marker = metadata_marker + "=\"sha256=" - if metadata_marker in maybe_metadata: - # Implement https://peps.python.org/pep-0714/ - _, _, tail = maybe_metadata.partition(metadata_marker) - metadata_sha256, _, _ = tail.partition("\"") - metadata_url = dist_url + ".metadata" - break + + # Dist-info is more common in modern PyPI + m_val = attrs.get("data-dist-info-metadata") or attrs.get("data-core-metadata") + if m_val and m_val != "false": + _, _, metadata_sha256 = m_val.partition("sha256=") + metadata_url = dist_url + ".metadata" + + # 5. Result object + dist = struct( + filename = filename, + version = version, + url = dist_url, + digest = digest, + metadata_sha256 = metadata_sha256, + metadata_url = metadata_url, + yanked = yanked, + ) if filename.endswith(".whl"): - whls[sha256] = struct( - filename = filename, - url = _absolute_url(url, dist_url), - sha256 = sha256, - metadata_sha256 = metadata_sha256, - metadata_url = _absolute_url(url, metadata_url) if metadata_url else "", - yanked = yanked, - ) + whls[digest] = dist else: - sdists[sha256] = struct( - filename = filename, - url = _absolute_url(url, dist_url), - sha256 = sha256, - metadata_sha256 = "", - metadata_url = "", - yanked = yanked, - ) + sdists[digest] = dist + + if parse_index: + return packages return struct( sdists = sdists, whls = whls, + hashes_by_version = hashes_by_version, ) -def _get_root_directory(url): - scheme_end = url.find("://") - if scheme_end == -1: - fail("Invalid URL format") +def _parse_attrs(attr_string): + """Parses attributes from a pre-sliced string.""" + attrs = {} + parts = attr_string.split('"') - scheme = url[:scheme_end] - host_end = url.find("/", scheme_end + 3) - if host_end == -1: - host_end = len(url) - host = url[scheme_end + 3:host_end] + for i in range(0, len(parts) - 1, 2): + raw_key = parts[i].strip() + if not raw_key: + continue - return "{}://{}".format(scheme, host) + key_parts = raw_key.split(" ") + current_key = key_parts[-1].rstrip("=") -def _is_downloadable(url): - """Checks if the URL would be accepted by the Bazel downloader. + # Batch handle booleans + for j in range(len(key_parts) - 1): + b = key_parts[j].strip() + if b: + attrs[b] = "" - This is based on Bazel's HttpUtils::isUrlSupportedByDownloader - """ - return url.startswith("http://") or url.startswith("https://") or url.startswith("file://") + attrs[current_key] = parts[i + 1] -def _absolute_url(index_url, candidate): - if candidate == "": - return candidate + # Final trailing boolean check + last = parts[-1].strip() + if last: + for b in last.split(" "): + if b: + attrs[b] = "" + return attrs - if _is_downloadable(candidate): - return candidate +def _unescape_pypi_html(text): + """Unescape HTML text. - if candidate.startswith("/"): - # absolute path - root_directory = _get_root_directory(index_url) - return "{}{}".format(root_directory, candidate) + Decodes standard HTML entities used in the Simple API. + Specifically targets characters used in URLs and attribute values. - if candidate.startswith(".."): - # relative path with up references - candidate_parts = candidate.split("..") - last = candidate_parts[-1] - for _ in range(len(candidate_parts) - 1): - index_url, _, _ = index_url.rstrip("/").rpartition("/") + Args: + text: {type}`str` The text to replace. - return "{}/{}".format(index_url, last.strip("/")) + Returns: + A string with unescaped characters + """ - # relative path without up-references - return "{}/{}".format(index_url, candidate) + # 1. Short circuit for the most common case + if not text or "&" not in text: + return text + + # 2. Check for the most frequent PEP 503 entities first (version constraints). + # Re-ordering based on frequency reduces unnecessary checks for rare entities. + if ">" in text: + text = text.replace(">", ">") + if "<" in text: + text = text.replace("<", "<") + + # 3. Grouped check for numeric entities. + # If '&#' isn't there, we skip 4 distinct string scans. + if "&#" in text: + if "'" in text: + text = text.replace("'", "'") + if "'" in text: + text = text.replace("'", "'") + if " " in text: + text = text.replace(" ", "\n") + if " " in text: + text = text.replace(" ", "\r") + + if """ in text: + text = text.replace(""", '"') + + # 4. Handle ampersands last to prevent double-decoding. + if "&" in text: + text = text.replace("&", "&") + + return text diff --git a/python/private/pypi/patch_whl.bzl b/python/private/pypi/patch_whl.bzl index a7da224321..e8d76eeba5 100644 --- a/python/private/pypi/patch_whl.bzl +++ b/python/private/pypi/patch_whl.bzl @@ -12,25 +12,26 @@ # See the License for the specific language governing permissions and # limitations under the License. -"""A small utility to patch a file in the repository context and repackage it using a Python interpreter - -Note, because we are patching a wheel file and we need a new RECORD file, this -function will print a diff of the RECORD and will ask the user to include a -RECORD patch in their patches that they maintain. This is to ensure that we can -satisfy the following usecases: -* Patch an invalid RECORD file. -* Patch files within a wheel. - -If we were silently regenerating the RECORD file, we may be vulnerable to supply chain -attacks (it is a very small chance) and keeping the RECORD patches next to the -other patches ensures that the users have overview on exactly what has changed -within the wheel. +"""A small utility to patch a file in the repository context natively. + +This replaces the previous approach that depended on running Python to +repack patched wheels. Instead, after extracting and patching, we fix +the RECORD file natively in Starlark and repack using system zip. """ +load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") load("//python/private:repo_utils.bzl", "repo_utils") load(":parse_whl_name.bzl", "parse_whl_name") -_rules_python_root = Label("//:BUILD.bazel") +_DUMMY_SHA256 = "sha256=0" + +_RECORD_EXCLUDES = { + "INSTALLER": None, + "RECORD": None, + "RECORD.jws": None, + "RECORD.p7s": None, + "REQUESTED": None, +} def patched_whl_name(original_whl_name): """Return the new filename to output the patched wheel. @@ -45,14 +46,6 @@ def patched_whl_name(original_whl_name): version = parsed_whl.version suffix = "patched" if "+" in version: - # This already has some local version, so we just append one more - # identifier here. We comply with the spec and mark the file as patched - # by adding a local version identifier at the end. - # - # By doing this we can still install the package using most of the package - # managers - # - # See https://packaging.python.org/en/latest/specifications/version-specifiers/#local-version-identifiers version = "{}.{}".format(version, suffix) else: version = "{}+{}".format(version, suffix) @@ -65,33 +58,30 @@ def patched_whl_name(original_whl_name): platform_tag = parsed_whl.platform_tag, ) -def patch_whl(rctx, *, python_interpreter, whl_path, patches, **kwargs): - """Patch a whl file and repack it to ensure that the RECORD metadata stays correct. +def patch_whl(rctx, *, whl_path, patches): + """Patch a whl file and repack it natively. + + The wheel is extracted, patched, missing RECORD entries are added + with dummy sha256 and size values, and finally the wheel is + repacked using system zip. Args: rctx: repository_ctx - python_interpreter: the python interpreter to use. whl_path: The whl file name to be patched. patches: a label-keyed-int dict that has the patch files as keys and the patch_strip as the value. - **kwargs: extras passed to repo_utils.execute_checked. Returns: value of the repackaging action. """ - - # extract files into the current directory for patching as rctx.patch - # does not support patching in another directory. whl_input = rctx.path(whl_path) - # symlink to a zip file to use bazel's extract so that we can use bazel's - # repository_ctx patch implementation. The whl file may be in a different - # external repository. - whl_file_zip = whl_input.basename + ".zip" - rctx.symlink(whl_input, whl_file_zip) - rctx.extract(whl_file_zip) - if not rctx.delete(whl_file_zip): - fail("Failed to remove the symlink after extracting") + repo_utils.extract( + rctx, + archive = whl_input, + supports_whl_extraction = rp_config.supports_whl_extraction, + extract_needs_chmod = rp_config.extract_needs_chmod, + ) if not patches: fail("Trying to patch wheel without any patches") @@ -99,39 +89,159 @@ def patch_whl(rctx, *, python_interpreter, whl_path, patches, **kwargs): for patch_file, patch_strip in patches.items(): rctx.patch(patch_file, strip = patch_strip) - record_patch = rctx.path("RECORD.patch") + _fix_record(rctx) + whl_patched = patched_whl_name(whl_input.basename) + rctx.delete(whl_input) + _repack_whl(rctx, output = whl_patched) + + return rctx.path(whl_patched) + +def _fix_record(rctx): + """Add missing file entries to RECORD with dummy sha256 and file size.""" + for entry in rctx.path(".").readdir(): + if not entry.basename.endswith(".dist-info"): + continue + + record_path = entry.get_child("RECORD") + if not record_path.exists: + continue + + all_files = _collect_files(rctx) + record_rel = repo_utils.repo_root_relative_path(rctx, record_path) + + new_content = fix_record_content( + record_content = rctx.read(record_path), + all_files = all_files, + record_rel = record_rel, + ) + if new_content != None: + rctx.file(record_path, new_content) + +def fix_record_content(record_content, all_files, record_rel): + """Add missing file entries to RECORD content with dummy sha256 and size. + + Args: + record_content: {type}`str` The existing RECORD file content. + all_files: {type}`dict[str, bool]` All files in the directory, keys + are repo-root-relative paths. + record_rel: {type}`str` The repo-root-relative path to the RECORD file. + + Returns: + {type}`str | None` The new RECORD content if entries were added, + or None if no changes were needed. + """ + has_trailing_newline = record_content.endswith("\n") + lines = record_content.split("\n") + + is_all_quoted = True + has_content = False + existing = {} + for line in lines: + stripped = line.strip() + if not stripped: + continue + has_content = True + if not stripped.startswith('"'): + is_all_quoted = False + parts = stripped.split(",") + if len(parts) >= 1: + fname = parts[0].strip('"') + existing[fname] = True + + if not has_content: + is_all_quoted = False + + existing[record_rel] = True + + added = [] + for fpath in sorted(all_files.keys()): + if fpath in existing: + continue + basename = fpath.split("/")[-1] if "/" in fpath else fpath + if basename in _RECORD_EXCLUDES: + continue + if fpath.endswith(".whl"): + continue + added.append(fpath) + + if not added: + return None + + new_lines = list(lines) + if not has_trailing_newline: + new_lines.append("") + if is_all_quoted: + for fpath in added: + new_lines.append('"{file}",{sha256},0'.format( + file = fpath, + sha256 = _DUMMY_SHA256, + )) + else: + for fpath in added: + new_lines.append("{file},{sha256},0".format( + file = fpath, + sha256 = _DUMMY_SHA256, + )) + new_lines.append("") + + return "\n".join(new_lines) + +def _collect_files(rctx): + """Collect all file paths relative to the repo root (iterative).""" + result = {} + paths = [(rctx.path("."), "")] + for _ in range(10000000): + if not paths: + break + path, prefix = paths.pop() + for entry in path.readdir(): + rel = prefix + "/" + entry.basename if prefix else entry.basename + if entry.is_dir: + paths.append((entry, rel)) + else: + result[rel] = True + return result + +def _repack_whl(rctx, *, output): + """Repack the current directory into a wheel file using system zip.""" + os_name = repo_utils.get_platforms_os_name(rctx) + if os_name == "windows": + _repack_whl_windows(rctx, output = output) + else: + _repack_whl_unix(rctx, output = output) + +def _repack_whl_unix(rctx, *, output): repo_utils.execute_checked( rctx, + op = "PatchWhl", arguments = [ - python_interpreter, - "-m", - "python.private.pypi.repack_whl", - "--record-patch", - record_patch, - whl_input, - whl_patched, + "zip", + "-0", + "-X", + str(output), + "-r", + ".", ], - environment = { - "PYTHONPATH": str(rctx.path(_rules_python_root).dirname), - }, - **kwargs ) - if record_patch.exists: - record_patch_contents = rctx.read(record_patch) - warning_msg = """WARNING: the resultant RECORD file of the patch wheel is different +def _repack_whl_windows(rctx, *, output): + powershell_exe = rctx.which("powershell.exe") or rctx.which("powershell") + if not powershell_exe: + fail("powershell not found on PATH") - If you are patching on Windows, you may see this warning because of - a known issue (bazelbuild/rules_python#1639) with file endings. + script_path = rctx.path(Label("//python/private/pypi:repack_whl.ps1")) - If you would like to silence the warning, you can apply the patch that is stored in - {record_patch}. The contents of the file are below: -{record_patch_contents}""".format( - record_patch = record_patch, - record_patch_contents = record_patch_contents, - ) - print(warning_msg) # buildifier: disable=print - - return rctx.path(whl_patched) + repo_utils.execute_checked( + rctx, + op = "PatchWhl", + arguments = [ + powershell_exe, + "-NoProfile", + "-File", + str(script_path), + "-Output", + str(output), + ], + ) diff --git a/python/private/pypi/pep508_deps.bzl b/python/private/pypi/pep508_deps.bzl new file mode 100644 index 0000000000..fd6d961bf5 --- /dev/null +++ b/python/private/pypi/pep508_deps.bzl @@ -0,0 +1,197 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""This module is for implementing PEP508 compliant METADATA deps parsing. +""" + +load("//python/private:normalize_name.bzl", "normalize_name") +load(":pep508_env.bzl", "create_env") +load(":pep508_evaluate.bzl", "evaluate") +load(":pep508_requirement.bzl", "requirement") + +def deps( + name, + *, + requires_dist, + extras = [], + excludes = [], + include = []): + """Parse the RequiresDist from wheel METADATA + + Args: + name: {type}`str` the name of the wheel. + requires_dist: {type}`list[str]` the list of RequiresDist lines from the + METADATA file. + excludes: {type}`list[str]` what packages should we exclude. + include: {type}`list[str]` what packages should we exclude. If it is not + specified, then we will include all deps from `requires_dist`. + extras: {type}`list[str]` the requested extras to generate targets for. + + Returns: + A struct with attributes: + * deps: {type}`list[str]` dependencies to include unconditionally. + * deps_select: {type}`dict[str, list[str]]` dependencies to include on particular + subset of target platforms. + """ + if not requires_dist: + return struct( + deps = [], + deps_select = {}, + ) + + reqs = sorted( + [requirement(r) for r in requires_dist], + key = lambda x: "{}:{}:".format(x.name, sorted(x.extras), x.marker), + ) + deps = {} + deps_select = {} + name = normalize_name(name) + want_extras = _resolve_extras(name, reqs, extras) + include = [normalize_name(n) for n in include] + + # drop self edges + excludes = [name] + [normalize_name(x) for x in excludes] + + reqs_by_name = {} + + for req in reqs: + if req.name_ in excludes: + continue + + if include and req.name_ not in include: + continue + + reqs_by_name.setdefault(req.name, []).append(req) + + for name, reqs in reqs_by_name.items(): + _add_reqs( + deps, + deps_select, + normalize_name(name), + reqs, + extras = want_extras, + ) + + return struct( + deps = sorted(deps), + deps_select = { + d: markers + for d, markers in sorted(deps_select.items()) + }, + ) + +def _add(deps, deps_select, dep, markers = None): + dep = normalize_name(dep) + + if not markers: + deps[dep] = True + elif len(markers) == 1: + deps_select[dep] = markers[0] + else: + deps_select[dep] = "({})".format(") or (".join(sorted(markers))) + +def _resolve_extras(self_name, reqs, extras): + """Resolve extras which are due to depending on self[some_other_extra]. + + Some packages may have cyclic dependencies resulting from extras being used, one example is + `etils`, where we have one set of extras as aliases for other extras + and we have an extra called 'all' that includes all other extras. + + Example: github.com/google/etils/blob/a0b71032095db14acf6b33516bca6d885fe09e35/pyproject.toml#L32. + + When the `requirements.txt` is generated by `pip-tools`, then it is likely that + this step is not needed, but for other `requirements.txt` files this may be useful. + + NOTE @aignas 2023-12-08: the extra resolution is not platform dependent, + but in order for it to become platform dependent we would have to have + separate targets for each extra in extras. + """ + + # Resolve any extra extras due to self-edges, empty string means no + # extras The empty string in the set is just a way to make the handling + # of no extras and a single extra easier and having a set of {"", "foo"} + # is equivalent to having {"foo"}. + # + # Use a dict as a set here to simplify operations. + extras = {x: None for x in (extras or [""])} + + self_reqs = [] + for req in reqs: + if req.name != self_name: + continue + + if req.marker == None: + # I am pretty sure we cannot reach this code as it does not + # make sense to specify packages in this way, but since it is + # easy to handle, lets do it. + # + # TODO @aignas 2023-12-08: add a test + extras = extras | {x: None for x in req.extras} + else: + # process these in a separate loop + self_reqs.append(req) + + for _ in range(10000): + # handles packages with up to 10000 recursive extras + new_extras = {} + for req in self_reqs: + if _evaluate_any(req, extras): + new_extras.update({x: None for x in req.extras}) + else: + continue + + num_extras_before = len(extras) + extras = extras | new_extras + num_extras_after = len(new_extras) + + if num_extras_before == num_extras_after: + break + + # Poor mans set + return sorted(extras) + +def _evaluate_any(req, extras): + env = create_env() + for extra in extras: + if evaluate(req.marker, env = env | {"extra": extra}): + return True + + return False + +def _add_reqs(deps, deps_select, dep, reqs, *, extras): + for req in reqs: + if not req.marker: + _add(deps, deps_select, dep) + return + + env = create_env() + markers = {} + found_unconditional = False + for req in reqs: + for x in extras: + m = evaluate(req.marker, env = env | {"extra": x}, strict = False) + if m == False: + continue + elif m == True: + _add(deps, deps_select, dep) + found_unconditional = True + break + else: + markers[m] = None + continue + if found_unconditional: + break + + if markers and not found_unconditional: + _add(deps, deps_select, dep, sorted(markers)) diff --git a/python/private/pypi/pep508_env.bzl b/python/private/pypi/pep508_env.bzl new file mode 100644 index 0000000000..9fe9dcaada --- /dev/null +++ b/python/private/pypi/pep508_env.bzl @@ -0,0 +1,253 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""This module is for implementing PEP508 environment definition. +""" + +load("//python/private:normalize_name.bzl", "normalize_name") +load("//python/private:version.bzl", "version") + +_DEFAULT = "//conditions:default" + +# Here we store the aliases in the platform so that the users can specify any valid target in +# there. +_cpu_aliases = { + "arm": "aarch32", + "arm64": "aarch64", +} +_os_aliases = { + "macos": "osx", +} + +# See https://stackoverflow.com/a/45125525 +platform_machine_aliases = { + # These pairs mean the same hardware, but different values may be used + # on different host platforms. + "amd64": "x86_64", + "arm64": "aarch64", + "i386": "x86_32", + "i686": "x86_32", +} + +# NOTE: There are many cpus, and unfortunately, the value isn't directly +# accessible to Starlark. Using CcToolchain.cpu might work, though. +# Some targets are aliases and are omitted below as their value is implied +# by the target they resolve to. +platform_machine_select_map = { + "@platforms//cpu:aarch32": "aarch32", + "@platforms//cpu:aarch64": "aarch64", + # @platforms//cpu:arm is an alias for @platforms//cpu:aarch32 + # @platforms//cpu:arm64 is an alias for @platforms//cpu:aarch64 + "@platforms//cpu:arm64_32": "arm64_32", + "@platforms//cpu:arm64e": "arm64e", + "@platforms//cpu:armv6-m": "armv6-m", + "@platforms//cpu:armv7": "armv7", + "@platforms//cpu:armv7-m": "armv7-m", + "@platforms//cpu:armv7e-m": "armv7e-m", + "@platforms//cpu:armv7e-mf": "armv7e-mf", + "@platforms//cpu:armv7k": "armv7k", + "@platforms//cpu:armv8-m": "armv8-m", + "@platforms//cpu:cortex-r52": "cortex-r52", + "@platforms//cpu:cortex-r82": "cortex-r82", + "@platforms//cpu:i386": "i386", + "@platforms//cpu:mips64": "mips64", + "@platforms//cpu:ppc": "ppc", + "@platforms//cpu:ppc32": "ppc32", + "@platforms//cpu:ppc64le": "ppc64le", + "@platforms//cpu:riscv32": "riscv32", + "@platforms//cpu:riscv64": "riscv64", + "@platforms//cpu:s390x": "s390x", + "@platforms//cpu:wasm32": "wasm32", + "@platforms//cpu:wasm64": "wasm64", + "@platforms//cpu:x86_32": "x86_32", + "@platforms//cpu:x86_64": "x86_64", + # The value is empty string if it cannot be determined: + # https://docs.python.org/3/library/platform.html#platform.machine + _DEFAULT: "", +} + +# Platform system returns results from the `uname` call. +_platform_system_values = { + # See https://peps.python.org/pep-0738/#platform + "android": "Android", + "freebsd": "FreeBSD", + # See https://peps.python.org/pep-0730/#platform + # NOTE: Per Pep 730, "iPadOS" is also an acceptable value + "ios": "iOS", + "linux": "Linux", + "netbsd": "NetBSD", + "openbsd": "OpenBSD", + "osx": "Darwin", # NOTE: macos is an alias to osx, we handle it through _os_aliases + "windows": "Windows", +} + +platform_system_select_map = { + "@platforms//os:{}".format(bazel_os): py_system + for bazel_os, py_system in _platform_system_values.items() +} | { + # The value is empty string if it cannot be determined: + # https://docs.python.org/3/library/platform.html#platform.machine + _DEFAULT: "", +} + +# The copy of SO [answer](https://stackoverflow.com/a/13874620) containing +# all of the platforms: +# ┍━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━┑ +# │ System │ Value │ +# ┝━━━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━┥ +# │ Linux │ linux or linux2 (*) │ +# │ Windows │ win32 │ +# │ Windows/Cygwin │ cygwin │ +# │ Windows/MSYS2 │ msys │ +# │ Mac OS X │ darwin │ +# │ OS/2 │ os2 │ +# │ OS/2 EMX │ os2emx │ +# │ RiscOS │ riscos │ +# │ AtheOS │ atheos │ +# │ FreeBSD 7 │ freebsd7 │ +# │ FreeBSD 8 │ freebsd8 │ +# │ FreeBSD N │ freebsdN │ +# │ OpenBSD 6 │ openbsd6 │ +# │ AIX │ aix (**) │ +# ┕━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━┙ +# +# (*) Prior to Python 3.3, the value for any Linux version is always linux2; after, it is linux. +# (**) Prior Python 3.8 could also be aix5 or aix7; use sys.platform.startswith() +# +# We are using only the subset that we actually support. +_sys_platform_values = { + # These values are decided by the sys.platform docs. + "android": "android", + "emscripten": "emscripten", + # NOTE: The below values are approximations. The sys.platform() docs + # don't have documented values for these OSes. Per docs, the + # sys.platform() value reflects the OS at the time Python was *built* + # instead of the runtime (target) OS value. + "freebsd": "freebsd", + "ios": "ios", + "linux": "linux", + "openbsd": "openbsd", + "osx": "darwin", # NOTE: macos is an alias to osx, we handle it through _os_aliases + "wasi": "wasi", + "windows": "win32", +} + +sys_platform_select_map = { + # These values are decided by the sys.platform docs. + "@platforms//os:{}".format(bazel_os): py_platform + for bazel_os, py_platform in _sys_platform_values.items() +} | { + # For lack of a better option, use empty string. No standard doc/spec + # about sys_platform value. + _DEFAULT: "", +} + +# The "java" value is documented, but with Jython defunct, +# shouldn't occur in practice. +# The os.name value is technically a property of the runtime, not the +# targetted runtime OS, but the distinction shouldn't matter if +# things are properly configured. +os_name_select_map = { + "@platforms//os:windows": "nt", + _DEFAULT: "posix", +} + +def _set_default(env, env_key, m, key): + """Set the default value in the env if it is not already set.""" + default = m.get(key, m[_DEFAULT]) + env.setdefault(env_key, default) + +def env(*, env = None, os, arch, python_version = "", extra = None): + """Return an env target platform + + NOTE: This is for use during the loading phase. For the analysis phase, + `env_marker_setting()` constructs the env dict. + + Args: + env: {type}`str` the environment. + os: {type}`str` the OS name. + arch: {type}`str` the CPU name. + python_version: {type}`str` the full python version. + extra: {type}`str` the extra value to be added into the env. + + Returns: + A dict that can be used as `env` in the marker evaluation. + """ + env = env or {} + env = env | create_env() + if extra != None: + env["extra"] = extra + + if python_version: + v = version.parse(python_version) + major = v.release[0] + minor = v.release[1] + micro = v.release[2] if len(v.release) > 2 else 0 + env = env | { + "implementation_version": "{}.{}.{}".format(major, minor, micro), + "python_full_version": "{}.{}.{}".format(major, minor, micro), + "python_version": "{}.{}".format(major, minor), + } + + if os: + os = "@platforms//os:{}".format(_os_aliases.get(os, os)) + _set_default(env, "os_name", os_name_select_map, os) + _set_default(env, "platform_system", platform_system_select_map, os) + _set_default(env, "sys_platform", sys_platform_select_map, os) + + if arch: + arch = "@platforms//cpu:{}".format(_cpu_aliases.get(arch, arch)) + _set_default(env, "platform_machine", platform_machine_select_map, arch) + + set_missing_env_defaults(env) + + return env + +def create_env(): + return { + # Per-variable normalization functions. Each entry maps a marker + # variable name to a function (value) -> normalized_value. + "_aliases": { + "extra": normalize_name, + "platform_machine": lambda x: platform_machine_aliases.get(x, x), + }, + } + +def set_missing_env_defaults(env): + """Sets defaults based on existing values. + + Args: + env: dict; NOTE: modified in-place + """ + if "implementation_name" not in env: + # Use cpython as the default because it's likely the correct value. + env["implementation_name"] = "cpython" + if "platform_python_implementation" not in env: + # The `platform_python_implementation` marker value is supposed to come + # from `platform.python_implementation()`, however, PEP 421 introduced + # `sys.implementation.name` and the `implementation_name` env marker to + # replace it. Per the platform.python_implementation docs, there's now + # essentially just two possible "registered" values: CPython or PyPy. + # Rather than add a field to the toolchain, we just special case the value + # from `sys.implementation.name` to handle the two documented values. + platform_python_impl = env["implementation_name"] + if platform_python_impl == "cpython": + platform_python_impl = "CPython" + elif platform_python_impl == "pypy": + platform_python_impl = "PyPy" + env["platform_python_implementation"] = platform_python_impl + if "platform_release" not in env: + env["platform_release"] = "" + if "platform_version" not in env: + env["platform_version"] = "0" diff --git a/python/private/pypi/pep508_evaluate.bzl b/python/private/pypi/pep508_evaluate.bzl new file mode 100644 index 0000000000..61e461a54f --- /dev/null +++ b/python/private/pypi/pep508_evaluate.bzl @@ -0,0 +1,501 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""This module is for implementing PEP508 in starlark as FeatureFlagInfo +""" + +load("//python/private:enum.bzl", "enum") +load("//python/private:version.bzl", "version") + +# The expression parsing and resolution for the PEP508 is below +# + +_STATE = enum( + STRING = "string", + VAR = "var", + OP = "op", + NONE = "none", +) +_BRACKETS = "()" +_OPCHARS = "<>!=~" +_QUOTES = "'\"" +_WSP = " \t" +_NON_VERSION_VAR_NAMES = [ + "implementation_name", + "os_name", + "platform_machine", + "platform_python_implementation", + "platform_release", + "platform_system", + "sys_platform", + "extra", +] +_AND = "and" +_OR = "or" +_NOT = "not" +_ENV_ALIASES = "_aliases" + +def tokenize(marker): + """Tokenize the input string. + + The output will have double-quoted values (i.e. the quoting will be normalized) and all of the whitespace will be trimmed. + + Args: + marker: {type}`str` The input to tokenize. + + Returns: + The {type}`str` that is the list of recognized tokens that should be parsed. + """ + if not marker: + return [] + + tokens = [] + token = "" + state = _STATE.NONE + char = "" + + # Due to the `continue` in the loop, we will be processing chars at a slower pace + for _ in range(2 * len(marker)): + if token and (state == _STATE.NONE or not marker): + if tokens and token == "in" and tokens[-1] == _NOT: + tokens[-1] += " " + token + else: + tokens.append(token) + token = "" + + if not marker: + return tokens + + char = marker[0] + if char in _BRACKETS: + state = _STATE.NONE + token = char + elif state == _STATE.STRING and char in _QUOTES: + state = _STATE.NONE + token = '"{}"'.format(token) + elif ( + (state == _STATE.VAR and not char.isalnum() and char != "_") or + (state == _STATE.OP and char not in _OPCHARS) + ): + state = _STATE.NONE + continue # Skip consuming the char below + elif state == _STATE.NONE: + # Transition from _STATE.NONE to something or stay in NONE + if char in _QUOTES: + state = _STATE.STRING + elif char.isalnum(): + state = _STATE.VAR + token += char + elif char in _OPCHARS: + state = _STATE.OP + token += char + elif char in _WSP: + state = _STATE.NONE + else: + fail("BUG: Cannot parse '{}' in {} ({})".format(char, state, marker)) + else: + token += char + + # Consume the char + marker = marker[1:] + + return fail("BUG: failed to process the marker in allocated cycles: {}".format(marker)) + +def evaluate(marker, *, env, strict = True, **kwargs): + """Evaluate the marker against a given env. + + Args: + marker: {type}`str` The string marker to evaluate. + env: {type}`dict[str, str]` The environment to evaluate the marker against. + strict: {type}`bool` A setting to not fail on missing values in the env. + **kwargs: Extra kwargs to be passed to the expression evaluator. + + Returns: + The {type}`bool | str` If the marker is compatible with the given env. If strict is + `False`, then the output type is `str` which will represent the remaining + expression that has not been evaluated. + """ + tokens = tokenize(marker) + + ast = _new_expr(marker = marker, **kwargs) + for _ in range(len(tokens) * 2): + if not tokens: + break + + tokens = ast.parse(env = env, tokens = tokens, strict = strict) + + if not tokens: + return ast.value() + + fail("Could not evaluate: {}".format(marker)) + +_STRING_REPLACEMENTS = { + "!=": "neq", + "(": "_", + ")": "_", + "<": "lt", + "<=": "lteq", + "==": "eq", + "===": "eeq", + ">": "gt", + ">=": "gteq", + "not in": "not_in", + "~==": "cmp", +} + +def to_string(marker): + return "_".join([ + _STRING_REPLACEMENTS.get(t, t) + for t in tokenize(marker) + ]).replace("\"", "") + +def _and_fn(x, y): + """Our custom `and` evaluation function. + + Allow partial evaluation if one of the values is a string, return the + string value because that means that `marker_expr` was set to + `strict = False` and we are only evaluating what we can. + """ + if not (x and y): + return False + + x_is_str = type(x) == type("") + y_is_str = type(y) == type("") + if x_is_str and y_is_str: + return "{} and {}".format(x, y) + elif x_is_str: + return x + else: + return y + +def _or_fn(x, y): + """Our custom `or` evaluation function. + + Allow partial evaluation if one of the values is a string, return the + string value because that means that `marker_expr` was set to + `strict = False` and we are only evaluating what we can. + """ + x_is_str = type(x) == type("") + y_is_str = type(y) == type("") + + if x_is_str and y_is_str: + return "{} or {}".format(x, y) if x and y else "" + elif x_is_str: + return "" if y else x + elif y_is_str: + return "" if x else y + else: + return x or y + +def _not_fn(x): + """Our custom `not` evaluation function. + + Allow partial evaluation if the value is a string. + """ + if type(x) == type(""): + return "not {}".format(x) + else: + return not x + +def _new_expr( + *, + marker, + and_fn = _and_fn, + or_fn = _or_fn, + not_fn = _not_fn): + # buildifier: disable=uninitialized + self = struct( + marker = marker, + tree = [], + parse = lambda **kwargs: _parse(self, **kwargs), + value = lambda: _value(self), + # This is a way for us to have a handle to the currently constructed + # expression tree branch. + current = lambda: self._current[-1] if self._current else None, + _current = [], + _and = and_fn, + _or = or_fn, + _not = not_fn, + ) + return self + +def _parse(self, *, env, tokens, strict = False): + """The parse function takes the consumed tokens and returns the remaining.""" + token, remaining = tokens[0], tokens[1:] + + if token == "(": + expr = _open_parenthesis(self) + elif token == ")": + expr = _close_parenthesis(self) + elif token == _AND: + expr = _and_expr(self) + elif token == _OR: + expr = _or_expr(self) + elif token == _NOT: + expr = _not_expr(self) + else: + expr = marker_expr(env = env, strict = strict, *tokens[:3]) + remaining = tokens[3:] + + _append(self, expr) + return remaining + +def _value(self): + """Evaluate the expression tree""" + if not self.tree: + # Basic case where no marker should evaluate to True + return True + + for _ in range(len(self.tree)): + if len(self.tree) == 1: + return self.tree[0] + + # Resolve all of the `or` expressions as it is safe to do now since all + # `and` and `not` expressions have been taken care of by now. + if getattr(self.tree[-2], "op", None) == _OR: + current = self.tree.pop() + self.tree[-1] = self.tree[-1].value(current) + else: + break + + fail("BUG: invalid state: {}".format(self.tree)) + +def marker_expr(left, op, right, *, env, strict = True): + """Evaluate a marker expression + + Args: + left: {type}`str` the env identifier or a value quoted in `"`. + op: {type}`str` the operation to carry out. + right: {type}`str` the env identifier or a value quoted in `"`. + strict: {type}`bool` if false, only evaluates the values that are present + in the environment, otherwise returns the original expression. + env: {type}`dict[str, str]` the `env` to substitute `env` identifiers in + the ` ` expression. Note, if `env` has a key + "_aliases", then we will do normalization so that we can ensure + that e.g. `aarch64` evaluation in the `platform_machine` works the + same way irrespective if the marker uses `arm64` or `aarch64` value + in the expression. + + Returns: + {type}`bool` if the expression evaluation result or {type}`str` if the expression + could not be evaluated. + """ + var_name = None + if right not in env and left not in env and not strict: + return "{} {} {}".format(left, op, right) + if left[0] == '"': + var_name = right + right = env[right] + left = left.strip("\"") + + if _ENV_ALIASES in env: + # Normalize the literal value using per-variable normalization + # functions. This handles platform aliases (e.g. arm64 -> aarch64) + # and PEP 685 extra name normalization (e.g. db-backend -> db_backend). + left = env.get(_ENV_ALIASES, {}).get(var_name, lambda x: x)(left) + + else: + var_name = left + left = env[left] + right = right.strip("\"") + + if _ENV_ALIASES in env: + # See the note above on normalization + right = env.get(_ENV_ALIASES, {}).get(var_name, lambda x: x)(right) + + if var_name in _NON_VERSION_VAR_NAMES: + return _env_expr(left, op, right) + elif var_name.endswith("_version"): + return _version_expr(left, op, right) + else: + # Do not fail here, just evaluate the expression to False. + return False + +def _env_expr(left, op, right): + """Evaluate a string comparison expression""" + if op == "==": + return left == right + elif op == "!=": + return left != right + elif op == "in": + return left in right + elif op == "not in": + return left not in right + elif op == "<": + return left < right + elif op == "<=": + return left <= right + elif op == ">": + return left > right + elif op == ">=": + return left >= right + else: + return fail("unsupported op: '{}' {} '{}'".format(left, op, right)) + +def _version_expr(left, op, right): + """Evaluate a version comparison expression""" + _left = version.parse(left) + _right = version.parse(right) + if _left == None or _right == None: + # Per spec, if either can't be normalized to a version, then + # fallback to simple string comparison. Usually this is `platform_version` + # or `platform_release`, which vary depending on platform. + return _env_expr(left, op, right) + + if op == "===": + return version.is_eeq(_left, _right) + elif op == "!=": + return version.is_ne(_left, _right) + elif op == "==": + return version.is_eq(_left, _right) + elif op == "<": + return version.is_lt(_left, _right) + elif op == ">": + return version.is_gt(_left, _right) + elif op == "<=": + return version.is_le(_left, _right) + elif op == ">=": + return version.is_ge(_left, _right) + elif op == "~=": + return version.is_compatible(_left, _right) + else: + return False # Let's just ignore the invalid ops + +# Code to allowing to combine expressions with logical operators + +def _append(self, value): + if value == None: + return + + current = self.current() or self + op = getattr(value, "op", None) + + if op == _NOT: + current.tree.append(value) + elif op in [_AND, _OR]: + value.append(current.tree[-1]) + current.tree[-1] = value + elif not current.tree: + current.tree.append(value) + elif hasattr(current.tree[-1], "append"): + current.tree[-1].append(value) + elif hasattr(current.tree, "_append"): + current.tree._append(value) + else: + fail("Cannot evaluate '{}' in '{}', current: {}".format(value, self.marker, current)) + +def _open_parenthesis(self): + """Add an extra node into the tree to perform evaluate inside parenthesis.""" + self._current.append(_new_expr( + marker = self.marker, + and_fn = self._and, + or_fn = self._or, + not_fn = self._not, + )) + +def _close_parenthesis(self): + """Backtrack and evaluate the expression within parenthesis.""" + value = self._current.pop().value() + if type(value) == type(""): + return "({})".format(value) + else: + return value + +def _not_expr(self): + """Add an extra node into the tree to perform an 'not' operation.""" + + def _append(value): + """Append a value to the not expression node. + + This codifies `not` precedence over `and` and performs backtracking to + evaluate any `not` statements and forward the value to the first `and` + statement if needed. + """ + + current = self.current() or self + current.tree[-1] = self._not(value) + + for _ in range(len(current.tree)): + if not len(current.tree) > 1: + break + + op = getattr(current.tree[-2], "op", None) + if op == None: + pass + elif op == _NOT: + value = current.tree.pop() + current.tree[-1] = self._not(value) + continue + elif op == _AND: + value = current.tree.pop() + current.tree[-1].append(value) + elif op != _OR: + fail("BUG: '{} not' compound is unsupported".format(current.tree[-1])) + + break + + return struct( + op = _NOT, + append = _append, + ) + +def _and_expr(self): + """Add an extra node into the tree to perform an 'and' operation""" + maybe_value = [None] + + def _append(value): + """Append a value to the and expression node. + + Here we backtrack, but we only evaluate the current `and` statement - + all of the `not` statements will be by now evaluated and `or` + statements need to be evaluated later. + """ + if maybe_value[0] == None: + maybe_value[0] = value + return + + current = self.current() or self + current.tree[-1] = self._and(maybe_value[0], value) + + return struct( + op = _AND, + append = _append, + # private fields that help debugging + _maybe_value = maybe_value, + ) + +def _or_expr(self): + """Add an extra node into the tree to perform an 'or' operation""" + maybe_value = [None] + + def _append(value): + """Append a value to the or expression node. + + Here we just append the extra values to the tree and the `or` + statements will be evaluated in the _value() function. + """ + if maybe_value[0] == None: + maybe_value[0] = value + return + + current = self.current() or self + current.tree.append(value) + + return struct( + op = _OR, + value = lambda x: self._or(maybe_value[0], x), + append = _append, + # private fields that help debugging + _maybe_value = maybe_value, + ) diff --git a/python/private/pypi/pep508_requirement.bzl b/python/private/pypi/pep508_requirement.bzl new file mode 100644 index 0000000000..7552642572 --- /dev/null +++ b/python/private/pypi/pep508_requirement.bzl @@ -0,0 +1,58 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""This module is for parsing PEP508 requires-dist and requirements lines. +""" + +load("//python/private:normalize_name.bzl", "normalize_name") + +_STRIP = ["(", " ", ">", "=", "<", "~", "!", "@"] + +def requirement(spec): + """Parse a PEP508 requirement line + + Args: + spec: {type}`str` requirement line that will be parsed. + + Returns: + A struct with the information. + """ + spec = spec.strip() + requires, _, maybe_hashes = spec.partition(";") + + version_start = requires.find("==") + version = None + if version_start != -1: + # Extract everything after '==' until the next space or end of the string + version, _, _ = requires[version_start + 2:].partition(" ") + + # Remove any trailing characters from the version string + version = version.strip(" ") + + marker, _, _ = maybe_hashes.partition("--hash") + requires, _, extras_unparsed = requires.partition("[") + extras_unparsed, _, _ = extras_unparsed.partition("]") + for char in _STRIP: + requires, _, _ = requires.partition(char) + extras = [normalize_name(e) for e in extras_unparsed.replace(" ", "").split(",") if e] + name = requires.strip(" ") + name = normalize_name(name) + + return struct( + name = name.replace("_", "-"), + name_ = name, + marker = marker.strip(" "), + extras = extras, + version = version, + ) diff --git a/python/private/pypi/pip_compile.bzl b/python/private/pypi/pip_compile.bzl index dc5b186a6a..1bec3435a4 100644 --- a/python/private/pypi/pip_compile.bzl +++ b/python/private/pypi/pip_compile.bzl @@ -19,7 +19,17 @@ NOTE @aignas 2024-06-23: We are using the implementation specific name here to make it possible to have multiple tools inside the `pypi` directory """ -load("//python:defs.bzl", _py_binary = "py_binary", _py_test = "py_test") +load("//python:py_binary.bzl", _py_binary = "py_binary") +load("//python:py_test.bzl", _py_test = "py_test") + +# The `data` attribute does not allow duplicate labels, but user-provided `data` +# can overlap with labels added from attributes like `src`. +def _dedupe_data(data): + res = [] + for d in data: + if d not in res: + res.append(d) + return res def pip_compile( name, @@ -37,16 +47,17 @@ def pip_compile( requirements_windows = None, visibility = ["//visibility:private"], tags = None, + constraints = [], + data = [], **kwargs): - """Generates targets for managing pip dependencies with pip-compile. + """Generates targets for managing pip dependencies with pip-compile (piptools). By default this rules generates a filegroup named "[name]" which can be included in the data of some other compile_pip_requirements rule that references these requirements (e.g. with `-r ../other/requirements.txt`). - It also generates two targets for running pip-compile: - - validate with `bazel test [name]_test` + - validate with `bazel test [name].test` - update with `bazel run [name].update` If you are using a version control system, the requirements.txt generated by this rule should @@ -58,13 +69,16 @@ def pip_compile( defaults to `["pyproject.toml"]`. Supported formats are: * a requirements text file, usually named `requirements.in` * A `.toml` file, where the `project.dependencies` list is used as per - [PEP621](https://peps.python.org/pep-0621/). + {pep}`PEP 621`. src: file containing inputs to dependency resolution. If not specified, defaults to `pyproject.toml`. Supported formats are: * a requirements text file, usually named `requirements.in` * A `.toml` file, where the `project.dependencies` list is used as per - [PEP621](https://peps.python.org/pep-0621/). - extra_args: passed to pip-compile. + {pep}`PEP 621`. + extra_args: passed to pip-compile (aka `piptools`). See the + [pip-compile docs](https://pip-tools.readthedocs.io/en/latest/cli/pip-compile) + for args and meaning (passing `-h` and/or `--version` can help + inform what args are available) extra_deps: extra dependencies passed to pip-compile. generate_hashes: whether to put hashes in the requirements_txt file. py_binary: the py_binary rule to be used. @@ -76,6 +90,8 @@ def pip_compile( requirements_windows: File of windows specific resolve output to check validate if requirement.in has changes. tags: tagging attribute common to all build rules, passed to both the _test and .update rules. visibility: passed to both the _test and .update rules. + constraints: a list of files containing constraints to pass to pip-compile with `--constraint`. + data: A list of labels to include as part of the `data` attribute in the generated `py_binary`. **kwargs: other bazel attributes passed to the "_test" rule. """ if len([x for x in [srcs, src, requirements_in] if x != None]) > 1: @@ -90,16 +106,18 @@ def pip_compile( requirements_txt = name + ".txt" if requirements_txt == None else requirements_txt + data = data or [] + # "Default" target produced by this macro # Allow a compile_pip_requirements rule to include another one in the data # for a requirements file that does `-r ../other/requirements.txt` native.filegroup( name = name, - srcs = kwargs.pop("data", []) + [requirements_txt], + srcs = data + [requirements_txt], visibility = visibility, ) - data = [name, requirements_txt] + srcs + [f for f in (requirements_linux, requirements_darwin, requirements_windows) if f != None] + data = _dedupe_data(data + [name, requirements_txt] + srcs + [f for f in (requirements_linux, requirements_darwin, requirements_windows) if f != None] + constraints) # Use the Label constructor so this is expanded in the context of the file # where it appears, which is to say, in @rules_python @@ -109,7 +127,7 @@ def pip_compile( args = ["--src=%s" % loc.format(src) for src in srcs] + [ loc.format(requirements_txt), - "//%s:%s.update" % (native.package_name(), name), + "//%s:%s" % (native.package_name(), name), "--resolver=backtracking", "--allow-unsafe", ] @@ -121,6 +139,8 @@ def pip_compile( args.append("--requirements-darwin={}".format(loc.format(requirements_darwin))) if requirements_windows: args.append("--requirements-windows={}".format(loc.format(requirements_windows))) + for constraint in constraints: + args.append("--constraint=$(location {})".format(constraint)) args.extend(extra_args) deps = [ @@ -155,17 +175,25 @@ def pip_compile( } env = kwargs.pop("env", {}) + env_inherit = kwargs.pop("env_inherit", []) + proxy_variables = ["https_proxy", "http_proxy", "no_proxy", "HTTPS_PROXY", "HTTP_PROXY", "NO_PROXY"] + + for var in proxy_variables: + if var not in env_inherit: + env_inherit.append(var) py_binary( name = name + ".update", env = env, + python_version = kwargs.get("python_version", None), + target_compatible_with = kwargs.get("target_compatible_with", []), **attrs ) timeout = kwargs.pop("timeout", "short") py_test( - name = name + "_test", + name = name + ".test", timeout = timeout, # setuptools (the default python build tool) attempts to find user # configuration in the user's home direcotory. This seems to work fine on @@ -176,6 +204,14 @@ def pip_compile( "@@platforms//os:windows": {"USERPROFILE": "Z:\\FakeSetuptoolsHomeDirectoryHack"}, "//conditions:default": {}, }) | env, + env_inherit = env_inherit, # kwargs could contain test-specific attributes like size **dict(attrs, **kwargs) ) + + native.alias( + name = "{}_test".format(name), + actual = ":{}.test".format(name), + deprecation = "Use '{}.test' instead. The '*_test' target will be removed in the next major release.".format(name), + tags = ["manual"], + ) diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index 47fa31f1bc..436af73c8d 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -16,14 +16,26 @@ load("@bazel_skylib//lib:sets.bzl", "sets") load("//python/private:normalize_name.bzl", "normalize_name") -load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR") +load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "repo_utils") load("//python/private:text_util.bzl", "render") -load(":evaluate_markers.bzl", "evaluate_markers", EVALUATE_MARKERS_SRCS = "SRCS") load(":parse_requirements.bzl", "host_platform", "parse_requirements", "select_requirement") +load(":pep508_env.bzl", "env") load(":pip_repository_attrs.bzl", "ATTRS") +load(":pypi_repo_utils.bzl", "pypi_repo_utils") load(":render_pkg_aliases.bzl", "render_pkg_aliases") load(":requirements_files_by_platform.bzl", "requirements_files_by_platform") +_CONFIG_REPO_TEMPLATE = """"{name}__config" + whl_config_repo( + name = "{name}__config", + repo_prefix = "{name}_", + groups = all_requirement_groups, + whl_map = {{ + p: "" + for p in all_whl_requirements_by_package + }}, + )""" + def _get_python_interpreter_attr(rctx): """A helper function for getting the `python_interpreter` attribute or it's default @@ -54,7 +66,7 @@ def use_isolated(ctx, attr): use_isolated = attr.isolated # The environment variable will take precedence over the attribute - isolated_env = ctx.os.environ.get("RULES_PYTHON_PIP_ISOLATED", None) + isolated_env = ctx.getenv("RULES_PYTHON_PIP_ISOLATED", None) if isolated_env != None: if isolated_env.lower() in ("0", "false"): use_isolated = False @@ -71,6 +83,33 @@ exports_files(["requirements.bzl"]) """ def _pip_repository_impl(rctx): + logger = repo_utils.logger(rctx) + python_interpreter = pypi_repo_utils.resolve_python_interpreter( + rctx, + python_interpreter = rctx.attr.python_interpreter, + python_interpreter_target = rctx.attr.python_interpreter_target, + ) + result = pypi_repo_utils.execute_checked( + rctx, + python = python_interpreter, + srcs = [], + op = "GetPythonVersion", + arguments = ["-c", "import sys; print(sys.version.split()[0])"], + ) + python_version = result.stdout.strip().splitlines()[-1] + platforms = [ + "linux_aarch64", + "linux_arm", + "linux_ppc", + "linux_riscv64", + "linux_s390x", + "linux_x86_64", + "osx_aarch64", + "osx_x86_64", + "windows_x86_64", + ] + + marker_env = env(os = rctx.os.name, arch = rctx.os.arch, python_version = python_version) requirements_by_platform = parse_requirements( rctx, requirements_by_platform = requirements_files_by_platform( @@ -80,28 +119,32 @@ def _pip_repository_impl(rctx): requirements_osx = rctx.attr.requirements_darwin, requirements_windows = rctx.attr.requirements_windows, extra_pip_args = rctx.attr.extra_pip_args, + platforms = platforms, ), extra_pip_args = rctx.attr.extra_pip_args, - evaluate_markers = lambda rctx, requirements: evaluate_markers( - rctx, - requirements = requirements, - python_interpreter = rctx.attr.python_interpreter, - python_interpreter_target = rctx.attr.python_interpreter_target, - srcs = rctx.attr._evaluate_markers_srcs, - ), + platforms = { + p: struct( + env = marker_env, + whl_abi_tags = [], + whl_platform_tags = [], + ) + for p in platforms + }, + extract_url_srcs = False, + logger = logger, ) selected_requirements = {} options = None repository_platform = host_platform(rctx) - for name, requirements in requirements_by_platform.items(): - r = select_requirement( - requirements, + for whl in requirements_by_platform: + requirement = select_requirement( + whl.srcs, platform = None if rctx.attr.download_only else repository_platform, ) - if not r: + if not requirement: continue - options = options or r.extra_pip_args - selected_requirements[name] = r.requirement_line + options = options or requirement.extra_pip_args + selected_requirements[whl.name] = requirement.requirement_line bzl_packages = sorted(selected_requirements.keys()) @@ -138,7 +181,7 @@ def _pip_repository_impl(rctx): imports = [ # NOTE: Maintain the order consistent with `buildifier` 'load("@rules_python//python:pip.bzl", "pip_utils")', - 'load("@rules_python//python/pip_install:pip_repository.bzl", "group_library", "whl_library")', + 'load("@rules_python//python/pip_install:pip_repository.bzl", "whl_config_repo", "whl_library")', ] annotations = {} @@ -167,30 +210,37 @@ def _pip_repository_impl(rctx): if rctx.attr.python_interpreter_target: config["python_interpreter_target"] = str(rctx.attr.python_interpreter_target) - if rctx.attr.experimental_target_platforms: - config["experimental_target_platforms"] = rctx.attr.experimental_target_platforms macro_tmpl = "@%s//{}:{}" % rctx.attr.name aliases = render_pkg_aliases( aliases = { pkg: rctx.attr.name + "_" + pkg - for pkg in bzl_packages or [] + for pkg in bzl_packages }, extra_hub_aliases = rctx.attr.extra_hub_aliases, + requirement_cycles = requirement_cycles, ) for path, contents in aliases.items(): rctx.file(path, contents) rctx.file("BUILD.bazel", _BUILD_FILE_CONTENTS) + if rctx.attr.use_hub_alias_dependencies: + rctx.template( + "config.bzl", + rctx.attr._config_template, + substitutions = { + "%%PACKAGES%%": render.dict({ + pkg: None + for pkg in bzl_packages + }, value_repr = lambda x: "None"), + }, + ) + config_repo_template = repr(rctx.attr.name) + else: + config_repo_template = _CONFIG_REPO_TEMPLATE.format(name = rctx.attr.name) + rctx.template("requirements.bzl", rctx.attr._template, substitutions = { - " # %%GROUP_LIBRARY%%": """\ - group_repo = "{name}__groups" - group_library( - name = group_repo, - repo_prefix = "{name}_", - groups = all_requirement_groups, - )""".format(name = rctx.attr.name) if not rctx.attr.use_hub_alias_dependencies else "", "%%ALL_DATA_REQUIREMENTS%%": render.list([ macro_tmpl.format(p, "data") for p in bzl_packages @@ -206,6 +256,7 @@ def _pip_repository_impl(rctx): }), "%%ANNOTATIONS%%": render.dict(dict(sorted(annotations.items()))), "%%CONFIG%%": render.dict(dict(sorted(config.items()))), + "%%CONFIG_REPO%%": config_repo_template, "%%EXTRA_PIP_ARGS%%": json.encode(options), "%%IMPORTS%%": "\n".join(imports), "%%MACRO_TMPL%%": macro_tmpl, @@ -226,20 +277,15 @@ pip_repository = repository_rule( doc = """\ Optional annotations to apply to packages. Keys should be package names, with capitalization matching the input requirements file, and values should be -generated using the `package_name` macro. For example usage, see [this WORKSPACE -file](https://github.com/bazelbuild/rules_python/blob/main/examples/pip_repository_annotations/WORKSPACE). +generated using the `package_name` macro. """, ), + _config_template = attr.label( + default = ":config.bzl.tmpl", + ), _template = attr.label( default = ":requirements.bzl.tmpl.workspace", ), - _evaluate_markers_srcs = attr.label_list( - default = EVALUATE_MARKERS_SRCS, - doc = """\ -The list of labels to use as SRCS for the marker evaluation code. This ensures that the -code will be re-evaluated when any of files in the default changes. -""", - ), **ATTRS ), doc = """Accepts a locked/compiled requirements file and installs the dependencies listed within. @@ -247,6 +293,9 @@ code will be re-evaluated when any of files in the default changes. Those dependencies become available in a generated `requirements.bzl` file. You can instead check this `requirements.bzl` file into your repo, see the "vendoring" section below. +For advanced use-cases, such as handling multi-platform dependencies, see the +[How-to: Multi-Platform PyPI Dependencies guide](/howto/multi-platform-pypi-deps). + In your WORKSPACE file: ```starlark @@ -303,30 +352,6 @@ functionality for exposing [entry points][whl_ep] as `py_binary` targets as well [whl_ep]: https://packaging.python.org/specifications/entry-points/ -```starlark -load("@pypi//:requirements.bzl", "entry_point") - -alias( - name = "pip-compile", - actual = entry_point( - pkg = "pip-tools", - script = "pip-compile", - ), -) -``` - -Note that for packages whose name and script are the same, only the name of the package -is needed when calling the `entry_point` macro. - -```starlark -load("@pip//:requirements.bzl", "entry_point") - -alias( - name = "flake8", - actual = entry_point("flake8"), -) -``` - :::{rubric} Vendoring the requirements.bzl file :heading-level: 3 ::: @@ -335,13 +360,13 @@ In some cases you may not want to generate the requirements.bzl file as a reposi while Bazel is fetching dependencies. For example, if you produce a reusable Bazel module such as a ruleset, you may want to include the requirements.bzl file rather than make your users install the WORKSPACE setup to generate it. -See https://github.com/bazelbuild/rules_python/issues/608 +See {gh-issue}`608` This is the same workflow as Gazelle, which creates `go_repository` rules with [`update-repos`](https://github.com/bazelbuild/bazel-gazelle#update-repos) To do this, use the "write to source file" pattern documented in -https://blog.aspect.dev/bazel-can-write-to-the-source-folder + to put a copy of the generated requirements.bzl into your project. Then load the requirements.bzl file directly rather than from the generated repository. See the example in rules_python/examples/pip_parse_vendored. diff --git a/python/private/pypi/pip_repository_attrs.bzl b/python/private/pypi/pip_repository_attrs.bzl index 23000869e9..4f0b26448b 100644 --- a/python/private/pypi/pip_repository_attrs.bzl +++ b/python/private/pypi/pip_repository_attrs.bzl @@ -21,6 +21,79 @@ repositories.""" load(":attrs.bzl", COMMON_ATTRS = "ATTRS") ATTRS = { + "experimental_requirement_cycles": attr.string_list_dict( + default = {}, + doc = """\ +A mapping of dependency cycle names to a list of requirements which form that cycle. + +Requirements which form cycles will be installed together and taken as +dependencies together in order to ensure that the cycle is always satisified. + +Example: + `sphinx` depends on `sphinxcontrib-serializinghtml` + When listing both as requirements, ala + + ``` + py_binary( + name = "doctool", + ... + deps = [ + "@pypi//sphinx:pkg", + "@pypi//sphinxcontrib_serializinghtml", + ] + ) + ``` + + Will produce a Bazel error such as + + ``` + ERROR: .../external/pypi_sphinxcontrib_serializinghtml/BUILD.bazel:44:6: in alias rule @pypi_sphinxcontrib_serializinghtml//:pkg: cycle in dependency graph: + //:doctool (...) + @pypi//sphinxcontrib_serializinghtml:pkg (...) + .-> @pypi_sphinxcontrib_serializinghtml//:pkg (...) + | @pypi_sphinxcontrib_serializinghtml//:_pkg (...) + | @pypi_sphinx//:pkg (...) + | @pypi_sphinx//:_pkg (...) + `-- @pypi_sphinxcontrib_serializinghtml//:pkg (...) + ``` + + Which we can resolve by configuring these two requirements to be installed together as a cycle + + ``` + pip_parse( + ... + experimental_requirement_cycles = { + "sphinx": [ + "sphinx", + "sphinxcontrib-serializinghtml", + ] + }, + ) + ``` + +Warning: + If a dependency participates in multiple cycles, all of those cycles must be + collapsed down to one. For instance `a <-> b` and `a <-> c` cannot be listed + as two separate cycles. +""", + ), + "extra_hub_aliases": attr.string_list_dict( + doc = """\ +Extra aliases to make for specific wheels in the hub repo. This is useful when +paired with the {attr}`whl_modifications`. + +:::{versionadded} 0.38.0 + +For `pip.parse` with bzlmod +::: + +:::{versionadded} 1.0.0 + +For `pip_parse` with workspace. +::: +""", + mandatory = False, + ), "requirements_by_platform": attr.label_keyed_string_dict( doc = """\ The requirements files and the comma delimited list of target platforms as values. diff --git a/python/private/pypi/pkg_aliases.bzl b/python/private/pypi/pkg_aliases.bzl index 5a3f84199b..b1c29c95ee 100644 --- a/python/private/pypi/pkg_aliases.bzl +++ b/python/private/pypi/pkg_aliases.bzl @@ -12,74 +12,66 @@ # See the License for the specific language governing permissions and # limitations under the License. -"""pkg_aliases is a macro to generate aliases for selecting the right wheel for the right target platform. +"""{obj}`pkg_aliases` is a macro to generate aliases for selecting the right wheel for the right target platform. -This is used in bzlmod and non-bzlmod setups.""" +If you see an error where the distribution selection error indicates the config setting names this +page may help to describe the naming convention and relationship between various flags and options +in `rules_python` and the error message contents. + +Definitions: +:minor_version: Python interpreter minor version that the distributions are compatible with. +:suffix: Can be either empty or `__`, which is usually used to distinguish multiple versions used for different target platforms. +:os: OS identifier that exists in `@platforms//os:`. +:cpu: CPU architecture identifier that exists in `@platforms//cpu:`. + +All of the config settings used by this macro are generated by +{obj}`config_settings`, for more detailed documentation on what each config +setting maps to and their precedence, refer to documentation on that page. + +`//_config:is_cp3` is used to select any target platforms. +""" load("@bazel_skylib//lib:selects.bzl", "selects") +load("//python/private:common_labels.bzl", "labels") load("//python/private:text_util.bzl", "render") load( ":labels.bzl", "DATA_LABEL", "DIST_INFO_LABEL", + "EXTRACTED_WHEEL_FILES", "PY_LIBRARY_IMPL_LABEL", "PY_LIBRARY_PUBLIC_LABEL", "WHEEL_FILE_IMPL_LABEL", "WHEEL_FILE_PUBLIC_LABEL", ) -load(":parse_whl_name.bzl", "parse_whl_name") -load(":whl_target_platforms.bzl", "whl_target_platforms") - -# This value is used as sentinel value in the alias/config setting machinery -# for libc and osx versions. If we encounter this version in this part of the -# code, then it means that we have a bug in rules_python and that we should fix -# it. It is more of an internal consistency check. -_VERSION_NONE = (0, 0) - -_CONFIG_SETTINGS_PKG = str(Label("//python/config_settings:BUILD.bazel")).partition(":")[0] _NO_MATCH_ERROR_TEMPLATE = """\ -No matching wheel for current configuration's Python version. +No matching wheel for current configuration's Python version and platform. The current build configuration's Python version doesn't match any of the Python wheels available for this distribution. This distribution supports the following Python configuration settings: {config_settings} +As configured by the `pip.parse.target_platforms` attribute. Note that +`requirements_by_platform` only affects the Bazel host platform unless +`target_platforms` is also set. + To determine the current configuration's Python version, run: `bazel config ` (shown further below) -and look for one of: - {settings_pkg}:python_version - {settings_pkg}:pip_whl - {settings_pkg}:pip_whl_glibc_version - {settings_pkg}:pip_whl_muslc_version - {settings_pkg}:pip_whl_osx_arch - {settings_pkg}:pip_whl_osx_version - {settings_pkg}:py_freethreaded - {settings_pkg}:py_linux_libc - -If the value is missing, then the default value is being used, see documentation: -{docs_url}/python/config_settings""" - -def _no_match_error(actual): - if type(actual) != type({}): - return None - - if "//conditions:default" in actual: - return None - - return _NO_MATCH_ERROR_TEMPLATE.format( - config_settings = render.indent( - "\n".join(sorted([ - value - for key in actual - for value in (key if type(key) == "tuple" else [key]) - ])), - ).lstrip(), - settings_pkg = _CONFIG_SETTINGS_PKG, - docs_url = "https://rules-python.readthedocs.io/en/latest/api/rules_python", - ) +For the current configuration value see the debug message above that is +printing the current flag values. If you can't see the message, then re-run the +build to make it a failure instead by running the build with: + --{current_flags}=fail + +However, the command above will hide the `bazel config ` message. +""" + +_LABEL_NONE = labels.NONE +_LABEL_CURRENT_CONFIG = Label("//python/config_settings:current_config") +_LABEL_CURRENT_CONFIG_NO_MATCH = Label("//python/config_settings:is_not_matching_current_config") +_INCOMPATIBLE = "_no_matching_repository" def pkg_aliases( *, @@ -87,11 +79,11 @@ def pkg_aliases( actual, group_name = None, extra_aliases = None, - native = native, - select = selects.with_or, **kwargs): """Create aliases for an actual package. + Exposed only to be used from the hub repositories created by `rules_python`. + Args: name: {type}`str` The name of the package. actual: {type}`dict[Label | tuple, str] | str` The name of the repo the @@ -100,11 +92,12 @@ def pkg_aliases( to bazel skylib's `selects.with_or`, so they can be tuples as well. group_name: {type}`str` The group name that the pkg belongs to. extra_aliases: {type}`list[str]` The extra aliases to be created. - native: {type}`struct` used in unit tests. - select: {type}`select` used in unit tests. - **kwargs: extra kwargs to pass to {bzl:obj}`get_filename_config_settings`. + **kwargs: extra kwargs to pass to {bzl:obj}`get_config_settings`. """ - native.alias( + alias = kwargs.pop("native", native).alias + select = kwargs.pop("select", selects.with_or) + + alias( name = name, actual = ":" + PY_LIBRARY_PUBLIC_LABEL, ) @@ -114,13 +107,33 @@ def pkg_aliases( WHEEL_FILE_PUBLIC_LABEL: WHEEL_FILE_IMPL_LABEL if group_name else WHEEL_FILE_PUBLIC_LABEL, DATA_LABEL: DATA_LABEL, DIST_INFO_LABEL: DIST_INFO_LABEL, + EXTRACTED_WHEEL_FILES: EXTRACTED_WHEEL_FILES, } | { x: x for x in extra_aliases or [] } actual = multiplatform_whl_aliases(aliases = actual, **kwargs) - no_match_error = _no_match_error(actual) + if type(actual) == type({}) and "//conditions:default" not in actual: + alias( + name = _INCOMPATIBLE, + actual = select( + {_LABEL_CURRENT_CONFIG_NO_MATCH: _LABEL_NONE}, + no_match_error = _NO_MATCH_ERROR_TEMPLATE.format( + config_settings = render.indent( + "\n".join(sorted([ + value + for key in actual + for value in (key if type(key) == "tuple" else [key]) + ])), + ).lstrip(), + current_flags = str(_LABEL_CURRENT_CONFIG), + ), + ), + visibility = ["//visibility:private"], + tags = ["manual"], + ) + actual["//conditions:default"] = _INCOMPATIBLE for name, target_name in target_names.items(): if type(actual) == type(""): @@ -134,10 +147,9 @@ def pkg_aliases( v: "@{repo}//:{target_name}".format( repo = repo, target_name = name, - ) + ) if repo != _INCOMPATIBLE else repo for v, repo in actual.items() }, - no_match_error = no_match_error, ) else: fail("The `actual` arg must be a dictionary or a string") @@ -146,49 +158,34 @@ def pkg_aliases( if target_name.startswith("_"): kwargs["visibility"] = ["//_groups:__subpackages__"] - native.alias( + alias( name = target_name, actual = _actual, **kwargs ) if group_name: - native.alias( + alias( name = PY_LIBRARY_PUBLIC_LABEL, actual = "//_groups:{}_pkg".format(group_name), ) - native.alias( + alias( name = WHEEL_FILE_PUBLIC_LABEL, actual = "//_groups:{}_whl".format(group_name), ) -def _normalize_versions(name, versions): - if not versions: - return [] - - if _VERSION_NONE in versions: - fail("a sentinel version found in '{}', check render_pkg_aliases for bugs".format(name)) - - return sorted(versions) - def multiplatform_whl_aliases( *, - aliases = [], - glibc_versions = [], - muslc_versions = [], - osx_versions = []): + aliases = []): """convert a list of aliases from filename to config_setting ones. + Exposed only for unit tests. + Args: - aliases: {type}`str | dict[whl_config_setting | str, str]`: The aliases + aliases: {type}`str | dict[struct | str, str]`: The aliases to process. Any aliases that have the filename set will be - converted to a dict of config settings to repo names. - glibc_versions: {type}`list[tuple[int, int]]` list of versions that can be - used in this hub repo. - muslc_versions: {type}`list[tuple[int, int]]` list of versions that can be - used in this hub repo. - osx_versions: {type}`list[tuple[int, int]]` list of versions that can be - used in this hub repo. + converted to a dict of config settings to repo names. The + struct is created by {bzl:obj}`whl_config_setting`. Returns: A dict with of config setting labels to repo names or the repo name itself. @@ -198,197 +195,55 @@ def multiplatform_whl_aliases( # We don't have any aliases, this is a repo name return aliases - # TODO @aignas 2024-11-17: we might be able to use FeatureFlagInfo and some - # code gen to create a version_lt_x target, which would allow us to check - # if the libc version is in a particular range. - glibc_versions = _normalize_versions("glibc_versions", glibc_versions) - muslc_versions = _normalize_versions("muslc_versions", muslc_versions) - osx_versions = _normalize_versions("osx_versions", osx_versions) - ret = {} - versioned_additions = {} for alias, repo in aliases.items(): if type(alias) != "struct": ret[alias] = repo continue - elif not (alias.filename or alias.target_platforms): - # This is an internal consistency check - fail("Expected to have either 'filename' or 'target_platforms' set, got: {}".format(alias)) - config_settings, all_versioned_settings = get_filename_config_settings( - filename = alias.filename or "", + # This is if we are using `whl_config_setting` struct + config_settings = get_config_settings( target_platforms = alias.target_platforms, python_version = alias.version, - # If we have multiple platforms but no wheel filename, lets use different - # config settings. - non_whl_prefix = "sdist" if alias.filename else "", - glibc_versions = glibc_versions, - muslc_versions = muslc_versions, - osx_versions = osx_versions, ) for setting in config_settings: ret["//_config" + setting] = repo - # Now for the versioned platform config settings, we need to select one - # that best fits the bill and if there are multiple wheels, e.g. - # manylinux_2_17_x86_64 and manylinux_2_28_x86_64, then we need to select - # the former when the glibc is in the range of [2.17, 2.28) and then chose - # the later if it is [2.28, ...). If the 2.28 wheel was not present in - # the hub, then we would need to use 2.17 for all the glibc version - # configurations. - # - # Here we add the version settings to a dict where we key the range of - # versions that the whl spans. If the wheel supports musl and glibc at - # the same time, we do this for each supported platform, hence the - # double dict. - for default_setting, versioned in all_versioned_settings.items(): - versions = sorted(versioned) - min_version = versions[0] - max_version = versions[-1] - - versioned_additions.setdefault(default_setting, {})[(min_version, max_version)] = struct( - repo = repo, - settings = versioned, - ) - - versioned = {} - for default_setting, candidates in versioned_additions.items(): - # Sort the candidates by the range of versions the span, so that we - # start with the lowest version. - for _, candidate in sorted(candidates.items()): - # Set the default with the first candidate, which gives us the highest - # compatibility. If the users want to use a higher-version than the default - # they can configure the glibc_version flag. - versioned.setdefault("//_config" + default_setting, candidate.repo) - - # We will be overwriting previously added entries, but that is intended. - for _, setting in candidate.settings.items(): - versioned["//_config" + setting] = candidate.repo - - ret.update(versioned) return ret -def get_filename_config_settings( +def get_config_settings( *, - filename, target_platforms, - python_version, - glibc_versions = None, - muslc_versions = None, - osx_versions = None, - non_whl_prefix = "sdist"): + python_version): """Get the filename config settings. + Exposed only for unit tests. + Args: - filename: the distribution filename (can be a whl or an sdist). target_platforms: list[str], target platforms in "{abi}_{os}_{cpu}" format. - glibc_versions: list[tuple[int, int]], list of versions. - muslc_versions: list[tuple[int, int]], list of versions. - osx_versions: list[tuple[int, int]], list of versions. python_version: the python version to generate the config_settings for. - non_whl_prefix: the prefix of the config setting when the whl we don't have - a filename ending with ".whl". Returns: A tuple: * A list of config settings that are generated by ./pip_config_settings.bzl * The list of default version settings. """ - prefixes = [] - suffixes = [] - setting_supported_versions = {} - - if filename.endswith(".whl"): - parsed = parse_whl_name(filename) - if parsed.python_tag == "py2.py3": - py = "py" - elif parsed.python_tag.startswith("cp"): - py = "cp3x" - else: - py = "py3" - if parsed.abi_tag.startswith("cp"): - abi = "cp" - else: - abi = parsed.abi_tag - - if parsed.platform_tag == "any": - prefixes = ["_{}_{}_any".format(py, abi)] - else: - prefixes = ["_{}_{}".format(py, abi)] - suffixes = _whl_config_setting_suffixes( - platform_tag = parsed.platform_tag, - glibc_versions = glibc_versions, - muslc_versions = muslc_versions, - osx_versions = osx_versions, - setting_supported_versions = setting_supported_versions, - ) - else: - prefixes = [""] if not non_whl_prefix else ["_" + non_whl_prefix] - - versioned = { - ":is_cp{}{}_{}".format(python_version, p, suffix): { - version: ":is_cp{}{}_{}".format(python_version, p, setting) - for version, setting in versions.items() - } - for p in prefixes - for suffix, versions in setting_supported_versions.items() - } + prefixes = [ + "cp{}".format(python_version).replace(".", ""), + ] - if suffixes or target_platforms or versioned: + if target_platforms: target_platforms = target_platforms or [] - suffixes = suffixes or [_non_versioned_platform(p) for p in target_platforms] + suffixes = [_non_versioned_platform(p) for p in target_platforms] return [ - ":is_cp{}{}_{}".format(python_version, p, s) - for p in prefixes - for s in suffixes - ], versioned + ":is_{}_{}".format(prefix, suffix) + for prefix in prefixes + for suffix in suffixes + ] else: - return [":is_cp{}{}".format(python_version, p) for p in prefixes], setting_supported_versions - -def _whl_config_setting_suffixes( - platform_tag, - glibc_versions, - muslc_versions, - osx_versions, - setting_supported_versions): - suffixes = [] - for platform_tag in platform_tag.split("."): - for p in whl_target_platforms(platform_tag): - prefix = p.os - suffix = p.cpu - if "manylinux" in platform_tag: - prefix = "manylinux" - versions = glibc_versions - elif "musllinux" in platform_tag: - prefix = "musllinux" - versions = muslc_versions - elif p.os in ["linux", "windows"]: - versions = [(0, 0)] - elif p.os == "osx": - versions = osx_versions - if "universal2" in platform_tag: - suffix += "_universal2" - else: - fail("Unsupported whl os: {}".format(p.os)) - - default_version_setting = "{}_{}".format(prefix, suffix) - supported_versions = {} - for v in versions: - if v == (0, 0): - suffixes.append(default_version_setting) - elif v >= p.version: - supported_versions[v] = "{}_{}_{}_{}".format( - prefix, - v[0], - v[1], - suffix, - ) - if supported_versions: - setting_supported_versions[default_version_setting] = supported_versions - - return suffixes + return [":is_{}".format(p) for p in prefixes] def _non_versioned_platform(p, *, strict = False): """A small utility function that converts 'cp311_linux_x86_64' to 'linux_x86_64'. diff --git a/python/private/pypi/platform.bzl b/python/private/pypi/platform.bzl new file mode 100644 index 0000000000..e8f36a980b --- /dev/null +++ b/python/private/pypi/platform.bzl @@ -0,0 +1,45 @@ +"""A common platform structure for using internally.""" + +def platform(*, name, arch_name, os_name, config_settings = [], env = {}, marker = "", whl_abi_tags = [], whl_platform_tags = []): + """A platform structure for using internally. + + Args: + name: {type}`str` the human friendly name of the platform. + arch_name: {type}`str` the @platforms//cpu: value. + os_name: {type}`str` the @platforms//os: value. + config_settings: {type}`list[Label|str]` The list of labels for selecting the + platform. + env: {type}`dict[str, str]` the PEP508 environment for marker evaluation. + marker: {type}`str` the env marker expression that is evaluated to determine if we + should use the platform. This is useful to turn on certain platforms for + particular python versions. + whl_abi_tags: {type}`list[str]` A list of values for matching abi tags. + whl_platform_tags: {type}`list[str]` A list of values for matching platform tags. + + Returns: + struct with the necessary values for pipstar implementation. + """ + + # NOTE @aignas 2025-07-08: the least preferred is the first item in the list + if "any" not in whl_platform_tags: + # the lowest priority one needs to be the first one + whl_platform_tags = ["any"] + whl_platform_tags + + whl_abi_tags = whl_abi_tags or ["abi3", "cp{major}{minor}"] + if "none" not in whl_abi_tags: + # the lowest priority one needs to be the first one + whl_abi_tags = ["none"] + whl_abi_tags + + return struct( + name = name, + arch_name = arch_name, + os_name = os_name, + config_settings = config_settings, + env = { + # defaults for env + "implementation_name": "cpython", + } | env, + marker = marker, + whl_abi_tags = whl_abi_tags, + whl_platform_tags = whl_platform_tags, + ) diff --git a/python/private/pypi/pypi_cache.bzl b/python/private/pypi/pypi_cache.bzl new file mode 100644 index 0000000000..4f6fa9ac37 --- /dev/null +++ b/python/private/pypi/pypi_cache.bzl @@ -0,0 +1,351 @@ +"""A cache for the PyPI index contents evaluation. + +This is design to work as the following: +- in-memory cache for results of PyPI index queries, so that we are not calling PyPI multiple times + for the same package for different hub repos. + +In the future the same will be used to: +- Store PyPI index query results as facts in the MODULE.bazel.lock file +""" + +load(":version_from_filename.bzl", "version_from_filename") + +# This value should be changed whenever the storage format changes. +# Changing it simply means the information cached in the lockfile has to be +# recomputed. +# +# v2: the `dist_hashes` values and the `dist_yanked` keys are canonical +# `:` strings instead of bare sha256 hex digests. +_FACT_VERSION = "v2" + +def pypi_cache(mctx = None, store = None): + """The cache for PyPI index queries. + + Currently the key is of the following structure: + (url, real_url, versions) + + Args: + mctx: The module context + store: The in-memory store, should implement dict interface for get and setdefault + + Returns: + A cache struct + """ + mcache = memory_cache(store) + fcache = facts_cache(getattr(mctx, "facts", None)) + + # buildifier: disable=uninitialized + self = struct( + _mcache = mcache, + _facts = fcache, + setdefault = lambda key, parsed_result: _pypi_cache_setdefault(self, key, parsed_result), + get = lambda key: _pypi_cache_get(self, key), + get_facts = lambda: _pypi_cache_get_facts(self), + ) + + # buildifier: enable=uninitialized + return self + +def _pypi_cache_setdefault(self, key, parsed_result): + """Store the value if not yet cached. + + Args: + self: {type}`struct` The self of this implementation. + key: {type}`str` The cache key, can be any string. + parsed_result: {type}`struct` The result of `parse_simpleapi_html` function. + + index_url and distribution is used to write to the MODULE.bazel.lock file as facts + real_index_url and distribution is used to write to in-memory cache to ensure that there are + no duplicate calls to the PyPI indexes + + Returns: + The `parse_result`. + """ + index_url, real_url, versions = key + self._mcache.setdefault(real_url, parsed_result) + if not versions or not self._facts: + return parsed_result + + # Filter the packages to only what is needed before writing to the facts cache + filtered = _filter_packages(parsed_result, versions) + return self._facts.setdefault(index_url, filtered) + +def _pypi_cache_get(self, key): + """Return the parsed result from the cache. + + Args: + self: {type}`struct` The self of this implementation. + key: {type}`str` The cache key, can be any string. + + Returns: + The {type}`struct` or `None` based on if the result is in the cache or not. + """ + index_url, real_url, versions = key + + # When retrieving from memory cache, filter down to only what is needed. If the + # cache is empty, we will attempt to read from facts, however, reading from memory + # first allows us to not parse the contents of the lock file that may add up. + cached = _filter_packages(self._mcache.get(real_url), versions) + if not self._facts: + return cached + + if not cached and versions: + # Could not get from in-memory, read from lockfile facts + cached = self._facts.get(index_url, versions) + else: + # We might be using something from memory that is not yet stored in facts (e.g. we processed + # the requirements.txt for one Python version and the deps got cached, but new python + # version means different deps, which may add extras. + self._facts.setdefault(index_url, cached) + + return cached + +def _pypi_cache_get_facts(self): + if not self._facts: + return {} + + return self._facts.facts + +def memory_cache(cache = None): + """SimpleAPI cache for making fewer calls. + + We are using the `real_url` as the key in the cache functions on purpose in order to get the + best possible cache hits. + + Args: + cache: the storage to store things in memory. + + Returns: + struct with 2 methods, `get` and `setdefault`. + """ + if cache == None: + cache = {} + + return struct( + get = lambda real_url: cache.get(real_url), + setdefault = lambda real_url, value: cache.setdefault(real_url, value), + ) + +def _filter_packages(dists, requested_versions): + if dists == None or not requested_versions: + return dists + + if type(dists) == "dict": + result = { + pkg: url + for pkg, url in dists.items() + if pkg in requested_versions + } + return result if result else None + + hashes_by_version = {} + whls = {} + sdists = {} + + for digest, d in dists.sdists.items(): + if d.version not in requested_versions: + continue + + sdists[digest] = d + hashes_by_version.setdefault(d.version, []).append(digest) + + for digest, d in dists.whls.items(): + if d.version not in requested_versions: + continue + + whls[digest] = d + hashes_by_version.setdefault(d.version, []).append(digest) + + if not whls and not sdists: + # TODO @aignas 2026-03-08: add logging + #print("WARN: no dists matched for versions {}".format(requested_versions)) + return None + + return struct( + whls = whls, + sdists = sdists, + hashes_by_version = { + k: sorted(v) + for k, v in hashes_by_version.items() + }, + ) + +def facts_cache(known_facts, facts_version = _FACT_VERSION): + """The facts cache. + + Here we have a way to store things as facts and the main thing to keep in mind is that we should + not use the real_url in case it contains credentials in it (e.g. is of form `https://:@`). + + Args: + known_facts: An opaque object coming from {obj}`module_ctx.facts`. + facts_version: {type}`str` the version of the facts schema, used for short-circuiting. + + Returns: + A struct that has: + * `get` method for getting values from the facts cache. + * `setdefault` method for setting values in the cache. + * `facts` attribute that should be passed to the {obj}`module_ctx.extension_metadata` to persist facts. + """ + if known_facts == None: + return None + + facts = {} + + return struct( + get = lambda index_url, versions: _get_from_facts( + facts, + known_facts, + index_url, + versions, + facts_version, + ), + setdefault = lambda url, value: _store_facts(facts, facts_version, url, value), + known_facts = known_facts, + facts = facts, + ) + +def _get_from_facts(facts, known_facts, index_url, requested_versions, facts_version): + if known_facts.get("fact_version") != facts_version: + # cannot trust known facts, different version that we know how to parse + return None + + if type(requested_versions) == "dict": + result = _filter_packages( + dists = known_facts.get("index_urls", {}).get(index_url, {}), + requested_versions = requested_versions, + ) + if result: + if len(result) != len(requested_versions): + # If the results are incomplete, return None, so that we can + # fetch sources from the internet again. + return None + + # Only persist the accessed (requested) packages. Packages that + # exist in known_facts but are not in requested_versions (e.g. + # removed from all requirements files) are dropped from the + # computed facts so they get cleaned up from the lockfile. + _store_facts(facts, facts_version, index_url, result) + return result + + known_sources = {} + + root_url, _, distribution = index_url.rstrip("/").rpartition("/") + distribution = distribution.rstrip("/") + root_url = root_url.rstrip("/") + + retrieved_versions = {} + + for url, digest in known_facts.get("dist_hashes", {}).get(root_url, {}).get(distribution, {}).items(): + filename = known_facts.get("dist_filenames", {}).get(root_url, {}).get(distribution, {}).get(url) + if not filename: + _, _, filename = url.rpartition("/") + + version = version_from_filename(filename) + if version not in requested_versions: + # TODO @aignas 2026-01-21: do the check by requested digests at some point + # We don't have sufficient info in the lock file, need to call the API + # + continue + + retrieved_versions[version] = True + + if filename.endswith(".whl"): + dists = known_sources.setdefault("whls", {}) + else: + dists = known_sources.setdefault("sdists", {}) + + known_sources.setdefault("hashes_by_version", {}).setdefault(version, []).append(digest) + + dists.setdefault(digest, struct( + digest = digest, + filename = filename, + version = version, + metadata_url = "", + metadata_sha256 = "", + url = url, + yanked = known_facts.get("dist_yanked", {}).get(root_url, {}).get(distribution, {}).get(digest), + )) + + if not known_sources: + # We found nothing in facts + return None + + if len(requested_versions) != len(retrieved_versions): + # If the results are incomplete, then return None, so that we can fetch sources from the + # internet again. + return None + + output = struct( + whls = known_sources.get("whls", {}), + sdists = known_sources.get("sdists", {}), + hashes_by_version = { + k: sorted(v) + for k, v in known_sources.get("hashes_by_version", {}).items() + }, + ) + + # Persist these facts for the next run because we have used them. + return _store_facts(facts, facts_version, index_url, output) + +def _store_facts(facts, fact_version, index_url, value): + """Store values as facts in the lock file. + + The main idea is to ensure that the lock file is small and it is only + storing what we would need to fetch from the internet. Any derivative + information we can get from this that can be achieved using pure Starlark + functions should be done in Starlark. + """ + if not value: + return value + + facts["fact_version"] = fact_version + + if type(value) == "dict": + # facts: { + # "index_urls": { + # "": { + # "": "", + # }, + # }, + # }, + for pkg, url in value.items(): + facts.setdefault("index_urls", {}).setdefault(index_url, {})[pkg] = url + return value + + root_url, _, distribution = index_url.rstrip("/").rpartition("/") + distribution = distribution.rstrip("/") + root_url = root_url.rstrip("/") + + # The schema is + # facts: { + # "dist_hashes": { + # "": { + # "": { + # # An empty string if the index does not advertise any digest. + # "": ":", + # }, + # }, + # }, + # "dist_filenames": { + # "": { + # "": { + # "": "", # if it is different from the URL + # }, + # }, + # }, + # "dist_yanked": { + # "": { + # "": { + # ":": "", # if the package is yanked + # }, + # }, + # }, + # }, + for digest, d in (value.sdists | value.whls).items(): + facts.setdefault("dist_hashes", {}).setdefault(root_url, {}).setdefault(distribution, {}).setdefault(d.url, digest) + if not d.url.endswith(d.filename): + facts.setdefault("dist_filenames", {}).setdefault(root_url, {}).setdefault(distribution, {}).setdefault(d.url, d.filename) + if d.yanked != None: + facts.setdefault("dist_yanked", {}).setdefault(root_url, {}).setdefault(distribution, {}).setdefault(digest, d.yanked) + + return value diff --git a/python/private/pypi/pypi_repo_utils.bzl b/python/private/pypi/pypi_repo_utils.bzl index 196431636f..8ec7bd1dbe 100644 --- a/python/private/pypi/pypi_repo_utils.bzl +++ b/python/private/pypi/pypi_repo_utils.bzl @@ -16,6 +16,7 @@ load("@bazel_skylib//lib:types.bzl", "types") load("//python/private:repo_utils.bzl", "repo_utils") +load("//python/private:util.bzl", "is_importable_name") def _get_python_interpreter_attr(mrctx, *, python_interpreter = None): """A helper function for getting the `python_interpreter` attribute or it's default @@ -104,11 +105,29 @@ def _construct_pypath(mrctx, *, entries): ]) return pypath -def _execute_checked(mrctx, *, srcs, **kwargs): +def _execute_prep(mrctx, *, python, srcs, **kwargs): + for src in srcs: + # This will ensure that we will re-evaluate the bzlmod extension or + # refetch the repository_rule when the srcs change. + mrctx.watch(mrctx.path(src)) + + environment = kwargs.pop("environment", {}) + pythonpath = environment.get("PYTHONPATH", "") + if pythonpath and not types.is_string(pythonpath): + environment["PYTHONPATH"] = _construct_pypath(mrctx, entries = pythonpath) + kwargs["environment"] = environment + + # -B is added to prevent the repo-phase invocation from creating timestamp + # based pyc files, which contributes to race conditions and non-determinism + kwargs["arguments"] = [python, "-B"] + kwargs.get("arguments", []) + return kwargs + +def _execute_checked(mrctx, *, python, srcs, **kwargs): """Helper function to run a python script and modify the PYTHONPATH to include external deps. Args: mrctx: Handle to the module_ctx or repository_ctx. + python: The python interpreter to use. srcs: The src files that the script depends on. This is important to ensure that the Bazel repository cache or the bzlmod lock file gets invalidated when any one file changes. It is advisable to use @@ -118,26 +137,70 @@ def _execute_checked(mrctx, *, srcs, **kwargs): the `environment` has a value `PYTHONPATH` and it is a list, then it will be passed to `construct_pythonpath` function. """ + return repo_utils.execute_checked( + mrctx, + **_execute_prep(mrctx, python = python, srcs = srcs, **kwargs) + ) - for src in srcs: - # This will ensure that we will re-evaluate the bzlmod extension or - # refetch the repository_rule when the srcs change. This should work on - # Bazel versions without `mrctx.watch` as well. - repo_utils.watch(mrctx, mrctx.path(src)) - - env = kwargs.pop("environment", {}) - pythonpath = env.get("PYTHONPATH", "") - if pythonpath and not types.is_string(pythonpath): - env["PYTHONPATH"] = _construct_pypath(mrctx, entries = pythonpath) +def _execute_checked_stdout(mrctx, *, python, srcs, **kwargs): + """Helper function to run a python script and modify the PYTHONPATH to include external deps. - return repo_utils.execute_checked( + Args: + mrctx: Handle to the module_ctx or repository_ctx. + python: The python interpreter to use. + srcs: The src files that the script depends on. This is important to + ensure that the Bazel repository cache or the bzlmod lock file gets + invalidated when any one file changes. It is advisable to use + `RECORD` files for external deps and the list of srcs from the + rules_python repo for any scripts. + **kwargs: Arguments forwarded to `repo_utils.execute_checked`. If + the `environment` has a value `PYTHONPATH` and it is a list, then + it will be passed to `construct_pythonpath` function. + """ + return repo_utils.execute_checked_stdout( mrctx, - environment = env, - **kwargs + **_execute_prep(mrctx, python = python, srcs = srcs, **kwargs) ) +def _find_namespace_package_files(rctx, install_dir): + """Finds all `__init__.py` files that belong to namespace packages. + + A `__init__.py` file belongs to a namespace package if it contains `__path__ =`, + `pkgutil`, and `extend_path(`. + + Args: + rctx (repository_ctx): The repository context. + install_dir (path): The path to the install directory. + + Returns: + list[str]: A list of relative paths to `__init__.py` files that belong + to namespace packages. + """ + + namespace_package_files = [] + for top_level_dir in install_dir.readdir(): + if not is_importable_name(top_level_dir.basename): + continue + init_py = top_level_dir.get_child("__init__.py") + if not init_py.exists: + continue + content = rctx.read(init_py) + + # Look for code resembling the pkgutil namespace setup code: + # __path__ = __import__("pkgutil").extend_path(__path__, __name__) + if ("__path__ =" in content and + "pkgutil" in content and + "extend_path(" in content): + namespace_package_files.append( + repo_utils.repo_root_relative_path(rctx, init_py), + ) + + return namespace_package_files + pypi_repo_utils = struct( construct_pythonpath = _construct_pypath, execute_checked = _execute_checked, + execute_checked_stdout = _execute_checked_stdout, + find_namespace_package_files = _find_namespace_package_files, resolve_python_interpreter = _resolve_python_interpreter, ) diff --git a/python/private/pypi/python_tag.bzl b/python/private/pypi/python_tag.bzl new file mode 100644 index 0000000000..224c5f96f0 --- /dev/null +++ b/python/private/pypi/python_tag.bzl @@ -0,0 +1,41 @@ +"A simple utility function to get the python_tag from the implementation name" + +load("//python/private:version.bzl", "version") + +# Taken from +# https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#python-tag +_PY_TAGS = { + # "py": Generic Python (does not require implementation-specific features) + "cpython": "cp", + "ironpython": "ip", + "jython": "jy", + "pypy": "pp", + "python": "py", +} +PY_TAG_GENERIC = "py" + +def python_tag(implementation_name, python_version = ""): + """Get the python_tag from the implementation_name. + + Args: + implementation_name: {type}`str` the implementation name, e.g. "cpython" + python_version: {type}`str` a version who can be parsed using PEP440 compliant + parser. + + Returns: + A {type}`str` that represents the python_tag with a version if the + python_version is given. + """ + if python_version: + v = version.parse(python_version, strict = True) + suffix = "{}{}".format( + v.release[0], + v.release[1] if len(v.release) > 1 else "", + ) + else: + suffix = "" + + return "{}{}".format( + _PY_TAGS.get(implementation_name, implementation_name), + suffix, + ) diff --git a/python/private/pypi/render_pkg_aliases.bzl b/python/private/pypi/render_pkg_aliases.bzl index 62c893d3e7..0a1c328491 100644 --- a/python/private/pypi/render_pkg_aliases.bzl +++ b/python/private/pypi/render_pkg_aliases.bzl @@ -22,8 +22,6 @@ load( ":generate_group_library_build_bazel.bzl", "generate_group_library_build_bazel", ) # buildifier: disable=bzl-visibility -load(":parse_whl_name.bzl", "parse_whl_name") -load(":whl_target_platforms.bzl", "whl_target_platforms") NO_MATCH_ERROR_MESSAGE_TEMPLATE = """\ No matching wheel for current configuration's Python version. @@ -48,19 +46,17 @@ def _repr_dict(*, value_repr = repr, **kwargs): return {k: value_repr(v) for k, v in kwargs.items() if v} def _repr_config_setting(alias): - if alias.filename or alias.target_platforms: + if alias.target_platforms: return render.call( "whl_config_setting", **_repr_dict( - filename = alias.filename, target_platforms = alias.target_platforms, - config_setting = alias.config_setting, version = alias.version, ) ) else: return repr( - alias.config_setting or ("//_config:is_python_" + alias.version), + "//_config:is_cp{}".format(alias.version.replace(".", "")), ) def _repr_actual(aliases): @@ -143,12 +139,26 @@ def render_pkg_aliases(*, aliases, requirement_cycles = None, extra_hub_aliases files["_groups/BUILD.bazel"] = generate_group_library_build_bazel("", requirement_cycles) return files -def render_multiplatform_pkg_aliases(*, aliases, **kwargs): +def _major_minor(python_version): + major, _, tail = python_version.partition(".") + minor, _, _ = tail.partition(".") + return "{}.{}".format(major, minor) + +def _major_minor_versions(python_versions): + if not python_versions: + return [] + + # Use a dict as a simple set + return sorted({_major_minor(v): None for v in python_versions}) + +def render_multiplatform_pkg_aliases(*, aliases, platform_config_settings = {}, **kwargs): """Render the multi-platform pkg aliases. Args: aliases: dict[str, list(whl_config_setting)] A list of aliases that will be transformed from ones having `filename` to ones having `config_setting`. + platform_config_settings: {type}`dict[str, list[str]]` contains all of the + target platforms and their appropriate `target_settings`. **kwargs: extra arguments passed to render_pkg_aliases. Returns: @@ -165,28 +175,26 @@ def render_multiplatform_pkg_aliases(*, aliases, **kwargs): contents = render_pkg_aliases( aliases = aliases, - glibc_versions = flag_versions.get("glibc_versions", []), - muslc_versions = flag_versions.get("muslc_versions", []), - osx_versions = flag_versions.get("osx_versions", []), **kwargs ) contents["_config/BUILD.bazel"] = _render_config_settings( - glibc_versions = flag_versions.get("glibc_versions", []), - muslc_versions = flag_versions.get("muslc_versions", []), - osx_versions = flag_versions.get("osx_versions", []), - python_versions = flag_versions.get("python_versions", []), - target_platforms = flag_versions.get("target_platforms", []), + python_versions = _major_minor_versions(flag_versions.get("python_versions", [])), + platform_config_settings = platform_config_settings, visibility = ["//:__subpackages__"], ) return contents -def _render_config_settings(**kwargs): +def _render_config_settings(platform_config_settings, **kwargs): return """\ load("@rules_python//python/private/pypi:config_settings.bzl", "config_settings") {}""".format(render.call( "config_settings", name = repr("config_settings"), + platform_config_settings = render.dict( + platform_config_settings, + value_repr = render.list, + ), **_repr_dict(value_repr = render.list, **kwargs) )) @@ -201,54 +209,21 @@ def get_whl_flag_versions(settings): * python_versions """ python_versions = {} - glibc_versions = {} target_platforms = {} - muslc_versions = {} - osx_versions = {} for setting in settings: - if not setting.version and not setting.filename: + if not setting.version: continue if setting.version: python_versions[setting.version] = None - if setting.filename and setting.filename.endswith(".whl") and not setting.filename.endswith("-any.whl"): - parsed = parse_whl_name(setting.filename) - else: - for plat in setting.target_platforms or []: - target_platforms[_non_versioned_platform(plat)] = None - continue - - for platform_tag in parsed.platform_tag.split("."): - parsed = whl_target_platforms(platform_tag) - - for p in parsed: - target_platforms[p.target_platform] = None - - if platform_tag.startswith("win") or platform_tag.startswith("linux"): - continue - - head, _, tail = platform_tag.partition("_") - major, _, tail = tail.partition("_") - minor, _, tail = tail.partition("_") - if tail: - version = (int(major), int(minor)) - if "many" in head: - glibc_versions[version] = None - elif "musl" in head: - muslc_versions[version] = None - elif "mac" in head: - osx_versions[version] = None - else: - fail(platform_tag) + for plat in setting.target_platforms or []: + target_platforms[_non_versioned_platform(plat)] = None return { k: sorted(v) for k, v in { - "glibc_versions": glibc_versions, - "muslc_versions": muslc_versions, - "osx_versions": osx_versions, "python_versions": python_versions, "target_platforms": target_platforms, }.items() diff --git a/python/private/pypi/repack_whl.ps1 b/python/private/pypi/repack_whl.ps1 new file mode 100644 index 0000000000..2f3b4ada0f --- /dev/null +++ b/python/private/pypi/repack_whl.ps1 @@ -0,0 +1,7 @@ +param( + [string]$Output +) + +$files = Get-ChildItem -Path . -Exclude 'tmp.zip', $Output +Compress-Archive -Path $files -DestinationPath 'tmp.zip' -Force +Move-Item -Path 'tmp.zip' -Destination $Output -Force diff --git a/python/private/pypi/repack_whl.py b/python/private/pypi/repack_whl.py deleted file mode 100644 index 519631f272..0000000000 --- a/python/private/pypi/repack_whl.py +++ /dev/null @@ -1,186 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -""" -Regenerate a whl file after patching and cleanup the patched contents. - -This script will take contents of the current directory and create a new wheel -out of it and will remove all files that were written to the wheel. -""" - -from __future__ import annotations - -import argparse -import csv -import difflib -import logging -import pathlib -import sys -import tempfile - -from tools.wheelmaker import _WhlFile - -# NOTE: Implement the following matching of what goes into the RECORD -# https://peps.python.org/pep-0491/#the-dist-info-directory -_EXCLUDES = [ - "RECORD", - "INSTALLER", - "RECORD.jws", - "RECORD.p7s", - "REQUESTED", -] - -_DISTINFO = "dist-info" - - -def _unidiff_output(expected, actual, record): - """ - Helper function. Returns a string containing the unified diff of two - multiline strings. - """ - - expected = expected.splitlines(1) - actual = actual.splitlines(1) - - diff = difflib.unified_diff( - expected, actual, fromfile=f"a/{record}", tofile=f"b/{record}" - ) - - return "".join(diff) - - -def _files_to_pack(dir: pathlib.Path, want_record: str) -> list[pathlib.Path]: - """Check that the RECORD file entries are correct and print a unified diff on failure.""" - - # First get existing files by using the RECORD file - got_files = [] - got_distinfos = [] - for row in csv.reader(want_record.splitlines()): - rec = row[0] - path = dir / rec - - if not path.exists(): - # skip files that do not exist as they won't be present in the final - # RECORD file. - continue - - if not path.parent.name.endswith(_DISTINFO): - got_files.append(path) - elif path.name not in _EXCLUDES: - got_distinfos.append(path) - - # Then get extra files present in the directory but not in the RECORD file - extra_files = [] - extra_distinfos = [] - for path in dir.rglob("*"): - if path.is_dir(): - continue - - elif path.parent.name.endswith(_DISTINFO): - if path.name in _EXCLUDES: - # NOTE: we implement the following matching of what goes into the RECORD - # https://peps.python.org/pep-0491/#the-dist-info-directory - continue - elif path not in got_distinfos: - extra_distinfos.append(path) - - elif path not in got_files: - extra_files.append(path) - - # sort the extra files for reproducibility - extra_files.sort() - extra_distinfos.sort() - - # This order ensures that the structure of the RECORD file is always the - # same and ensures smaller patchsets to the RECORD file in general - return got_files + extra_files + got_distinfos + extra_distinfos - - -def main(sys_argv): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument( - "whl_path", - type=pathlib.Path, - help="The original wheel file that we have patched.", - ) - parser.add_argument( - "--record-patch", - type=pathlib.Path, - help="The output path that we are going to write the RECORD file patch to.", - ) - parser.add_argument( - "output", - type=pathlib.Path, - help="The output path that we are going to write a new file to.", - ) - args = parser.parse_args(sys_argv) - - cwd = pathlib.Path.cwd() - logging.debug("=" * 80) - logging.debug("Repackaging the wheel") - logging.debug("=" * 80) - - with tempfile.TemporaryDirectory(dir=cwd) as tmpdir: - patched_wheel_dir = cwd / tmpdir - logging.debug(f"Created a tmpdir: {patched_wheel_dir}") - - excludes = [args.whl_path, patched_wheel_dir] - - logging.debug("Moving whl contents to the newly created tmpdir") - for p in cwd.glob("*"): - if p in excludes: - logging.debug(f"Ignoring: {p}") - continue - - rel_path = p.relative_to(cwd) - dst = p.rename(patched_wheel_dir / rel_path) - logging.debug(f"mv {p} -> {dst}") - - distinfo_dir = next(iter(patched_wheel_dir.glob("*dist-info"))) - logging.debug(f"Found dist-info dir: {distinfo_dir}") - record_path = distinfo_dir / "RECORD" - record_contents = record_path.read_text() if record_path.exists() else "" - distribution_prefix = distinfo_dir.with_suffix("").name - - with _WhlFile( - args.output, mode="w", distribution_prefix=distribution_prefix - ) as out: - for p in _files_to_pack(patched_wheel_dir, record_contents): - rel_path = p.relative_to(patched_wheel_dir) - out.add_file(str(rel_path), p) - - logging.debug(f"Writing RECORD file") - got_record = out.add_recordfile().decode("utf-8", "surrogateescape") - - if got_record == record_contents: - logging.info(f"Created a whl file: {args.output}") - return - - record_diff = _unidiff_output( - record_contents, - got_record, - out.distinfo_path("RECORD"), - ) - args.record_patch.write_text(record_diff) - logging.warning( - f"Please apply patch to the RECORD file ({args.record_patch}):\n{record_diff}" - ) - - -if __name__ == "__main__": - logging.basicConfig( - format="%(module)s: %(levelname)s: %(message)s", level=logging.DEBUG - ) - - sys.exit(main(sys.argv[1:])) diff --git a/python/private/pypi/requirements.bzl.tmpl.workspace b/python/private/pypi/requirements.bzl.tmpl.workspace index 2f4bcd6916..f61a5271f6 100644 --- a/python/private/pypi/requirements.bzl.tmpl.workspace +++ b/python/private/pypi/requirements.bzl.tmpl.workspace @@ -52,7 +52,8 @@ def install_deps(**whl_library_kwargs): for requirement in group_requirements } - # %%GROUP_LIBRARY%% + config_repo = %%CONFIG_REPO%% + config_load = "@{}//:config.bzl".format(config_repo) # Install wheels which may be participants in a group whl_config = dict(_config) @@ -68,5 +69,6 @@ def install_deps(**whl_library_kwargs): group_name = group_name, group_deps = group_deps, annotation = _get_annotation(requirement), + config_load = config_load, **whl_config ) diff --git a/python/private/pypi/requirements_files_by_platform.bzl b/python/private/pypi/requirements_files_by_platform.bzl index e3aafc083f..c6f53ae611 100644 --- a/python/private/pypi/requirements_files_by_platform.bzl +++ b/python/private/pypi/requirements_files_by_platform.bzl @@ -14,22 +14,10 @@ """Get the requirement files by platform.""" +load(":argparse.bzl", "argparse") load(":whl_target_platforms.bzl", "whl_target_platforms") -# TODO @aignas 2024-05-13: consider using the same platform tags as are used in -# the //python:versions.bzl -DEFAULT_PLATFORMS = [ - "linux_aarch64", - "linux_arm", - "linux_ppc", - "linux_s390x", - "linux_x86_64", - "osx_aarch64", - "osx_x86_64", - "windows_x86_64", -] - -def _default_platforms(*, filter): +def _default_platforms(*, filter, platforms): if not filter: fail("Must specific a filter string, got: {}".format(filter)) @@ -48,40 +36,20 @@ def _default_platforms(*, filter): fail("The filter can only contain '*' at the end of it") if not prefix: - return DEFAULT_PLATFORMS + return platforms - return [p for p in DEFAULT_PLATFORMS if p.startswith(prefix)] + match = [p for p in platforms if p.startswith(prefix) or ( + p.startswith("cp") and p.partition("_")[-1].startswith(prefix) + )] else: - return [p for p in DEFAULT_PLATFORMS if filter in p] + match = [p for p in platforms if filter in p] -def _platforms_from_args(extra_pip_args): - platform_values = [] - - if not extra_pip_args: - return platform_values - - for arg in extra_pip_args: - if platform_values and platform_values[-1] == "": - platform_values[-1] = arg - continue - - if arg == "--platform": - platform_values.append("") - continue - - if not arg.startswith("--platform"): - continue - - _, _, plat = arg.partition("=") - if not plat: - _, _, plat = arg.partition(" ") - if plat: - platform_values.append(plat) - else: - platform_values.append("") + return match +def _platforms_from_args(extra_pip_args): + platform_values = argparse.platform(extra_pip_args, []) if not platform_values: - return [] + return platform_values platforms = { p.target_platform: None @@ -91,13 +59,12 @@ def _platforms_from_args(extra_pip_args): return list(platforms.keys()) def _platform(platform_string, python_version = None): - if not python_version or platform_string.startswith("cp3"): + if not python_version or platform_string.startswith("cp"): return platform_string - _, _, tail = python_version.partition(".") - minor, _, _ = tail.partition(".") + major, _, tail = python_version.partition(".") - return "cp3{}_{}".format(minor, platform_string) + return "cp{}{}_{}".format(major, tail, platform_string) def requirements_files_by_platform( *, @@ -106,6 +73,8 @@ def requirements_files_by_platform( requirements_linux = None, requirements_lock = None, requirements_windows = None, + uv_lock = None, + platforms, extra_pip_args = None, python_version = None, logger = None, @@ -120,10 +89,13 @@ def requirements_files_by_platform( requirements_linux (label): The requirements file for the linux OS. requirements_lock (label): The requirements file for all OSes, or used as a fallback. requirements_windows (label): The requirements file for windows OS. + uv_lock (label): The uv.lock file, or used as primary source. extra_pip_args (string list): Extra pip arguments to perform extra validations and to be joined with args fined in files. python_version: str or None. This is needed when the get_index_urls is specified. It should be of the form "3.x.x", + platforms: {type}`list[str]` the list of human-friendly platform labels that should + be used for the evaluation. logger: repo_utils.logger or None, a simple struct to log diagnostic messages. fail_fn (Callable[[str], None]): A failure function used in testing failure cases. @@ -136,20 +108,26 @@ def requirements_files_by_platform( requirements_linux or requirements_osx or requirements_windows or - requirements_by_platform + requirements_by_platform or + uv_lock ): fail_fn( - "A 'requirements_lock' attribute must be specified, a platform-specific lockfiles " + + "A 'requirements_lock' or 'uv_lock' attribute must be specified, a platform-specific lockfiles " + "via 'requirements_by_platform' or an os-specific lockfiles must be specified " + "via 'requirements_*' attributes", ) return None - platforms = _platforms_from_args(extra_pip_args) + platforms_from_args = _platforms_from_args(extra_pip_args) + if logger: + logger.debug(lambda: "Platforms from pip args: {} (from {})".format(platforms_from_args, extra_pip_args)) + + input_platforms = platforms + default_platforms = [_platform(p, python_version) for p in platforms] if logger: - logger.debug(lambda: "Platforms from pip args: {}".format(platforms)) + logger.debug(lambda: "Input platforms: {}".format(input_platforms)) - if platforms: + if platforms_from_args: lock_files = [ f for f in [ @@ -168,9 +146,12 @@ def requirements_files_by_platform( fail_fn("only a single 'requirements_lock' file can be used when using '--platform' pip argument, consider specifying it via 'requirements_lock' attribute") return None - files_by_platform = [ - (lock_files[0], platforms), - ] + if not lock_files: + files_by_platform = [] + else: + files_by_platform = [ + (lock_files[0], platforms_from_args), + ] if logger: logger.debug(lambda: "Files by platform with the platform set in the args: {}".format(files_by_platform)) else: @@ -178,7 +159,8 @@ def requirements_files_by_platform( file: [ platform for filter_or_platform in specifier.split(",") - for platform in (_default_platforms(filter = filter_or_platform) if filter_or_platform.endswith("*") else [filter_or_platform]) + for platform in (_default_platforms(filter = filter_or_platform, platforms = platforms) if filter_or_platform.endswith("*") else [filter_or_platform]) + if _platform(platform, python_version) in default_platforms ] for file, specifier in requirements_by_platform.items() }.items() @@ -189,9 +171,9 @@ def requirements_files_by_platform( for f in [ # If the users need a greater span of the platforms, they should consider # using the 'requirements_by_platform' attribute. - (requirements_linux, _default_platforms(filter = "linux_*")), - (requirements_osx, _default_platforms(filter = "osx_*")), - (requirements_windows, _default_platforms(filter = "windows_*")), + (requirements_linux, _default_platforms(filter = "linux_*", platforms = platforms)), + (requirements_osx, _default_platforms(filter = "osx_*", platforms = platforms)), + (requirements_windows, _default_platforms(filter = "windows_*", platforms = platforms)), (requirements_lock, None), ]: if f[0]: @@ -216,8 +198,7 @@ def requirements_files_by_platform( return None configured_platforms[p] = file - else: - default_platforms = [_platform(p, python_version) for p in DEFAULT_PLATFORMS] + elif plats == None: plats = [ p for p in default_platforms @@ -232,6 +213,13 @@ def requirements_files_by_platform( for p in plats: configured_platforms[p] = file + elif logger: + logger.info(lambda: "File {} will be ignored because there are no configured platforms: {} out of {}".format( + file, + default_platforms, + input_platforms, + )) + if logger: logger.debug(lambda: "Configured platforms for file {} are {}".format(file, plats)) @@ -253,6 +241,10 @@ def requirements_files_by_platform( ret = {} for plat, file in requirements.items(): - ret.setdefault(file, []).append(plat) + ret.setdefault(file, []).append(_platform(plat, python_version = python_version)) + + for file, _plats in files_by_platform: + if file not in ret and _plats != None: + ret[file] = [] return ret diff --git a/python/private/pypi/requirements_parser/resolve_target_platforms.py b/python/private/pypi/requirements_parser/resolve_target_platforms.py index c899a943cc..96607ee240 100755 --- a/python/private/pypi/requirements_parser/resolve_target_platforms.py +++ b/python/private/pypi/requirements_parser/resolve_target_platforms.py @@ -21,7 +21,6 @@ import pathlib from packaging.requirements import Requirement - from python.private.pypi.whl_installer.platform import Platform INPUT_HELP = """\ @@ -50,8 +49,8 @@ def main(): hashes = prefix + hashes req = Requirement(entry) - for p in target_platforms: - (platform,) = Platform.from_string(p) + for p, triple in target_platforms.items(): + (platform,) = Platform.from_string(triple) if not req.marker or req.marker.evaluate(platform.env_markers("")): response.setdefault(requirement_line, []).append(p) diff --git a/python/private/pypi/select_whl.bzl b/python/private/pypi/select_whl.bzl new file mode 100644 index 0000000000..b32fc68f01 --- /dev/null +++ b/python/private/pypi/select_whl.bzl @@ -0,0 +1,296 @@ +"Select a single wheel that fits the parameters of a target platform." + +load("//python/private:version.bzl", "version") +load(":parse_whl_name.bzl", "parse_whl_name") +load(":python_tag.bzl", "PY_TAG_GENERIC", "python_tag") + +_ANDROID = "android" +_IOS = "ios" +_MANYLINUX = "manylinux" +_MACOSX = "macosx" +_MUSLLINUX = "musllinux" + +# Taken from https://peps.python.org/pep-0600/ +_LEGACY_ALIASES = { + "manylinux1_i686": "manylinux_2_5_i686", + "manylinux1_x86_64": "manylinux_2_5_x86_64", + "manylinux2010_i686": "manylinux_2_12_i686", + "manylinux2010_x86_64": "manylinux_2_12_x86_64", + "manylinux2014_aarch64": "manylinux_2_17_aarch64", + "manylinux2014_armv7l": "manylinux_2_17_armv7l", + "manylinux2014_i686": "manylinux_2_17_i686", + "manylinux2014_ppc64": "manylinux_2_17_ppc64", + "manylinux2014_ppc64le": "manylinux_2_17_ppc64le", + "manylinux2014_s390x": "manylinux_2_17_s390x", + "manylinux2014_x86_64": "manylinux_2_17_x86_64", +} + +def _value_priority(*, tag, values): + keys = [] + for priority, wp in enumerate(values): + if tag == wp: + keys.append(priority) + + return max(keys) if keys else None + +def _is_platform_tag_versioned(tag): + return ( + tag.startswith(_ANDROID) or + tag.startswith(_IOS) or + tag.startswith(_MACOSX) or + tag.startswith(_MANYLINUX) or + tag.startswith(_MUSLLINUX) + ) + +def _parse_platform_tags(tags): + """A helper function that parses all of the platform tags. + + The main idea is to make this more robust and have better debug messages about which will + is compatible and which is not with the target platform. + """ + ret = [] + replacements = {} + for tag in tags: + tag = _LEGACY_ALIASES.get(tag, tag) + + if not _is_platform_tag_versioned(tag): + ret.append(tag) + continue + + want_os, sep, tail = tag.partition("_") + if not sep: + fail("could not parse the tag: {}".format(tag)) + + want_major, _, tail = tail.partition("_") + if want_major == "*": + # the expected match is any version + want_arch = tail + elif want_os.startswith(_ANDROID): + want_arch = tail + else: + # drop the minor version segment + _, _, want_arch = tail.partition("_") + + placeholder = "{}_*_{}".format(want_os, want_arch) + replacements[placeholder] = tag + if placeholder in ret: + ret.remove(placeholder) + + ret.append(placeholder) + + return [ + replacements.get(p, p) + for p in ret + ] + +def _platform_tag_priority(*, tag, values): + # Implements matching platform tag + # https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/ + + if not _is_platform_tag_versioned(tag): + res = _value_priority(tag = tag, values = values) + if res == None: + return res + + return (res, (0, 0)) + + # Only android, ios, macosx, manylinux or musllinux platforms should be considered + + os, _, tail = tag.partition("_") + major, _, tail = tail.partition("_") + if not tag.startswith(_ANDROID): + minor, _, arch = tail.partition("_") + else: + minor = "0" + arch = tail + version = (int(major), int(minor)) + + keys = [] + for priority, wp in enumerate(values): + want_os, sep, tail = wp.partition("_") + if not sep: + # if there is no `_` separator, then it means that we have something like `win32` or + # similar wheels that we are considering, this means that it should be discarded because + # we are dealing only with platforms that have `_`. + continue + + if want_os != os: + # os should always match exactly for us to match and assign a priority + continue + + want_major, _, tail = tail.partition("_") + if want_major == "*": + # the expected match is any version + want_major = "" + want_minor = "" + want_arch = tail + elif tag.startswith(_ANDROID): + # we set it to `0` above, so setting the `want_minor` her to `0` will make things + # consistent. + want_minor = "0" + want_arch = tail + else: + # here we parse the values from the given platform + want_minor, _, want_arch = tail.partition("_") + + if want_arch != arch: + # the arch should match exactly + continue + + # if want_major is defined, then we know that we don't have a `*` in the matcher. + want_version = (int(want_major), int(want_minor)) if want_major else None + if not want_version or version <= want_version: + keys.append((priority, (-version[0], -version[1]))) + + return max(keys) if keys else None + +def _python_tag_priority(*, tag, implementation, py_version): + if tag.startswith(PY_TAG_GENERIC): + ver_str = tag[len(PY_TAG_GENERIC):] + elif tag.startswith(implementation): + ver_str = tag[len(implementation):] + else: + return None + + # Add a 0 at the end in case it is a single digit + ver_str = "{}.{}".format(ver_str[0], ver_str[1:] or "0") + + ver = version.parse(ver_str) + if not version.is_compatible(py_version, ver): + return None + + return ( + tag.startswith(implementation), + version.key(ver), + ) + +def _candidates_by_priority( + *, + whls, + implementation_name, + python_version, + whl_abi_tags, + whl_platform_tags, + logger): + """Calculate the priority of each wheel + + Returns: + A dictionary where keys are priority tuples which allows us to sort and pick the + last item. + """ + py_version = version.parse(python_version, strict = True) + implementation = python_tag(implementation_name) + + ret = {} + for whl in whls: + parsed = parse_whl_name(whl.filename) + priority = None + + # See https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#compressed-tag-sets + for platform in parsed.platform_tag.split("."): + platform = _platform_tag_priority(tag = platform, values = whl_platform_tags) + if platform == None: + logger.debug(lambda: "The platform_tag in '{}' does not match given list: {}".format( + whl.filename, + whl_platform_tags, + )) + continue + + for py in parsed.python_tag.split("."): + py = _python_tag_priority( + tag = py, + implementation = implementation, + py_version = py_version, + ) + if py == None: + logger.debug(lambda: "The python_tag in '{}' does not match implementation or version: {} {}".format( + whl.filename, + implementation, + py_version.string, + )) + continue + + for abi in parsed.abi_tag.split("."): + abi = _value_priority( + tag = abi, + values = whl_abi_tags, + ) + if abi == None: + logger.debug(lambda: "The abi_tag in '{}' does not match given list: {}".format( + whl.filename, + whl_abi_tags, + )) + continue + + # 1. Prefer platform wheels + # 2. Then prefer implementation/python version + # 3. Then prefer more specific ABI wheels + candidate = (platform, py, abi) + priority = priority or candidate + if candidate > priority: + priority = candidate + + if priority == None: + logger.debug(lambda: "The whl '{}' is incompatible".format( + whl.filename, + )) + continue + + ret[priority] = whl + + return ret + +def select_whl( + *, + whls, + python_version, + whl_platform_tags, + whl_abi_tags, + implementation_name = "cpython", + limit = 1, + logger): + """Select a whl that is the most suitable for the given platform. + + Args: + whls: {type}`list[struct]` a list of candidates which have a `filename` + attribute containing the `whl` filename. + python_version: {type}`str` the target python version. + implementation_name: {type}`str` the `implementation_name` from the target_platform env. + whl_abi_tags: {type}`list[str]` The whl abi tags to select from. The preference is + for wheels that have ABI values appearing later in the `whl_abi_tags` list. + whl_platform_tags: {type}`list[str]` The whl platform tags to select from. + The platform tag may contain `*` and this means that if the platform tag is + versioned (e.g. `manylinux`), then we will select the highest available + platform version, e.g. if `manylinux_2_17` and `manylinux_2_5` wheels are both + compatible, we will select `manylinux_2_17`. Otherwise for versioned platform + tags we select the highest *compatible* version, e.g. if `manylinux_2_6` + support is requested, then we would select `manylinux_2_5` in the previous + example. This allows us to pass the same filtering parameters when selecting + all of the whl dependencies irrespective of what actual platform tags they + contain. + limit: {type}`int` number of wheels to return. Defaults to 1. + logger: {type}`struct` the logger instance. + + Returns: + {type}`list[struct] | struct | None`, a single struct from the `whls` input + argument or `None` if a match is not found. If the `limit` is greater than + one, then we will return a list. + """ + candidates = _candidates_by_priority( + whls = whls, + implementation_name = implementation_name, + python_version = python_version, + whl_abi_tags = whl_abi_tags, + whl_platform_tags = _parse_platform_tags(whl_platform_tags), + logger = logger, + ) + + if not candidates: + return None + + res = [i[1] for i in sorted(candidates.items())] + logger.debug(lambda: "Sorted candidates:\n{}".format( + "\n".join([c.filename for c in res]), + )) + + return res[-1] if limit == 1 else res[-limit:] diff --git a/python/private/pypi/simpleapi_download.bzl b/python/private/pypi/simpleapi_download.bzl index c730c20439..b5f7032b0a 100644 --- a/python/private/pypi/simpleapi_download.bzl +++ b/python/private/pypi/simpleapi_download.bzl @@ -16,126 +16,203 @@ A file that houses private functions used in the `bzlmod` extension with the same name. """ -load("@bazel_features//:features.bzl", "bazel_features") -load("//python/private:auth.bzl", "get_auth") +load("//python/private:auth.bzl", _get_auth = "get_auth") load("//python/private:envsubst.bzl", "envsubst") load("//python/private:normalize_name.bzl", "normalize_name") load(":parse_simpleapi_html.bzl", "parse_simpleapi_html") +load(":urllib.bzl", "urllib") -def simpleapi_download(ctx, *, attr, cache, parallel_download = True): +def simpleapi_download( + ctx, + *, + attr, + cache, + parallel_download = True, + read_simpleapi = None, + get_auth = None, + _fail = fail): """Download Simple API HTML. + First it queries all of the indexes for available packages and then it downloads the contents of + the per-package URLs and hash digest values. This is to enable us to use bazel_downloader with + `requirements.txt` files. As a side effect we also are able to "cross-compile" by fetching the + right wheel for the right target platform through the information that we retrieve here. + Args: ctx: The module_ctx or repository_ctx. attr: Contains the parameters for the download. They are grouped into a struct for better clarity. It must have attributes: - * index_url: str, the index. - * index_url_overrides: dict[str, str], the index overrides for - separate packages. - * extra_index_urls: Extra index URLs that will be looked up after - the main is looked up. + * index_url: str, the index, or if `extra_index_urls` are passed, the default index. + * index_url_overrides: dict[str, str], the index overrides for separate packages. + * extra_index_urls: Will be looked at in the order they are defined and the first match + wins. This is similar to what uv does, see + https://docs.astral.sh/uv/concepts/indexes/#searching-across-multiple-indexes. + PRs for implementing other strategies are welcome. * sources: list[str], the sources to download things for. Each value is the contents of requirements files. * envsubst: list[str], the envsubst vars for performing substitution in index url. * netrc: The netrc parameter for ctx.download, see http_file for docs. * auth_patterns: The auth_patterns parameter for ctx.download, see http_file for docs. - cache: A dictionary that can be used as a cache between calls during a - single evaluation of the extension. We use a dictionary as a cache - so that we can reuse calls to the simple API when evaluating the - extension. Using the canonical_id parameter of the module_ctx would - deposit the simple API responses to the bazel cache and that is - undesirable because additions to the PyPI index would not be - reflected when re-evaluating the extension unless we do - `bazel clean --expunge`. + cache: An opaque object used to cache call results. For implementation + see ./pypi_cache.bzl file. We use the canonical_id parameter for the key + value to ensure that distribution fetches from different indexes do not cause + cache collisions, because the index may return different locations from where + the files should be downloaded. We are not using the built-in cache in the + `download` function because the index may get updated at any time and we need + to be able to refresh the data. parallel_download: A boolean to enable usage of bazel 7.1 non-blocking downloads. + read_simpleapi: a function for reading and parsing of the SimpleAPI contents. + Used in tests. + get_auth: A function to get auth information passed to read_simpleapi. Used in tests. + _fail: a function to print a failure. Used in tests. Returns: dict of pkg name to the parsed HTML contents - a list of structs. """ + if not attr.sources: + return {} + index_url_overrides = { normalize_name(p): i for p, i in (attr.index_url_overrides or {}).items() } + sources = { + normalize_name(pkg): versions + for pkg, versions in attr.sources.items() + } - download_kwargs = {} - if bazel_features.external_deps.download_has_block_param: - download_kwargs["block"] = not parallel_download + read_simpleapi = read_simpleapi or _read_simpleapi - # NOTE @aignas 2024-03-31: we are not merging results from multiple indexes - # to replicate how `pip` would handle this case. - async_downloads = {} + ctx.report_progress("Fetch package lists from PyPI index") + + # NOTE: we are not merging results from multiple indexes to replicate how `pip` would + # handle this case. What we do is we select a particular index to download the packages + dist_urls = _get_dist_urls( + ctx, + default_index = attr.index_url, + index_urls = attr.extra_index_urls, + index_url_overrides = index_url_overrides, + sources = sources, + read_simpleapi = read_simpleapi, + cache = cache, + get_auth = get_auth, + attr = attr, + block = not parallel_download, + _fail = _fail, + ) + + ctx.report_progress("Fetching package URLs from PyPI index") + + downloads = {} contents = {} - index_urls = [attr.index_url] + attr.extra_index_urls - for pkg in attr.sources: - pkg_normalized = normalize_name(pkg) - - success = False - for index_url in index_urls: - result = _read_simpleapi( - ctx = ctx, - url = "{}/{}/".format( - index_url_overrides.get(pkg_normalized, index_url).rstrip("/"), - pkg, - ), - attr = attr, - cache = cache, - **download_kwargs - ) - if hasattr(result, "wait"): - # We will process it in a separate loop: - async_downloads.setdefault(pkg_normalized, []).append( - struct( - pkg_normalized = pkg_normalized, - wait = result.wait, - ), - ) - continue + for pkg, url in dist_urls.items(): + result = read_simpleapi( + ctx = ctx, + attr = attr, + url = url, + cache = cache, + versions = sources[pkg], + get_auth = get_auth, + block = not parallel_download, + parse_index = False, + ) + if hasattr(result, "wait"): + # We will process it in a separate loop: + downloads[pkg] = result + else: + contents[pkg] = _with_index_url(url, result.output) + + for pkg, d in downloads.items(): + # If we use `block` == False, then we need to have a second loop that is + # collecting all of the results as they were being downloaded in parallel. + contents[pkg] = _with_index_url(dist_urls[pkg], d.wait().output) + + return contents - if result.success: - contents[pkg_normalized] = result.output - success = True - break +def _get_dist_urls(ctx, *, default_index, index_urls, index_url_overrides, sources, read_simpleapi, attr, block, _fail = fail, **kwargs): + # Ensure the value is not frozen + index_urls = [] + (index_urls or []) + if default_index not in index_urls: + index_urls.append(default_index) - if not async_downloads and not success: - fail("Failed to download metadata from urls: {}".format( - ", ".join(index_urls), + index_url_overrides = index_url_overrides or {} + if index_url_overrides or len(index_urls) == 1: + # Let's not call the index at all and just assume that all of the overrides have been + # specified or there is only a single index and there is no need to download anything + return { + pkg: _normalize_url("{}/{}/".format( + index_url_overrides.get(pkg, default_index), + pkg.replace("_", "-"), # Use the official normalization for URLs )) + for pkg in sources + } + + downloads = {} + results = {} + + for index_url in index_urls: + download = read_simpleapi( + ctx = ctx, + attr = attr, + url = _normalize_url("{index_url}/".format(index_url = index_url)), + parse_index = True, + versions = {pkg: None for pkg in sources}, + block = block, + **kwargs + ) + if hasattr(download, "wait"): + downloads[index_url] = download + else: + results[index_url] = download + + for index_url, download in downloads.items(): + results[index_url] = download.wait() - if not async_downloads: - return contents + found_on_index = {} + for index_url, result in results.items(): + for pkg in sources: + if pkg in found_on_index: + # We have already found the package, skip searching for it in + # other indexes. + # + # If we wanted to merge all of the index results, we would have to continue here + # and in the outer function process merging of the results. + continue - # If we use `block` == False, then we need to have a second loop that is - # collecting all of the results as they were being downloaded in parallel. - for pkg, downloads in async_downloads.items(): - success = False - for download in downloads: - result = download.wait() + found = result.output.get(pkg) + if not found: + continue - if result.success and download.pkg_normalized not in contents: - contents[download.pkg_normalized] = result.output - success = True + # Ignore the URL here because we know how to construct it. - if not success: - fail("Failed to download metadata from urls: {}".format( - ", ".join(index_urls), + found_on_index[pkg] = _normalize_url("{}/{}/".format( + index_url, + pkg.replace("_", "-"), # Use the official normalization for URLs )) - return contents + return found_on_index -def _read_simpleapi(ctx, url, attr, cache, **download_kwargs): +def _normalize_url(url): + return urllib.strip_empty_path_segments(url) + +def _read_simpleapi(ctx, url, attr, cache, versions, parse_index, get_auth = None, **download_kwargs): """Read SimpleAPI. Args: ctx: The module_ctx or repository_ctx. - url: str, the url parameter that can be passed to ctx.download. + url: {type}`str`, the url parameter that can be passed to ctx.download. attr: The attribute that contains necessary info for downloading. The following attributes must be present: - * envsubst: The envsubst values for performing substitutions in the URL. - * netrc: The netrc parameter for ctx.download, see http_file for docs. + * envsubst: {type}`dict[str, str]` for performing substitutions in the URL. + * netrc: The netrc parameter for ctx.download, see {obj}`http_file` for docs. * auth_patterns: The auth_patterns parameter for ctx.download, see - http_file for docs. - cache: A dict for storing the results. + {obj}`http_file` for docs. + cache: {type}`struct` the `pypi_cache` instance. + versions: {type}`list[str] The versions that have been requested. + get_auth: A function to get auth information. Used in tests. + parse_index: {type}`bool` Whether to parse the content as a root index page + (e.g. `/simple/`) instead of a package-specific page. **download_kwargs: Any extra params to ctx.download. Note that output and auth will be passed for you. @@ -143,20 +220,12 @@ def _read_simpleapi(ctx, url, attr, cache, **download_kwargs): A similar object to what `download` would return except that in result.out will be the parsed simple api contents. """ - # NOTE @aignas 2024-03-31: some of the simple APIs use relative URLs for - # the whl location and we cannot handle multiple URLs at once by passing - # them to ctx.download if we want to correctly handle the relative URLs. - # TODO: Add a test that env subbed index urls do not leak into the lock file. + real_url = _normalize_url(envsubst(url, attr.envsubst, ctx.getenv)) - real_url = envsubst( - url, - attr.envsubst, - ctx.getenv if hasattr(ctx, "getenv") else ctx.os.environ.get, - ) - - cache_key = real_url - if cache_key in cache: - return struct(success = True, output = cache[cache_key]) + cache_key = (url, real_url, versions) + cached_result = cache.get(cache_key) + if cached_result: + return struct(success = True, output = cached_result) output_str = envsubst( url, @@ -173,32 +242,58 @@ def _read_simpleapi(ctx, url, attr, cache, **download_kwargs): output = ctx.path(output_str.strip("_").lower() + ".html") + get_auth = get_auth or _get_auth + # NOTE: this may have block = True or block = False in the download_kwargs download = ctx.download( url = [real_url], output = output, auth = get_auth(ctx, [real_url], ctx_attr = attr), - allow_fail = True, **download_kwargs ) if download_kwargs.get("block") == False: # Simulate the same API as ctx.download has return struct( - wait = lambda: _read_index_result(ctx, download.wait(), output, real_url, cache, cache_key), + wait = lambda: _read_index_result( + ctx, + result = download.wait(), + output = output, + cache = cache, + cache_key = cache_key, + parse_index = parse_index, + ), ) - return _read_index_result(ctx, download, output, real_url, cache, cache_key) + return _read_index_result( + ctx, + result = download, + output = output, + cache = cache, + cache_key = cache_key, + parse_index = parse_index, + ) -def _read_index_result(ctx, result, output, url, cache, cache_key): +def _read_index_result(ctx, *, result, output, cache, cache_key, parse_index): if not result.success: return struct(success = False) content = ctx.read(output) - output = parse_simpleapi_html(url = url, content = content) + output = parse_simpleapi_html(content = content, parse_index = parse_index) if output: cache.setdefault(cache_key, output) - return struct(success = True, output = output, cache_key = cache_key) + return struct(success = True, output = output) else: return struct(success = False) + +def _with_index_url(index_url, values): + if not values: + return values + + return struct( + sdists = values.sdists, + whls = values.whls, + hashes_by_version = values.hashes_by_version, + index_url = index_url, + ) diff --git a/python/private/pypi/unified_hub_repo.bzl b/python/private/pypi/unified_hub_repo.bzl new file mode 100644 index 0000000000..f88db75f19 --- /dev/null +++ b/python/private/pypi/unified_hub_repo.bzl @@ -0,0 +1,111 @@ +"""Repository rule for creating the Unified PyPI Hub.""" + +load("//python/private:normalize_name.bzl", "normalize_name") +load("//python/private:text_util.bzl", "render") + +_ROOT_BUILD_TMPL = """\ +load("@rules_python//python/private/pypi:unified_hub_setup.bzl", "define_venv_flag_config_settings") + +package(default_visibility = ["//visibility:public"]) + +define_venv_flag_config_settings( + name = "venv_config_settings", + hubs = {hubs}, +) +""" + +_PKG_BUILD_TMPL = """\ +load("@rules_python//python/private/pypi:unified_hub_setup.bzl", "define_pypi_package_targets") + +package(default_visibility = ["//visibility:public"]) + +define_pypi_package_targets( + name = "{pkg_name}", + default_hub = {default_hub}, + extra_aliases = {extra_aliases}, + hubs = {hubs}, + pkg_hubs = {pkg_hubs}, +) +""" + +def _unified_hub_repo_impl(rctx): + hubs = rctx.attr.hubs + default_hub = rctx.attr.default_hub or None + + # 1. Generate Root BUILD.bazel with shared config settings + rctx.file( + "BUILD.bazel", + _ROOT_BUILD_TMPL.format(hubs = hubs), + ) + + # 2. Organize extra aliases by package + extra_aliases_by_pkg = {} + for qual_alias, alias_hubs in rctx.attr.extra_aliases.items(): + if ":" not in qual_alias: + fail("extra_aliases keys must be in 'pkg:alias' format.") + pkg, alias = qual_alias.split(":", 1) + extra_aliases_by_pkg.setdefault(pkg, {})[alias] = alias_hubs + + # 3. Generate package subpackages + for pkg_name, pkg_hubs in rctx.attr.packages.items(): + extra_aliases = extra_aliases_by_pkg.get(pkg_name, {}) + rctx.file( + pkg_name + "/BUILD.bazel", + _PKG_BUILD_TMPL.format( + default_hub = render.str(default_hub), + extra_aliases = extra_aliases, + hubs = hubs, + pkg_hubs = pkg_hubs, + pkg_name = pkg_name, + ), + ) + +unified_hub_repo = repository_rule( + implementation = _unified_hub_repo_impl, + attrs = { + "default_hub": attr.string( + doc = "The PyPI hub to use when no other hub's conditions match.", + ), + "extra_aliases": attr.string_list_dict( + doc = "Dictionary mapping 'package:alias' to a list of hubs that support it.", + ), + "hubs": attr.string_list( + mandatory = True, + doc = "List of all concrete PyPI hub names.", + ), + "packages": attr.string_list_dict( + mandatory = True, + doc = "Dictionary mapping package names to a list of hubs that contain them.", + ), + }, + doc = "Private repository rule creating the automatic Unified PyPI Hub.", +) + +def unified_workspace_hub_repo(name, hubs, default_hub = None, extra_aliases = {}): + """Creates a Unified PyPI Hub repository for WORKSPACE mode by loading requirements from hubs. + + Args: + name: Name of the repository rule (e.g. "pypi"). + hubs: Dict mapping hub name to its `all_requirements` list or dict. + e.g. {"dev_pip": dev_pip_requirements, "pypi_alpha": pypi_alpha_requirements} + default_hub: Optional default hub name. + extra_aliases: Dictionary mapping 'package:alias' to a list of hubs that support it. + """ + packages = {} + for hub_name, req_map in hubs.items(): + req_list = req_map.keys() if type(req_map) == type({}) else req_map + for req in req_list: + pkg_name = req.split("//")[-1].split(":")[0] + norm_pkg = normalize_name(pkg_name) + if norm_pkg not in packages: + packages[norm_pkg] = [] + if hub_name not in packages[norm_pkg]: + packages[norm_pkg].append(hub_name) + + unified_hub_repo( + name = name, + default_hub = default_hub, + extra_aliases = extra_aliases, + hubs = sorted(hubs.keys()), + packages = packages, + ) diff --git a/python/private/pypi/unified_hub_setup.bzl b/python/private/pypi/unified_hub_setup.bzl new file mode 100644 index 0000000000..b19410fd4b --- /dev/null +++ b/python/private/pypi/unified_hub_setup.bzl @@ -0,0 +1,110 @@ +"""Helper functions for setting up targets within the Unified PyPI Hub repository.""" + +load( + "@rules_python//python/private/pypi:labels.bzl", + "DATA_LABEL", + "DIST_INFO_LABEL", + "EXTRACTED_WHEEL_FILES", + "PY_LIBRARY_PUBLIC_LABEL", + "WHEEL_FILE_PUBLIC_LABEL", +) +load("@rules_python//python/private/pypi:missing_package.bzl", "missing_package_error") + +def define_venv_flag_config_settings(name, hubs): + """Defines the root config_settings for each PyPI spoke hub. + + Args: + name: unused macro name required by buildifier. + hubs: list of concrete hub names. + """ + for hub in hubs: + native.config_setting( + name = "_is_venv_" + hub, + flag_values = {"@rules_python//python/config_settings:venv": hub}, + ) + +_STANDARD_ALIASES = [ + PY_LIBRARY_PUBLIC_LABEL, + WHEEL_FILE_PUBLIC_LABEL, + DATA_LABEL, + DIST_INFO_LABEL, + EXTRACTED_WHEEL_FILES, +] + +def define_pypi_package_targets(name, pkg_hubs, extra_aliases, hubs, default_hub = None): + """Define the targets for a PyPI package in the unified PyPI hub. + + Args: + name: normalized PyPI package name, serving as the main target name. + pkg_hubs: list of hubs that contain this package. + extra_aliases: dict mapping extra alias names to lists of hubs that support them. + hubs: list of all concrete hub names. + default_hub: the hub to use by default. + """ + pkg_name = name + + # Main apparent package target delegates to :pkg + native.alias( + name = pkg_name, + actual = ":pkg", + ) + + all_aliases = _STANDARD_ALIASES + sorted(extra_aliases.keys()) + missing_errors = {} + + for alias_name in all_aliases: + select_map = {} + for hub in hubs: + is_supported = ( + (alias_name in _STANDARD_ALIASES and hub in pkg_hubs) or + (alias_name not in _STANDARD_ALIASES and hub in extra_aliases.get(alias_name, [])) + ) + + if is_supported: + select_map["//:_is_venv_" + hub] = "@{hub}//{pkg}:{alias}".format( + hub = hub, + pkg = pkg_name, + alias = alias_name, + ) + else: + err_target = "_missing_{alias}_in_{hub}".format(alias = alias_name, hub = hub) + if err_target not in missing_errors: + missing_errors[err_target] = { + "hub_name": hub, + "package_name": pkg_name if alias_name in _STANDARD_ALIASES else (pkg_name + ":" + alias_name), + } + select_map["//:_is_venv_" + hub] = ":{}".format(err_target) + + # //conditions:default fallback + default_supported = ( + default_hub and + ((alias_name in _STANDARD_ALIASES and default_hub in pkg_hubs) or + (alias_name not in _STANDARD_ALIASES and default_hub in extra_aliases.get(alias_name, []))) + ) + + if default_supported: + select_map["//conditions:default"] = "@{hub}//{pkg}:{alias}".format( + hub = default_hub, + pkg = pkg_name, + alias = alias_name, + ) + else: + err_target = "_missing_{alias}_in_default".format(alias = alias_name) + if err_target not in missing_errors: + missing_errors[err_target] = { + "hub_name": default_hub or "", + "package_name": pkg_name if alias_name in _STANDARD_ALIASES else (pkg_name + ":" + alias_name), + } + select_map["//conditions:default"] = ":{}".format(err_target) + + native.alias( + name = alias_name, + actual = select(select_map), + ) + + # Generate missing package error targets + for err_name, err_args in missing_errors.items(): + missing_package_error( + name = err_name, + **err_args + ) diff --git a/python/private/pypi/urllib.bzl b/python/private/pypi/urllib.bzl new file mode 100644 index 0000000000..0754334de9 --- /dev/null +++ b/python/private/pypi/urllib.bzl @@ -0,0 +1,95 @@ +"""Utilities for getting an absolute URL from index_url and the URL we find on PyPI index.""" + +load("//python/private:envsubst.bzl", _envsubst = "envsubst") + +def _get_root_directory(url): + scheme_end = url.find("://") + if scheme_end == -1: + fail("Invalid URL format: '{}'".format(url)) + + scheme = url[:scheme_end] + host_end = url.find("/", scheme_end + 3) + if host_end == -1: + host_end = len(url) + host = url[scheme_end + 3:host_end] + + return "{}://{}".format(scheme, host) + +def _is_downloadable(url): + """Checks if the URL would be accepted by the Bazel downloader. + + This is based on Bazel's HttpUtils::isUrlSupportedByDownloader + """ + return url.startswith("http://") or url.startswith("https://") or url.startswith("file://") + +def _absolute_url(index_url, candidate): + """Convert into an absolute URL. + + Args: + index_url: The index URL where the file has been found. + candidate: The candidate URL which may be not absolute. + + Returns: + An absolute URL + """ + if candidate == "": + return candidate + + if _is_downloadable(candidate): + return candidate + + if candidate.startswith("/"): + # absolute path + root_directory = _get_root_directory(index_url) + return "{}{}".format(root_directory, candidate) + + if candidate.startswith(".."): + # relative path with up references + candidate_parts = candidate.split("..") + last = candidate_parts[-1] + for _ in range(len(candidate_parts) - 1): + index_url, _, _ = index_url.rstrip("/").rpartition("/") + + return "{}/{}".format(index_url, last.strip("/")) + + # relative path without up-references + return "{}/{}".format(index_url.rstrip("/"), candidate) + +def _with_envsubst(url, envsubst = None, getenv = None): + if not url or not envsubst or not getenv: + return url + + return _envsubst(url, envsubst, getenv) + +def _strip_empty_path_segments(url): + """Removes empty path segments from a URL. Does nothing for urls with no scheme. + + Public only for testing. + + Args: + url: The url to remove empty path segments from + + Returns: + The url with empty path segments removed and any trailing slash preserved. + If the url had no scheme it is returned unchanged. + """ + sep = "://" + scheme, _, rest = url.partition(sep) + if scheme.lower() == "file" and rest.startswith("/"): + sep = sep + "/" + rest = rest[1:] + + if rest == "": + return url + stripped = "/".join([p for p in rest.split("/") if p]) + if url.endswith("/"): + return "{}{}{}/".format(scheme, sep, stripped) + else: + return "{}{}{}".format(scheme, sep, stripped) + +urllib = struct( + is_absolute = _is_downloadable, + # Ensure that we strip empty path segments when making an absolute URL + absolute_url = lambda index_url, candidate, *, envsubst = None, getenv = None: _strip_empty_path_segments(_with_envsubst(_absolute_url(index_url, candidate), envsubst, getenv)), + strip_empty_path_segments = _strip_empty_path_segments, +) diff --git a/python/private/pypi/venv_entry_point.bzl b/python/private/pypi/venv_entry_point.bzl new file mode 100644 index 0000000000..32cb6f55a5 --- /dev/null +++ b/python/private/pypi/venv_entry_point.bzl @@ -0,0 +1,50 @@ +"""Rule for generating venv entry point scripts.""" + +load("//python/private:attributes.bzl", "WINDOWS_CONSTRAINTS_ATTRS") +load("//python/private:common.bzl", "is_windows_platform") +load("//python/private:rule_builders.bzl", "ruleb") + +def _venv_entry_point_impl(ctx): + is_windows = is_windows_platform(ctx) + + out_name = ctx.label.name + python_exe = "" + if is_windows: + out_name += ".bat" + python_exe = "pythonw.exe" if ctx.attr.group == "gui_scripts" else "python.exe" + + out = ctx.actions.declare_file(out_name) + + ctx.actions.expand_template( + template = ctx.file._template, + output = out, + substitutions = { + "{ATTRIBUTE}": ctx.attr.attribute, + "{MODULE}": ctx.attr.module, + "{PYTHON_EXE}": python_exe, + }, + is_executable = True, + ) + + return [DefaultInfo( + files = depset([out]), + executable = out, + )] + +_builder = ruleb.Rule( + implementation = _venv_entry_point_impl, + executable = True, +) +_builder.attrs.update({ + "attribute": attr.string(mandatory = False, doc = "The attribute to call"), + "extras": attr.string(mandatory = False, doc = "The extras for the entry point"), + "group": attr.string(mandatory = False, doc = "The entry point group (e.g. console_scripts)"), + "module": attr.string(mandatory = True, doc = "The module to import"), + "_template": attr.label( + default = Label("//python/private/pypi:venv_entry_point_template"), + allow_single_file = True, + ), +}) +_builder.attrs.update(WINDOWS_CONSTRAINTS_ATTRS) + +venv_entry_point = _builder.build() diff --git a/python/private/pypi/venv_entry_point_template.bat b/python/private/pypi/venv_entry_point_template.bat new file mode 100644 index 0000000000..36a7dd3b41 --- /dev/null +++ b/python/private/pypi/venv_entry_point_template.bat @@ -0,0 +1,8 @@ +@setlocal enabledelayedexpansion & "%~dp0{PYTHON_EXE}" -x "%~f0" %* & exit /b !ERRORLEVEL! +# -*- coding: utf-8 -*- +import re +import sys +from {MODULE} import {ATTRIBUTE} +if __name__ == "__main__": + sys.argv[0] = re.sub(r"(-script\.pyw|\.exe)?$", "", sys.argv[0]) + sys.exit({ATTRIBUTE}()) diff --git a/python/private/pypi/venv_entry_point_template.sh b/python/private/pypi/venv_entry_point_template.sh new file mode 100644 index 0000000000..d40c31adc4 --- /dev/null +++ b/python/private/pypi/venv_entry_point_template.sh @@ -0,0 +1,10 @@ +#!/bin/sh +'''exec' "$(dirname "$0")/python3" "$0" "$@" +' ''' +# -*- coding: utf-8 -*- +import re +import sys +from {MODULE} import {ATTRIBUTE} +if __name__ == '__main__': + sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) + sys.exit({ATTRIBUTE}()) diff --git a/python/private/pypi/venv_rewrite_shebang.bzl b/python/private/pypi/venv_rewrite_shebang.bzl new file mode 100644 index 0000000000..c653211850 --- /dev/null +++ b/python/private/pypi/venv_rewrite_shebang.bzl @@ -0,0 +1,82 @@ +"""Rule for rewriting portable shebangs.""" + +load("//python/private:attributes.bzl", "WINDOWS_CONSTRAINTS_ATTRS") +load("//python/private:common.bzl", "is_windows_platform", "runfiles_root_path") +load("//python/private:py_info.bzl", "PyInfoBuilder", "VenvSymlinkEntry", "VenvSymlinkKind") +load("//python/private:rule_builders.bzl", "ruleb") + +def _venv_rewrite_shebang_impl(ctx): + is_windows = is_windows_platform(ctx) + + out_name = ctx.label.name + if is_windows: + out_name += ".bat" + + out_file = ctx.actions.declare_file(out_name) + in_file = ctx.file.src + + action_args = ctx.actions.args() + rewriter_file = ctx.files._venv_shebang_rewriter[0] + inputs = depset([in_file, rewriter_file]) + + if rewriter_file.path.endswith(".ps1"): + action_exe = "powershell.exe" + action_args.add_all([ + "-ExecutionPolicy", + "Bypass", + "-NoProfile", + "-File", + rewriter_file, + ]) + else: + action_exe = ctx.attr._venv_shebang_rewriter[DefaultInfo].files_to_run + + action_args.add(in_file) + action_args.add(out_file) + action_args.add("windows" if is_windows else "unix") + + ctx.actions.run( + inputs = inputs, + outputs = [out_file], + executable = action_exe, + arguments = [action_args], + mnemonic = "PyVenvRewriteBin", + progress_message = "Rewriting venv bin script %{input}", + toolchain = None, + ) + + symlink = VenvSymlinkEntry( + kind = VenvSymlinkKind.BIN, + link_to_path = runfiles_root_path(ctx, out_file.short_path), + link_to_file = out_file, + venv_path = out_name, + package = ctx.attr.package, + version = ctx.attr.version, + files = depset([out_file]), + ) + builder = PyInfoBuilder.new() + builder.venv_symlinks.add([symlink]) + py_info = builder.build() + + return [ + DefaultInfo(files = depset([out_file]), executable = out_file), + py_info, + ] + +_builder = ruleb.Rule( + implementation = _venv_rewrite_shebang_impl, + executable = True, +) +_builder.attrs.update({ + "package": attr.string(), + "src": attr.label(mandatory = True, allow_single_file = True), + "version": attr.string(), + "_venv_shebang_rewriter": attr.label( + default = "//python/private/pypi:venv_shebang_rewriter", + allow_files = True, + cfg = "exec", + ), +}) +_builder.attrs.update(WINDOWS_CONSTRAINTS_ATTRS) + +venv_rewrite_shebang = _builder.build() diff --git a/python/private/pypi/venv_shebang_rewriter.ps1 b/python/private/pypi/venv_shebang_rewriter.ps1 new file mode 100644 index 0000000000..fb6077b407 --- /dev/null +++ b/python/private/pypi/venv_shebang_rewriter.ps1 @@ -0,0 +1,44 @@ +[CmdletBinding()] +param( + [Parameter(Position=0, Mandatory=$true)] + [string]$InFile, + + [Parameter(Position=1, Mandatory=$true)] + [string]$OutFile, + + [Parameter(Position=2, Mandatory=$true)] + [string]$TargetOs +) + +$ErrorActionPreference = "Stop" + +$firstLine = Get-Content -Path $InFile -TotalCount 1 -ErrorAction SilentlyContinue +$content = Get-Content -Path $InFile | Select-Object -Skip 1 + +$Utf8NoBom = New-Object System.Text.UTF8Encoding $False + +if ($TargetOs -eq "windows") { + if ($firstLine -match "^#!pythonw") { + $pythonExe = "pythonw.exe" + } else { + $pythonExe = "python.exe" + } + # A Batch-Python polyglot. Batch executes the first line and exits, + # while Python (via -x) ignores the first line and executes the rest. + $wrapper = "@setlocal enabledelayedexpansion & `"%~dp0$pythonExe`" -x `"%~f0`" %* & exit /b !ERRORLEVEL!" + [System.IO.File]::WriteAllText($OutFile, $wrapper + "`r`n", $Utf8NoBom) +} else { + # A Shell-Python polyglot. The shell executes the triple-quoted 'exec' + # command, re-running the script with python3 from the scripts directory. + # Python ignores the triple-quoted string and continues. + $wrapper = @" +#!/bin/sh +'''exec' "`$(dirname "`$0")/python3" "`$0" "`$@" +' ''' +"@ + [System.IO.File]::WriteAllText($OutFile, $wrapper + "`n", $Utf8NoBom) +} + +if ($null -ne $content) { + [System.IO.File]::AppendAllLines($OutFile, [string[]]$content, $Utf8NoBom) +} diff --git a/python/private/pypi/venv_shebang_rewriter.sh b/python/private/pypi/venv_shebang_rewriter.sh new file mode 100755 index 0000000000..d4391d3352 --- /dev/null +++ b/python/private/pypi/venv_shebang_rewriter.sh @@ -0,0 +1,27 @@ +#!/bin/sh +set -eu + +IN="$1" +OUT="$2" +TARGET_OS="$3" + +FIRST_LINE=$(head -n 1 "$IN") + +if [ "$TARGET_OS" = "windows" ]; then + case "$FIRST_LINE" in + "#!pythonw"*) PYTHON_EXE="pythonw.exe" ;; + *) PYTHON_EXE="python.exe" ;; + esac + # A Batch-Python polyglot. Batch executes the first line and exits, + # while Python (via -x) ignores the first line and executes the rest. + printf "@setlocal enabledelayedexpansion & \"%%~dp0$PYTHON_EXE\" -x \"%%~f0\" %%* & exit /b !ERRORLEVEL!\r\n" > "$OUT" +else + printf "#!/bin/sh\n" > "$OUT" + # A Shell-Python polyglot. The shell executes the triple-quoted 'exec' + # command, re-running the script with python3 from the scripts directory. + # Python ignores the triple-quoted string and continues. + printf "'''exec' \"\$(dirname \"\$0\")/python3\" \"\$0\" \"\$@\"\n' '''\n" >> "$OUT" +fi + +tail -n +2 "$IN" >> "$OUT" +chmod +x "$OUT" diff --git a/python/private/pypi/version_from_filename.bzl b/python/private/pypi/version_from_filename.bzl new file mode 100644 index 0000000000..d0b6e3105d --- /dev/null +++ b/python/private/pypi/version_from_filename.bzl @@ -0,0 +1,42 @@ +"""Parse the version of the thing just from the filename. This is useful for selecting files based on the requested version.""" + +_SDIST_EXTS = [ + ".tar", # handles any compression + ".zip", +] + +def version_from_filename(filename, _fail = None): + """Parse the version of the filename. + + Args: + filename: {type}`str` the filename. + _fail: The fail function. + + Returns: + A string version or None if we could not parse the version. + """ + # See https://packaging.python.org/en/latest/specifications/binary-distribution-format/#binary-distribution-format + + if filename.endswith(".whl"): + # The format is {name}-{version}-{whl_specifiers}.whl + _, _, version = filename.partition("-") + version, _, _ = version.partition("-") + return version + + # NOTE @aignas 2025-03-29: most of the files are wheels, so this is not the common path + + # {name}-{version}.{ext} + head = "" + for ext in _SDIST_EXTS: + head, _, _ = filename.rpartition(ext) # build or name + if head: + break + + if not head: + if _fail: + _fail("Unsupported sdist extension: {filename}".format(filename = filename)) + return None + + # Based on PEP440 the version number cannot include dashes + _, _, version = head.rpartition("-") + return version diff --git a/python/private/pypi/whl_config_repo.bzl b/python/private/pypi/whl_config_repo.bzl new file mode 100644 index 0000000000..b7cea5a8b7 --- /dev/null +++ b/python/private/pypi/whl_config_repo.bzl @@ -0,0 +1,46 @@ +"""whl_config_library implementation for WORKSPACE setups.""" + +load("//python/private:text_util.bzl", "render") +load(":generate_group_library_build_bazel.bzl", "generate_group_library_build_bazel") + +def _whl_config_repo_impl(rctx): + build_file_contents = generate_group_library_build_bazel( + repo_prefix = rctx.attr.repo_prefix, + groups = rctx.attr.groups, + ) + rctx.file("_groups/BUILD.bazel", build_file_contents) + rctx.file("BUILD.bazel", "") + rctx.template( + "config.bzl", + rctx.attr._config_template, + substitutions = { + "%%PACKAGES%%": render.dict(rctx.attr.whl_map or {}, value_repr = lambda x: "None"), + }, + ) + +whl_config_repo = repository_rule( + attrs = { + "groups": attr.string_list_dict( + doc = "A mapping of group names to requirements within that group.", + ), + "repo_prefix": attr.string( + doc = "Prefix used for the whl_library created components of each group", + ), + "whl_map": attr.string_dict( + doc = """\ +The wheel map where values are json.encoded strings of the whl_map constructed +in the pip.parse tag class. +""", + ), + "_config_template": attr.label( + default = ":config.bzl.tmpl", + ), + }, + doc = """ +Create a package containing only wrapper py_library and whl_library rules for implementing dependency groups. +This is an implementation detail of dependency groups and should not be used alone. + +PRIVATE USE ONLY, only used in WORKSPACE. + """, + implementation = _whl_config_repo_impl, +) diff --git a/python/private/pypi/whl_config_setting.bzl b/python/private/pypi/whl_config_setting.bzl index d966206372..1d868b1b65 100644 --- a/python/private/pypi/whl_config_setting.bzl +++ b/python/private/pypi/whl_config_setting.bzl @@ -14,34 +14,44 @@ "A small function to create an alias for a whl distribution" -def whl_config_setting(*, version = None, config_setting = None, filename = None, target_platforms = None): +def whl_config_setting(*, version = None, target_platforms = None): """The bzl_packages value used by by the render_pkg_aliases function. This contains the minimum amount of information required to generate correct aliases in a hub repository. Args: - version: optional(str), the version of the python toolchain that this + version: {type}`str | None`the version of the python toolchain that this whl alias is for. If not set, then non-version aware aliases will be constructed. This is mainly used for better error messages when there is no match found during a select. - config_setting: optional(Label or str), the config setting that we should use. Defaults - to "//_config:is_python_{version}". - filename: optional(str), the distribution filename to derive the config_setting. - target_platforms: optional(list[str]), the list of target_platforms for this + target_platforms: {type}`list[str] | None` the list of target_platforms for this distribution. Returns: a struct with the validated and parsed values. """ + + # FIXME @aignas 2025-07-26: There is still a potential that there will be ambigous match + # if the user is trying to have different packages for 3.X.Y and 3.X.Z versions of python + # in the same hub repository. Consider just removing this processing as I am not sure it + # has much value. if target_platforms: - for p in target_platforms: + target_platforms_input = target_platforms + target_platforms = [] + for p in target_platforms_input: if not p.startswith("cp"): fail("target_platform should start with 'cp' denoting the python version, got: " + p) + abi, _, tail = p.partition("_") + + # drop the micro version here, currently there is no usecase to use + # multiple python interpreters with the same minor version but + # different micro version. + abi, _, _ = abi.partition(".") + target_platforms.append("{}_{}".format(abi, tail)) + return struct( - config_setting = config_setting, - filename = filename, # Make the struct hashable target_platforms = tuple(target_platforms) if target_platforms else None, version = version, diff --git a/python/private/pypi/whl_extract.bzl b/python/private/pypi/whl_extract.bzl new file mode 100644 index 0000000000..0d61b9a07b --- /dev/null +++ b/python/private/pypi/whl_extract.bzl @@ -0,0 +1,108 @@ +"""A simple whl extractor.""" + +load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") +load("//python/private:repo_utils.bzl", "repo_utils") +load(":whl_metadata.bzl", "find_whl_metadata") + +def whl_extract(rctx, *, whl_path, logger): + """Extract whls in Starlark. + + Args: + rctx: the repository ctx. + whl_path: the whl path to extract. + logger: The logger to use + """ + install_dir_path = rctx.path("site-packages") + repo_utils.extract( + rctx, + archive = whl_path, + output = install_dir_path, + supports_whl_extraction = rp_config.supports_whl_extraction, + extract_needs_chmod = rp_config.extract_needs_chmod, + ) + + metadata_file = find_whl_metadata( + install_dir = install_dir_path, + logger = logger, + ) + + # Get the .dist_info dir name + dist_info_dir = metadata_file.dirname + rctx.file( + dist_info_dir.get_child("INSTALLER"), + "https://github.com/bazel-contrib/rules_python#pipstar", + ) + + # Get the .dist_info dir name + data_dir = dist_info_dir.dirname.get_child(dist_info_dir.basename[:-len(".dist-info")] + ".data") + if data_dir.exists: + for prefix, dest_prefix in { + # https://docs.python.org/3/library/sysconfig.html#posix-prefix + # We are taking this from the legacy whl installer config + "data": "data", + "headers": "include", + # In theory there may be directory collisions here, so it would be best to + # merge the paths here. We are doing for quite a few levels deep. What is + # more, this code has to be reasonably efficient because some packages like + # to not put everything to the top level, but to indicate explicitly if + # something is in `platlib` or `purelib` (e.g. libclang wheel). + "platlib": "site-packages", + "purelib": "site-packages", + "scripts": "bin", + }.items(): + src = data_dir.get_child(prefix) + if not src.exists: + # The prefix does not exist in the wheel, we can continue + continue + + dest_dir = rctx.path(dest_prefix) + repo_utils.mkdir(rctx, dest_dir) + for (src, dest) in merge_trees(src, dest_dir): + logger.debug(lambda: "Renaming: {} -> {}".format(src, dest)) + repo_utils.rename(rctx, src, dest) + + # Ensure that there is no data dir left + rctx.delete(data_dir) + +def merge_trees(src, dest): + """Merge src into the destination path. + + This will attempt to merge-move src files to the destination directory if there are + existing files. Fails at directory depth is 10000 or if there are collisions. + + Args: + src: {type}`path` a src path to rename. + dest: {type}`path` a dest path to rename to. + + Returns: + A list of tuples for src and destination paths. + """ + ret = [] + remaining = [(src, dest)] + collisions = [] + for _ in range(10000): + if collisions or not remaining: + break + + tmp = [] + for (s, d) in remaining: + if not d.exists: + ret.append((s, d)) + continue + + if not s.is_dir or not d.is_dir: + collisions.append(s) + continue + + for file_or_dir in s.readdir(): + tmp.append((file_or_dir, d.get_child(file_or_dir.basename))) + + remaining = tmp + + if remaining: + fail("Exceeded maximum directory depth of 10000 during tree merge.") + + if collisions: + fail("Detected collisions between {} and {}: {}".format(src, dest, collisions)) + + return ret diff --git a/python/private/pypi/whl_installer/BUILD.bazel b/python/private/pypi/whl_installer/BUILD.bazel index 5bce1a5bcc..1912b34af8 100644 --- a/python/private/pypi/whl_installer/BUILD.bazel +++ b/python/private/pypi/whl_installer/BUILD.bazel @@ -1,12 +1,10 @@ -load("//python:defs.bzl", "py_binary", "py_library") +load("//python:py_binary.bzl", "py_binary") +load("//python:py_library.bzl", "py_library") py_library( name = "lib", srcs = [ "arguments.py", - "namespace_pkgs.py", - "platform.py", - "wheel.py", "wheel_installer.py", ], visibility = [ diff --git a/python/private/pypi/whl_installer/arguments.py b/python/private/pypi/whl_installer/arguments.py index 29bea8026e..8471c94ffe 100644 --- a/python/private/pypi/whl_installer/arguments.py +++ b/python/private/pypi/whl_installer/arguments.py @@ -14,11 +14,8 @@ import argparse import json -import pathlib from typing import Any, Dict, Set -from python.private.pypi.whl_installer.platform import Platform - def parser(**kwargs: Any) -> argparse.ArgumentParser: """Create a parser for the wheel_installer tool.""" @@ -41,22 +38,11 @@ def parser(**kwargs: Any) -> argparse.ArgumentParser: action="store", help="Extra arguments to pass down to pip.", ) - parser.add_argument( - "--platform", - action="extend", - type=Platform.from_string, - help="Platforms to target dependencies. Can be used multiple times.", - ) parser.add_argument( "--pip_data_exclude", action="store", help="Additional data exclusion parameters to add to the pip packages BUILD file.", ) - parser.add_argument( - "--enable_implicit_namespace_pkgs", - action="store_true", - help="Disables conversion of implicit namespace packages into pkg-util style packages.", - ) parser.add_argument( "--environment", action="store", @@ -68,11 +54,6 @@ def parser(**kwargs: Any) -> argparse.ArgumentParser: help="Use 'pip download' instead of 'pip wheel'. Disables building wheels from source, but allows use of " "--platform, --python-version, --implementation, and --abi in --extra_pip_args.", ) - parser.add_argument( - "--whl-file", - type=pathlib.Path, - help="Extract a whl file to be used within Bazel.", - ) return parser diff --git a/python/private/pypi/whl_installer/namespace_pkgs.py b/python/private/pypi/whl_installer/namespace_pkgs.py deleted file mode 100644 index 7d23c0e34b..0000000000 --- a/python/private/pypi/whl_installer/namespace_pkgs.py +++ /dev/null @@ -1,121 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""Utility functions to discover python package types""" -import os -import textwrap -from pathlib import Path # supported in >= 3.4 -from typing import List, Optional, Set - - -def implicit_namespace_packages( - directory: str, ignored_dirnames: Optional[List[str]] = None -) -> Set[Path]: - """Discovers namespace packages implemented using the 'native namespace packages' method. - - AKA 'implicit namespace packages', which has been supported since Python 3.3. - See: https://packaging.python.org/guides/packaging-namespace-packages/#native-namespace-packages - - Args: - directory: The root directory to recursively find packages in. - ignored_dirnames: A list of directories to exclude from the search - - Returns: - The set of directories found under root to be packages using the native namespace method. - """ - namespace_pkg_dirs: Set[Path] = set() - standard_pkg_dirs: Set[Path] = set() - directory_path = Path(directory) - ignored_dirname_paths: List[Path] = [Path(p) for p in ignored_dirnames or ()] - # Traverse bottom-up because a directory can be a namespace pkg because its child contains module files. - for dirpath, dirnames, filenames in map( - lambda t: (Path(t[0]), *t[1:]), os.walk(directory_path, topdown=False) - ): - if "__init__.py" in filenames: - standard_pkg_dirs.add(dirpath) - continue - elif ignored_dirname_paths: - is_ignored_dir = dirpath in ignored_dirname_paths - child_of_ignored_dir = any( - d in dirpath.parents for d in ignored_dirname_paths - ) - if is_ignored_dir or child_of_ignored_dir: - continue - - dir_includes_py_modules = _includes_python_modules(filenames) - parent_of_namespace_pkg = any( - Path(dirpath, d) in namespace_pkg_dirs for d in dirnames - ) - parent_of_standard_pkg = any( - Path(dirpath, d) in standard_pkg_dirs for d in dirnames - ) - parent_of_pkg = parent_of_namespace_pkg or parent_of_standard_pkg - if ( - (dir_includes_py_modules or parent_of_pkg) - and - # The root of the directory should never be an implicit namespace - dirpath != directory_path - ): - namespace_pkg_dirs.add(dirpath) - return namespace_pkg_dirs - - -def add_pkgutil_style_namespace_pkg_init(dir_path: Path) -> None: - """Adds 'pkgutil-style namespace packages' init file to the given directory - - See: https://packaging.python.org/guides/packaging-namespace-packages/#pkgutil-style-namespace-packages - - Args: - dir_path: The directory to create an __init__.py for. - - Raises: - ValueError: If the directory already contains an __init__.py file - """ - ns_pkg_init_filepath = os.path.join(dir_path, "__init__.py") - - if os.path.isfile(ns_pkg_init_filepath): - raise ValueError("%s already contains an __init__.py file." % dir_path) - - with open(ns_pkg_init_filepath, "w") as ns_pkg_init_f: - # See https://packaging.python.org/guides/packaging-namespace-packages/#pkgutil-style-namespace-packages - ns_pkg_init_f.write( - textwrap.dedent( - """\ - # __path__ manipulation added by bazelbuild/rules_python to support namespace pkgs. - __path__ = __import__('pkgutil').extend_path(__path__, __name__) - """ - ) - ) - - -def _includes_python_modules(files: List[str]) -> bool: - """ - In order to only transform directories that Python actually considers namespace pkgs - we need to detect if a directory includes Python modules. - - Which files are loadable as modules is extension based, and the particular set of extensions - varies by platform. - - See: - 1. https://github.com/python/cpython/blob/7d9d25dbedfffce61fc76bc7ccbfa9ae901bf56f/Lib/importlib/machinery.py#L19 - 2. PEP 420 -- Implicit Namespace Packages, Specification - https://www.python.org/dev/peps/pep-0420/#specification - 3. dynload_shlib.c and dynload_win.c in python/cpython. - """ - module_suffixes = { - ".py", # Source modules - ".pyc", # Compiled bytecode modules - ".so", # Unix extension modules - ".pyd", # https://docs.python.org/3/faq/windows.html#is-a-pyd-file-the-same-as-a-dll - } - return any(Path(f).suffix in module_suffixes for f in files) diff --git a/python/private/pypi/whl_installer/platform.py b/python/private/pypi/whl_installer/platform.py deleted file mode 100644 index 83e42b0e46..0000000000 --- a/python/private/pypi/whl_installer/platform.py +++ /dev/null @@ -1,302 +0,0 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""Utility class to inspect an extracted wheel directory""" - -import platform -import sys -from dataclasses import dataclass -from enum import Enum -from typing import Any, Dict, Iterator, List, Optional, Union - - -class OS(Enum): - linux = 1 - osx = 2 - windows = 3 - darwin = osx - win32 = windows - - @classmethod - def interpreter(cls) -> "OS": - "Return the interpreter operating system." - return cls[sys.platform.lower()] - - def __str__(self) -> str: - return self.name.lower() - - -class Arch(Enum): - x86_64 = 1 - x86_32 = 2 - aarch64 = 3 - ppc = 4 - s390x = 5 - arm = 6 - amd64 = x86_64 - arm64 = aarch64 - i386 = x86_32 - i686 = x86_32 - x86 = x86_32 - ppc64le = ppc - - @classmethod - def interpreter(cls) -> "Arch": - "Return the currently running interpreter architecture." - # FIXME @aignas 2023-12-13: Hermetic toolchain on Windows 3.11.6 - # is returning an empty string here, so lets default to x86_64 - return cls[platform.machine().lower() or "x86_64"] - - def __str__(self) -> str: - return self.name.lower() - - -def _as_int(value: Optional[Union[OS, Arch]]) -> int: - """Convert one of the enums above to an int for easier sorting algorithms. - - Args: - value: The value of an enum or None. - - Returns: - -1 if we get None, otherwise, the numeric value of the given enum. - """ - if value is None: - return -1 - - return int(value.value) - - -def host_interpreter_minor_version() -> int: - return sys.version_info.minor - - -@dataclass(frozen=True) -class Platform: - os: Optional[OS] = None - arch: Optional[Arch] = None - minor_version: Optional[int] = None - - @classmethod - def all( - cls, - want_os: Optional[OS] = None, - minor_version: Optional[int] = None, - ) -> List["Platform"]: - return sorted( - [ - cls(os=os, arch=arch, minor_version=minor_version) - for os in OS - for arch in Arch - if not want_os or want_os == os - ] - ) - - @classmethod - def host(cls) -> List["Platform"]: - """Use the Python interpreter to detect the platform. - - We extract `os` from sys.platform and `arch` from platform.machine - - Returns: - A list of parsed values which makes the signature the same as - `Platform.all` and `Platform.from_string`. - """ - return [ - Platform( - os=OS.interpreter(), - arch=Arch.interpreter(), - minor_version=host_interpreter_minor_version(), - ) - ] - - def all_specializations(self) -> Iterator["Platform"]: - """Return the platform itself and all its unambiguous specializations. - - For more info about specializations see - https://bazel.build/docs/configurable-attributes - """ - yield self - if self.arch is None: - for arch in Arch: - yield Platform(os=self.os, arch=arch, minor_version=self.minor_version) - if self.os is None: - for os in OS: - yield Platform(os=os, arch=self.arch, minor_version=self.minor_version) - if self.arch is None and self.os is None: - for os in OS: - for arch in Arch: - yield Platform(os=os, arch=arch, minor_version=self.minor_version) - - def __lt__(self, other: Any) -> bool: - """Add a comparison method, so that `sorted` returns the most specialized platforms first.""" - if not isinstance(other, Platform) or other is None: - raise ValueError(f"cannot compare {other} with Platform") - - self_arch, self_os = _as_int(self.arch), _as_int(self.os) - other_arch, other_os = _as_int(other.arch), _as_int(other.os) - - if self_os == other_os: - return self_arch < other_arch - else: - return self_os < other_os - - def __str__(self) -> str: - if self.minor_version is None: - if self.os is None and self.arch is None: - return "//conditions:default" - - if self.arch is None: - return f"@platforms//os:{self.os}" - else: - return f"{self.os}_{self.arch}" - - if self.arch is None and self.os is None: - return f"@//python/config_settings:is_python_3.{self.minor_version}" - - if self.arch is None: - return f"cp3{self.minor_version}_{self.os}_anyarch" - - if self.os is None: - return f"cp3{self.minor_version}_anyos_{self.arch}" - - return f"cp3{self.minor_version}_{self.os}_{self.arch}" - - @classmethod - def from_string(cls, platform: Union[str, List[str]]) -> List["Platform"]: - """Parse a string and return a list of platforms""" - platform = [platform] if isinstance(platform, str) else list(platform) - ret = set() - for p in platform: - if p == "host": - ret.update(cls.host()) - continue - - abi, _, tail = p.partition("_") - if not abi.startswith("cp"): - # The first item is not an abi - tail = p - abi = "" - os, _, arch = tail.partition("_") - arch = arch or "*" - - minor_version = int(abi[len("cp3") :]) if abi else None - - if arch != "*": - ret.add( - cls( - os=OS[os] if os != "*" else None, - arch=Arch[arch], - minor_version=minor_version, - ) - ) - - else: - ret.update( - cls.all( - want_os=OS[os] if os != "*" else None, - minor_version=minor_version, - ) - ) - - return sorted(ret) - - # NOTE @aignas 2023-12-05: below is the minimum number of accessors that are defined in - # https://peps.python.org/pep-0496/ to make rules_python generate dependencies. - # - # WARNING: It may not work in cases where the python implementation is different between - # different platforms. - - # derived from OS - @property - def os_name(self) -> str: - if self.os == OS.linux or self.os == OS.osx: - return "posix" - elif self.os == OS.windows: - return "nt" - else: - return "" - - @property - def sys_platform(self) -> str: - if self.os == OS.linux: - return "linux" - elif self.os == OS.osx: - return "darwin" - elif self.os == OS.windows: - return "win32" - else: - return "" - - @property - def platform_system(self) -> str: - if self.os == OS.linux: - return "Linux" - elif self.os == OS.osx: - return "Darwin" - elif self.os == OS.windows: - return "Windows" - else: - return "" - - # derived from OS and Arch - @property - def platform_machine(self) -> str: - """Guess the target 'platform_machine' marker. - - NOTE @aignas 2023-12-05: this may not work on really new systems, like - Windows if they define the platform markers in a different way. - """ - if self.arch == Arch.x86_64: - return "x86_64" - elif self.arch == Arch.x86_32 and self.os != OS.osx: - return "i386" - elif self.arch == Arch.x86_32: - return "" - elif self.arch == Arch.aarch64 and self.os == OS.linux: - return "aarch64" - elif self.arch == Arch.aarch64: - # Assuming that OSX and Windows use this one since the precedent is set here: - # https://github.com/cgohlke/win_arm64-wheels - return "arm64" - elif self.os != OS.linux: - return "" - elif self.arch == Arch.ppc64le: - return "ppc64le" - elif self.arch == Arch.s390x: - return "s390x" - else: - return "" - - def env_markers(self, extra: str) -> Dict[str, str]: - # If it is None, use the host version - minor_version = self.minor_version or host_interpreter_minor_version() - - return { - "extra": extra, - "os_name": self.os_name, - "sys_platform": self.sys_platform, - "platform_machine": self.platform_machine, - "platform_system": self.platform_system, - "platform_release": "", # unset - "platform_version": "", # unset - "python_version": f"3.{minor_version}", - # FIXME @aignas 2024-01-14: is putting zero last a good idea? Maybe we should - # use `20` or something else to avoid having weird issues where the full version is used for - # matching and the author decides to only support 3.y.5 upwards. - "implementation_version": f"3.{minor_version}.0", - "python_full_version": f"3.{minor_version}.0", - # we assume that the following are the same as the interpreter used to setup the deps: - # "implementation_name": "cpython" - # "platform_python_implementation: "CPython", - } diff --git a/python/private/pypi/whl_installer/wheel.py b/python/private/pypi/whl_installer/wheel.py deleted file mode 100644 index 0f6bd27cdd..0000000000 --- a/python/private/pypi/whl_installer/wheel.py +++ /dev/null @@ -1,383 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""Utility class to inspect an extracted wheel directory""" - -import email -import re -from collections import defaultdict -from dataclasses import dataclass -from pathlib import Path -from typing import Dict, List, Optional, Set, Tuple - -import installer -from packaging.requirements import Requirement -from pip._vendor.packaging.utils import canonicalize_name - -from python.private.pypi.whl_installer.platform import ( - Platform, - host_interpreter_minor_version, -) - - -@dataclass(frozen=True) -class FrozenDeps: - deps: List[str] - deps_select: Dict[str, List[str]] - - -class Deps: - """Deps is a dependency builder that has a build() method to return FrozenDeps.""" - - def __init__( - self, - name: str, - requires_dist: List[str], - *, - extras: Optional[Set[str]] = None, - platforms: Optional[Set[Platform]] = None, - ): - """Create a new instance and parse the requires_dist - - Args: - name (str): The name of the whl distribution - requires_dist (list[Str]): The Requires-Dist from the METADATA of the whl - distribution. - extras (set[str], optional): The list of requested extras, defaults to None. - platforms (set[Platform], optional): The list of target platforms, defaults to - None. If the list of platforms has multiple `minor_version` values, it - will change the code to generate the select statements using - `@rules_python//python/config_settings:is_python_3.y` conditions. - """ - self.name: str = Deps._normalize(name) - self._platforms: Set[Platform] = platforms or set() - self._target_versions = {p.minor_version for p in platforms or {}} - self._default_minor_version = None - if platforms and len(self._target_versions) > 2: - # TODO @aignas 2024-06-23: enable this to be set via a CLI arg - # for being more explicit. - self._default_minor_version = host_interpreter_minor_version() - - if None in self._target_versions and len(self._target_versions) > 2: - raise ValueError( - f"all python versions need to be specified explicitly, got: {platforms}" - ) - - # Sort so that the dictionary order in the FrozenDeps is deterministic - # without the final sort because Python retains insertion order. That way - # the sorting by platform is limited within the Platform class itself and - # the unit-tests for the Deps can be simpler. - reqs = sorted( - (Requirement(wheel_req) for wheel_req in requires_dist), - key=lambda x: f"{x.name}:{sorted(x.extras)}", - ) - - want_extras = self._resolve_extras(reqs, extras) - - # Then add all of the requirements in order - self._deps: Set[str] = set() - self._select: Dict[Platform, Set[str]] = defaultdict(set) - for req in reqs: - self._add_req(req, want_extras) - - def _add(self, dep: str, platform: Optional[Platform]): - dep = Deps._normalize(dep) - - # Self-edges are processed in _resolve_extras - if dep == self.name: - return - - if not platform: - self._deps.add(dep) - - # If the dep is in the platform-specific list, remove it from the select. - pop_keys = [] - for p, deps in self._select.items(): - if dep not in deps: - continue - - deps.remove(dep) - if not deps: - pop_keys.append(p) - - for p in pop_keys: - self._select.pop(p) - return - - if dep in self._deps: - # If the dep is already in the main dependency list, no need to add it in the - # platform-specific dependency list. - return - - # Add the platform-specific dep - self._select[platform].add(dep) - - # Add the dep to specializations of the given platform if they - # exist in the select statement. - for p in platform.all_specializations(): - if p not in self._select: - continue - - self._select[p].add(dep) - - if len(self._select[platform]) == 1: - # We are adding a new item to the select and we need to ensure that - # existing dependencies from less specialized platforms are propagated - # to the newly added dependency set. - for p, deps in self._select.items(): - # Check if the existing platform overlaps with the given platform - if p == platform or platform not in p.all_specializations(): - continue - - self._select[platform].update(self._select[p]) - - def _maybe_add_common_dep(self, dep): - if len(self._target_versions) < 2: - return - - platforms = [Platform()] + [ - Platform(minor_version=v) for v in self._target_versions - ] - - # If the dep is targeting all target python versions, lets add it to - # the common dependency list to simplify the select statements. - for p in platforms: - if p not in self._select: - return - - if dep not in self._select[p]: - return - - # All of the python version-specific branches have the dep, so lets add - # it to the common deps. - self._deps.add(dep) - for p in platforms: - self._select[p].remove(dep) - if not self._select[p]: - self._select.pop(p) - - @staticmethod - def _normalize(name: str) -> str: - return re.sub(r"[-_.]+", "_", name).lower() - - def _resolve_extras( - self, reqs: List[Requirement], extras: Optional[Set[str]] - ) -> Set[str]: - """Resolve extras which are due to depending on self[some_other_extra]. - - Some packages may have cyclic dependencies resulting from extras being used, one example is - `etils`, where we have one set of extras as aliases for other extras - and we have an extra called 'all' that includes all other extras. - - Example: github.com/google/etils/blob/a0b71032095db14acf6b33516bca6d885fe09e35/pyproject.toml#L32. - - When the `requirements.txt` is generated by `pip-tools`, then it is likely that - this step is not needed, but for other `requirements.txt` files this may be useful. - - NOTE @aignas 2023-12-08: the extra resolution is not platform dependent, - but in order for it to become platform dependent we would have to have - separate targets for each extra in extras. - """ - - # Resolve any extra extras due to self-edges, empty string means no - # extras The empty string in the set is just a way to make the handling - # of no extras and a single extra easier and having a set of {"", "foo"} - # is equivalent to having {"foo"}. - extras = extras or {""} - - self_reqs = [] - for req in reqs: - if Deps._normalize(req.name) != self.name: - continue - - if req.marker is None: - # I am pretty sure we cannot reach this code as it does not - # make sense to specify packages in this way, but since it is - # easy to handle, lets do it. - # - # TODO @aignas 2023-12-08: add a test - extras = extras | req.extras - else: - # process these in a separate loop - self_reqs.append(req) - - # A double loop is not strictly optimal, but always correct without recursion - for req in self_reqs: - if any(req.marker.evaluate({"extra": extra}) for extra in extras): - extras = extras | req.extras - else: - continue - - # Iterate through all packages to ensure that we include all of the extras from previously - # visited packages. - for req_ in self_reqs: - if any(req_.marker.evaluate({"extra": extra}) for extra in extras): - extras = extras | req_.extras - - return extras - - def _add_req(self, req: Requirement, extras: Set[str]) -> None: - if req.marker is None: - self._add(req.name, None) - return - - marker_str = str(req.marker) - - if not self._platforms: - if any(req.marker.evaluate({"extra": extra}) for extra in extras): - self._add(req.name, None) - return - - # NOTE @aignas 2023-12-08: in order to have reasonable select statements - # we do have to have some parsing of the markers, so it begs the question - # if packaging should be reimplemented in Starlark to have the best solution - # for now we will implement it in Python and see what the best parsing result - # can be before making this decision. - match_os = any( - tag in marker_str - for tag in [ - "os_name", - "sys_platform", - "platform_system", - ] - ) - match_arch = "platform_machine" in marker_str - match_version = "version" in marker_str - - if not (match_os or match_arch or match_version): - if any(req.marker.evaluate({"extra": extra}) for extra in extras): - self._add(req.name, None) - return - - for plat in self._platforms: - if not any( - req.marker.evaluate(plat.env_markers(extra)) for extra in extras - ): - continue - - if match_arch and self._default_minor_version: - self._add(req.name, plat) - if plat.minor_version == self._default_minor_version: - self._add(req.name, Platform(plat.os, plat.arch)) - elif match_arch: - self._add(req.name, Platform(plat.os, plat.arch)) - elif match_os and self._default_minor_version: - self._add(req.name, Platform(plat.os, minor_version=plat.minor_version)) - if plat.minor_version == self._default_minor_version: - self._add(req.name, Platform(plat.os)) - elif match_os: - self._add(req.name, Platform(plat.os)) - elif match_version and self._default_minor_version: - self._add(req.name, Platform(minor_version=plat.minor_version)) - if plat.minor_version == self._default_minor_version: - self._add(req.name, Platform()) - elif match_version: - self._add(req.name, None) - - # Merge to common if possible after processing all platforms - self._maybe_add_common_dep(req.name) - - def build(self) -> FrozenDeps: - return FrozenDeps( - deps=sorted(self._deps), - deps_select={str(p): sorted(deps) for p, deps in self._select.items()}, - ) - - -class Wheel: - """Representation of the compressed .whl file""" - - def __init__(self, path: Path): - self._path = path - - @property - def path(self) -> Path: - return self._path - - @property - def name(self) -> str: - # TODO Also available as installer.sources.WheelSource.distribution - name = str(self.metadata["Name"]) - return canonicalize_name(name) - - @property - def metadata(self) -> email.message.Message: - with installer.sources.WheelFile.open(self.path) as wheel_source: - metadata_contents = wheel_source.read_dist_info("METADATA") - metadata = installer.utils.parse_metadata_file(metadata_contents) - return metadata - - @property - def version(self) -> str: - # TODO Also available as installer.sources.WheelSource.version - return str(self.metadata["Version"]) - - def entry_points(self) -> Dict[str, Tuple[str, str]]: - """Returns the entrypoints defined in the current wheel - - See https://packaging.python.org/specifications/entry-points/ for more info - - Returns: - Dict[str, Tuple[str, str]]: A mapping of the entry point's name to it's module and attribute - """ - with installer.sources.WheelFile.open(self.path) as wheel_source: - if "entry_points.txt" not in wheel_source.dist_info_filenames: - return dict() - - entry_points_mapping = dict() - entry_points_contents = wheel_source.read_dist_info("entry_points.txt") - entry_points = installer.utils.parse_entrypoints(entry_points_contents) - for script, module, attribute, script_section in entry_points: - if script_section == "console": - entry_points_mapping[script] = (module, attribute) - - return entry_points_mapping - - def dependencies( - self, - extras_requested: Set[str] = None, - platforms: Optional[Set[Platform]] = None, - ) -> FrozenDeps: - return Deps( - self.name, - extras=extras_requested, - platforms=platforms, - requires_dist=self.metadata.get_all("Requires-Dist", []), - ).build() - - def unzip(self, directory: str) -> None: - installation_schemes = { - "purelib": "/site-packages", - "platlib": "/site-packages", - "headers": "/include", - "scripts": "/bin", - "data": "/data", - } - destination = installer.destinations.SchemeDictionaryDestination( - installation_schemes, - # TODO Should entry_point scripts also be handled by installer rather than custom code? - interpreter="/dev/null", - script_kind="posix", - destdir=directory, - bytecode_optimization_levels=[], - ) - - with installer.sources.WheelFile.open(self.path) as wheel_source: - installer.install( - source=wheel_source, - destination=destination, - additional_metadata={ - "INSTALLER": b"https://github.com/bazelbuild/rules_python", - }, - ) diff --git a/python/private/pypi/whl_installer/wheel_installer.py b/python/private/pypi/whl_installer/wheel_installer.py index ef8181c30d..9aa3a4a375 100644 --- a/python/private/pypi/whl_installer/wheel_installer.py +++ b/python/private/pypi/whl_installer/wheel_installer.py @@ -18,16 +18,12 @@ import glob import json import os -import re import subprocess import sys from pathlib import Path from tempfile import NamedTemporaryFile -from typing import Dict, List, Optional, Set, Tuple -from pip._vendor.packaging.utils import canonicalize_name - -from python.private.pypi.whl_installer import arguments, namespace_pkgs, wheel +from python.private.pypi.whl_installer import arguments def _configure_reproducible_wheels() -> None: @@ -55,96 +51,6 @@ def _configure_reproducible_wheels() -> None: os.environ["PYTHONHASHSEED"] = "0" -def _parse_requirement_for_extra( - requirement: str, -) -> Tuple[Optional[str], Optional[Set[str]]]: - """Given a requirement string, returns the requirement name and set of extras, if extras specified. - Else, returns (None, None) - """ - - # https://www.python.org/dev/peps/pep-0508/#grammar - extras_pattern = re.compile( - r"^\s*([0-9A-Za-z][0-9A-Za-z_.\-]*)\s*\[\s*([0-9A-Za-z][0-9A-Za-z_.\-]*(?:\s*,\s*[0-9A-Za-z][0-9A-Za-z_.\-]*)*)\s*\]" - ) - - matches = extras_pattern.match(requirement) - if matches: - return ( - canonicalize_name(matches.group(1)), - {extra.strip() for extra in matches.group(2).split(",")}, - ) - - return None, None - - -def _setup_namespace_pkg_compatibility(wheel_dir: str) -> None: - """Converts native namespace packages to pkgutil-style packages - - Namespace packages can be created in one of three ways. They are detailed here: - https://packaging.python.org/guides/packaging-namespace-packages/#creating-a-namespace-package - - 'pkgutil-style namespace packages' (2) and 'pkg_resources-style namespace packages' (3) works in Bazel, but - 'native namespace packages' (1) do not. - - We ensure compatibility with Bazel of method 1 by converting them into method 2. - - Args: - wheel_dir: the directory of the wheel to convert - """ - - namespace_pkg_dirs = namespace_pkgs.implicit_namespace_packages( - wheel_dir, - ignored_dirnames=["%s/bin" % wheel_dir], - ) - - for ns_pkg_dir in namespace_pkg_dirs: - namespace_pkgs.add_pkgutil_style_namespace_pkg_init(ns_pkg_dir) - - -def _extract_wheel( - wheel_file: str, - extras: Dict[str, Set[str]], - enable_implicit_namespace_pkgs: bool, - platforms: List[wheel.Platform], - installation_dir: Path = Path("."), -) -> None: - """Extracts wheel into given directory and creates py_library and filegroup targets. - - Args: - wheel_file: the filepath of the .whl - installation_dir: the destination directory for installation of the wheel. - extras: a list of extras to add as dependencies for the installed wheel - enable_implicit_namespace_pkgs: if true, disables conversion of implicit namespace packages and will unzip as-is - """ - - whl = wheel.Wheel(wheel_file) - whl.unzip(installation_dir) - - if not enable_implicit_namespace_pkgs: - _setup_namespace_pkg_compatibility(installation_dir) - - extras_requested = extras[whl.name] if whl.name in extras else set() - - dependencies = whl.dependencies(extras_requested, platforms) - - with open(os.path.join(installation_dir, "metadata.json"), "w") as f: - metadata = { - "name": whl.name, - "version": whl.version, - "deps": dependencies.deps, - "deps_by_platform": dependencies.deps_select, - "entry_points": [ - { - "name": name, - "module": module, - "attribute": attribute, - } - for name, (module, attribute) in sorted(whl.entry_points().items()) - ], - } - json.dump(metadata, f) - - def main() -> None: args = arguments.parser(description=__doc__).parse_args() deserialized_args = dict(vars(args)) @@ -152,19 +58,6 @@ def main() -> None: _configure_reproducible_wheels() - if args.whl_file: - whl = Path(args.whl_file) - - name, extras_for_pkg = _parse_requirement_for_extra(args.requirement) - extras = {name: extras_for_pkg} if extras_for_pkg and name else dict() - _extract_wheel( - wheel_file=whl, - extras=extras, - enable_implicit_namespace_pkgs=args.enable_implicit_namespace_pkgs, - platforms=arguments.get_platforms(args), - ) - return - pip_args = ( [sys.executable, "-m", "pip"] + (["--isolated"] if args.isolated else []) diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 612ca2cfdf..47b05468ef 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -17,25 +17,27 @@ load("//python/private:auth.bzl", "AUTH_ATTRS", "get_auth") load("//python/private:envsubst.bzl", "envsubst") load("//python/private:is_standalone_interpreter.bzl", "is_standalone_interpreter") +load("//python/private:normalize_name.bzl", "normalize_name") load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "repo_utils") load(":attrs.bzl", "ATTRS", "use_isolated") -load(":deps.bzl", "all_repo_names") +load(":deps.bzl", "all_repo_names", "record_files") load(":generate_whl_library_build_bazel.bzl", "generate_whl_library_build_bazel") -load(":parse_whl_name.bzl", "parse_whl_name") load(":patch_whl.bzl", "patch_whl") +load(":pep508_requirement.bzl", "requirement") load(":pypi_repo_utils.bzl", "pypi_repo_utils") -load(":whl_target_platforms.bzl", "whl_target_platforms") +load(":urllib.bzl", "urllib") +load(":whl_extract.bzl", "whl_extract") +load(":whl_metadata.bzl", "parse_entry_points", "whl_metadata") _CPPFLAGS = "CPPFLAGS" _COMMAND_LINE_TOOLS_PATH_SLUG = "commandlinetools" -_WHEEL_ENTRY_POINT_PREFIX = "rules_python_wheel_entry_point" -def _get_xcode_location_cflags(rctx): +def _get_xcode_location_cflags(rctx, logger = None): """Query the xcode sdk location to update cflags Figure out if this interpreter target comes from rules_python, and patch the xcode sdk location if so. - Pip won't be able to compile c extensions from sdists with the pre built python distributions from indygreg - otherwise. See https://github.com/indygreg/python-build-standalone/issues/103 + Pip won't be able to compile c extensions from sdists with the pre built python distributions from astral-sh + otherwise. See https://github.com/astral-sh/python-build-standalone/issues/103 """ # Only run on MacOS hosts @@ -46,6 +48,7 @@ def _get_xcode_location_cflags(rctx): rctx, op = "GetXcodeLocation", arguments = [repo_utils.which_checked(rctx, "xcode-select"), "--print-path"], + logger = logger, ) if xcode_sdk_location.return_code != 0: return [] @@ -55,16 +58,44 @@ def _get_xcode_location_cflags(rctx): # This is a full xcode installation somewhere like /Applications/Xcode13.0.app/Contents/Developer # so we need to change the path to to the macos specific tools which are in a different relative # path than xcode installed command line tools. - xcode_root = "{}/Platforms/MacOSX.platform/Developer".format(xcode_root) + xcode_sdks_json = repo_utils.execute_checked( + rctx, + op = "LocateXCodeSDKs", + arguments = [ + repo_utils.which_checked(rctx, "xcrun"), + "xcodebuild", + "-showsdks", + "-json", + ], + environment = { + "DEVELOPER_DIR": xcode_root, + }, + logger = logger, + ).stdout + xcode_sdks = json.decode(xcode_sdks_json) + potential_sdks = [ + sdk + for sdk in xcode_sdks + if "productName" in sdk and + sdk["productName"] == "macOS" and + "darwinos" not in sdk["canonicalName"] + ] + + # Now we'll get two entries here (one for internal and another one for public) + # It shouldn't matter which one we pick. + xcode_sdk_path = potential_sdks[0]["sdkPath"] + else: + xcode_sdk_path = "{}/SDKs/MacOSX.sdk".format(xcode_root) + return [ - "-isysroot {}/SDKs/MacOSX.sdk".format(xcode_root), + "-isysroot {}".format(xcode_sdk_path), ] def _get_toolchain_unix_cflags(rctx, python_interpreter, logger = None): """Gather cflags from a standalone toolchain for unix systems. - Pip won't be able to compile c extensions from sdists with the pre built python distributions from indygreg - otherwise. See https://github.com/indygreg/python-build-standalone/issues/103 + Pip won't be able to compile c extensions from sdists with the pre built python distributions from astral-sh + otherwise. See https://github.com/astral-sh/python-build-standalone/issues/103 """ # Only run on Unix systems @@ -75,14 +106,24 @@ def _get_toolchain_unix_cflags(rctx, python_interpreter, logger = None): if not is_standalone_interpreter(rctx, python_interpreter, logger = logger): return [] - stdout = repo_utils.execute_checked_stdout( + stdout = pypi_repo_utils.execute_checked_stdout( rctx, op = "GetPythonVersionForUnixCflags", + # python_interpreter by default points to a symlink, however when using bazel in vendor mode, + # and the vendored directory moves around, the execution of python fails, as it's getting confused + # where it's running from. More to the fact that we are executing it in isolated mode "-I", which + # results in PYTHONHOME being ignored. The solution is to run python from it's real directory. + python = python_interpreter.realpath, arguments = [ - python_interpreter, + # Run the interpreter in isolated mode, this options implies -E, -P and -s. + # Ensures environment variables are ignored that are set in userspace, such as PYTHONPATH, + # which may interfere with this invocation. + "-I", "-c", "import sys; print(f'{sys.version_info[0]}.{sys.version_info[1]}', end='')", ], + srcs = [], + logger = logger, ) _python_version = stdout include_path = "{}/include/python{}".format( @@ -108,21 +149,13 @@ def _parse_optional_attrs(rctx, args, extra_pip_args = None): if use_isolated(rctx, rctx.attr): args.append("--isolated") - # Bazel version 7.1.0 and later (and rolling releases from version 8.0.0-pre.20240128.3) - # support rctx.getenv(name, default): When building incrementally, any change to the value of - # the variable named by name will cause this repository to be re-fetched. - if "getenv" in dir(rctx): - getenv = rctx.getenv - else: - getenv = rctx.os.environ.get - # Check for None so we use empty default types from our attrs. # Some args want to be list, and some want to be dict. if extra_pip_args != None: args += [ "--extra_pip_args", json.encode(struct(arg = [ - envsubst(pip_arg, rctx.attr.envsubst, getenv) + envsubst(pip_arg, rctx.attr.envsubst, rctx.getenv) for pip_arg in extra_pip_args ])), ] @@ -136,17 +169,66 @@ def _parse_optional_attrs(rctx, args, extra_pip_args = None): json.encode(struct(arg = rctx.attr.pip_data_exclude)), ] - if rctx.attr.enable_implicit_namespace_pkgs: - args.append("--enable_implicit_namespace_pkgs") - + env = {} if rctx.attr.environment != None: - args += [ - "--environment", - json.encode(struct(arg = rctx.attr.environment)), + for key, value in rctx.attr.environment.items(): + env[key] = value + + # This is super hacky, but working out something nice is tricky. + # This is in particular needed for psycopg2 which attempts to link libpython.a, + # in order to point the linker at the correct python intepreter. + if rctx.attr.add_libdir_to_library_search_path: + if "LDFLAGS" in env: + fail("Can't set both environment LDFLAGS and add_libdir_to_library_search_path") + command = [ + pypi_repo_utils.resolve_python_interpreter(rctx), + "-c", + "import sys ; sys.stdout.write('{}/lib'.format(sys.exec_prefix))", ] + result = rctx.execute(command) + if result.return_code != 0: + fail("Failed to get LDFLAGS path: command: {}, exit code: {}, stdout: {}, stderr: {}".format(command, result.return_code, result.stdout, result.stderr)) + libdir = result.stdout + env["LDFLAGS"] = "-L{}".format(libdir) + + args += [ + "--environment", + json.encode(struct(arg = env)), + ] return args +def _get_python_home(rctx, python_interpreter, logger = None): + """Get the PYTHONHOME directory from the selected python interpretter + + Args: + rctx (repository_ctx): The repository context. + python_interpreter (path): The resolved python interpreter. + logger: Optional logger to use for operations. + Returns: + String of PYTHONHOME directory. + """ + + return pypi_repo_utils.execute_checked_stdout( + rctx, + op = "GetPythonHome", + # python_interpreter by default points to a symlink, however when using bazel in vendor mode, + # and the vendored directory moves around, the execution of python fails, as it's getting confused + # where it's running from. More to the fact that we are executing it in isolated mode "-I", which + # results in PYTHONHOME being ignored. The solution is to run python from it's real directory. + python = python_interpreter.realpath, + arguments = [ + # Run the interpreter in isolated mode, this options implies -E, -P and -s. + # Ensures environment variables are ignored that are set in userspace, such as PYTHONPATH, + # which may interfere with this invocation. + "-I", + "-c", + "import sys; print(f'{sys.prefix}', end='')", + ], + srcs = [], + logger = logger, + ) + def _create_repository_execution_environment(rctx, python_interpreter, logger = None): """Create a environment dictionary for processes we spawn with rctx.execute. @@ -158,228 +240,314 @@ def _create_repository_execution_environment(rctx, python_interpreter, logger = Dictionary of environment variable suitable to pass to rctx.execute. """ - # Gather any available CPPFLAGS values - cppflags = [] - cppflags.extend(_get_xcode_location_cflags(rctx)) - cppflags.extend(_get_toolchain_unix_cflags(rctx, python_interpreter, logger = logger)) - env = { + "PYTHONHOME": _get_python_home(rctx, python_interpreter, logger), "PYTHONPATH": pypi_repo_utils.construct_pythonpath( rctx, entries = rctx.attr._python_path_entries, ), - _CPPFLAGS: " ".join(cppflags), } + # Gather any available CPPFLAGS values + # + # We may want to build in an environment without a cc toolchain. + # In those cases, we're limited to --download-only, but we should respect that here. + is_wheel = rctx.attr.filename and rctx.attr.filename.endswith(".whl") + if not (rctx.attr.download_only or is_wheel): + cppflags = [] + cppflags.extend(_get_xcode_location_cflags(rctx, logger = logger)) + cppflags.extend(_get_toolchain_unix_cflags(rctx, python_interpreter, logger = logger)) + env[_CPPFLAGS] = " ".join(cppflags) return env -def _whl_library_impl(rctx): - logger = repo_utils.logger(rctx) - python_interpreter = pypi_repo_utils.resolve_python_interpreter( - rctx, - python_interpreter = rctx.attr.python_interpreter, - python_interpreter_target = rctx.attr.python_interpreter_target, +def _get_entry_points(rctx, install_dir_path, metadata): + dist_info_dir = "{}-{}.dist-info".format( + metadata.name.replace("-", "_"), + metadata.version.replace("-", "_"), ) - args = [ - python_interpreter, - "-m", - "python.private.pypi.whl_installer.wheel_installer", - "--requirement", - rctx.attr.requirement, - ] - extra_pip_args = [] - extra_pip_args.extend(rctx.attr.extra_pip_args) + entry_points_txt = install_dir_path.get_child(dist_info_dir).get_child("entry_points.txt") + if entry_points_txt.exists: + return parse_entry_points(rctx.read(entry_points_txt)) + return {} + +def _move_scripts_needing_shebang_rewrite(rctx, entry_points): + bin_dir = rctx.path("bin") + if not bin_dir.exists: + return + + ep_names = {name.lower(): True for name in entry_points} + for script in bin_dir.readdir(): + if script.is_dir: + continue + if script.basename.lower() in ep_names: + rctx.delete(script) + continue + if script.basename.endswith(".exe") or script.basename.endswith(".dll"): + continue + content = rctx.read(script) + if content.startswith("#!python"): + rewrite_bin_dir = rctx.path("rewrite-bin") + repo_utils.mkdir(rctx, rewrite_bin_dir) + repo_utils.rename(rctx, script, rctx.path("rewrite-bin/" + script.basename)) + +def _to_purl(*, index, metadata, filename): + """ + Produce a PyPI PURL from the metadata. - # Manually construct the PYTHONPATH since we cannot use the toolchain here - environment = _create_repository_execution_environment(rctx, python_interpreter, logger = logger) + https://github.com/package-url/purl-spec/blob/main/types-doc/pypi-definition.md + """ + + # https://github.com/package-url/purl-spec/blob/main/types-doc/pypi-definition.md#name-definition + name = normalize_name(metadata.name).replace("_", "-") + + qualifiers = {} + if index: + qualifiers["repository_url"] = index + if filename: + qualifiers["file_name"] = filename + + return "pkg:pypi/{}@{}?{}".format(name, metadata.version, "&".join(["{}={}".format(key, val) for key, val in qualifiers.items()])) + +def _whl_extract(rctx, *, whl_path, logger, sdist_filename = None): + """Extract the wheel, apply patches and generate BUILD.bazel files.""" + if rctx.attr.whl_patches: + patches = {} + for patch_file, json_args in rctx.attr.whl_patches.items(): + patch_dst = struct(**json.decode(json_args)) + if whl_path.basename in patch_dst.whls: + patches[patch_file] = patch_dst.patch_strip + + if patches: + whl_path = patch_whl( + rctx, + whl_path = whl_path, + patches = patches, + ) + + whl_extract(rctx, whl_path = whl_path, logger = logger) + + install_dir_path = whl_path.dirname.get_child("site-packages") + metadata = whl_metadata( + install_dir = install_dir_path, + read_fn = rctx.read, + logger = logger, + ) + rctx.file("metadata.json", json.encode_indent({ + "name": metadata.name, + "provides_extra": metadata.provides_extra, + "requires_dist": metadata.requires_dist, + "version": metadata.version, + })) + namespace_package_files = pypi_repo_utils.find_namespace_package_files(rctx, install_dir_path) + + entry_points = _get_entry_points(rctx, install_dir_path, metadata) + _move_scripts_needing_shebang_rewrite(rctx, entry_points) + + build_file_contents = generate_whl_library_build_bazel( + name = whl_path.basename, + dep_template = rctx.attr.dep_template or "@{}{{name}}//:{{target}}".format( + rctx.attr.repo_prefix, + ), + sdist_filename = sdist_filename, + config_load = rctx.attr.config_load, + metadata_name = metadata.name, + metadata_version = metadata.version, + requires_dist = metadata.requires_dist, + # TODO @aignas 2025-05-17: maybe have a build flag for this instead + enable_implicit_namespace_pkgs = rctx.attr.enable_implicit_namespace_pkgs, + # TODO @aignas 2025-04-14: load through the hub: + annotation = None if not rctx.attr.annotation else struct(**json.decode(rctx.read(rctx.attr.annotation))), + data_exclude = rctx.attr.pip_data_exclude, + group_deps = rctx.attr.group_deps, + group_name = rctx.attr.group_name, + namespace_package_files = namespace_package_files, + extras = requirement(rctx.attr.requirement).extras, + entry_points = entry_points, + purl = _to_purl( + index = rctx.attr.index_url, + metadata = metadata, + filename = sdist_filename or whl_path.basename, + ), + ) + + # Delete these in case the wheel had them. They generally don't cause + # a problem, but let's avoid the chance of that happening. + rctx.file("WORKSPACE") + rctx.file("WORKSPACE.bazel") + rctx.file("MODULE.bazel") + rctx.file("REPO.bazel", """\ +repo( + default_package_metadata = [ + "//:package_metadata", + ], +) +""") + + # BUILD files interfere with globbing and Bazel package boundaries. + _remove_files(rctx, "BUILD", "BUILD.bazel") + rctx.file("BUILD.bazel", build_file_contents) + + if hasattr(rctx, "repo_metadata"): + return rctx.repo_metadata(reproducible = True) + + return None + +def _whl_archive_impl(rctx): + logger = repo_utils.logger(rctx) whl_path = None if rctx.attr.whl_file: + rctx.watch(rctx.attr.whl_file) whl_path = rctx.path(rctx.attr.whl_file) # Simulate the behaviour where the whl is present in the current directory. rctx.symlink(whl_path, whl_path.basename) whl_path = rctx.path(whl_path.basename) - elif rctx.attr.urls: + elif rctx.attr.urls and rctx.attr.filename: filename = rctx.attr.filename urls = rctx.attr.urls - if not filename: - _, _, filename = urls[0].rpartition("/") + urls = [ + urllib.absolute_url( + rctx.attr.index_url, + url, + envsubst = rctx.attr.envsubst, + getenv = rctx.getenv, + ) + for url in urls + ] + result = rctx.download( + url = urls, + output = filename, + sha256 = rctx.attr.sha256, + integrity = rctx.attr.integrity if not rctx.attr.sha256 else "", + auth = get_auth(rctx, urls), + ) + if not rctx.attr.sha256 and not rctx.attr.integrity: + # this is only seen when there is a direct URL reference without a hash + logger.warn("Please update the requirement line to include the hash:\n{} \\\n --hash=sha256:{}".format( + rctx.attr.requirement, + result.sha256, + )) - if not (filename.endswith(".whl") or filename.endswith("tar.gz") or filename.endswith(".zip")): - if rctx.attr.filename: - msg = "got '{}'".format(filename) - else: - msg = "detected '{}' from url:\n{}".format(filename, urls[0]) - fail("Only '.whl', '.tar.gz' or '.zip' files are supported, {}".format(msg)) + if not result.success: + fail("could not download the '{}' from {}:\n{}".format(filename, urls, result)) + if filename.endswith(".whl"): + whl_path = rctx.path(filename) + else: + fail("Only wheels are supported") + else: + fail("Either 'whl_file' or 'urls' and 'filename' needs to be specified") + + return _whl_extract(rctx, whl_path = whl_path, logger = logger) + +def _pip_archive_impl(rctx): + logger = repo_utils.logger(rctx) + + sdist_filename = None + extra_pip_args = [] + extra_pip_args.extend(rctx.attr.extra_pip_args) + if rctx.attr.urls and rctx.attr.filename: + filename = rctx.attr.filename + urls = rctx.attr.urls + urls = [ + urllib.absolute_url( + rctx.attr.index_url, + url, + envsubst = rctx.attr.envsubst, + getenv = rctx.getenv, + ) + for url in urls + ] result = rctx.download( url = urls, output = filename, sha256 = rctx.attr.sha256, + integrity = rctx.attr.integrity if not rctx.attr.sha256 else "", auth = get_auth(rctx, urls), ) + if not rctx.attr.sha256 and not rctx.attr.integrity: + # this is only seen when there is a direct URL reference without a hash + logger.warn("Please update the requirement line to include the hash:\n{} \\\n --hash=sha256:{}".format( + rctx.attr.requirement, + result.sha256, + )) if not result.success: fail("could not download the '{}' from {}:\n{}".format(filename, urls, result)) if filename.endswith(".whl"): - whl_path = rctx.path(rctx.attr.filename) + fail("Only sdists are supported") else: + sdist_filename = filename + # It is an sdist and we need to tell PyPI to use a file in this directory # and, allow getting build dependencies from PYTHONPATH, which we # setup in this repository rule, but still download any necessary # build deps from PyPI (e.g. `flit_core`) if they are missing. extra_pip_args.extend(["--find-links", "."]) + # When we already have a wheel, Python isn't used, + # so there's no need to setup env vars to run Python, unless we need to + # build an sdist or resolve a requirement. + python_interpreter = pypi_repo_utils.resolve_python_interpreter( + rctx, + python_interpreter = rctx.attr.python_interpreter, + python_interpreter_target = rctx.attr.python_interpreter_target, + ) + args = [ + "-m", + "python.private.pypi.whl_installer.wheel_installer", + "--requirement", + rctx.attr.requirement, + ] args = _parse_optional_attrs(rctx, args, extra_pip_args) - if not whl_path: - if rctx.attr.urls: - op_tmpl = "whl_library.BuildWheelFromSource({name}, {requirement})" - elif rctx.attr.download_only: - op_tmpl = "whl_library.DownloadWheel({name}, {requirement})" - else: - op_tmpl = "whl_library.ResolveRequirement({name}, {requirement})" - - repo_utils.execute_checked( - rctx, - # truncate the requirement value when logging it / reporting - # progress since it may contain several ' --hash=sha256:... - # --hash=sha256:...' substrings that fill up the console - op = op_tmpl.format(name = rctx.attr.name, requirement = rctx.attr.requirement.split(" ", 1)[0]), - arguments = args, - environment = environment, - quiet = rctx.attr.quiet, - timeout = rctx.attr.timeout, - logger = logger, - ) - - whl_path = rctx.path(json.decode(rctx.read("whl_file.json"))["whl_file"]) - if not rctx.delete("whl_file.json"): - fail("failed to delete the whl_file.json file") - - if rctx.attr.whl_patches: - patches = {} - for patch_file, json_args in rctx.attr.whl_patches.items(): - patch_dst = struct(**json.decode(json_args)) - if whl_path.basename in patch_dst.whls: - patches[patch_file] = patch_dst.patch_strip + # Manually construct the PYTHONPATH since we cannot use the toolchain here + environment = _create_repository_execution_environment(rctx, python_interpreter, logger = logger) - if patches: - whl_path = patch_whl( - rctx, - op = "whl_library.PatchWhl({}, {})".format(rctx.attr.name, rctx.attr.requirement), - python_interpreter = python_interpreter, - whl_path = whl_path, - patches = patches, - quiet = rctx.attr.quiet, - timeout = rctx.attr.timeout, - ) + if rctx.attr.urls: + op_tmpl = "whl_library.BuildWheelFromSource({name}, {requirement})" + elif rctx.attr.download_only: + op_tmpl = "whl_library.DownloadWheel({name}, {requirement})" + else: + op_tmpl = "whl_library.ResolveRequirement({name}, {requirement})" - target_platforms = rctx.attr.experimental_target_platforms - if target_platforms: - parsed_whl = parse_whl_name(whl_path.basename) - if parsed_whl.platform_tag != "any": - # NOTE @aignas 2023-12-04: if the wheel is a platform specific - # wheel, we only include deps for that target platform - target_platforms = [ - p.target_platform - for p in whl_target_platforms( - platform_tag = parsed_whl.platform_tag, - abi_tag = parsed_whl.abi_tag, - ) - ] - - repo_utils.execute_checked( + pypi_repo_utils.execute_checked( rctx, - op = "whl_library.ExtractWheel({}, {})".format(rctx.attr.name, whl_path), - arguments = args + [ - "--whl-file", - whl_path, - ] + ["--platform={}".format(p) for p in target_platforms], + # truncate the requirement value when logging it / reporting + # progress since it may contain several ' --hash=sha256:... + # --hash=sha256:...' substrings that fill up the console + python = python_interpreter, + op = op_tmpl.format(name = rctx.attr.name, requirement = rctx.attr.requirement.split(" ", 1)[0]), + arguments = args, environment = environment, + srcs = rctx.attr._python_srcs, quiet = rctx.attr.quiet, timeout = rctx.attr.timeout, logger = logger, ) - metadata = json.decode(rctx.read("metadata.json")) - rctx.delete("metadata.json") + whl_path = rctx.path(json.decode(rctx.read("whl_file.json"))["whl_file"]) + if not rctx.delete("whl_file.json"): + fail("failed to delete the whl_file.json file") - # NOTE @aignas 2024-06-22: this has to live on until we stop supporting - # passing `twine` as a `:pkg` library via the `WORKSPACE` builds. - # - # See ../../packaging.bzl line 190 - entry_points = {} - for item in metadata["entry_points"]: - name = item["name"] - module = item["module"] - attribute = item["attribute"] - - # There is an extreme edge-case with entry_points that end with `.py` - # See: https://github.com/bazelbuild/bazel/blob/09c621e4cf5b968f4c6cdf905ab142d5961f9ddc/src/test/java/com/google/devtools/build/lib/rules/python/PyBinaryConfiguredTargetTest.java#L174 - entry_point_without_py = name[:-3] + "_py" if name.endswith(".py") else name - entry_point_target_name = ( - _WHEEL_ENTRY_POINT_PREFIX + "_" + entry_point_without_py - ) - entry_point_script_name = entry_point_target_name + ".py" + return _whl_extract(rctx, whl_path = whl_path, logger = logger, sdist_filename = sdist_filename) - rctx.file( - entry_point_script_name, - _generate_entry_point_contents(module, attribute), - ) - entry_points[entry_point_without_py] = entry_point_script_name +def _remove_files(rctx, *basenames): + paths = list(rctx.path(".").readdir()) + for _ in range(10000000): + if not paths: + break + path = paths.pop() - build_file_contents = generate_whl_library_build_bazel( - name = whl_path.basename, - dep_template = rctx.attr.dep_template or "@{}{{name}}//:{{target}}".format(rctx.attr.repo_prefix), - dependencies = metadata["deps"], - dependencies_by_platform = metadata["deps_by_platform"], - group_name = rctx.attr.group_name, - group_deps = rctx.attr.group_deps, - data_exclude = rctx.attr.pip_data_exclude, - tags = [ - "pypi_name=" + metadata["name"], - "pypi_version=" + metadata["version"], - ], - entry_points = entry_points, - annotation = None if not rctx.attr.annotation else struct(**json.decode(rctx.read(rctx.attr.annotation))), - ) - rctx.file("BUILD.bazel", build_file_contents) - - return - -def _generate_entry_point_contents( - module, - attribute, - shebang = "#!/usr/bin/env python3"): - """Generate the contents of an entry point script. - - Args: - module (str): The name of the module to use. - attribute (str): The name of the attribute to call. - shebang (str, optional): The shebang to use for the entry point python - file. - - Returns: - str: A string of python code. - """ - contents = """\ -{shebang} -import sys -from {module} import {attribute} -if __name__ == "__main__": - sys.exit({attribute}()) -""".format( - shebang = shebang, - module = module, - attribute = attribute, - ) - return contents + if path.basename in basenames: + rctx.delete(path) + elif path.is_dir: + paths.extend(path.readdir()) # NOTE @aignas 2024-03-21: The usage of dict({}, **common) ensures that all args to `dict` are unique -whl_library_attrs = dict({ +_pip_archive_attrs = dict({ "annotation": attr.label( doc = ( "Optional json encoded file containing annotation to apply to the extracted wheel. " + @@ -387,11 +555,17 @@ whl_library_attrs = dict({ ), allow_files = True, ), + "config_load": attr.string( + doc = "The load location for configuration for pipstar.", + ), "dep_template": attr.string( doc = """ The dep template to use for referencing the dependencies. It should have `{name}` and `{target}` tokens that will be replaced with the normalized distribution name and the target that we need respectively. + +For example if your whl depends on `numpy` and your Python package repo is named +`pip` so that you would normally do `@pip//numpy`, then this should be: `@pip//{name}`. """, ), "filename": attr.string( @@ -404,8 +578,20 @@ and the target that we need respectively. "group_name": attr.string( doc = "Name of the group, if any.", ), + "index_url": attr.string( + doc = "The index_url that the package will be downloaded from.", + ), + "integrity": attr.string( + doc = """\ +The expected checksum of the downloaded whl in Subresource Integrity format +(e.g. `sha256-...` or `sha512-...`). Only used when `urls` is passed. If +`sha256` is also set, it takes precedence over this attribute. + +:::{versionadded} 2.3.0 +::: +""", + ), "repo": attr.string( - mandatory = True, doc = "Pointer to parent repo name. Used to make these rules rerun if the parent repo changes.", ), "repo_prefix": attr.string( @@ -427,15 +613,30 @@ DEPRECATED. Only left for people who vendor requirements.bzl. The list of urls of the whl to be downloaded using bazel downloader. Using this attr makes `extra_pip_args` and `download_only` ignored.""", ), - "whl_file": attr.label( - doc = "The whl file that should be used instead of downloading or building the whl.", - ), "whl_patches": attr.label_keyed_string_dict( - doc = """a label-keyed-string dict that has - json.encode(struct([whl_file], patch_strip]) as values. This - is to maintain flexibility and correct bzlmod extension interface - until we have a better way to define whl_library and move whl - patching to a separate place. INTERNAL USE ONLY.""", + doc = """ +A label-keyed-string dict with patch files as keys and json-strings as values. + +The keys are labels to the patch file to apply. + +The values describe what to apply the patch to and how to apply it. +It is encoded as `json.encode(struct([whls], patch_strip])`, +where `whls` is a `list[str`] of wheel filenames, and `patch_strip` +is a number. + +So it will look something like this: +``` +"//path/to/package:my.patch": json.encode(struct( + whls = ["something-2.7.1-py3-none-any.whl"], + patch_strip = 1, +)), +``` +The patch is applied within the scope of the .whl file. +I.e. you should create the patch from the same place you unziped the wheel. + + +This is to maintain flexibility and correct bzlmod extension interface until we have a better +way to define whl_library and move whl patching to a separate place. INTERNAL USE ONLY.""", ), "_python_path_entries": attr.label_list( # Get the root directory of these rules and keep them as a default attribute @@ -450,18 +651,165 @@ attr makes `extra_pip_args` and `download_only` ignored.""", for repo in all_repo_names ], ), + "_python_srcs": attr.label_list( + # Used as a default value in a rule to ensure we fetch the dependencies. + default = [ + Label("//python/private/pypi/whl_installer:wheel_installer.py"), + Label("//python/private/pypi/whl_installer:arguments.py"), + ] + record_files.values(), + ), "_rule_name": attr.string(default = "whl_library"), }, **ATTRS) -whl_library_attrs.update(AUTH_ATTRS) +_pip_archive_attrs.update(AUTH_ATTRS) -whl_library = repository_rule( - attrs = whl_library_attrs, +pip_archive = repository_rule( + attrs = _pip_archive_attrs, doc = """ Download and extracts a single wheel based into a bazel repo based on the requirement string passed in. -Instantiated from pip_repository and inherits config options from there.""", - implementation = _whl_library_impl, +Instantiated from pip_repository and inherits config options from there. + +:::{versionchanged} 1.9.0 +The `whl_library` is marked as reproducible if using starlark to extract and parse the +wheel contents without building an `sdist` first. +::: + +:::{versionchanged} 2.3.0 +The whl-only pure Starlark operations have been refactored into {obj}`whl_archive` and the +previously named {obj}`whl_library` repository became renamed to `pip_archive`. +::: +""", + implementation = _pip_archive_impl, environ = [ "RULES_PYTHON_PIP_ISOLATED", REPO_DEBUG_ENV_VAR, ], ) + +whl_archive = repository_rule( + attrs = { + k: _pip_archive_attrs[k] + for k in [ + "annotation", + "config_load", + "dep_template", + "filename", + "group_deps", + "group_name", + "index_url", + "integrity", + "repo_prefix", + "requirement", + "sha256", + "urls", + "whl_patches", + "enable_implicit_namespace_pkgs", + "envsubst", + "pip_data_exclude", + ] + } | { + "whl_file": attr.label( + doc = "The whl file that should be used instead of downloading or building the whl.", + ), + } | AUTH_ATTRS, + doc = """ +Download and extracts a single wheel based into a bazel repo based on the requirement string passed in. + +Does not depend on any python. +""", + implementation = _whl_archive_impl, + environ = [ + REPO_DEBUG_ENV_VAR, + ], +) + +def _whl_deps_library_impl(rctx): + logger = repo_utils.logger(rctx) + + if rctx.attr.metadata_file and rctx.attr.metadata: + logger.fail("Only one of 'metadata_file' and 'metadata' can be specified") + return + if not (rctx.attr.metadata_file or rctx.attr.metadata): + logger.fail("At least one of 'metadata_file' and 'metadata' must be specified") + return + + if rctx.attr.metadata_file: + metadata_contents = rctx.read(rctx.attr.metadata_file) + else: + metadata_contents = rctx.attr.metadata + + metadata = struct(**json.decode(metadata_contents)) + + build_file_contents = generate_whl_library_build_bazel( + dep_template = rctx.attr.dep_template or "@{}{{name}}//:{{target}}".format( + rctx.attr.repo_prefix, + ), + config_load = rctx.attr.config_load, + metadata_name = metadata.name, + metadata_version = metadata.version, + requires_dist = metadata.requires_dist, + group_deps = rctx.attr.group_deps, + group_name = rctx.attr.group_name, + repo = rctx.attr.repo or ( + str(rctx.attr.metadata_file) if rctx.attr.metadata_file else None + ), + extras = requirement(rctx.attr.requirement).extras, + ) + rctx.file("BUILD.bazel", build_file_contents) + +whl_deps_library = repository_rule( + attrs = { + k: _pip_archive_attrs[k] + for k in [ + "config_load", + "dep_template", + "group_deps", + "group_name", + "requirement", + ] + } | { + "metadata": attr.string( + doc = """ +The subset of the METADATA contents that is needed for generation of the dependencies. +* name: {type}`str` +* version: {type}`str` +* provides_extra: {type}`list[str]` +* requires_dist: {type}`list[str]` +""", + ), + "metadata_file": attr.label(doc = "An alternative way to pass {attr}`metadata` but as a file."), + "repo": attr.label(doc = "A label at the root of the repo to get stuff from."), + }, + doc = """ +A repo rule that reuses the sources from a different place and then creates the necessary targets +so that this can be used in the repo. + +Does not depend on any python. +""", + implementation = _whl_deps_library_impl, + environ = [REPO_DEBUG_ENV_VAR], +) + +def whl_library(name, repo = None, **kwargs): + """Create a whl_library. + + This proxies to one of the underlying implementations: + * {obj}`whl_archive` + * {obj}`pip_archive` + + Args: + name: {type}`str` The name of the repo. + repo: Unused, will be dropped in the next major release. + **kwargs: The args passed to the underlying implementation. + + Returns: + the repo metadata. + """ + _ = repo # buildifier: disable=unused-variable + + whl_file = kwargs.get("whl_file") + urls = kwargs.get("urls", []) + filename = kwargs.get("filename") + if whl_file or (urls and filename and filename.endswith(".whl")): + whl_archive(name = name, **kwargs) + else: + pip_archive(name = name, **kwargs) diff --git a/python/private/pypi/whl_library_alias.bzl b/python/private/pypi/whl_library_alias.bzl index d34b34a51a..66c3504d90 100644 --- a/python/private/pypi/whl_library_alias.bzl +++ b/python/private/pypi/whl_library_alias.bzl @@ -18,7 +18,7 @@ load("//python/private:full_version.bzl", "full_version") load(":render_pkg_aliases.bzl", "NO_MATCH_ERROR_MESSAGE_TEMPLATE") def _whl_library_alias_impl(rctx): - rules_python = rctx.attr._rules_python_workspace.workspace_name + rules_python = rctx.attr._rules_python_workspace.repo_name if rctx.attr.default_version: default_repo_prefix = rctx.attr.version_map[rctx.attr.default_version] else: diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index a303bdcd9a..ee5c781b3d 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -17,65 +17,163 @@ load("@bazel_skylib//rules:copy_file.bzl", "copy_file") load("//python:py_binary.bzl", "py_binary") load("//python:py_library.bzl", "py_library") -load("//python/private:glob_excludes.bzl", "glob_excludes") load("//python/private:normalize_name.bzl", "normalize_name") +load(":env_marker_setting.bzl", "env_marker_setting") load( ":labels.bzl", "DATA_LABEL", "DIST_INFO_LABEL", + "EXTRACTED_WHEEL_FILES", "PY_LIBRARY_IMPL_LABEL", "PY_LIBRARY_PUBLIC_LABEL", - "WHEEL_ENTRY_POINT_PREFIX", + "PY_SRCS_LABEL", + "WHEEL_FILE", "WHEEL_FILE_IMPL_LABEL", "WHEEL_FILE_PUBLIC_LABEL", ) +load(":namespace_pkgs.bzl", _create_inits = "create_inits") +load(":pep508_deps.bzl", "deps") +load(":venv_entry_point.bzl", "venv_entry_point") +load(":venv_rewrite_shebang.bzl", "venv_rewrite_shebang") + +# Files that are special to the Bazel processing of things. +_BAZEL_REPO_FILE_GLOBS = [ + "BUILD", + "BUILD.bazel", + "REPO.bazel", + "WORKSPACE", + "WORKSPACE.bzlmod", + "WORKSPACE.bazel", +] + +_IS_VENV_SITE_PACKAGES_YES = Label("//python/config_settings:_is_venvs_site_packages_yes") +_VENV_SITE_PACKAGES_FLAG = Label("//python/config_settings:venvs_site_packages") def whl_library_targets( *, name, - dep_template, + metadata_name = "", + requires_dist = [], + extras = [], + entry_points = {}, + include = [], + group_deps = [], + group_name = None, + dep_template = None, + data_exclude = [], + enable_implicit_namespace_pkgs = False, + sdist_filename = None, + namespace_package_files = [], + filegroups = None, + copy_files = {}, + copy_executables = {}, + srcs_exclude = [], + data = [], + visibility = ["//visibility:public"], + **kwargs): + """The macro to create whl targets from the METADATA. + + Args: + name: {type}`str` The wheel filename + metadata_name: {type}`str` The package name as written in wheel `METADATA`. + group_deps: {type}`list[str]` names of fellow members of the group (if + any). These will be excluded from generated deps lists so as to avoid + direct cycles. These dependencies will be provided at runtime by the + group rules which wrap this library and its fellows together. + requires_dist: {type}`list[str]` The list of `Requires-Dist` values from + the whl `METADATA`. + extras: {type}`list[str]` The list of requested extras. This essentially includes extra transitive dependencies in the final targets depending on the wheel `METADATA`. + entry_points: {type}`list[dict]` A list of parsed entry point definitions. + include: {type}`list[str]` The list of packages to include. + group_name: {type}`str | None` name of the dependency group (if any). + dep_template: {type}`str | None` The dep_template to use. + data_exclude: {type}`list[str]` The globs for data attribute exclusion. + enable_implicit_namespace_pkgs: {type}`boolean` generate __init__.py files for namespace pkgs. + sdist_filename: {type}`str | None` The filename of the sdist. + namespace_package_files: {type}`list[str]` A list of labels of files whose directories are namespace packages. + filegroups: {type}`dict[str, list[str]] | None` A dictionary of the target names and the glob matches. + copy_files: {type}`dict[str, str]` The mapping between src and dest locations. + copy_executables: {type}`dict[str, str]` The mapping between src and dest locations for executables. + srcs_exclude: {type}`list[str]` The globs for srcs attribute exclusion. + data: {type}`list[str]` A list of labels to include as part of the `data` attribute. + visibility: {type}`list[str]` The visibility of the targets. + **kwargs: Extra args passed to the {obj}`whl_library_deps_targets` and {obj}`whl_library_srcs`. + """ + create_extra_targets = bool(requires_dist or group_name) and dep_template + whl_library_srcs( + name = name, + sdist_filename = sdist_filename, + data_exclude = data_exclude, + srcs_exclude = srcs_exclude, + filegroups = filegroups, + entry_points = entry_points, + visibility = visibility, + data = data, + copy_files = copy_files, + copy_executables = copy_executables, + enable_implicit_namespace_pkgs = enable_implicit_namespace_pkgs, + namespace_package_files = namespace_package_files, + # If there are no dependencies, then let's create the targets with public labels. + # Note, we are not supporting grouping the packages in this case, but that is fine. + whl_name = WHEEL_FILE if create_extra_targets else WHEEL_FILE_PUBLIC_LABEL, + pkg_name = PY_SRCS_LABEL if create_extra_targets else PY_LIBRARY_PUBLIC_LABEL, + **kwargs + ) + + if create_extra_targets: + whl_library_deps_targets( + name = name, + metadata_name = metadata_name, + requires_dist = requires_dist, + group_deps = group_deps, # only needed if requires_dist is present + extras = extras, # only needed if requires_dist is present + include = include, # only needed if requires_dist is present + group_name = group_name, # only needed if requires_dist is present + dep_template = dep_template, # only needed if requires_dist is present + repo = None, # set aliases in the same repo + aliases = {}, + **kwargs + ) + +def whl_library_srcs( + *, + name, + sdist_filename = None, data_exclude = [], srcs_exclude = [], tags = [], - filegroups = { - DIST_INFO_LABEL: ["site-packages/*.dist-info/**"], - DATA_LABEL: ["data/**"], - }, - dependencies = [], - dependencies_by_platform = {}, - group_deps = [], - group_name = "", + filegroups = None, + entry_points = {}, data = [], copy_files = {}, copy_executables = {}, - entry_points = {}, native = native, + enable_implicit_namespace_pkgs = False, + namespace_package_files = [], + whl_name = WHEEL_FILE, + pkg_name = PY_SRCS_LABEL, + visibility = ["//visibility:public"], rules = struct( copy_file = copy_file, py_binary = py_binary, py_library = py_library, + venv_entry_point = venv_entry_point, + venv_rewrite_shebang = venv_rewrite_shebang, + env_marker_setting = env_marker_setting, + create_inits = _create_inits, )): """Create all of the whl_library targets. Args: name: {type}`str` The file to match for including it into the `whl` filegroup. This may be also parsed to generate extra metadata. - dep_template: {type}`str` The dep_template to use for dependency - interpolation. + sdist_filename: {type}`str | None` If the wheel was built from an sdist, + the filename of the sdist. + visibility: {type}`list[str]` The visibility of the source targets. tags: {type}`list[str]` The tags set on the `py_library`. - dependencies: {type}`list[str]` A list of dependencies. - dependencies_by_platform: {type}`dict[str, list[str]]` A list of - dependencies by platform key. - filegroups: {type}`dict[str, list[str]]` A dictionary of the target - names and the glob matches. - group_name: {type}`str` name of the dependency group (if any) which - contains this library. If set, this library will behave as a shim - to group implementation rules which will provide simultaneously - installed dependencies which would otherwise form a cycle. - group_deps: {type}`list[str]` names of fellow members of the group (if - any). These will be excluded from generated deps lists so as to avoid - direct cycles. These dependencies will be provided at runtime by the - group rules which wrap this library and its fellows together. + entry_points: {type}`list[dict]` A list of parsed entry point definitions. + filegroups: {type}`dict[str, list[str]] | None` A dictionary of the target + names and the glob matches. If `None`, defaults will be used. copy_executables: {type}`dict[str, str]` The mapping between src and dest locations for the targets. copy_files: {type}`dict[str, str]` The mapping between src and @@ -85,26 +183,75 @@ def whl_library_targets( srcs_exclude: {type}`list[str]` The globs for srcs attribute exclusion in `py_library`. data: {type}`list[str]` A list of labels to include as part of the `data` attribute in `py_library`. - entry_points: {type}`dict[str, str]` The mapping between the script - name and the python file to use. DEPRECATED. + enable_implicit_namespace_pkgs: {type}`boolean` generate __init__.py + files for namespace pkgs. + namespace_package_files: {type}`list[str]` A list of labels of files whose + directories are namespace packages. + whl_name: {type}`str` The label name to use for the wheel filegroup target. + pkg_name: {type}`str` The label name to use for the py_library target. native: {type}`native` The native struct for overriding in tests. rules: {type}`struct` A struct with references to rules for creating targets. """ - _ = name # buildifier: @unused - - dependencies = sorted([normalize_name(d) for d in dependencies]) - dependencies_by_platform = { - platform: sorted([normalize_name(d) for d in deps]) - for platform, deps in dependencies_by_platform.items() - } tags = sorted(tags) data = [] + data - for filegroup_name, glob in filegroups.items(): + bins_for_data_label = [] + + for ep_dict in entry_points.values(): + kwargs = dict(ep_dict) + ep_name = kwargs.pop("name") + ep_target_name = "bin/{}".format(ep_name) + rules.venv_entry_point( + name = ep_target_name, + **kwargs + ) + bins_for_data_label.append(ep_target_name) + data.append(ep_target_name) + + existing_bin_names = {ep["name"].lower(): None for ep in entry_points.values()} + for p in native.glob(["bin/*"], allow_empty = True): + existing_bin_names[p[len("bin/"):].lower()] = None + + for src_path in native.glob(["rewrite-bin/*"], allow_empty = True): + script_name = src_path[len("rewrite-bin/"):] + if script_name.lower() in existing_bin_names: + continue + rewrite_target_name = "bin/{}".format(script_name) + rules.venv_rewrite_shebang( + name = rewrite_target_name, + src = src_path, + package = name, + ) + bins_for_data_label.append(rewrite_target_name) + data.append(rewrite_target_name) + + if filegroups == None: + filegroups = { + EXTRACTED_WHEEL_FILES: dict( + include = ["**"], + # The Bazel repo files are always excluded; only the sdist + # filename is conditional on `sdist_filename`. + exclude = _BAZEL_REPO_FILE_GLOBS + ( + [sdist_filename] if sdist_filename else [] + ), + ), + DIST_INFO_LABEL: dict( + include = ["site-packages/*.dist-info/**"], + ), + DATA_LABEL: dict( + include = ["data/**", "bin/**", "include/**"], + ), + } + + for filegroup_name, glob_kwargs in filegroups.items(): + glob_kwargs = {"allow_empty": True} | glob_kwargs + srcs = native.glob(**glob_kwargs) + if filegroup_name == DATA_LABEL: + srcs = srcs + bins_for_data_label native.filegroup( name = filegroup_name, - srcs = native.glob(glob, allow_empty = True), - visibility = ["//visibility:public"], + srcs = srcs, + visibility = visibility, ) for src, dest in copy_files.items(): @@ -112,7 +259,7 @@ def whl_library_targets( name = dest + ".copy", src = src, out = dest, - visibility = ["//visibility:public"], + visibility = visibility, ) data.append(dest) for src, dest in copy_executables.items(): @@ -121,48 +268,136 @@ def whl_library_targets( src = src, out = dest, is_executable = True, - visibility = ["//visibility:public"], + visibility = visibility, ) data.append(dest) - _config_settings( - dependencies_by_platform.keys(), - native = native, - visibility = ["//visibility:private"], - ) + if hasattr(native, "filegroup"): + native.filegroup( + name = whl_name, + srcs = [name], + visibility = visibility, + ) + + if hasattr(rules, "py_library"): + srcs = native.glob( + ["site-packages/**/*.py"], + exclude = srcs_exclude, + # Empty sources are allowed to support wheels that don't have any + # pure-Python code, e.g. pymssql, which is written in Cython. + allow_empty = True, + ) + + # NOTE: pyi files should probably be excluded because they're carried + # by the pyi_srcs attribute. However, historical behavior included + # them in data and some tools currently rely on that. + _data_exclude = [ + "**/*.py", + "**/*.pyc", + "**/*.pyc.*", # During pyc creation, temp files named *.pyc.NNNN are created + ] + if sdist_filename: + _data_exclude.append("**/*.dist-info/RECORD") + for item in data_exclude: + if item not in _data_exclude: + _data_exclude.append(item) + + data = data + native.glob( + ["site-packages/**/*"], + exclude = _data_exclude, + allow_empty = True, + ) + + pyi_srcs = native.glob( + ["site-packages/**/*.pyi"], + allow_empty = True, + ) + + if not enable_implicit_namespace_pkgs: + generated_namespace_package_files = select({ + _IS_VENV_SITE_PACKAGES_YES: [], + "//conditions:default": rules.create_inits( + srcs = srcs + data + pyi_srcs, + ignored_dirnames = [], # If you need to ignore certain folders, you can patch rules_python here to do so. + root = "site-packages", + ), + }) + namespace_package_files += generated_namespace_package_files + srcs = srcs + generated_namespace_package_files + + # This is done after create_inits() is called so that the data scheme + # files don't have such files created in their directories. + data = data + [DATA_LABEL] - # TODO @aignas 2024-10-25: remove the entry_point generation once - # `py_console_script_binary` is the only way to use entry points. - for entry_point, entry_point_script_name in entry_points.items(): - rules.py_binary( - name = "{}_{}".format(WHEEL_ENTRY_POINT_PREFIX, entry_point), - # Ensure that this works on Windows as well - script may have Windows path separators. - srcs = [entry_point_script_name.replace("\\", "/")], + rules.py_library( + name = pkg_name, + srcs = srcs, + pyi_srcs = pyi_srcs, + data = data, # This makes this directory a top-level in the python import # search path for anything that depends on this. - imports = ["."], - deps = [":" + PY_LIBRARY_PUBLIC_LABEL], - visibility = ["//visibility:public"], + imports = ["site-packages"], + tags = tags, + visibility = visibility, + experimental_venvs_site_packages = _VENV_SITE_PACKAGES_FLAG, + namespace_package_files = namespace_package_files, ) - # Ensure this list is normalized - # Note: mapping used as set - group_deps = { - normalize_name(d): True - for d in group_deps - } - - dependencies = [ - d - for d in dependencies - if d not in group_deps - ] - dependencies_by_platform = { - p: deps - for p, deps in dependencies_by_platform.items() - for deps in [[d for d in deps if d not in group_deps]] - if deps - } +def whl_library_deps_targets( + *, + name = None, + repo, + aliases = None, + metadata_name, + requires_dist, + extras, + include = [], + group_deps = [], + group_name = None, + dep_template, + tags = [], + visibility = ["//visibility:public"], + native = native, + rules = struct( + copy_file = copy_file, + py_binary = py_binary, + py_library = py_library, + venv_entry_point = venv_entry_point, + venv_rewrite_shebang = venv_rewrite_shebang, + env_marker_setting = env_marker_setting, + create_inits = _create_inits, + )): + """Create all of the whl_library targets. + + Args: + name: {type}`str` The wheel filename + metadata_name: {type}`str` The package name as written in wheel `METADATA`. + group_deps: {type}`list[str]` names of fellow members of the group (if + any). These will be excluded from generated deps lists so as to avoid + direct cycles. These dependencies will be provided at runtime by the + group rules which wrap this library and its fellows together. + requires_dist: {type}`list[str]` The list of `Requires-Dist` values from + the whl `METADATA`. + extras: {type}`list[str]` The list of requested extras. This essentially includes extra transitive dependencies in the final targets depending on the wheel `METADATA`. + include: {type}`list[str]` The list of packages to include. + group_name: {type}`str | None` name of the dependency group (if any). + dep_template: {type}`str | None` The dep_template to use. + tags: {type}`list[str]` The tags set on the targets. + repo: {type}`str | Label | None` The BUILD.bazel label to the parent repo that has the + sources. If none, then will take the targets from the current dir. + aliases: {type}`dict[str, str] | None` The list of aliases to create in the parent repo. If None, will create + the default values. Empty list means no aliases. + visibility: {type}`list[str]` The visibility of the targets. + native: {type}`native` The native struct for overriding in tests. + rules: {type}`struct` A struct with references to rules for creating targets. + """ + repo_label = Label(repo).same_package_label if repo else (lambda x: x) + if aliases == None: + aliases = { + EXTRACTED_WHEEL_FILES: repo_label(EXTRACTED_WHEEL_FILES), + DIST_INFO_LABEL: repo_label(DIST_INFO_LABEL), + DATA_LABEL: repo_label(DATA_LABEL), + } # If this library is a member of a group, its public label aliases need to # point to the group implementation rule not the implementation rules. We @@ -170,174 +405,142 @@ def whl_library_targets( # implementation. if group_name and "//:" in dep_template: # This is the legacy behaviour where the group library is outside the hub repo + # + # It is expected to disappear when we drop WORKSPACE or drop the vendoring of + # pip_parse `requirements.bzl` in WORKSPACE. The alternative would be to add + # another argument to the macro, but it is already full of arguments. label_tmpl = dep_template.format( - name = "_groups", + name = "_config", target = normalize_name(group_name) + "_{}", + ).replace( + "//:", + "//_groups:", ) + aliases = aliases | { + PY_LIBRARY_PUBLIC_LABEL: label_tmpl.format(PY_LIBRARY_PUBLIC_LABEL), + WHEEL_FILE_PUBLIC_LABEL: label_tmpl.format(WHEEL_FILE_PUBLIC_LABEL), + } impl_vis = [dep_template.format( - name = "_groups", + name = "_config", target = "__pkg__", + ).replace( + "//:", + "//_groups:", )] - native.alias( - name = PY_LIBRARY_PUBLIC_LABEL, - actual = label_tmpl.format(PY_LIBRARY_PUBLIC_LABEL), - visibility = ["//visibility:public"], - ) - native.alias( - name = WHEEL_FILE_PUBLIC_LABEL, - actual = label_tmpl.format(WHEEL_FILE_PUBLIC_LABEL), - visibility = ["//visibility:public"], - ) py_library_label = PY_LIBRARY_IMPL_LABEL whl_file_label = WHEEL_FILE_IMPL_LABEL - - elif group_name: - py_library_label = PY_LIBRARY_PUBLIC_LABEL - whl_file_label = WHEEL_FILE_PUBLIC_LABEL - impl_vis = [dep_template.format(name = "", target = "__subpackages__")] - else: py_library_label = PY_LIBRARY_PUBLIC_LABEL whl_file_label = WHEEL_FILE_PUBLIC_LABEL - impl_vis = ["//visibility:public"] + if group_name: + impl_vis = [dep_template.format(name = "", target = "__subpackages__")] + else: + impl_vis = visibility + + if not requires_dist: + # If the package is in a group but has no deps, we still need the public labels to + # point at the srcs targets so that the group implementation can use them. We don't + # need any of the extra targets, so just create the aliases. + aliases = aliases | { + py_library_label: repo_label(PY_SRCS_LABEL), + whl_file_label: repo_label(WHEEL_FILE), + } + + for alias, actual in aliases.items(): + native.alias( + name = alias, + actual = actual, + visibility = visibility, + ) + + if not requires_dist: + # If there are extras, then they will be visible in requires_dist. + return + + package_deps = _parse_requires_dist( + name = metadata_name, + requires_dist = requires_dist, + excludes = group_deps, + extras = extras, + include = include, + ) + + _config_settings( + dependencies_with_markers = package_deps.deps_select, + rules = rules, + visibility = ["//visibility:private"], + ) if hasattr(native, "filegroup"): + # We include the whl file as srcs so that `$(location :whl)` expands to the whl file. + # The transitive dependencies are available via the `data` attribute. native.filegroup( name = whl_file_label, - srcs = [name], + srcs = [repo_label(WHEEL_FILE)], data = _deps( - deps = dependencies, - deps_by_platform = dependencies_by_platform, + deps = [], + package_deps = package_deps, tmpl = dep_template.format(name = "{}", target = WHEEL_FILE_PUBLIC_LABEL), - # NOTE @aignas 2024-10-28: Actually, `select` is not part of - # `native`, but in order to support bazel 6.4 in unit tests, I - # have to somehow pass the `select` implementation in the unit - # tests and I chose this to be routed through the `native` - # struct. So, tests` will be successful in `getattr` and the - # real code will use the fallback provided here. - select = getattr(native, "select", select), ), visibility = impl_vis, ) if hasattr(rules, "py_library"): - _data_exclude = [ - "**/*.py", - "**/*.pyc", - "**/*.pyc.*", # During pyc creation, temp files named *.pyc.NNNN are created - # RECORD is known to contain sha256 checksums of files which might include the checksums - # of generated files produced when wheels are installed. The file is ignored to avoid - # Bazel caching issues. - "**/*.dist-info/RECORD", - ] + glob_excludes.version_dependent_exclusions() - for item in data_exclude: - if item not in _data_exclude: - _data_exclude.append(item) - rules.py_library( name = py_library_label, - srcs = native.glob( - ["site-packages/**/*.py"], - exclude = srcs_exclude, - # Empty sources are allowed to support wheels that don't have any - # pure-Python code, e.g. pymssql, which is written in Cython. - allow_empty = True, - ), - data = data + native.glob( - ["site-packages/**/*"], - exclude = _data_exclude, - ), - # This makes this directory a top-level in the python import - # search path for anything that depends on this. - imports = ["site-packages"], + # We include as srcs to ensure that the (locations :pkg) works as expected. + srcs = [repo_label(PY_SRCS_LABEL)], deps = _deps( - deps = dependencies, - deps_by_platform = dependencies_by_platform, + # We include as deps, so that `PyInfo` and friends (e.g. `pyi_srcs`) get + # propagated. Just passing the target as `srcs` is not enough to propagate + # `pyi_srcs`, see `tests/base_rules/py_library`. + deps = [repo_label(PY_SRCS_LABEL)], + package_deps = package_deps, tmpl = dep_template.format(name = "{}", target = PY_LIBRARY_PUBLIC_LABEL), - select = getattr(native, "select", select), ), tags = tags, visibility = impl_vis, ) -def _config_settings(dependencies_by_platform, native = native, **kwargs): +def _parse_requires_dist( + *, + name, + requires_dist, + excludes, + include, + extras): + return deps( + name = normalize_name(name), + requires_dist = requires_dist, + excludes = excludes, + include = include, + extras = extras, + ) + +def _config_settings(dependencies_with_markers, rules, **kwargs): """Generate config settings for the targets. Args: - dependencies_by_platform: {type}`list[str]` platform keys, can be - one of the following formats: - * `//conditions:default` - * `@platforms//os:{value}` - * `@platforms//cpu:{value}` - * `@//python/config_settings:is_python_3.{minor_version}` - * `{os}_{cpu}` - * `cp3{minor_version}_{os}_{cpu}` - native: {type}`native` The native struct for overriding in tests. + dependencies_with_markers: {type}`dict[str, str]` The markers to evaluate by + each dep. + rules: used for testing **kwargs: Extra kwargs to pass to the rule. """ - for p in dependencies_by_platform: - if p.startswith("@") or p.endswith("default"): - continue - - abi, _, tail = p.partition("_") - if not abi.startswith("cp"): - tail = p - abi = "" - - os, _, arch = tail.partition("_") - os = "" if os == "anyos" else os - arch = "" if arch == "anyarch" else arch - - _kwargs = dict(kwargs) - if arch: - _kwargs.setdefault("constraint_values", []).append("@platforms//cpu:{}".format(arch)) - if os: - _kwargs.setdefault("constraint_values", []).append("@platforms//os:{}".format(os)) - - if abi: - _kwargs["flag_values"] = { - "@rules_python//python/config_settings:python_version_major_minor": "3.{minor_version}".format( - minor_version = abi[len("cp3"):], - ), - } - - native.config_setting( - name = "is_{name}".format( - name = p.replace("cp3", "python_3."), - ), - **_kwargs + for dep, expression in dependencies_with_markers.items(): + rules.env_marker_setting( + name = "include_{}".format(dep), + expression = expression, + **kwargs ) -def _plat_label(plat): - if plat.endswith("default"): - return plat - elif plat.startswith("@//"): - return Label(plat.strip("@")) - elif plat.startswith("@"): - return plat - else: - return ":is_" + plat.replace("cp3", "python_3.") - -def _deps(deps, deps_by_platform, tmpl, select = select): - deps = [tmpl.format(d) for d in sorted(deps)] +def _deps(deps, package_deps, tmpl): + deps = [] + deps + [tmpl.format(d) for d in sorted(package_deps.deps)] - if not deps_by_platform: - return deps + for dep in package_deps.deps_select: + deps = deps + select({ + ":is_include_{}_true".format(dep): [tmpl.format(dep)], + "//conditions:default": [], + }) - deps_by_platform = { - _plat_label(p): [ - tmpl.format(d) - for d in sorted(deps) - ] - for p, deps in sorted(deps_by_platform.items()) - } - - # Add the default, which means that we will be just using the dependencies in - # `deps` for platforms that are not handled in a special way by the packages - deps_by_platform.setdefault("//conditions:default", []) - - if not deps: - return select(deps_by_platform) - else: - return deps + select(deps_by_platform) + return deps diff --git a/python/private/pypi/whl_metadata.bzl b/python/private/pypi/whl_metadata.bzl new file mode 100644 index 0000000000..2981a5d92f --- /dev/null +++ b/python/private/pypi/whl_metadata.bzl @@ -0,0 +1,168 @@ +"""A simple function to find the METADATA file and parse it""" + +_NAME = "Name: " +_PROVIDES_EXTRA = "Provides-Extra: " +_REQUIRES_DIST = "Requires-Dist: " +_VERSION = "Version: " + +def whl_metadata(*, install_dir, read_fn, logger): + """Find and parse the METADATA file in the extracted whl contents dir. + + Args: + install_dir: {type}`path` location where the wheel has been extracted. + read_fn: the function used to read files. + logger: the function used to log failures. + + Returns: + A struct with parsed values: + * `name`: {type}`str` the name of the wheel. + * `version`: {type}`str` the version of the wheel. + * `requires_dist`: {type}`list[str]` the list of requirements. + * `provides_extra`: {type}`list[str]` the list of extras that this package + provides. + """ + metadata_file = find_whl_metadata(install_dir = install_dir, logger = logger) + contents = read_fn(metadata_file) + + result = parse_whl_metadata(contents) + + if not (result.name and result.version): + logger.fail("Failed to parse the wheel METADATA file:\n{}\n{}\n{}".format( + 80 * "=", + contents.rstrip("\n"), + 80 * "=", + )) + return None + + return result + +def parse_whl_metadata(contents): + """Parse .whl METADATA file + + Args: + contents: {type}`str` the contents of the file. + + Returns: + A struct with parsed values: + * `name`: {type}`str` the name of the wheel. + * `version`: {type}`str` the version of the wheel. + * `requires_dist`: {type}`list[str]` the list of requirements. + * `provides_extra`: {type}`list[str]` the list of extras that this package + provides. + """ + parsed = { + "name": "", + "provides_extra": [], + "requires_dist": [], + "version": "", + } + for line in contents.strip().split("\n"): + if not line: + # Stop parsing on first empty line, which marks the end of the + # headers containing the metadata. + break + + if line.startswith(_NAME): + _, _, value = line.partition(_NAME) + parsed["name"] = value.strip() + elif line.startswith(_VERSION): + _, _, value = line.partition(_VERSION) + parsed["version"] = value.strip() + elif line.startswith(_REQUIRES_DIST): + _, _, value = line.partition(_REQUIRES_DIST) + parsed["requires_dist"].append(value.strip(" ")) + elif line.startswith(_PROVIDES_EXTRA): + _, _, value = line.partition(_PROVIDES_EXTRA) + parsed["provides_extra"].append(value.strip(" ")) + + return struct( + name = parsed["name"], + provides_extra = parsed["provides_extra"], + requires_dist = parsed["requires_dist"], + version = parsed["version"], + ) + +def find_whl_metadata(*, install_dir, logger): + """Find the whl METADATA file in the install_dir. + + Args: + install_dir: {type}`path` location where the wheel has been extracted. + logger: the function used to log failures. + + Returns: + {type}`path` The path to the METADATA file. + """ + dist_info = None + for maybe_dist_info in install_dir.readdir(): + # first find the ".dist-info" folder + if not (maybe_dist_info.is_dir and maybe_dist_info.basename.endswith(".dist-info")): + continue + + dist_info = maybe_dist_info + metadata_file = dist_info.get_child("METADATA") + + if metadata_file.exists: + return metadata_file + + break + + if dist_info: + logger.fail("The METADATA file for the wheel could not be found in '{}/{}'".format(install_dir.basename, dist_info.basename)) + else: + logger.fail("The '*.dist-info' directory could not be found in '{}'".format(install_dir.basename)) + return None + +def parse_entry_points(contents): + """Parses entry_points.txt contents and returns console_scripts and gui_scripts entries. + + Args: + contents: {type}`str` The contents of the entry_points.txt file. + + Returns: + {type}`dict[str, dict]` A dict keyed by the original entry point name. + """ + entries = {} + seen_lower_names = {} + current_group = None + current_group_lower = None + for line in contents.splitlines(): + line = line.strip() + if not line or line.startswith("#"): + continue + if line.startswith("[") and line.endswith("]"): + current_group = line[1:-1].strip() + current_group_lower = current_group.lower() + continue + + if current_group_lower in ("console_scripts", "gui_scripts"): + name, _, ref = line.partition("=") + name = name.strip() + + # Names are case-insensitive. + # See https://packaging.python.org/en/latest/specifications/entry-points/#data-model + # Entry points must be unique for a given name because they turn + # into files and may be on a case-insensitive file system. + lower_name = name.lower() + if lower_name in seen_lower_names: + continue + seen_lower_names[lower_name] = True + + # remove inline comments + ref, _, _ = ref.partition("#") + ref = ref.strip() + + extras = "" + if "[" in ref and ref.endswith("]"): + ref, _, extras_part = ref.partition("[") + extras = extras_part[:-1].strip() + ref = ref.strip() + + module, _, attribute = ref.partition(":") + entries[name] = { + "attribute": attribute.strip(), + "extras": extras, + "group": current_group, + "module": module.strip(), + "name": name, + } + return entries diff --git a/python/private/pypi/whl_repo_name.bzl b/python/private/pypi/whl_repo_name.bzl index 38ed600cd1..739ff2a4a1 100644 --- a/python/private/pypi/whl_repo_name.bzl +++ b/python/private/pypi/whl_repo_name.bzl @@ -18,26 +18,41 @@ load("//python/private:normalize_name.bzl", "normalize_name") load(":parse_whl_name.bzl", "parse_whl_name") -def whl_repo_name(prefix, filename, sha256): +def whl_repo_name(filename, digest, *target_platforms): """Return a valid whl_library repo name given a distribution filename. Args: - prefix: {type}`str` the prefix of the whl_library. filename: {type}`str` the filename of the distribution. - sha256: {type}`str` the sha256 of the distribution. + digest: {type}`str` the digest of the distribution, either as a + canonical `:` string or a bare hex digest. + *target_platforms: {type}`list[str]` the extra suffixes to append. + Only used when we need to support different extras per version. Returns: a string that can be used in {obj}`whl_library`. """ - parts = [prefix] + + # Strip the `:` prefix so that the name only contains the hex digest + # and stays the same as it has always been for bare sha256 values. + digest = digest.rpartition(":")[2] + + parts = [] if not filename.endswith(".whl"): # Then the filename is basically foo-3.2.1. - parts.append(normalize_name(filename.rpartition("-")[0])) - parts.append("sdist") + name, _, tail = filename.rpartition("-") + parts.append(normalize_name(name)) + if digest: + parts.append("sdist") + version = "" + else: + for ext in [".tar", ".zip"]: + tail, _, _ = tail.partition(ext) + version = tail.replace(".", "_").replace("!", "_") else: parsed = parse_whl_name(filename) name = normalize_name(parsed.distribution) + version = parsed.version.replace(".", "_").replace("!", "_").replace("+", "_").replace("%", "_") python_tag, _, _ = parsed.python_tag.partition(".") abi_tag, _, _ = parsed.abi_tag.partition(".") platform_tag, _, _ = parsed.platform_tag.partition(".") @@ -47,15 +62,19 @@ def whl_repo_name(prefix, filename, sha256): parts.append(abi_tag) parts.append(platform_tag) - parts.append(sha256[:8]) + if digest: + parts.append(digest[:8]) + elif version: + parts.insert(1, version) + + parts.extend([p.partition("_")[-1] for p in target_platforms]) return "_".join(parts) -def pypi_repo_name(prefix, whl_name, *target_platforms): +def pypi_repo_name(whl_name, *target_platforms): """Return a valid whl_library given a requirement line. Args: - prefix: {type}`str` the prefix of the whl_library. whl_name: {type}`str` the whl_name to use. *target_platforms: {type}`list[str]` the target platforms to use in the name. @@ -63,7 +82,6 @@ def pypi_repo_name(prefix, whl_name, *target_platforms): {type}`str` that can be used in {obj}`whl_library`. """ parts = [ - prefix, normalize_name(whl_name), ] parts.extend([p.partition("_")[-1] for p in target_platforms]) diff --git a/python/private/pypi/whl_target_platforms.bzl b/python/private/pypi/whl_target_platforms.bzl index bdc44c697a..28547c679c 100644 --- a/python/private/pypi/whl_target_platforms.bzl +++ b/python/private/pypi/whl_target_platforms.bzl @@ -16,8 +16,6 @@ A starlark implementation of the wheel platform tag parsing to get the target platform. """ -load(":parse_whl_name.bzl", "parse_whl_name") - # The order of the dictionaries is to keep definitions with their aliases next to each # other _CPU_ALIASES = { @@ -31,7 +29,8 @@ _CPU_ALIASES = { "arm64": "aarch64", "ppc": "ppc", "ppc64": "ppc", - "ppc64le": "ppc", + "ppc64le": "ppc64le", + "riscv64": "riscv64", "s390x": "s390x", "arm": "arm", "armv6l": "arm", @@ -46,129 +45,6 @@ _OS_PREFIXES = { "win": "windows", } # buildifier: disable=unsorted-dict-items -def select_whls(*, whls, want_platforms = [], logger = None): - """Select a subset of wheels suitable for target platforms from a list. - - Args: - whls(list[struct]): A list of candidates which have a `filename` - attribute containing the `whl` filename. - want_platforms(str): The platforms in "{abi}_{os}_{cpu}" or "{os}_{cpu}" format. - logger: A logger for printing diagnostic messages. - - Returns: - A filtered list of items from the `whls` arg where `filename` matches - the selected criteria. If no match is found, an empty list is returned. - """ - if not whls: - return [] - - want_abis = { - "abi3": None, - "none": None, - } - - _want_platforms = {} - version_limit = None - - for p in want_platforms: - if not p.startswith("cp3"): - fail("expected all platforms to start with ABI, but got: {}".format(p)) - - abi, _, os_cpu = p.partition("_") - _want_platforms[os_cpu] = None - _want_platforms[p] = None - - version_limit_candidate = int(abi[3:]) - if not version_limit: - version_limit = version_limit_candidate - if version_limit and version_limit != version_limit_candidate: - fail("Only a single python version is supported for now") - - # For some legacy implementations the wheels may target the `cp3xm` ABI - _want_platforms["{}m_{}".format(abi, os_cpu)] = None - want_abis[abi] = None - want_abis[abi + "m"] = None - - want_platforms = sorted(_want_platforms) - - candidates = {} - for whl in whls: - parsed = parse_whl_name(whl.filename) - - if logger: - logger.trace(lambda: "Deciding whether to use '{}'".format(whl.filename)) - - supported_implementations = {} - whl_version_min = 0 - for tag in parsed.python_tag.split("."): - supported_implementations[tag[:2]] = None - - if tag.startswith("cp3") or tag.startswith("py3"): - version = int(tag[len("..3"):] or 0) - else: - # In this case it should be eithor "cp2" or "py2" and we will default - # to `whl_version_min` = 0 - continue - - if whl_version_min == 0 or version < whl_version_min: - whl_version_min = version - - if not ("cp" in supported_implementations or "py" in supported_implementations): - if logger: - logger.trace(lambda: "Discarding the whl because the whl does not support CPython, whl supported implementations are: {}".format(supported_implementations)) - continue - - if want_abis and parsed.abi_tag not in want_abis: - # Filter out incompatible ABIs - if logger: - logger.trace(lambda: "Discarding the whl because the whl abi did not match") - continue - - if whl_version_min > version_limit: - if logger: - logger.trace(lambda: "Discarding the whl because the whl supported python version is too high") - continue - - compatible = False - if parsed.platform_tag == "any": - compatible = True - else: - for p in whl_target_platforms(parsed.platform_tag, abi_tag = parsed.abi_tag.strip("m") if parsed.abi_tag.startswith("cp") else None): - if p.target_platform in want_platforms: - compatible = True - break - - if not compatible: - if logger: - logger.trace(lambda: "Discarding the whl because the whl does not support the desired platforms: {}".format(want_platforms)) - continue - - for implementation in supported_implementations: - candidates.setdefault( - ( - parsed.abi_tag, - parsed.platform_tag, - ), - {}, - ).setdefault( - ( - # prefer cp implementation - implementation == "cp", - # prefer higher versions - whl_version_min, - # prefer abi3 over none - parsed.abi_tag != "none", - # prefer cpx abi over abi3 - parsed.abi_tag != "abi3", - ), - [], - ).append(whl) - - return [ - candidates[key][sorted(v)[-1]][-1] - for key, v in candidates.items() - ] - def whl_target_platforms(platform_tag, abi_tag = ""): """Parse the wheel abi and platform tags and return (os, cpu) tuples. diff --git a/python/private/pyproject_utils.bzl b/python/private/pyproject_utils.bzl new file mode 100644 index 0000000000..e96ffd2023 --- /dev/null +++ b/python/private/pyproject_utils.bzl @@ -0,0 +1,51 @@ +"""Utilities for reading values from pyproject.toml.""" + +load("@toml.bzl", "toml") +load(":version.bzl", "version") + +def read_pyproject(module_ctx, pyproject): + """Read a pyproject.toml file and return the relevant fields. + + The file is parsed with a pure-Starlark TOML decoder; no Python + interpreter is required. The raw `requires-python` value is returned + as-is so that callers can decide how to interpret it. + + Args: + module_ctx: the module extension context (needs `path` and `read`). + pyproject: {type}`Label` pointing at the pyproject.toml file. + + Returns: + {type}`struct` with the attributes: + * `requires_python`: {type}`str | None` the raw `requires-python` + value (e.g. `"==3.13.9"`), or `None` if it is not set. + """ + data = toml.decode(module_ctx.read(module_ctx.path(pyproject), watch = "yes")) + return struct( + requires_python = data.get("project", {}).get("requires-python"), + ) + +def version_from_requires_python(requires_python): + """Derive a concrete Python version from a `requires-python` value. + + Currently only an exact `==X.Y.Z` specifier is supported. The value is + validated and normalized via {obj}`//python/private:version.bzl` so that + malformed input fails in a consistent way. Broader specifier support + (e.g. `>=`, `X.Y`) can be layered on here in the future. + + Args: + requires_python: {type}`str` the raw `requires-python` value. + + Returns: + {type}`str` the normalized version string (e.g. `"3.13.9"`). + """ + if not requires_python: + fail("`requires-python` must be specified") + + if not requires_python.startswith("=="): + fail("`requires-python` must pin an exact version with `==`, got: {}".format(requires_python)) + + bare_version = requires_python[len("=="):].strip() + + # Parse strictly so malformed versions fail cleanly, then normalize. + version.parse(bare_version, strict = True) + return version.normalize(bare_version) diff --git a/python/private/python.bzl b/python/private/python.bzl index ec6f73e41f..8932d00e1b 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -15,39 +15,50 @@ "Python toolchain module extensions for use with bzlmod." load("@bazel_features//:features.bzl", "bazel_features") -load("//python:versions.bzl", "DEFAULT_RELEASE_BASE_URL", "PLATFORMS", "TOOL_VERSIONS") +load("//python:versions.bzl", "DEFAULT_RELEASE_BASE_URLS", "PLATFORMS") load(":auth.bzl", "AUTH_ATTRS") load(":full_version.bzl", "full_version") +load(":pbs_manifest.bzl", "parse_runtime_manifest") +load(":platform_info.bzl", "platform_info") +load(":pyproject_utils.bzl", "read_pyproject", "version_from_requires_python") load(":python_register_toolchains.bzl", "python_register_toolchains") load(":pythons_hub.bzl", "hub_repo") load(":repo_utils.bzl", "repo_utils") -load(":semver.bzl", "semver") -load(":text_util.bzl", "render") -load(":toolchains_repo.bzl", "multi_toolchain_aliases") -load(":util.bzl", "IS_BAZEL_6_4_OR_HIGHER") - -# This limit can be increased essentially arbitrarily, but doing so will cause a rebuild of all -# targets using any of these toolchains due to the changed repository name. -_MAX_NUM_TOOLCHAINS = 9999 -_TOOLCHAIN_INDEX_PAD_LENGTH = len(str(_MAX_NUM_TOOLCHAINS)) +load( + ":toolchains_repo.bzl", + "host_compatible_python_repo", + "multi_toolchain_aliases", + "sorted_host_platform_names", + "sorted_host_platforms", +) +load(":version.bzl", "version") -def parse_modules(*, module_ctx, _fail = fail): +def parse_modules(*, module_ctx, logger = None, _fail = fail): """Parse the modules and return a struct for registrations. Args: module_ctx: {type}`module_ctx` module context. + logger: {type}`repo_utils.logger` A logger to use. _fail: {type}`function` the failure function, mainly for testing. Returns: A struct with the following attributes: - * `toolchains`: The list of toolchains to register. The last - element is special and is treated as the default toolchain. - * `defaults`: The default `kwargs` passed to - {bzl:obj}`python_register_toolchains`. - * `debug_info`: {type}`None | dict` extra information to be passed - to the debug repo. + * `toolchains`: {type}`list[ToolchainConfig]` The list of toolchains to + register. The last element is special and is treated as the default + toolchain. + * `config`: Various toolchain config, see `_get_toolchain_config`. + * `debug_info`: {type}`None | dict` extra information to be passed + to the debug repo. + * `platforms`: {type}`dict[str, platform_info]` of the base set of + platforms toolchains should be created for, if possible. + + ToolchainConfig struct: + * python_version: str, full python version string + * name: str, the base toolchain name, e.g., "python_3_10", no + platform suffix. + * register_coverage_tool: bool """ - if module_ctx.os.environ.get("RULES_PYTHON_BZLMOD_DEBUG", "0") == "1": + if module_ctx.getenv("RULES_PYTHON_BZLMOD_DEBUG", "0") == "1": debug_info = { "toolchains_registered": [], } @@ -67,16 +78,9 @@ def parse_modules(*, module_ctx, _fail = fail): # Map of string Major.Minor or Major.Minor.Patch to the toolchain_info struct global_toolchain_versions = {} - ignore_root_user_error = None - - logger = repo_utils.logger(module_ctx, "python") - - # if the root module does not register any toolchain then the - # ignore_root_user_error takes its default value: False - if not module_ctx.modules[0].tags.toolchain: - ignore_root_user_error = False + config = _get_toolchain_config(mctx = module_ctx, modules = module_ctx.modules, _fail = _fail) - config = _get_toolchain_config(modules = module_ctx.modules, _fail = _fail) + default_python_version = _compute_default_python_version(module_ctx) seen_versions = {} for mod in module_ctx.modules: @@ -85,6 +89,7 @@ def parse_modules(*, module_ctx, _fail = fail): mod = mod, seen_versions = seen_versions, config = config, + default_python_version = default_python_version, ) for toolchain_attr in toolchain_attr_structs: @@ -104,26 +109,17 @@ def parse_modules(*, module_ctx, _fail = fail): # * rules_python needs to set a soft default in case the root module doesn't, # e.g. if the root module doesn't use Python itself. # * The root module is allowed to override the rules_python default. - is_default = toolchain_attr.is_default + is_default = default_python_version == toolchain_version - # Also only the root module should be able to decide ignore_root_user_error. - # Modules being depended upon don't know the final environment, so they aren't - # in the right position to know or decide what the correct setting is. - - # If an inconsistency in the ignore_root_user_error among multiple toolchains is detected, fail. - if ignore_root_user_error != None and toolchain_attr.ignore_root_user_error != ignore_root_user_error: - fail("Toolchains in the root module must have consistent 'ignore_root_user_error' attributes") - - ignore_root_user_error = toolchain_attr.ignore_root_user_error elif mod.name == "rules_python" and not default_toolchain: - # We don't do the len() check because we want the default that rules_python - # sets to be clearly visible. - is_default = toolchain_attr.is_default + # This branch handles when the root module doesn't declare a + # Python toolchain + is_default = default_python_version == toolchain_version else: is_default = False if is_default and default_toolchain != None: - _fail_multiple_default_toolchains( + _fail_multiple_default_toolchains_chosen( first = default_toolchain.name, second = toolchain_name, ) @@ -144,7 +140,7 @@ def parse_modules(*, module_ctx, _fail = fail): first = first, second_toolchain_name = toolchain_name, second_module_name = mod.name, - logger = logger, + logger = logger or repo_utils.logger(module_ctx, "python", mod = mod), ) toolchain_info = None else: @@ -157,7 +153,6 @@ def parse_modules(*, module_ctx, _fail = fail): global_toolchain_versions[toolchain_version] = toolchain_info if debug_info: debug_info["toolchains_registered"].append({ - "ignore_root_user_error": ignore_root_user_error, "module": {"is_root": mod.is_root, "name": mod.name}, "name": toolchain_name, }) @@ -176,12 +171,10 @@ def parse_modules(*, module_ctx, _fail = fail): elif toolchain_info: toolchains.append(toolchain_info) - config.default.setdefault("ignore_root_user_error", ignore_root_user_error) - # A default toolchain is required so that the non-version-specific rules # are able to match a toolchain. if default_toolchain == None: - fail("No default Python toolchain configured. Is rules_python missing `is_default=True`?") + fail("No default Python toolchain configured. Is rules_python missing `python.defaults()`?") elif default_toolchain.python_version not in global_toolchain_versions: fail('Default version "{python_version}" selected by module ' + '"{module_name}", but no toolchain with that version registered'.format( @@ -193,13 +186,25 @@ def parse_modules(*, module_ctx, _fail = fail): # toolchain. We need the default last. toolchains.append(default_toolchain) - if len(toolchains) > _MAX_NUM_TOOLCHAINS: - fail("more than {} python versions are not supported".format(_MAX_NUM_TOOLCHAINS)) + # sort the toolchains so that the toolchain versions that are in the + # `minor_mapping` are coming first. This ensures that `python_version = + # "3.X"` transitions work as expected. + minor_version_toolchains = [] + other_toolchains = [] + minor_mapping = list(config.minor_mapping.values()) + for t in toolchains: + # FIXME @aignas 2025-04-04: How can we unit test that this ordering is + # consistent with what would actually work? + if config.minor_mapping.get(t.python_version, t.python_version) in minor_mapping: + minor_version_toolchains.append(t) + else: + other_toolchains.append(t) + toolchains = minor_version_toolchains + other_toolchains return struct( config = config, debug_info = debug_info, - default_python_version = toolchains[-1].python_version, + default_python_version = default_toolchain.python_version, toolchains = [ struct( python_version = t.python_version, @@ -213,13 +218,54 @@ def parse_modules(*, module_ctx, _fail = fail): def _python_impl(module_ctx): py = parse_modules(module_ctx = module_ctx) - loaded_platforms = {} - for toolchain_info in py.toolchains: + # For all other processing (after parsing the modules) let's use a single logger. + logger = repo_utils.logger(module_ctx, "python", mod = module_ctx.modules[0]) + + # Host compatible runtime repos + # dict[str version, struct] where struct has: + # * full_python_version: str + # * platform: platform_info struct + # * platform_name: str platform name + # * impl_repo_name: str repo name of the runtime's python_repository() repo + all_host_compatible_impls = {} + + # Host compatible repos that still need to be created because, when + # creating the actual runtime repo, there wasn't a host-compatible + # variant defined for it. + # dict[str reponame, struct] where struct has: + # * compatible_version: str, e.g. 3.10 or 3.10.1. The version the host + # repo should be compatible with + # * full_python_version: str, e.g. 3.10.1, the full python version of + # the toolchain that still needs a host repo created. + needed_host_repos = {} + + # list of structs; see inline struct call within the loop below. + toolchain_impls = [] + + # list[str] of the repo names for host compatible repos + all_host_compatible_repo_names = [] + + # Create the underlying python_repository repos that contain the + # python runtimes and their toolchain implementation definitions. + for i, toolchain_info in enumerate(py.toolchains): + is_last = (i + 1) == len(py.toolchains) + # Ensure that we pass the full version here. full_python_version = full_version( version = toolchain_info.python_version, minor_mapping = py.config.minor_mapping, + fail_on_err = False, ) + if not full_python_version: + logger.info(lambda: ( + "The actual toolchain for python_version '{version}' " + + "has not been registered, but was requested, please configure a toolchain " + + "to be actually downloaded and setup" + ).format( + version = toolchain_info.python_version, + )) + continue + kwargs = { "python_version": full_python_version, "register_coverage_tool": toolchain_info.register_coverage_tool, @@ -229,36 +275,176 @@ def _python_impl(module_ctx): kwargs.update(py.config.kwargs.get(toolchain_info.python_version, {})) kwargs.update(py.config.kwargs.get(full_python_version, {})) kwargs.update(py.config.default) - loaded_platforms[full_python_version] = python_register_toolchains( + register_result = python_register_toolchains( name = toolchain_info.name, _internal_bzlmod_toolchain_call = True, **kwargs ) + if not register_result.impl_repos: + continue + + host_platforms = {} + for repo_name, (platform_name, platform_info) in register_result.impl_repos.items(): + toolchain_impls.append(struct( + # str: The base name to use for the toolchain() target + name = repo_name, + # str: The repo name the toolchain() target points to. + impl_repo_name = repo_name, + # str: platform key in the passed-in platforms dict + platform_name = platform_name, + # struct: platform_info() struct + platform = platform_info, + # str: Major.Minor.Micro python version + full_python_version = full_python_version, + # bool: whether to implicitly add the python version constraint + # to the toolchain's target_settings. + # The last toolchain is the default; it can't have version constraints + set_python_version_constraint = is_last, + )) + if _is_compatible_with_host(module_ctx, platform_info): + host_compat_entry = struct( + full_python_version = full_python_version, + platform = platform_info, + platform_name = platform_name, + impl_repo_name = repo_name, + ) + host_platforms[platform_name] = host_compat_entry + all_host_compatible_impls.setdefault(full_python_version, []).append( + host_compat_entry, + ) + parsed_version = version.parse(full_python_version) + all_host_compatible_impls.setdefault( + "{}.{}".format(*parsed_version.release[0:2]), + [], + ).append(host_compat_entry) + + host_repo_name = toolchain_info.name + "_host" + if host_platforms: + all_host_compatible_repo_names.append(host_repo_name) + host_platforms = sorted_host_platforms(host_platforms) + entries = host_platforms.values() + host_compatible_python_repo( + name = host_repo_name, + base_name = host_repo_name, + # NOTE: Order matters. The first found to be compatible is + # (usually) used. + platforms = host_platforms.keys(), + os_names = {str(i): e.platform.os_name for i, e in enumerate(entries)}, + arch_names = {str(i): e.platform.arch for i, e in enumerate(entries)}, + python_versions = {str(i): e.full_python_version for i, e in enumerate(entries)}, + impl_repo_names = {str(i): e.impl_repo_name for i, e in enumerate(entries)}, + ) + else: + needed_host_repos[host_repo_name] = struct( + compatible_version = toolchain_info.python_version, + full_python_version = full_python_version, + ) + + if needed_host_repos: + for key, entries in all_host_compatible_impls.items(): + all_host_compatible_impls[key] = sorted( + entries, + reverse = True, + key = lambda e: version.key(version.parse(e.full_python_version)), + ) + + for host_repo_name, info in needed_host_repos.items(): + choices = [] + if info.compatible_version not in all_host_compatible_impls: + logger.warn("No host compatible runtime found compatible with version {}".format(info.compatible_version)) + continue + + choices = all_host_compatible_impls[info.compatible_version] + platform_keys = [ + # We have to prepend the offset because the same platform + # name might occur across different versions + "{}_{}".format(i, entry.platform_name) + for i, entry in enumerate(choices) + ] + platform_keys = sorted_host_platform_names(platform_keys) + + all_host_compatible_repo_names.append(host_repo_name) + host_compatible_python_repo( + name = host_repo_name, + base_name = host_repo_name, + platforms = platform_keys, + impl_repo_names = { + str(i): entry.impl_repo_name + for i, entry in enumerate(choices) + }, + os_names = {str(i): entry.platform.os_name for i, entry in enumerate(choices)}, + arch_names = {str(i): entry.platform.arch for i, entry in enumerate(choices)}, + python_versions = {str(i): entry.full_python_version for i, entry in enumerate(choices)}, + ) + + # list[str] The infix to use for the resulting toolchain() `name` arg. + toolchain_names = [] + + # dict[str i, str repo]; where repo is the full repo name + # ("python_3_10_unknown-linux-x86_64") for the toolchain + # i corresponds to index `i` in toolchain_names + toolchain_repo_names = {} + + # dict[str i, list[str] constraints]; where constraints is a list + # of labels for target_compatible_with + # i corresponds to index `i` in toolchain_names + toolchain_tcw_map = {} + + # dict[str i, list[str] settings]; where settings is a list + # of labels for target_settings + # i corresponds to index `i` in toolchain_names + toolchain_ts_map = {} + + # dict[str i, str set_constraint]; where set_constraint is the string + # "True" or "False". + # i corresponds to index `i` in toolchain_names + toolchain_set_python_version_constraints = {} + + # dict[str i, str python_version]; where python_version is the full + # python version ("3.4.5"). + toolchain_python_versions = {} + + # dict[str i, str platform_key]; where platform_key is the key within + # the PLATFORMS global for this toolchain + toolchain_platform_keys = {} + + # Extra target_settings to add to every registered toolchain, e.g. for + # gating the default toolchains behind a custom config_setting. + global_add_target_settings = py.config.add_target_settings + + # Split the toolchain info into separate objects so they can be passed onto + # the repository rule. + for entry in toolchain_impls: + key = str(len(toolchain_names)) + + toolchain_names.append(entry.name) + toolchain_repo_names[key] = entry.impl_repo_name + toolchain_tcw_map[key] = entry.platform.compatible_with + + # The target_settings attribute may not be present for users + # patching python/versions.bzl. + toolchain_ts_map[key] = getattr(entry.platform, "target_settings", []) + global_add_target_settings + toolchain_platform_keys[key] = entry.platform_name + toolchain_python_versions[key] = entry.full_python_version + + # Repo rules can't accept dict[str, bool], so encode them as a string value. + toolchain_set_python_version_constraints[key] = ( + "True" if entry.set_python_version_constraint else "False" + ) - # Create the pythons_hub repo for the interpreter meta data and the - # the various toolchains. hub_repo( name = "pythons_hub", - # Last toolchain is default + toolchain_names = toolchain_names, + toolchain_repo_names = toolchain_repo_names, + toolchain_target_compatible_with_map = toolchain_tcw_map, + toolchain_target_settings_map = toolchain_ts_map, + toolchain_platform_keys = toolchain_platform_keys, + toolchain_python_versions = toolchain_python_versions, + toolchain_set_python_version_constraints = toolchain_set_python_version_constraints, + host_compatible_repo_names = sorted(all_host_compatible_repo_names), default_python_version = py.default_python_version, minor_mapping = py.config.minor_mapping, python_versions = list(py.config.default["tool_versions"].keys()), - toolchain_prefixes = [ - render.toolchain_prefix(index, toolchain.name, _TOOLCHAIN_INDEX_PAD_LENGTH) - for index, toolchain in enumerate(py.toolchains) - ], - toolchain_python_versions = [ - full_version(version = t.python_version, minor_mapping = py.config.minor_mapping) - for t in py.toolchains - ], - # The last toolchain is the default; it can't have version constraints - # Despite the implication of the arg name, the values are strs, not bools - toolchain_set_python_version_constraints = [ - "True" if i != len(py.toolchains) - 1 else "False" - for i in range(len(py.toolchains)) - ], - toolchain_user_repository_names = [t.name for t in py.toolchains], - loaded_platforms = loaded_platforms, ) # This is require in order to support multiple version py_test @@ -282,6 +468,24 @@ def _python_impl(module_ctx): else: return None +def _is_compatible_with_host(mctx, platform_info): + os_name = repo_utils.get_platforms_os_name(mctx) + cpu_name = repo_utils.get_platforms_cpu_name(mctx) + return platform_info.os_name == os_name and platform_info.arch == cpu_name + +def _one_or_the_same(first, second, *, onerror = None): + if not first: + return second + if not second or second == first: + return first + if onerror: + return onerror(first, second) + else: + fail("Unique value needed, got both '{}' and '{}', which are different".format( + first, + second, + )) + def _fail_duplicate_module_toolchain_version(version, module): fail(("Duplicate module toolchain version: module '{module}' attempted " + "to use version '{version}' multiple times in itself").format( @@ -305,24 +509,62 @@ def _warn_duplicate_global_toolchain_version(version, first, second_toolchain_na version = version, )) -def _fail_multiple_default_toolchains(first, second): +def _fail_multiple_defaults_python_version(first, second): + fail(("Multiple python_version entries in defaults: " + + "First default was python_version '{first}'. " + + "Second was python_version '{second}'").format( + first = first, + second = second, + )) + +def _fail_multiple_defaults_python_version_file(first, second): + fail(("Multiple python_version_file entries in defaults: " + + "First default was python_version_file '{first}'. " + + "Second was python_version_file '{second}'").format( + first = first, + second = second, + )) + +def _fail_multiple_defaults_python_version_env(first, second): + fail(("Multiple python_version_env entries in defaults: " + + "First default was python_version_env '{first}'. " + + "Second was python_version_env '{second}'").format( + first = first, + second = second, + )) + +def _fail_multiple_default_toolchains_chosen(first, second): fail(("Multiple default toolchains: only one toolchain " + - "can have is_default=True. First default " + + "can be chosen as a default. First default " + "was toolchain '{first}'. Second was '{second}'").format( first = first, second = second, )) -def _validate_version(*, version, _fail = fail): - parsed = semver(version) - if parsed.patch == None or parsed.build or parsed.pre_release: - _fail("The 'python_version' attribute needs to specify an 'X.Y.Z' semver-compatible version, got: '{}'".format(version)) +def _fail_multiple_default_toolchains_in_module(mod, toolchain_attrs): + fail(("Multiple default toolchains: only one toolchain " + + "can have is_default=True.\n" + + "Module '{module}' contains {count} toolchains with " + + "is_default=True: {versions}").format( + module = mod.name, + count = len(toolchain_attrs), + versions = ", ".join(sorted([v.python_version for v in toolchain_attrs])), + )) + +def _validate_version(version_str, *, _fail = fail): + v = version.parse(version_str, strict = True, _fail = _fail) + if v == None: + # Only reachable in tests + return False + + if len(v.release) < 3: + _fail("The 'python_version' attribute needs to specify the full version in at least 'X.Y.Z' format, got: '{}'".format(v.string)) return False return True def _process_single_version_overrides(*, tag, _fail = fail, default): - if not _validate_version(version = tag.python_version, _fail = _fail): + if not _validate_version(tag.python_version, _fail = _fail): return available_versions = default["tool_versions"] @@ -334,9 +576,9 @@ def _process_single_version_overrides(*, tag, _fail = fail, default): return for platform in (tag.sha256 or []): - if platform not in PLATFORMS: + if platform not in default["platforms"]: _fail("The platform must be one of {allowed} but got '{got}'".format( - allowed = sorted(PLATFORMS), + allowed = sorted(default["platforms"]), got = platform, )) return @@ -372,7 +614,7 @@ def _process_single_version_overrides(*, tag, _fail = fail, default): kwargs.setdefault(tag.python_version, {})["distutils"] = tag.distutils def _process_single_version_platform_overrides(*, tag, _fail = fail, default): - if not _validate_version(version = tag.python_version, _fail = _fail): + if not _validate_version(tag.python_version, _fail = _fail): return available_versions = default["tool_versions"] @@ -393,9 +635,56 @@ def _process_single_version_platform_overrides(*, tag, _fail = fail, default): available_versions[tag.python_version].setdefault("sha256", {})[tag.platform] = tag.sha256 if tag.strip_prefix: available_versions[tag.python_version].setdefault("strip_prefix", {})[tag.platform] = tag.strip_prefix + if tag.urls: available_versions[tag.python_version].setdefault("url", {})[tag.platform] = tag.urls + # If platform is customized, or doesn't exist, (re)define one. + if ((tag.target_compatible_with or tag.target_settings or tag.os_name or tag.arch) or + tag.platform not in default["platforms"]): + os_name = tag.os_name + arch = tag.arch + + if not tag.target_compatible_with: + target_compatible_with = [] + if os_name: + target_compatible_with.append("@platforms//os:{}".format( + repo_utils.get_platforms_os_name(os_name), + )) + if arch: + target_compatible_with.append("@platforms//cpu:{}".format( + repo_utils.get_platforms_cpu_name(arch), + )) + else: + target_compatible_with = tag.target_compatible_with + + # For lack of a better option, give a bogus value. It only affects + # if the runtime is considered host-compatible. + if not os_name: + os_name = "UNKNOWN_CUSTOM_OS" + if not arch: + arch = "UNKNOWN_CUSTOM_ARCH" + + # Move the override earlier in the ordering -- the platform key ordering + # becomes the toolchain ordering within the version. This allows the + # override to have a superset of constraints from a regular runtimes + # (e.g. same platform, but with a custom flag required). + override_first = { + tag.platform: platform_info( + compatible_with = target_compatible_with, + target_settings = tag.target_settings, + os_name = os_name, + arch = arch, + ), + } + for key, value in default["platforms"].items(): + # Don't replace our override with the old value + if key in override_first: + continue + override_first[key] = value + + default["platforms"] = override_first + def _process_global_overrides(*, tag, default, _fail = fail): if tag.available_python_versions: available_versions = default["tool_versions"] @@ -413,18 +702,20 @@ def _process_global_overrides(*, tag, default, _fail = fail): if tag.minor_mapping: for minor_version, full_version in tag.minor_mapping.items(): - parsed = semver(minor_version) - if parsed.patch != None or parsed.build or parsed.pre_release: - fail("Expected the key to be of `X.Y` format but got `{}`".format(minor_version)) - parsed = semver(full_version) - if parsed.patch == None: - fail("Expected the value to at least be of `X.Y.Z` format but got `{}`".format(minor_version)) + parsed = version.parse(minor_version, strict = True, _fail = _fail) + if len(parsed.release) > 2 or parsed.pre or parsed.post or parsed.dev or parsed.local: + fail("Expected the key to be of `X.Y` format but got `{}`".format(parsed.string)) + + # Ensure that the version is valid + version.parse(full_version, strict = True, _fail = _fail) default["minor_mapping"] = tag.minor_mapping + if tag.add_target_settings: + default["add_target_settings"] = list(tag.add_target_settings) + forwarded_attrs = sorted(AUTH_ATTRS) + [ - "ignore_root_user_error", - "base_url", + "base_urls", "register_all_versions", ] for key in forwarded_attrs: @@ -452,26 +743,160 @@ def _override_defaults(*overrides, modules, _fail = fail, default): override.fn(tag = tag, _fail = _fail, default = default) -def _get_toolchain_config(*, modules, _fail = fail): +def _manifest_entry_sort_key(entry): + flavor_rank = {"full": 3, "install_only": 1, "install_only_stripped": 2}.get(entry.archive_flavor, 4) + microarch = entry.microarch + if not microarch: + microarch_rank = 0 + elif microarch.startswith("v") and microarch[1:].isdigit(): + microarch_rank = int(microarch[1:]) + else: + microarch_rank = 999 + return (flavor_rank, microarch_rank) + +def _populate_from_pbs_manifest( + *, + mctx, + add_runtime_manifest_urls = [], + add_runtime_manifest_files = [], + runtime_manifest_sha = "", + base_urls = [], + available_versions, + _fail): + manifest_contents = [] + + if add_runtime_manifest_urls: + manifest_path = mctx.path("runtime_manifest") + result = mctx.download( + url = add_runtime_manifest_urls, + output = manifest_path, + sha256 = runtime_manifest_sha, + ) + if not result.success: + _fail("Failed to download manifest from {}: {}".format(add_runtime_manifest_urls, result)) + return + manifest_contents.append(mctx.read(manifest_path)) + + for manifest_file in add_runtime_manifest_files: + manifest_contents.append(mctx.read(manifest_file, watch = "yes")) + + if not manifest_contents: + return + + base_download_urls = [url.rpartition("/")[0] for url in add_runtime_manifest_urls] + if not base_download_urls and base_urls: + base_download_urls = list(base_urls) + + entries = [] + for content in manifest_contents: + entries.extend(parse_runtime_manifest(content)) + + # We don't model archive_flavor via flags yet, so have to pick one. + # Preference is given to install_only because its smaller + entries = sorted( + entries, + key = _manifest_entry_sort_key, + ) + + for entry in entries: + location = entry.location + sha256 = entry.sha256 + py_version = entry.python_version + + # Fallback to matching against PLATFORMS keys as before to ensure compatibility + # with rules_python expected platform keys. + matched_platform = "{}-{}-{}".format(entry.arch, entry.vendor, entry.os) + if entry.libc: + matched_platform += "-" + entry.libc + if entry.freethreaded: + matched_platform += "-freethreaded" + + if matched_platform not in PLATFORMS: + continue + + if entry.archive_flavor not in ["install_only", "install_only_stripped", "full"]: + continue + + v_dict = available_versions.setdefault(py_version, {}) + if matched_platform in v_dict.get("sha256", {}): + continue + + if "://" in location: + urls = [location] + else: + urls = ["{}/{}".format(b_url, location) for b_url in base_download_urls] + + strip_prefix = "python/install" if entry.archive_flavor == "full" else "python" + + v_dict.setdefault("sha256", {})[matched_platform] = sha256 + v_dict.setdefault("url", {})[matched_platform] = urls + v_dict.setdefault("strip_prefix", {})[matched_platform] = strip_prefix + +def _get_toolchain_config(*, mctx, modules, _fail = fail): + """Computes the configs for toolchains. + + Args: + mctx: The module context. + modules: The modules from module_ctx + _fail: Function to call for failing; only used for testing. + + Returns: + A struct with the following: + * `kwargs`: {type}`dict[str, dict[str, object]` custom kwargs to pass to + `python_register_toolchains`, keyed by python version. + The first key is either a Major.Minor or Major.Minor.Patch + string. + * `minor_mapping`: {type}`dict[str, str]` the mapping of Major.Minor + to Major.Minor.Patch. + * `default`: {type}`dict[str, object]` of kwargs passed along to + `python_register_toolchains`. These keys take final precedence. + * `register_all_versions`: {type}`bool` whether all known versions + should be registered. + """ + # Items that can be overridden - available_versions = { - version: { - # Use a dicts straight away so that we could do URL overrides for a - # single version. - "sha256": dict(item["sha256"]), - "strip_prefix": { - platform: item["strip_prefix"] - for platform in item["sha256"] - } if type(item["strip_prefix"]) == type("") else item["strip_prefix"], - "url": { - platform: [item["url"]] - for platform in item["sha256"] - } if type(item["url"]) == type("") else item["url"], - } - for version, item in TOOL_VERSIONS.items() - } + available_versions = {} + _populate_from_pbs_manifest( + mctx = mctx, + add_runtime_manifest_files = [Label("//python/private:runtimes_manifest.txt")], + base_urls = DEFAULT_RELEASE_BASE_URLS, + available_versions = available_versions, + _fail = _fail, + ) + + # Check for add_runtime_manifest_urls or add_runtime_manifest_files in override tags in root module + root_module = modules[0] if modules else None + if root_module and root_module.is_root: + for tag in root_module.tags.override: + if tag.add_runtime_manifest_urls or tag.add_runtime_manifest_files: + _populate_from_pbs_manifest( + mctx = mctx, + add_runtime_manifest_urls = tag.add_runtime_manifest_urls, + add_runtime_manifest_files = tag.add_runtime_manifest_files, + runtime_manifest_sha = tag.runtime_manifest_sha, + base_urls = tag.base_urls, + available_versions = available_versions, + _fail = _fail, + ) + + # Check for add_runtime_manifest_urls or add_runtime_manifest_files in override tags in root module + root_module = modules[0] if modules else None + if root_module and root_module.is_root: + for tag in root_module.tags.override: + if tag.add_runtime_manifest_urls or tag.add_runtime_manifest_files: + _populate_from_pbs_manifest( + mctx = mctx, + add_runtime_manifest_urls = tag.add_runtime_manifest_urls, + add_runtime_manifest_files = tag.add_runtime_manifest_files, + runtime_manifest_sha = tag.runtime_manifest_sha, + base_urls = tag.base_urls, + available_versions = available_versions, + _fail = _fail, + ) + default = { - "base_url": DEFAULT_RELEASE_BASE_URL, + "base_urls": DEFAULT_RELEASE_BASE_URLS, + "platforms": dict(PLATFORMS), # Copy so it's mutable. "tool_versions": available_versions, } @@ -502,12 +927,16 @@ def _get_toolchain_config(*, modules, _fail = fail): ) register_all_versions = default.pop("register_all_versions", False) + add_target_settings = default.pop("add_target_settings", []) kwargs = default.pop("kwargs", {}) versions = {} for version_string in available_versions: - v = semver(version_string) - versions.setdefault("{}.{}".format(v.major, v.minor), []).append((int(v.patch), version_string)) + v = version.parse(version_string, strict = True) + versions.setdefault( + "{}.{}".format(v.release[0], v.release[1]), + [], + ).append((version.key(v), v.string)) minor_mapping = { major_minor: max(subset)[1] @@ -524,11 +953,124 @@ def _get_toolchain_config(*, modules, _fail = fail): minor_mapping = minor_mapping, default = default, register_all_versions = register_all_versions, + add_target_settings = add_target_settings, + ) + +def _compute_default_python_version(mctx): + default_python_version = None + for mod in mctx.modules: + # Only the root module and rules_python are allowed to specify the default + # toolchain for a couple reasons: + # * It prevents submodules from specifying different defaults and only + # one of them winning. + # * rules_python needs to set a soft default in case the root module doesn't, + # e.g. if the root module doesn't use Python itself. + # * The root module is allowed to override the rules_python default. + if not (mod.is_root or mod.name == "rules_python"): + continue + + defaults_attr_structs = _create_defaults_attr_structs(mod = mod) + default_python_version_env = None + default_python_version_file = None + pyproject_toml_label = None + + for defaults_attr in defaults_attr_structs: + pyproject_toml_label = _one_or_the_same( + pyproject_toml_label, + defaults_attr.pyproject_toml, + onerror = lambda: fail("Multiple pyproject.toml files specified in defaults"), + ) + + default_python_version = _one_or_the_same( + default_python_version, + defaults_attr.python_version, + onerror = _fail_multiple_defaults_python_version, + ) + default_python_version_env = _one_or_the_same( + default_python_version_env, + defaults_attr.python_version_env, + onerror = _fail_multiple_defaults_python_version_env, + ) + default_python_version_file = _one_or_the_same( + default_python_version_file, + defaults_attr.python_version_file, + onerror = _fail_multiple_defaults_python_version_file, + ) + + # Priority order: ENV > pyproject_toml > python_version_file > python_version + if default_python_version_file: + default_python_version = _one_or_the_same( + default_python_version, + mctx.read(default_python_version_file, watch = "yes").strip(), + ) + if pyproject_toml_label: + pyproject = read_pyproject(mctx, pyproject_toml_label) + if pyproject.requires_python: + default_python_version = version_from_requires_python(pyproject.requires_python) + if default_python_version_env: + default_python_version = mctx.getenv( + default_python_version_env, + default_python_version, + ) + + if default_python_version: + break + + # Otherwise, look at legacy python.toolchain() calls for a default + toolchain_attrs = mod.tags.toolchain + + # Convenience: if one python.toolchain() call exists, treat it as + # the default. + if len(toolchain_attrs) == 1: + default_python_version = toolchain_attrs[0].python_version + else: + sets_default = [v for v in toolchain_attrs if v.is_default] + if len(sets_default) == 1: + default_python_version = sets_default[0].python_version + elif len(sets_default) > 1: + _fail_multiple_default_toolchains_in_module(mod, toolchain_attrs) + + if default_python_version: + break + + return default_python_version + +def _create_defaults_attr_structs(*, mod): + arg_structs = [] + + for tag in mod.tags.defaults: + arg_structs.append(_create_defaults_attr_struct(tag = tag)) + + return arg_structs + +def _create_defaults_attr_struct(*, tag): + return struct( + python_version = getattr(tag, "python_version", None), + python_version_env = getattr(tag, "python_version_env", None), + python_version_file = getattr(tag, "python_version_file", None), + pyproject_toml = getattr(tag, "pyproject_toml", None), ) -def _create_toolchain_attr_structs(*, mod, config, seen_versions): +def _create_toolchain_attr_structs(*, mod, config, seen_versions, default_python_version): arg_structs = [] + # Auto-register a toolchain for the default version if not already + # registered via an explicit python.toolchain() call. + # This works for any default source: pyproject_toml, python_version_file, + # python_version_env, or python_version. + has_explicit_toolchain = default_python_version and any([ + tag.python_version == default_python_version + for tag in mod.tags.toolchain + ]) + if (default_python_version and + default_python_version not in seen_versions and + mod.is_root and not has_explicit_toolchain): + arg_structs.append(_create_toolchain_attrs_struct( + python_version = default_python_version, + toolchain_tag_count = 1, + )) + seen_versions[default_python_version] = True + for tag in mod.tags.toolchain: arg_structs.append(_create_toolchain_attrs_struct( tag = tag, @@ -546,7 +1088,11 @@ def _create_toolchain_attr_structs(*, mod, config, seen_versions): return arg_structs -def _create_toolchain_attrs_struct(*, tag = None, python_version = None, toolchain_tag_count = None): +def _create_toolchain_attrs_struct( + *, + tag = None, + python_version = None, + toolchain_tag_count = None): if tag and python_version: fail("Only one of tag and python version can be specified") if tag: @@ -559,16 +1105,61 @@ def _create_toolchain_attrs_struct(*, tag = None, python_version = None, toolcha is_default = is_default, python_version = python_version if python_version else tag.python_version, configure_coverage_tool = getattr(tag, "configure_coverage_tool", False), - ignore_root_user_error = getattr(tag, "ignore_root_user_error", False), ) -def _get_bazel_version_specific_kwargs(): - kwargs = {} +_defaults = tag_class( + doc = """Tag class to specify the default Python version.""", + attrs = { + "pyproject_toml": attr.label( + mandatory = False, + doc = """\ +Label pointing to pyproject.toml file to read the default Python version from. +When specified, reads the `requires-python` field from pyproject.toml. +The version must be specified as `==X.Y.Z` (exact version with full semver). - if IS_BAZEL_6_4_OR_HIGHER: - kwargs["environ"] = ["RULES_PYTHON_BZLMOD_DEBUG"] +:::{versionadded} 2.3.0 +::: +""", + ), + "python_version": attr.string( + mandatory = False, + doc = """\ +String saying what the default Python version should be. If the string +matches the {attr}`python_version` attribute of a toolchain, this +toolchain is the default version. If this attribute is set, the +{attr}`is_default` attribute of the toolchain is ignored. - return kwargs +:::{versionadded} 1.4.0 +::: +""", + ), + "python_version_env": attr.string( + mandatory = False, + doc = """\ +Environment variable saying what the default Python version should be. +If the string matches the {attr}`python_version` attribute of a +toolchain, this toolchain is the default version. If this attribute is +set, the {attr}`is_default` attribute of the toolchain is ignored. + +:::{versionadded} 1.4.0 +::: +""", + ), + "python_version_file": attr.label( + mandatory = False, + allow_single_file = True, + doc = """\ +File saying what the default Python version should be. If the contents +of the file match the {attr}`python_version` attribute of a toolchain, +this toolchain is the default version. If this attribute is set, the +{attr}`is_default` attribute of the toolchain is ignored. + +:::{versionadded} 1.4.0 +::: +""", + ), + }, +) _toolchain = tag_class( doc = """Tag class used to register Python toolchains. @@ -619,10 +1210,8 @@ In order to use a different name than the above, you can use the following `MODU syntax: ```starlark python = use_extension("@rules_python//python/extensions:python.bzl", "python") -python.toolchain( - is_default = True, - python_version = "3.11", -) +python.defaults(python_version = "3.11") +python.toolchain(python_version = "3.11") use_repo(python, my_python_name = "python_3_11") ``` @@ -636,22 +1225,24 @@ Then the python interpreter will be available as `my_python_name`. doc = "Whether or not to configure the default coverage tool provided by `rules_python` for the compatible toolchains.", ), "ignore_root_user_error": attr.bool( - default = False, + default = True, doc = """\ -If `False`, the Python runtime installation will be made read only. This improves -the ability for Bazel to cache it, but prevents the interpreter from creating -`.pyc` files for the standard library dynamically at runtime as they are loaded. - -If `True`, the Python runtime installation is read-write. This allows the -interpreter to create `.pyc` files for the standard library, but, because they are -created as needed, it adversely affects Bazel's ability to cache the runtime and -can result in spurious build failures. +:::{versionchanged} 1.8.0 +Noop, will be removed in the next major release. +::: """, mandatory = False, ), "is_default": attr.bool( mandatory = False, - doc = "Whether the toolchain is the default version", + doc = """\ +Whether the toolchain is the default version. + +:::{versionchanged} 1.4.0 +This setting is ignored if the default version is set using the `defaults` +tag class (encouraged). +::: +""", ), "python_version": attr.string( mandatory = True, @@ -670,6 +1261,66 @@ _override = tag_class( ::: """, attrs = { + "add_runtime_manifest_files": attr.label_list( + mandatory = False, + allow_files = True, + doc = """ +Labels pointing to local python-build-standalone manifest files (e.g., `SHA256SUMS`). + +Example: +`//my/custom/manifest:SHA256SUMS` + +:::{seealso} +[Manifest file format documentation](https://rules-python.readthedocs.io/en/latest/toolchains.html#manifest-file-format) +::: + +:::{versionadded} 2.1.0 +::: +""", + ), + "add_runtime_manifest_urls": attr.string_list( + mandatory = False, + doc = """ +URLs pointing to python-build-standalone manifest files (e.g., SHA256SUMS). + +Example: +`https://github.com/astral-sh/python-build-standalone/releases/download/20260414/SHA256SUMS` + +Note that `/latest/` can be used in place of a specific release date (e.g., `20260414`) to automatically use the latest release: +`https://github.com/astral-sh/python-build-standalone/releases/latest/download/SHA256SUMS` + +:::{seealso} +[Manifest file format documentation](https://rules-python.readthedocs.io/en/latest/toolchains.html#manifest-file-format) +::: + +:::{versionadded} 2.1.0 +::: +""", + ), + "add_target_settings": attr.string_list( + mandatory = False, + doc = """\ +A list of `config_setting` labels to add to the `target_settings` of every +toolchain registered by this module extension. This is useful for creating +separate "families" of toolchains gated behind custom build settings. + +For example, to ensure the default prebuilt toolchains are only resolved when +a `prebuilt` config setting is active: + +```starlark +python.override( + add_target_settings = ["@@//:python_toolchain_family_prebuilt"], +) +``` + +These settings are appended to the `target_settings` of all toolchains +registered by the extension, including any that already have settings +from `python.single_version_platform_override`. + +:::{versionadded} 2.1.0 +::: +""", + ), "available_python_versions": attr.string_list( mandatory = False, doc = """\ @@ -684,23 +1335,14 @@ This attribute is usually used in order to ensure that no unexpected transitive dependencies are introduced. """, ), - "base_url": attr.string( + "base_urls": attr.string_list( mandatory = False, - doc = "The base URL to be used when downloading toolchains.", - default = DEFAULT_RELEASE_BASE_URL, + doc = "The base URLs to be used when downloading toolchains.", + default = DEFAULT_RELEASE_BASE_URLS, ), "ignore_root_user_error": attr.bool( - default = False, - doc = """\ -If `False`, the Python runtime installation will be made read only. This improves -the ability for Bazel to cache it, but prevents the interpreter from creating -`.pyc` files for the standard library dynamically at runtime as they are loaded. - -If `True`, the Python runtime installation is read-write. This allows the -interpreter to create `.pyc` files for the standard library, but, because they are -created as needed, it adversely affects Bazel's ability to cache the runtime and -can result in spurious build failures. -""", + default = True, + doc = """Deprecated; do not use. This attribute has no effect.""", mandatory = False, ), "minor_mapping": attr.string_dict( @@ -724,6 +1366,15 @@ The values in this mapping override the default values and do not replace them. default = {}, ), "register_all_versions": attr.bool(default = False, doc = "Add all versions"), + "runtime_manifest_sha": attr.string( + mandatory = False, + doc = """ +SHA256 hash for the add_runtime_manifest_urls. + +:::{versionadded} 2.1.0 +::: +""", + ), } | AUTH_ATTRS, ) @@ -814,10 +1465,48 @@ configuration, please use {obj}`single_version_override`. ::: """, attrs = { + "arch": attr.string( + doc = """ +The arch (cpu) the runtime is compatible with. + +If not set, then the runtime cannot be used as a `python_X_Y_host` runtime. + +If set, the `os_name`, `target_compatible_with` and `target_settings` attributes +should also be set. + +The values should be one of the values in `@platforms//cpu` + +:::{seealso} +Docs for [Registering custom runtimes] +::: + +:::{versionadded} 1.5.0 +::: +""", + ), "coverage_tool": attr.label( doc = """\ The coverage tool to be used for a particular Python interpreter. This can override `rules_python` defaults. +""", + ), + "os_name": attr.string( + doc = """ +The host OS the runtime is compatible with. + +If not set, then the runtime cannot be used as a `python_X_Y_host` runtime. + +If set, the `os_name`, `target_compatible_with` and `target_settings` attributes +should also be set. + +The values should be one of the values in `@platforms//os` + +:::{seealso} +Docs for [Registering custom runtimes] +::: + +:::{versionadded} 1.5.0 +::: """, ), "patch_strip": attr.int( @@ -831,8 +1520,20 @@ The coverage tool to be used for a particular Python interpreter. This can overr ), "platform": attr.string( mandatory = True, - values = PLATFORMS.keys(), - doc = "The platform to override the values for, must be one of:\n{}.".format("\n".join(sorted(["* `{}`".format(p) for p in PLATFORMS]))), + doc = """ +The platform to override the values for, typically one of:\n +{platforms} + +Other values are allowed, in which case, `target_compatible_with`, +`target_settings`, `os_name`, and `arch` should be specified so the toolchain is +only used when appropriate. + +:::{{versionchanged}} 1.5.0 +Arbitrary platform strings allowed. +::: +""".format( + platforms = "\n".join(sorted(["* `{}`".format(p) for p in PLATFORMS])), + ), ), "python_version": attr.string( mandatory = True, @@ -847,6 +1548,36 @@ The coverage tool to be used for a particular Python interpreter. This can overr doc = "The 'strip_prefix' for the archive, defaults to 'python'.", default = "python", ), + "target_compatible_with": attr.string_list( + doc = """ +The `target_compatible_with` values to use for the toolchain definition. + +If not set, then `os_name` and `arch` will be used to populate it. + +If set, `target_settings`, `os_name`, and `arch` should also be set. + +:::{seealso} +Docs for [Registering custom runtimes] +::: + +:::{versionadded} 1.5.0 +::: +""", + ), + "target_settings": attr.string_list( + doc = """ +The `target_setings` values to use for the toolchain definition. + +If set, `target_compatible_with`, `os_name`, and `arch` should also be set. + +:::{seealso} +Docs for [Registering custom runtimes] +::: + +:::{versionadded} 1.5.0 +::: +""", + ), "urls": attr.string_list( mandatory = False, doc = "The URL template to fetch releases for this Python version. If the URL template results in a relative fragment, default base URL is going to be used. Occurrences of `{python_version}`, `{platform}` and `{build}` will be interpolated based on the contents in the override and the known {attr}`platform` values.", @@ -859,12 +1590,13 @@ python = module_extension( """, implementation = _python_impl, tag_classes = { + "defaults": _defaults, "override": _override, "single_version_override": _single_version_override, "single_version_platform_override": _single_version_platform_override, "toolchain": _toolchain, }, - **_get_bazel_version_specific_kwargs() + environ = ["RULES_PYTHON_BZLMOD_DEBUG"], ) _DEBUG_BUILD_CONTENT = """ diff --git a/python/private/python_bootstrap_template.txt b/python/private/python_bootstrap_template.txt index 0f9c90b3b3..482918c038 100644 --- a/python/private/python_bootstrap_template.txt +++ b/python/private/python_bootstrap_template.txt @@ -1,10 +1,5 @@ %shebang% - -# This script must retain compatibility with a wide variety of Python versions -# since it is run for every py_binary target. Currently we guarantee support -# going back to Python 2.7, and try to support even Python 2.6 on a best-effort -# basis. We might abandon 2.6 support once users have the ability to control the -# above shebang string via the Python toolchain (#8685). +# vim: syntax=python from __future__ import absolute_import from __future__ import division @@ -12,31 +7,129 @@ from __future__ import print_function import sys -# The Python interpreter unconditionally prepends the directory containing this -# script (following symlinks) to the import path. This is the cause of #9239, -# and is a special case of #7091. We therefore explicitly delete that entry. -# TODO(#7091): Remove this hack when no longer necessary. -del sys.path[0] - +# By default, Python prepends the directory containing this script to +# sys.path. The stage 1 bootstrap only needs stdlib modules before it +# re-execs into the configured runtime, so avoid resolving imports from the +# target's output or runfiles package directory. This matters when that +# directory contains files with stdlib names, such as shutil.py or types.py. +# +# Python 3.11 introduced PYTHONSAFEPATH (-P), which disables the unsafe prepend. +# Isolated mode (-I) also disables it, including on older interpreters without +# safe_path. In either case, sys.path[0] is not the script directory and should +# be preserved. +if ( + not getattr(sys.flags, "safe_path", False) and + not getattr(sys.flags, "isolated", False) and + sys.path +): + del sys.path[0] + +# Generated file from @rules_python//python/private:python_bootstrap_template.txt + +from os.path import abspath, dirname, join, basename, normpath import os +import shutil import subprocess -import uuid -def IsRunningFromZip(): - return %is_zipfile% +# NOTE: The sentinel strings are split (e.g., "%stage2" + "_bootstrap%") so that +# the substitution logic won't replace them. This allows runtime detection of +# unsubstituted placeholders, which occurs when native py_binary is used in +# external repositories. In that case, we fall back to %main% which Bazel's +# native rule does substitute. +_STAGE2_BOOTSTRAP_SENTINEL = "%stage2" + "_bootstrap%" +# runfiles-root-relative path +STAGE2_BOOTSTRAP="%stage2_bootstrap%" + +# NOTE: The fallback logic from stage2_bootstrap to main is only present +# as a courtesy for an older, unsupported, configuration. It can be removed +# when that case is unlikely to be a concern anymore. +# See https://github.com/bazel-contrib/rules_python/pull/3495 +if STAGE2_BOOTSTRAP == _STAGE2_BOOTSTRAP_SENTINEL: + _MAIN_SENTINEL = "%main" + "%" + _main = "%main%" + if _main != _MAIN_SENTINEL and _main: + STAGE2_BOOTSTRAP = _main + else: + STAGE2_BOOTSTRAP = "" + +if not STAGE2_BOOTSTRAP: + print("ERROR: %stage2_bootstrap% (or %main%) was not substituted.", file=sys.stderr) + sys.exit(1) + +# runfiles-root-relative path to venv's python interpreter +# Empty string if a venv is not setup. +PYTHON_BINARY = '%python_binary%' + +# The path to the actual interpreter that is used. +# Typically PYTHON_BINARY is a symlink pointing to this. +# runfiles-root-relative path, absolute path, or single word. +# Used to create a venv at runtime, or when a venv isn't setup. +PYTHON_BINARY_ACTUAL = "%python_binary_actual%" + +# 0 or 1. +# 1 if this bootstrap was created for placement within a zipfile. 0 otherwise. +IS_ZIPFILE = "%is_zipfile%" == "1" +# 0 or 1. +# If 1, then a venv will be created at runtime that replicates what would have +# been the build-time structure. +RECREATE_VENV_AT_RUNTIME = "%recreate_venv_at_runtime%" == "1" +# 0 or 1 +# If 1, then the path to python will be resolved by running +# PYTHON_BINARY_ACTUAL to determine the actual underlying interpreter. +RESOLVE_PYTHON_BINARY_AT_RUNTIME = "%resolve_python_binary_at_runtime%" == "1" +# venv-relative path to the site-packages +# e.g. lib/python3.12t/site-packages +VENV_REL_SITE_PACKAGES = "%venv_rel_site_packages%" + +WORKSPACE_NAME = "%workspace_name%" + +# Target-specific interpreter args. +# Sentinel split to detect unsubstituted placeholder (see STAGE2_BOOTSTRAP above). +_INTERPRETER_ARGS_SENTINEL = "%interpreter" + "_args%" +_INTERPRETER_ARGS_RAW = "%interpreter_args%" +if _INTERPRETER_ARGS_RAW == _INTERPRETER_ARGS_SENTINEL: + INTERPRETER_ARGS = [] +else: + INTERPRETER_ARGS = [arg for arg in _INTERPRETER_ARGS_RAW.split("\n") if arg] + +# Symlinks to create at runtime in the venv. +# Line delimited entries +# Each entry is "venv_relative_path|path_to_symlink_to" +RUNTIME_VENV_SYMLINKS = """ +%runtime_venv_symlinks% +""".strip().split("\n") +RUNTIME_VENV_SYMLINKS = dict(line.split("|") for line in RUNTIME_VENV_SYMLINKS if line) + +ADDITIONAL_INTERPRETER_ARGS = os.environ.get("RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS", "") +EXTRACT_ROOT = os.environ.get("RULES_PYTHON_EXTRACT_ROOT") -if IsRunningFromZip(): +if IS_ZIPFILE: import shutil import tempfile import zipfile else: import re -# Return True if running on Windows -def IsWindows(): - return os.name == 'nt' +IS_WINDOWS = os.name == "nt" + +BIN_DIR_NAME = "bin" if not IS_WINDOWS else "Scripts" +LIB_DIR_NAME = "lib" if not IS_WINDOWS else "Lib" + +# Windows APIs can be picky about slashes depending on the context, +# so convert to backslashes to avoid any issues. +if IS_WINDOWS: + def norm_slashes(s): + return s.replace("/", "\\") -def GetWindowsPathWithUNCPrefix(path): + STAGE2_BOOTSTRAP = norm_slashes(STAGE2_BOOTSTRAP) + PYTHON_BINARY = norm_slashes(PYTHON_BINARY) + PYTHON_BINARY_ACTUAL = norm_slashes(PYTHON_BINARY_ACTUAL) + RUNTIME_VENV_SYMLINKS = { + norm_slashes(k): norm_slashes(v) + for k, v in RUNTIME_VENV_SYMLINKS.items() + } + +def get_windows_path_with_unc_prefix(path): """Adds UNC prefix after getting a normalized absolute Windows path. No-op for non-Windows platforms or if running under python2. @@ -45,14 +138,22 @@ def GetWindowsPathWithUNCPrefix(path): # No need to add prefix for non-Windows platforms. # And \\?\ doesn't work in python 2 or on mingw - if not IsWindows() or sys.version_info[0] < 3: + if not IS_WINDOWS or sys.version_info[0] < 3: return path # Starting in Windows 10, version 1607(OS build 14393), MAX_PATH limitations have been # removed from common Win32 file and directory functions. # Related doc: https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=cmd#enable-long-paths-in-windows-10-version-1607-and-later import platform - if platform.win32_ver()[1] >= '10.0.14393': + win32_version = None + # Windows 2022 with Python 3.12.8 gives flakey errors, so try a couple times. + for _ in range(3): + try: + win32_version = platform.win32_ver()[1] + break + except (ValueError, KeyError): + pass + if win32_version and win32_version >= '10.0.14393': return path # import sysconfig only now to maintain python 2.6 compatibility @@ -65,56 +166,51 @@ def GetWindowsPathWithUNCPrefix(path): if path.startswith(unicode_prefix): return path - # os.path.abspath returns a normalized absolute path - return unicode_prefix + os.path.abspath(path) + # abspath returns a normalized absolute path + return unicode_prefix + abspath(path) -def HasWindowsExecutableExtension(path): - return path.endswith('.exe') or path.endswith('.com') or path.endswith('.bat') - -PYTHON_BINARY = '%python_binary%' -if IsWindows() and not HasWindowsExecutableExtension(PYTHON_BINARY): - PYTHON_BINARY = PYTHON_BINARY + '.exe' - -def SearchPath(name): +def search_path(name): """Finds a file in a given search path.""" search_path = os.getenv('PATH', os.defpath).split(os.pathsep) for directory in search_path: if directory: - path = os.path.join(directory, name) + path = join(directory, name) if os.path.isfile(path) and os.access(path, os.X_OK): return path return None -def FindPythonBinary(module_space): +def find_python_binary(runfiles_root): """Finds the real Python binary if it's not a normal absolute path.""" - return FindBinary(module_space, PYTHON_BINARY) - -def PrintVerbose(*args): - if os.environ.get("RULES_PYTHON_BOOTSTRAP_VERBOSE"): - print("bootstrap:", *args, file=sys.stderr, flush=True) - -def PrintVerboseCoverage(*args): - """Print output if VERBOSE_COVERAGE is non-empty in the environment.""" - if os.environ.get("VERBOSE_COVERAGE"): - print(*args, file=sys.stderr) - -def IsVerboseCoverage(): - """Returns True if VERBOSE_COVERAGE is non-empty in the environment.""" - return os.environ.get("VERBOSE_COVERAGE") - -def FindCoverageEntryPoint(module_space): - cov_tool = '%coverage_tool%' - if cov_tool: - PrintVerboseCoverage('Using toolchain coverage_tool %r' % cov_tool) + if PYTHON_BINARY: + return find_binary(runfiles_root, PYTHON_BINARY) else: - cov_tool = os.environ.get('PYTHON_COVERAGE') - if cov_tool: - PrintVerboseCoverage('PYTHON_COVERAGE: %r' % cov_tool) - if cov_tool: - return FindBinary(module_space, cov_tool) - return None + return find_binary(runfiles_root, PYTHON_BINARY_ACTUAL) + + +def print_verbose(*args, mapping=None, values=None): + if not os.environ.get("RULES_PYTHON_BOOTSTRAP_VERBOSE"): + return + + if mapping is not None: + for key, value in sorted((mapping or {}).items()): + print( + "bootstrap: stage 1:", + *(list(args) + ["{}={}".format(key, repr(value))]), + file=sys.stderr, + flush=True + ) + elif values is not None: + for i, v in enumerate(values): + print( + "bootstrap: stage 1:", + *(list(args) + ["[{}] {}".format(i, repr(v))]), + file=sys.stderr, + flush=True + ) + else: + print("bootstrap: stage 1:", *args, file=sys.stderr, flush=True) -def FindBinary(module_space, bin_name): +def find_binary(runfiles_root, bin_name): """Finds the real binary if it's not a normal absolute path.""" if not bin_name: return None @@ -129,16 +225,12 @@ def FindBinary(module_space, bin_name): # Use normpath() to convert slashes to os.sep on Windows. elif os.sep in os.path.normpath(bin_name): # Case 3: Path is relative to the repo root. - return os.path.join(module_space, bin_name) + return join(runfiles_root, bin_name) else: # Case 4: Path has to be looked up in the search path. - return SearchPath(bin_name) - -def CreatePythonPathEntries(python_imports, module_space): - parts = python_imports.split(':') - return [module_space] + ['%s/%s' % (module_space, path) for path in parts] + return search_path(bin_name) -def FindModuleSpace(main_rel_path): +def find_runfiles_root(main_rel_path): """Finds the runfiles tree.""" # When the calling process used the runfiles manifest to resolve the # location of this stub script, the path may be expanded. This means @@ -153,19 +245,32 @@ def FindModuleSpace(main_rel_path): runfiles_dir = runfiles_manifest_file[:-9] # Be defensive: the runfiles dir should contain our main entry point. If # it doesn't, then it must not be our runfiles directory. - if runfiles_dir and os.path.exists(os.path.join(runfiles_dir, main_rel_path)): + if runfiles_dir and os.path.exists(join(runfiles_dir, main_rel_path)): return runfiles_dir + # Clear RUNFILES_DIR & RUNFILES_MANIFEST_FILE since the runfiles dir was + # not found. These can be correctly set for a parent Python process, but + # inherited by the child, and not correct for it. Later bootstrap code + # assumes they're are correct if set. + os.environ.pop('RUNFILES_DIR', None) + os.environ.pop('RUNFILES_MANIFEST_FILE', None) + stub_filename = sys.argv[0] + # On Windows, the path may contain both forward and backslashes. + # Normalize to the OS separator because the regex used later assumes + # the OS-specific separator. + if IS_WINDOWS: + stub_filename = stub_filename.replace("/", os.sep) + if not os.path.isabs(stub_filename): - stub_filename = os.path.join(os.getcwd(), stub_filename) + stub_filename = join(os.getcwd(), stub_filename) while True: - module_space = stub_filename + ('.exe' if IsWindows() else '') + '.runfiles' - if os.path.isdir(module_space): - return module_space + runfiles_root = stub_filename + ('.exe' if IS_WINDOWS else '') + '.runfiles' + if os.path.isdir(runfiles_root): + return runfiles_root - runfiles_pattern = r'(.*\.runfiles)' + (r'\\' if IsWindows() else '/') + '.*' + runfiles_pattern = r'(.*\.runfiles)' + (r'\\' if IS_WINDOWS else '/') + '.*' matchobj = re.match(runfiles_pattern, stub_filename) if matchobj: return matchobj.group(1) @@ -176,11 +281,11 @@ def FindModuleSpace(main_rel_path): if os.path.isabs(target): stub_filename = target else: - stub_filename = os.path.join(os.path.dirname(stub_filename), target) + stub_filename = join(os.path.dirname(stub_filename), target) raise AssertionError('Cannot find .runfiles directory for %s' % sys.argv[0]) -def ExtractZip(zip_path, dest_dir): +def extract_zip(zip_path, dest_dir): """Extracts the contents of a zip file, preserving the unix file mode bits. These include the permission bits, and in particular, the executable bit. @@ -192,38 +297,156 @@ def ExtractZip(zip_path, dest_dir): zip_path: The path to the zip file to extract dest_dir: The path to the destination directory """ - zip_path = GetWindowsPathWithUNCPrefix(zip_path) - dest_dir = GetWindowsPathWithUNCPrefix(dest_dir) + zip_path = get_windows_path_with_unc_prefix(zip_path) + dest_dir = get_windows_path_with_unc_prefix(dest_dir) with zipfile.ZipFile(zip_path) as zf: for info in zf.infolist(): zf.extract(info, dest_dir) # UNC-prefixed paths must be absolute/normalized. See # https://docs.microsoft.com/en-us/windows/desktop/fileio/naming-a-file#maximum-path-length-limitation - file_path = os.path.abspath(os.path.join(dest_dir, info.filename)) + file_path = abspath(join(dest_dir, info.filename)) # The Unix st_mode bits (see "man 7 inode") are stored in the upper 16 - # bits of external_attr. Of those, we set the lower 12 bits, which are the - # file mode bits (since the file type bits can't be set by chmod anyway). + # bits of external_attr. attrs = info.external_attr >> 16 - if attrs != 0: # Rumor has it these can be 0 for zips created on Windows. + # Symlink bit in st_mode is 0o120000. + if (attrs & 0o170000) == 0o120000: + with open(file_path, "r") as f: + target = f.read() + os.remove(file_path) + os.symlink(target, file_path) + # Of those, we set the lower 12 bits, which are the + # file mode bits (since the file type bits can't be set by chmod anyway). + elif attrs != 0: # Rumor has it these can be 0 for zips created on Windows. os.chmod(file_path, attrs & 0o7777) # Create the runfiles tree by extracting the zip file -def CreateModuleSpace(): +def create_runfiles_root(): temp_dir = tempfile.mkdtemp('', 'Bazel.runfiles_') - ExtractZip(os.path.dirname(__file__), temp_dir) - # IMPORTANT: Later code does `rm -fr` on dirname(module_space) -- it's + extract_zip(os.path.dirname(__file__), temp_dir) + # IMPORTANT: Later code does `rm -fr` on dirname(runfiles_root) -- it's # important that deletion code be in sync with this directory structure - return os.path.join(temp_dir, 'runfiles') - -# Returns repository roots to add to the import path. -def GetRepositoriesImports(module_space, import_all): - if import_all: - repo_dirs = [os.path.join(module_space, d) for d in os.listdir(module_space)] - repo_dirs.sort() - return [d for d in repo_dirs if os.path.isdir(d)] - return [os.path.join(module_space, '%workspace_name%')] + return join(temp_dir, 'runfiles') + +def _symlink_tree(link_from, link_to): + # Ensure the source is an absolute path to simplify symlinking + link_to_root = abspath(link_to) + link_from_root = abspath(link_from) + + # This is non-optimal because it recreates the entire + # venv site-packages tree (as opposed to finding a highest-common + # directory to symlink). But its easy and understandable. + for root, dirs, files in os.walk(link_to_root): + rel_path = os.path.relpath(root, link_to_root) + link_from_dir = join(link_from_root, rel_path) + + if not os.path.exists(link_from_dir): + os.makedirs(link_from_dir) + + for name in files: + link_to = join(root, name) + link_from = join(link_from_dir, name) + _symlink_exist_ok(from_=link_from, to=link_to) + +def _create_venv(runfiles_root, delete_dirs): + rel_runfiles_venv = dirname(dirname(PYTHON_BINARY)) + runfiles_venv = join(runfiles_root, rel_runfiles_venv) + print_verbose("create_venv: runfiles venv:", runfiles_venv) + if EXTRACT_ROOT: + venv = join(EXTRACT_ROOT, rel_runfiles_venv) + os.makedirs(venv, exist_ok=True) + else: + import tempfile + venv = tempfile.mkdtemp("", f"bazel.{basename(runfiles_venv)}.") + delete_dirs.append(venv) + + print_verbose("create_venv: created venv:", venv) + + python_exe_actual = find_binary(runfiles_root, PYTHON_BINARY_ACTUAL) + if python_exe_actual is None: + raise AssertionError('Could not find python binary: ' + repr(PYTHON_BINARY_ACTUAL)) + + # See stage1_bootstrap_template.sh for details on this code path. In short, + # this handles when the build-time python version doesn't match runtime + # and if the initially resolved python_exe_actual is a wrapper script. + if RESOLVE_PYTHON_BINARY_AT_RUNTIME: + venv_src = venv.replace("\\", "\\\\") # Escape backslashes + src = f""" +import sys, site +print(sys.executable) +print(sys.base_prefix) +print(site.getsitepackages(["{venv_src}"])[-1]) + """ + print_verbose("prog:", src) + output = subprocess.check_output([python_exe_actual, "-I"], shell=True, + encoding = "utf8", input=src) + output = output.strip().split("\n") + python_exe_actual = output[0] + python_home = output[1] if IS_WINDOWS else None + venv_lib = output[2] + os.makedirs(dirname(venv_lib), exist_ok=True) + runfiles_venv_lib = join(runfiles_venv, VENV_REL_SITE_PACKAGES) + else: + # On unixy, Python can find home based on the symlink. + python_home = dirname(python_exe_actual) if IS_WINDOWS else None + venv_lib = join(venv, LIB_DIR_NAME) + runfiles_venv_lib = join(runfiles_venv, LIB_DIR_NAME) + + venv_bin = join(venv, BIN_DIR_NAME) + try: + os.mkdir(venv_bin) + except FileExistsError as e: + pass + # Match the basename; some tools, e.g. pyvenv key off the executable name + venv_python_exe = join(venv_bin, os.path.basename(python_exe_actual)) + _symlink_exist_ok(from_=venv_python_exe, to=python_exe_actual) + + # Windows requires supporting .dll files in the venv bin dir, but + # they aren't known in advance when the interpreter is resolved at runtime. + if RESOLVE_PYTHON_BINARY_AT_RUNTIME and IS_WINDOWS: + files = os.listdir(python_home) + for f in files: + if not f.endswith((".dll", ".pdb")): continue + venv_path = join(venv, BIN_DIR_NAME, f) + target = join(python_home, f) + _symlink_exist_ok(from_=venv_path, to=target) + + runfiles_venv_bin = join(runfiles_venv, BIN_DIR_NAME) + # The runfiles bin directory may not exist if it's empty, e.g. + # supports_build_time_venv=False and the interpreter is resolved at + # runtime. + if os.path.exists(runfiles_venv_bin): + # Add any missing build-time entries under bin/. Do this before + # the manual symlink creation to better mimic what the + # regular runfiles directory looks like. + for f_basename in os.listdir(runfiles_venv_bin): + venv_path = join(venv, BIN_DIR_NAME, f_basename) + target = join(runfiles_venv_bin, f_basename) + _symlink_exist_ok(from_=venv_path, to=target) + + # Recreate correct relative symlinks + for venv_rel_path, link_to_rf_path in RUNTIME_VENV_SYMLINKS.items(): + venv_abs_path = join(venv, venv_rel_path) + link_to = normpath(join(runfiles_root, link_to_rf_path)) + os.makedirs(dirname(venv_abs_path), exist_ok=True) + _symlink_exist_ok(from_=venv_abs_path, to=link_to) + + # Do this last to handle non-relative symlinks artifacts + if os.path.exists(runfiles_venv_lib): + _symlink_tree(link_from=venv_lib, link_to=runfiles_venv_lib) + + if IS_WINDOWS: + print_verbose("create_venv: pyvenv.cfg home: ", python_home) + venv_pyvenv_cfg = join(venv, "pyvenv.cfg") + with open(venv_pyvenv_cfg, "w") as fp: + # Until Windows supports a build-time generated venv using symlinks + # to directories, we have to write the full, absolute, path to PYTHONHOME + # so that support directories (e.g. DLLs, libs) can be found. + fp.write("home = {}\n".format(python_home)) + else: + _symlink_exist_ok(from_=join(venv, "pyvenv.cfg"), to=join(runfiles_venv, "pyvenv.cfg")) + return venv_python_exe -def RunfilesEnvvar(module_space): +def runfiles_envvar(runfiles_root): """Finds the runfiles manifest or the runfiles directory. Returns: @@ -242,11 +465,11 @@ def RunfilesEnvvar(module_space): return ('RUNFILES_DIR', runfiles) # If running from a zip, there's no manifest file. - if IsRunningFromZip(): - return ('RUNFILES_DIR', module_space) + if IS_ZIPFILE: + return ('RUNFILES_DIR', runfiles_root) # Look for the runfiles "output" manifest, argv[0] + ".runfiles_manifest" - runfiles = module_space + '_manifest' + runfiles = runfiles_root + '_manifest' if os.path.exists(runfiles): return ('RUNFILES_MANIFEST_FILE', runfiles) @@ -254,74 +477,19 @@ def RunfilesEnvvar(module_space): # Normally .runfiles_manifest and MANIFEST are both present, but the # former will be missing for zip-based builds or if someone copies the # runfiles tree elsewhere. - runfiles = os.path.join(module_space, 'MANIFEST') + runfiles = join(runfiles_root, 'MANIFEST') if os.path.exists(runfiles): return ('RUNFILES_MANIFEST_FILE', runfiles) # If running in a sandbox and no environment variables are set, then # Look for the runfiles next to the binary. - if module_space.endswith('.runfiles') and os.path.isdir(module_space): - return ('RUNFILES_DIR', module_space) + if runfiles_root.endswith('.runfiles') and os.path.isdir(runfiles_root): + return ('RUNFILES_DIR', runfiles_root) return (None, None) -def Deduplicate(items): - """Efficiently filter out duplicates, keeping the first element only.""" - seen = set() - for it in items: - if it not in seen: - seen.add(it) - yield it - -def InstrumentedFilePaths(): - """Yields tuples of realpath of each instrumented file with the relative path.""" - manifest_filename = os.environ.get('COVERAGE_MANIFEST') - if not manifest_filename: - return - with open(manifest_filename, "r") as manifest: - for line in manifest: - filename = line.strip() - if not filename: - continue - try: - realpath = os.path.realpath(filename) - except OSError: - print( - "Could not find instrumented file {}".format(filename), - file=sys.stderr) - continue - if realpath != filename: - PrintVerboseCoverage("Fixing up {} -> {}".format(realpath, filename)) - yield (realpath, filename) - -def UnresolveSymlinks(output_filename): - # type: (str) -> None - """Replace realpath of instrumented files with the relative path in the lcov output. - - Though we are asking coveragepy to use relative file names, currently - ignore that for purposes of generating the lcov report (and other reports - which are not the XML report), so we need to go and fix up the report. - - This function is a workaround for that issue. Once that issue is fixed - upstream and the updated version is widely in use, this should be removed. - - See https://github.com/nedbat/coveragepy/issues/963. - """ - substitutions = list(InstrumentedFilePaths()) - if substitutions: - unfixed_file = output_filename + '.tmp' - os.rename(output_filename, unfixed_file) - with open(unfixed_file, "r") as unfixed: - with open(output_filename, "w") as output_file: - for line in unfixed: - if line.startswith('SF:'): - for (realpath, filename) in substitutions: - line = line.replace(realpath, filename) - output_file.write(line) - os.unlink(unfixed_file) - -def ExecuteFile(python_program, main_filename, args, env, module_space, - coverage_entrypoint, workspace, delete_module_space): +def execute_file(python_program, main_filename, args, env, runfiles_root, + workspace, delete_dirs): # type: (str, str, list[str], dict[str, str], str, str|None, str|None) -> ... """Executes the given Python file using the various environment settings. @@ -333,13 +501,20 @@ def ExecuteFile(python_program, main_filename, args, env, module_space, main_filename: (str) The Python file to execute args: (list[str]) Additional args to pass to the Python file env: (dict[str, str]) A dict of environment variables to set for the execution - module_space: (str) Path to the module space/runfiles tree directory - coverage_entrypoint: (str|None) Path to the coverage tool entry point file. + runfiles_root: (str) Path to the runfiles root directory workspace: (str|None) Name of the workspace to execute in. This is expected to be a directory under the runfiles tree. - delete_module_space: (bool), True if the module space should be deleted - after a successful (exit code zero) program run, False if not. + delete_dirs: (list[str]) directories that should be deleted after the user + program has finished running. """ + argv = [python_program] + argv.extend(INTERPRETER_ARGS) + additional_interpreter_args = os.environ.pop("RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS", "") + if additional_interpreter_args: + import shlex + argv.extend(shlex.split(additional_interpreter_args)) + argv.append(main_filename) + argv.extend(args) # We want to use os.execv instead of subprocess.call, which causes # problems with signal passing (making it difficult to kill # Bazel). However, these conditions force us to run via @@ -353,145 +528,96 @@ def ExecuteFile(python_program, main_filename, args, env, module_space, # - If we may need to emit a host config warning after execution, we # can't execv because we need control to return here. This only # happens for targets built in the host config. - # - For coverage targets, at least coveragepy requires running in - # two invocations, which also requires control to return here. # - if not (IsWindows() or workspace or coverage_entrypoint or delete_module_space): - _RunExecv(python_program, main_filename, args, env) + if not (IS_WINDOWS or workspace or delete_dirs): + _run_execv(python_program, argv, env) - if coverage_entrypoint is not None: - ret_code = _RunForCoverage(python_program, main_filename, args, env, - coverage_entrypoint, workspace) - else: - ret_code = subprocess.call( - [python_program, main_filename] + args, - env=env, - cwd=workspace - ) + print_verbose("run: subproc: environ:", mapping=os.environ) + print_verbose("run: subproc: cwd:", workspace) + print_verbose("run: subproc: argv:", values=argv) + ret_code = subprocess.call( + argv, env=env, cwd=workspace) + print_verbose("run: subproc: exit code:", ret_code) - if delete_module_space: - # NOTE: dirname() is called because CreateModuleSpace() creates a - # sub-directory within a temporary directory, and we want to remove the - # whole temporary directory. - shutil.rmtree(os.path.dirname(module_space), True) + if delete_dirs: + for delete_dir in delete_dirs: + print_verbose("cleanup: rmtree:", delete_dir) + shutil.rmtree(delete_dir, True) sys.exit(ret_code) -def _RunExecv(python_program, main_filename, args, env): - # type: (str, str, list[str], dict[str, str]) -> ... +def _run_execv(python_program, argv, env): + # type: (str, list[str], dict[str, str]) -> ... """Executes the given Python file using the various environment settings.""" os.environ.update(env) - PrintVerbose("RunExecv: environ:", os.environ) - argv = [python_program, main_filename] + args - PrintVerbose("RunExecv: argv:", python_program, argv) - os.execv(python_program, argv) - -def _RunForCoverage(python_program, main_filename, args, env, - coverage_entrypoint, workspace): - # type: (str, str, list[str], dict[str, str], str, str|None) -> int - """Collects coverage infomration for the given Python file. - - Args: - python_program: (str) Path to the Python binary to use for execution - main_filename: (str) The Python file to execute - args: (list[str]) Additional args to pass to the Python file - env: (dict[str, str]) A dict of environment variables to set for the execution - coverage_entrypoint: (str|None) Path to the coverage entry point to execute with. - workspace: (str|None) Name of the workspace to execute in. This is expected to be a - directory under the runfiles tree, and will recursively delete the - runfiles directory if set. - """ - # We need for coveragepy to use relative paths. This can only be configured - unique_id = uuid.uuid4() - rcfile_name = os.path.join(os.environ['COVERAGE_DIR'], ".coveragerc_{}".format(unique_id)) - with open(rcfile_name, "w") as rcfile: - rcfile.write('''[run] -relative_files = True -''') - PrintVerboseCoverage('Coverage entrypoint:', coverage_entrypoint) - # First run the target Python file via coveragepy to create a .coverage - # database file, from which we can later export lcov. - ret_code = subprocess.call( - [ - python_program, - coverage_entrypoint, - "run", - "--rcfile=" + rcfile_name, - "--append", - "--branch", - main_filename - ] + args, - env=env, - cwd=workspace - ) - output_filename = os.path.join(os.environ['COVERAGE_DIR'], 'pylcov.dat') - - PrintVerboseCoverage('Converting coveragepy database to lcov:', output_filename) - # Run coveragepy again to convert its .coverage database file into lcov. - # Under normal conditions running lcov outputs to stdout/stderr, which causes problems for `coverage`. - params = [python_program, coverage_entrypoint, "lcov", "--rcfile=" + rcfile_name, "-o", output_filename, "--quiet"] - kparams = {"env": env, "cwd": workspace, "stdout": subprocess.DEVNULL, "stderr": subprocess.DEVNULL} - if IsVerboseCoverage(): - # reconnect stdout/stderr to lcov generation. Should be useful for debugging `coverage` issues. - params.remove("--quiet") - kparams['stdout'] = sys.stderr - kparams['stderr'] = sys.stderr - - ret_code = subprocess.call( - params, - **kparams - ) or ret_code + print_verbose("RunExecv: environ:", mapping=os.environ) + print_verbose("RunExecv: python:", python_program) + print_verbose("RunExecv: argv:", values=argv) + try: + os.execv(python_program, argv) + except: + with open(python_program, 'rb') as f: + print_verbose("pyprog head:" + str(f.read(50))) + raise +def _symlink_exist_ok(*, from_, to): try: - os.unlink(rcfile_name) - except OSError as err: - # It's possible that the profiled program might execute another Python - # binary through a wrapper that would then delete the rcfile. Not much - # we can do about that, besides ignore the failure here. - PrintVerboseCoverage('Error removing temporary coverage rc file:', err) - if os.path.isfile(output_filename): - UnresolveSymlinks(output_filename) - return ret_code - -def Main(): + # On Windows, symlinks have to be told whether they're + # pointing to a file or directory. Python is supposed to auto-detect + # this, but in practice, this doesn't reliably happen. + os.symlink(to, from_, target_is_directory=os.path.isdir(to)) + except FileExistsError: + pass + + +def main(): + print_verbose("sys.version:", sys.version) + print_verbose("sys.executable:", sys.executable) + print_verbose("initial argv:", values=sys.argv) + print_verbose("initial cwd:", os.getcwd()) + print_verbose("initial environ:", mapping=os.environ) + print_verbose("initial sys.path:", values=sys.path) + print_verbose("IS_ZIPFILE:", IS_ZIPFILE) + print_verbose("PYTHON_BINARY:", PYTHON_BINARY) + print_verbose("PYTHON_BINARY_ACTUAL:", PYTHON_BINARY_ACTUAL) + print_verbose("RECREATE_VENV_AT_RUNTIME:", RECREATE_VENV_AT_RUNTIME) + print_verbose("RESOLVE_PYTHON_BINARY_AT_RUNTIME:", RESOLVE_PYTHON_BINARY_AT_RUNTIME) + print_verbose("RUNTIME_VENV_SYMLINKS size:", len(RUNTIME_VENV_SYMLINKS)) + print_verbose("STAGE2_BOOTSTRAP:", STAGE2_BOOTSTRAP) + print_verbose("VENV_REL_SITE_PACKAGES:", VENV_REL_SITE_PACKAGES) + print_verbose("WORKSPACE_NAME:", WORKSPACE_NAME ) + print_verbose("bootstrap sys.executable:", sys.executable) + print_verbose("bootstrap sys._base_executable:", getattr(sys, "_base_executable", "unknown")) + print_verbose("bootstrap sys.version:", sys.version) + args = sys.argv[1:] new_env = {} - # The main Python source file. - # The magic string percent-main-percent is replaced with the runfiles-relative - # filename of the main file of the Python binary in BazelPythonSemantics.java. - main_rel_path = '%main%' - if IsWindows(): - main_rel_path = main_rel_path.replace('/', os.sep) + # NOTE: We call normpath for two reasons: + # 1. Transform Bazel `foo/bar` to Windows `foo\bar` + # 2. Transform `_main/../foo/main.py` to simply `foo/main.py`, which + # matters if `_main` doesn't exist (which can occur if a binary + # is packaged and needs no artifacts from the main repo) + main_rel_path = os.path.normpath(STAGE2_BOOTSTRAP) + print_verbose("main_rel_path:", main_rel_path) + + delete_dirs = [] - if IsRunningFromZip(): - module_space = CreateModuleSpace() - delete_module_space = True + if IS_ZIPFILE: + runfiles_root = create_runfiles_root() + # NOTE: dirname() is called because create_runfiles_root() creates a + # sub-directory within a temporary directory, and we want to remove the + # whole temporary directory. + delete_dirs.append(dirname(runfiles_root)) else: - module_space = FindModuleSpace(main_rel_path) - delete_module_space = False - - python_imports = '%imports%' - python_path_entries = CreatePythonPathEntries(python_imports, module_space) - python_path_entries += GetRepositoriesImports(module_space, %import_all%) - # Remove duplicates to avoid overly long PYTHONPATH (#10977). Preserve order, - # keep first occurrence only. - python_path_entries = [ - GetWindowsPathWithUNCPrefix(d) - for d in python_path_entries - ] - - old_python_path = os.environ.get('PYTHONPATH') - if old_python_path: - python_path_entries += old_python_path.split(os.pathsep) - - python_path = os.pathsep.join(Deduplicate(python_path_entries)) - - if IsWindows(): - python_path = python_path.replace('/', os.sep) - - new_env['PYTHONPATH'] = python_path - runfiles_envkey, runfiles_envvalue = RunfilesEnvvar(module_space) + runfiles_root = find_runfiles_root(main_rel_path) + + print_verbose("runfiles root:", runfiles_root) + + if os.environ.get("RULES_PYTHON_TESTING_TELL_RUNFILES_ROOT"): + new_env["RULES_PYTHON_TESTING_RUNFILES_ROOT"] = runfiles_root + + runfiles_envkey, runfiles_envvalue = runfiles_envvar(runfiles_root) if runfiles_envkey: new_env[runfiles_envkey] = runfiles_envvalue @@ -499,48 +625,26 @@ def Main(): # See: https://docs.python.org/3.11/using/cmdline.html#envvar-PYTHONSAFEPATH new_env['PYTHONSAFEPATH'] = '1' - main_filename = os.path.join(module_space, main_rel_path) - main_filename = GetWindowsPathWithUNCPrefix(main_filename) + main_filename = join(runfiles_root, main_rel_path) + main_filename = get_windows_path_with_unc_prefix(main_filename) assert os.path.exists(main_filename), \ 'Cannot exec() %r: file not found.' % main_filename assert os.access(main_filename, os.R_OK), \ 'Cannot exec() %r: file not readable.' % main_filename - program = python_program = FindPythonBinary(module_space) - if python_program is None: - raise AssertionError('Could not find python binary: ' + PYTHON_BINARY) - - # COVERAGE_DIR is set if coverage is enabled and instrumentation is configured - # for something, though it could be another program executing this one or - # one executed by this one (e.g. an extension module). - if os.environ.get('COVERAGE_DIR'): - cov_tool = FindCoverageEntryPoint(module_space) - if cov_tool is None: - PrintVerboseCoverage('Coverage was enabled, but python coverage tool was not configured.') - else: - # Inhibit infinite recursion: - if 'PYTHON_COVERAGE' in os.environ: - del os.environ['PYTHON_COVERAGE'] - - if not os.path.exists(cov_tool): - raise EnvironmentError( - 'Python coverage tool %r not found. ' - 'Try running with VERBOSE_COVERAGE=1 to collect more information.' - % cov_tool - ) - - # coverage library expects sys.path[0] to contain the library, and replaces - # it with the directory of the program it starts. Our actual sys.path[0] is - # the runfiles directory, which must not be replaced. - # CoverageScript.do_execute() undoes this sys.path[0] setting. - # - # Update sys.path such that python finds the coverage package. The coverage - # entry point is coverage.coverage_main, so we need to do twice the dirname. - python_path_entries = new_env['PYTHONPATH'].split(os.pathsep) - python_path_entries.append(os.path.dirname(os.path.dirname(cov_tool))) - new_env['PYTHONPATH'] = os.pathsep.join(Deduplicate(python_path_entries)) + if RECREATE_VENV_AT_RUNTIME: + # When the venv is created at runtime, python_program is PYTHON_BINARY_ACTUAL + # so we have to re-point it to the symlink in the venv + python_program = _create_venv(runfiles_root, delete_dirs) else: - cov_tool = None + python_program = find_python_binary(runfiles_root) + if python_program is None: + raise AssertionError("Could not find python binary: {} or {}".format( + repr(PYTHON_BINARY), + repr(PYTHON_BINARY_ACTUAL) + )) + + python_program = get_windows_path_with_unc_prefix(python_program) # Some older Python versions on macOS (namely Python 3.7) may unintentionally # leave this environment variable set after starting the interpreter, which @@ -552,20 +656,20 @@ def Main(): new_env.update((key, val) for key, val in os.environ.items() if key not in new_env) workspace = None - if IsRunningFromZip(): + if IS_ZIPFILE: # If RUN_UNDER_RUNFILES equals 1, it means we need to # change directory to the right runfiles directory. # (So that the data files are accessible) if os.environ.get('RUN_UNDER_RUNFILES') == '1': - workspace = os.path.join(module_space, '%workspace_name%') + workspace = join(runfiles_root, WORKSPACE_NAME) try: sys.stdout.flush() - # NOTE: ExecuteFile may call execve() and lines after this will never run. - ExecuteFile( - python_program, main_filename, args, new_env, module_space, - cov_tool, workspace, - delete_module_space = delete_module_space, + # NOTE: execute_file may call execve() and lines after this will never run. + execute_file( + python_program, main_filename, args, new_env, runfiles_root, + workspace, + delete_dirs = delete_dirs, ) except EnvironmentError: @@ -573,8 +677,8 @@ def Main(): e = sys.exc_info()[1] # This exception occurs when os.execv() fails for some reason. if not getattr(e, 'filename', None): - e.filename = program # Add info to error message + e.filename = python_program # Add info to error message raise if __name__ == '__main__': - Main() + main() diff --git a/python/private/python_register_toolchains.bzl b/python/private/python_register_toolchains.bzl index cd3e9cbed7..c6f827be22 100644 --- a/python/private/python_register_toolchains.bzl +++ b/python/private/python_register_toolchains.bzl @@ -17,7 +17,7 @@ load( "//python:versions.bzl", - "DEFAULT_RELEASE_BASE_URL", + "DEFAULT_RELEASE_BASE_URLS", "MINOR_MAPPING", "PLATFORMS", "TOOL_VERSIONS", @@ -28,7 +28,7 @@ load(":full_version.bzl", "full_version") load(":python_repository.bzl", "python_repository") load( ":toolchains_repo.bzl", - "host_toolchain", + "host_compatible_python_repo", "toolchain_aliases", "toolchains_repo", ) @@ -41,13 +41,14 @@ def python_register_toolchains( register_coverage_tool = False, set_python_version_constraint = False, tool_versions = None, + platforms = PLATFORMS, minor_mapping = None, **kwargs): """Convenience macro for users which does typical setup. With `bzlmod` enabled, this function is not needed since `rules_python` is handling everything. In order to override the default behaviour from the - root module one can see the docs for the {rule}`python` extension. + root module one can see the docs for the {obj}`python` extension. - Create a repository for each built-in platform like "python_3_8_linux_amd64" - this repository is lazily fetched when Python is needed for that platform. @@ -70,18 +71,25 @@ def python_register_toolchains( tool_versions: {type}`dict` contains a mapping of version with SHASUM and platform info. If not supplied, the defaults in python/versions.bzl will be used. + platforms: {type}`dict[str, struct]` platforms to create toolchain + repositories for. Keys are platform names, and values are platform_info + structs. Note that only a subset is created, depending on what's + available in `tool_versions`. minor_mapping: {type}`dict[str, str]` contains a mapping from `X.Y` to `X.Y.Z` version. **kwargs: passed to each {obj}`python_repository` call. Returns: - On bzlmod this returns the loaded platform labels. Otherwise None. + On workspace, returns None. + + On bzlmod, returns a `dict[str, platform_info]`, which is the + subset of `platforms` that it created repositories for. """ bzlmod_toolchain_call = kwargs.pop("_internal_bzlmod_toolchain_call", False) if bzlmod_toolchain_call: register_toolchains = False - base_url = kwargs.pop("base_url", DEFAULT_RELEASE_BASE_URL) + base_urls = kwargs.pop("base_urls", DEFAULT_RELEASE_BASE_URLS) tool_versions = tool_versions or TOOL_VERSIONS minor_mapping = minor_mapping or MINOR_MAPPING @@ -89,29 +97,23 @@ def python_register_toolchains( toolchain_repo_name = "{name}_toolchains".format(name = name) - # When using unreleased Bazel versions, the version is an empty string - if native.bazel_version: - bazel_major = int(native.bazel_version.split(".")[0]) - if bazel_major < 6: - if register_coverage_tool: - # buildifier: disable=print - print(( - "WARNING: ignoring register_coverage_tool=True when " + - "registering @{name}: Bazel 6+ required, got {version}" - ).format( - name = name, - version = native.bazel_version, - )) - register_coverage_tool = False - + # list[str] of the platform names that were used loaded_platforms = [] - for platform in PLATFORMS.keys(): + + # dict[str repo name, tuple[str, platform_info]] + impl_repos = {} + for platform, platform_info in platforms.items(): sha256 = tool_versions[python_version]["sha256"].get(platform, None) if not sha256: continue loaded_platforms.append(platform) - (release_filename, urls, strip_prefix, patches, patch_strip) = get_release_info(platform, python_version, base_url, tool_versions) + (release_filename, urls, strip_prefix, patches, patch_strip) = get_release_info( + platform, + python_version, + base_urls = base_urls, + tool_versions = tool_versions, + ) # allow passing in a tool version coverage_tool = None @@ -130,11 +132,10 @@ def python_register_toolchains( )], ) + impl_repo_name = "{}_{}".format(name, platform) + impl_repos[impl_repo_name] = (platform, platform_info) python_repository( - name = "{name}_{platform}".format( - name = name, - platform = platform, - ), + name = impl_repo_name, sha256 = sha256, patches = patches, patch_strip = patch_strip, @@ -160,12 +161,6 @@ def python_register_toolchains( platform = platform, )) - host_toolchain( - name = name + "_host", - platforms = loaded_platforms, - python_version = python_version, - ) - toolchain_aliases( name = name, python_version = python_version, @@ -173,9 +168,18 @@ def python_register_toolchains( platforms = loaded_platforms, ) - # in bzlmod we write out our own toolchain repos + # in bzlmod we write out our own toolchain repos and host repos if bzlmod_toolchain_call: - return loaded_platforms + return struct( + # dict[str name, tuple[str platform_name, platform_info]] + impl_repos = impl_repos, + ) + + host_compatible_python_repo( + name = name + "_host", + platforms = loaded_platforms, + python_version = python_version, + ) toolchains_repo( name = toolchain_repo_name, diff --git a/python/private/python_repository.bzl b/python/private/python_repository.bzl index c7407c8f2c..9c44971117 100644 --- a/python/private/python_repository.bzl +++ b/python/private/python_repository.bzl @@ -15,7 +15,7 @@ """This file contains repository rules and macros to support toolchain registration. """ -load("//python:versions.bzl", "FREETHREADED", "INSTALL_ONLY", "PLATFORMS") +load("//python:versions.bzl", "FREETHREADED", "INSTALL_ONLY") load(":auth.bzl", "get_auth") load(":repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "repo_utils") load(":text_util.bzl", "render") @@ -52,6 +52,98 @@ def is_standalone_interpreter(rctx, python_interpreter_path, *, logger = None): logger = logger, ).return_code == 0 +def _get_pycache_root(rctx): + """Calculates and creates the pycache root directory. + + Returns: + {type}`path | None` The path to the pycache root, or None if it couldn't + be created. + """ + os_name = repo_utils.get_platforms_os_name(rctx) + is_windows = os_name == "windows" + + # 1. RULES_PYTHON_PYCACHE_DIR + res = rctx.getenv("RULES_PYTHON_PYCACHE_DIR") + if res: + res = res + "/" + rctx.name + return repo_utils.mkdir(rctx, res) + + # Suffix for cases 2-4 + # The first level directory is static and documented so that it is easy to + # use with e.g. --sandbox_add_mount_pair=/tmp/rules_python_pycache + suffix = "rules_python_pycache/{}/{}".format(hash(str(rctx.workspace_root)), rctx.name) + + # 2. XDG_CACHE_HOME + res = rctx.getenv("XDG_CACHE_HOME") + if res: + path = repo_utils.mkdir(rctx, rctx.path(res).get_child(suffix)) + if path: + return path + + # 3. TMP or TEMP + res = rctx.getenv("TMP") or rctx.getenv("TEMP") + if res: + path = repo_utils.mkdir(rctx, rctx.path(res).get_child(suffix)) + if path: + return path + + # 4. /tmp or Windows equivalent + if is_windows: + path = rctx.path("C:/Temp").get_child(suffix) + else: + path = rctx.path("/tmp").get_child(suffix) + + return repo_utils.mkdir(rctx, path) + +def _create_pycache_symlinks(rctx, logger): + """Finds all directories with a .py file and creates __pycache__ symlinks. + + Args: + rctx: {type}`repository_ctx` The repository rule's context object. + logger: Optional logger to use for operations. + """ + pycache_root = _get_pycache_root(rctx) + logger.info(lambda: "pycache root: {}".format(pycache_root)) + pycache_root_str = str(pycache_root) if pycache_root else None + + os_name = repo_utils.get_platforms_os_name(rctx) + null_device = "NUL" if os_name == "windows" else "/dev/null" + + queue = [rctx.path(".")] + + # Starlark doesn't support recursion, use a loop with a queue. + # Using a large range as a safeguard. + for _ in range(1000000): + if not queue: + break + p = queue.pop() + + has_py = False + for child in p.readdir(): + # Skip hidden files and directories + if child.basename.startswith("."): + continue + + if child.is_dir: + if child.basename == "__pycache__" or str(child) == pycache_root_str: + continue + queue.append(child) + elif child.basename.endswith(".py"): + has_py = True + + if has_py: + pycache_dir = p.get_child("__pycache__") + if pycache_root: + pycache_relative = repo_utils.repo_root_relative_path(rctx, pycache_dir) + target_dir = pycache_root.get_child(pycache_relative) + + repo_utils.mkdir(rctx, target_dir) + rctx.delete(pycache_dir) + rctx.symlink(target_dir, pycache_dir) + else: + rctx.delete(pycache_dir) + rctx.symlink(null_device, pycache_dir) + def _python_repository_impl(rctx): if rctx.attr.distutils and rctx.attr.distutils_content: fail("Only one of (distutils, distutils_content) should be set.") @@ -123,46 +215,11 @@ def _python_repository_impl(rctx): logger = logger, ) - # Make the Python installation read-only. This is to prevent issues due to - # pycs being generated at runtime: - # * The pycs are not deterministic (they contain timestamps) - # * Multiple processes trying to write the same pycs can result in errors. - if not rctx.attr.ignore_root_user_error: - if "windows" not in platform: - lib_dir = "lib" if "windows" not in platform else "Lib" - - repo_utils.execute_checked( - rctx, - op = "python_repository.MakeReadOnly", - arguments = [repo_utils.which_checked(rctx, "chmod"), "-R", "ugo-w", lib_dir], - logger = logger, - ) - exec_result = repo_utils.execute_unchecked( - rctx, - op = "python_repository.TestReadOnly", - arguments = [repo_utils.which_checked(rctx, "touch"), "{}/.test".format(lib_dir)], - logger = logger, - ) - - # The issue with running as root is the installation is no longer - # read-only, so the problems due to pyc can resurface. - if exec_result.return_code == 0: - stdout = repo_utils.execute_checked_stdout( - rctx, - op = "python_repository.GetUserId", - arguments = [repo_utils.which_checked(rctx, "id"), "-u"], - logger = logger, - ) - uid = int(stdout.strip()) - if uid == 0: - fail("The current user is root, please run as non-root when using the hermetic Python interpreter. See https://github.com/bazelbuild/rules_python/pull/713.") - else: - fail("The current user has CAP_DAC_OVERRIDE set, please drop this capability when using the hermetic Python interpreter. See https://github.com/bazelbuild/rules_python/pull/713.") - + _create_pycache_symlinks(rctx, logger) python_bin = "python.exe" if ("windows" in platform) else "bin/python3" if "linux" in platform: - # Workaround around https://github.com/indygreg/python-build-standalone/issues/231 + # Workaround around https://github.com/astral-sh/python-build-standalone/issues/231 for url in urls: head_and_release, _, _ = url.rpartition("/") _, _, release = head_and_release.rpartition("/") @@ -178,21 +235,20 @@ def _python_repository_impl(rctx): # building on. # # Link to the first affected release: - # https://github.com/indygreg/python-build-standalone/releases/tag/20240224 + # https://github.com/astral-sh/python-build-standalone/releases/tag/20240224 rctx.delete("share/terminfo") break glob_include = [] - glob_exclude = [] - if rctx.attr.ignore_root_user_error or "windows" in platform: - glob_exclude += [ - # These pycache files are created on first use of the associated python files. - # Exclude them from the glob because otherwise between the first time and second time a python toolchain is used," - # the definition of this filegroup will change, and depending rules will get invalidated." - # See https://github.com/bazelbuild/rules_python/issues/1008 for unconditionally adding these to toolchains so we can stop ignoring them." - "**/__pycache__/*.pyc", - "**/__pycache__/*.pyo", - ] + glob_exclude = [ + # These pycache files are created on first use of the associated python files. + # Exclude them from the glob because otherwise between the first time and second time a python toolchain is used," + # the definition of this filegroup will change, and depending rules will get invalidated." + # See https://github.com/bazel-contrib/rules_python/issues/1008 for unconditionally adding these to toolchains so we can stop ignoring them." + # pyc* is ignored because pyc creation creates temporary .pyc.NNNN files + "**/__pycache__/*.pyc*", + "**/__pycache__/*.pyo*", + ] if "windows" in platform: glob_include += [ @@ -245,7 +301,6 @@ define_hermetic_runtime_toolchain_impl( "coverage_tool": rctx.attr.coverage_tool, "distutils": rctx.attr.distutils, "distutils_content": rctx.attr.distutils_content, - "ignore_root_user_error": rctx.attr.ignore_root_user_error, "name": rctx.attr.name, "netrc": rctx.attr.netrc, "patch_strip": rctx.attr.patch_strip, @@ -262,7 +317,15 @@ define_hermetic_runtime_toolchain_impl( else: attrs["urls"] = urls - return attrs + # Bazel <8.3.0 lacks repository_ctx.repo_metadata + if not hasattr(rctx, "repo_metadata"): + return attrs + + reproducible = rctx.attr.sha256 != "" + return rctx.repo_metadata( + reproducible = reproducible, + attrs_for_reproducibility = {} if reproducible else attrs, + ) python_repository = repository_rule( _python_repository_impl, @@ -294,8 +357,8 @@ For more information see {attr}`py_runtime.coverage_tool`. mandatory = False, ), "ignore_root_user_error": attr.bool( - default = False, - doc = "Whether the check for root should be ignored or not. This causes cache misses with .pyc files.", + default = True, + doc = "Noop, will be removed in the next major release", mandatory = False, ), "netrc": attr.string( @@ -323,7 +386,6 @@ function defaults (e.g. `single_version_override` for `MODULE.bazel` files. "platform": attr.string( doc = "The platform name for the Python interpreter tarball.", mandatory = True, - values = PLATFORMS.keys(), ), "python_version": attr.string( doc = "The Python version.", diff --git a/python/private/pythons_hub.bzl b/python/private/pythons_hub.bzl index ac928ffc96..c64abf9887 100644 --- a/python/private/pythons_hub.bzl +++ b/python/private/pythons_hub.bzl @@ -15,8 +15,9 @@ "Repo rule used by bzlmod extension to create a repo that has a map of Python interpreters and their labels" load("//python:versions.bzl", "PLATFORMS") +load(":pbs_manifest.bzl", "parse_runtime_manifest") load(":text_util.bzl", "render") -load(":toolchains_repo.bzl", "python_toolchain_build_file_content") +load(":toolchains_repo.bzl", "toolchain_suite_content") def _have_same_length(*lists): if not lists: @@ -24,17 +25,19 @@ def _have_same_length(*lists): return len({len(length): None for length in lists}) == 1 _HUB_BUILD_FILE_TEMPLATE = """\ -load("@bazel_skylib//:bzl_library.bzl", "bzl_library") +# Generated by @rules_python//python/private:pythons_hub.bzl + load("@@{rules_python}//python/private:py_toolchain_suite.bzl", "py_toolchain_suite") +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") bzl_library( - name = "interpreters_bzl", + name = "interpreters", srcs = ["interpreters.bzl"], visibility = ["@rules_python//:__subpackages__"], ) bzl_library( - name = "versions_bzl", + name = "versions", srcs = ["versions.bzl"], visibility = ["@rules_python//:__subpackages__"], ) @@ -42,54 +45,47 @@ bzl_library( {toolchains} """ -def _hub_build_file_content( - prefixes, - python_versions, - set_python_version_constraints, - user_repository_names, - workspace_location, - loaded_platforms): - """This macro iterates over each of the lists and returns the toolchain content. - - python_toolchain_build_file_content is called to generate each of the toolchain - definitions. - """ - - if not _have_same_length(python_versions, set_python_version_constraints, user_repository_names): +def _hub_build_file_content(rctx): + # Verify a precondition. If these don't match, then something went wrong. + if not _have_same_length( + rctx.attr.toolchain_names, + rctx.attr.toolchain_platform_keys, + rctx.attr.toolchain_repo_names, + rctx.attr.toolchain_target_compatible_with_map, + rctx.attr.toolchain_target_settings_map, + rctx.attr.toolchain_set_python_version_constraints, + rctx.attr.toolchain_python_versions, + ): fail("all lists must have the same length") - # Iterate over the length of python_versions and call - # build the toolchain content by calling python_toolchain_build_file_content - toolchains = "\n".join( - [ - python_toolchain_build_file_content( - prefix = prefixes[i], - python_version = python_versions[i], - set_python_version_constraint = set_python_version_constraints[i], - user_repository_name = user_repository_names[i], - loaded_platforms = { - k: v - for k, v in PLATFORMS.items() - if k in loaded_platforms[python_versions[i]] - }, - ) - for i in range(len(python_versions)) - ], - ) + #pad_length = len(str(len(rctx.attr.toolchain_names))) + 1 + pad_length = 4 + toolchains = [] + for i, base_name in enumerate(rctx.attr.toolchain_names): + key = str(i) + platform = rctx.attr.toolchain_platform_keys[key] + if platform in PLATFORMS: + flag_values = PLATFORMS[platform].flag_values + else: + flag_values = {} + + toolchains.append(toolchain_suite_content( + prefix = "_{}_{}".format(render.left_pad_zero(i, pad_length), base_name), + user_repository_name = rctx.attr.toolchain_repo_names[key], + target_compatible_with = rctx.attr.toolchain_target_compatible_with_map[key], + flag_values = flag_values, + target_settings = rctx.attr.toolchain_target_settings_map[key], + set_python_version_constraint = rctx.attr.toolchain_set_python_version_constraints[key], + python_version = rctx.attr.toolchain_python_versions[key], + )) return _HUB_BUILD_FILE_TEMPLATE.format( - toolchains = toolchains, - rules_python = workspace_location.workspace_name, + toolchains = "\n".join(toolchains), + rules_python = rctx.attr._rules_python_workspace.repo_name, ) _interpreters_bzl_template = """ -INTERPRETER_LABELS = {{ -{interpreter_labels} -}} -""" - -_line_for_hub_template = """\ - "{name}_host": Label("@{name}_host//:python"), +INTERPRETER_LABELS = {labels} """ _versions_bzl_template = """ @@ -103,41 +99,44 @@ def _hub_repo_impl(rctx): # write them to the BUILD file. rctx.file( "BUILD.bazel", - _hub_build_file_content( - rctx.attr.toolchain_prefixes, - rctx.attr.toolchain_python_versions, - rctx.attr.toolchain_set_python_version_constraints, - rctx.attr.toolchain_user_repository_names, - rctx.attr._rules_python_workspace, - rctx.attr.loaded_platforms, - ), + _hub_build_file_content(rctx), executable = False, ) # Create a dict that is later used to create # a symlink to a interpreter. - interpreter_labels = "".join([ - _line_for_hub_template.format(name = name) - for name in rctx.attr.toolchain_user_repository_names - ]) - rctx.file( "interpreters.bzl", _interpreters_bzl_template.format( - interpreter_labels = interpreter_labels, + labels = render.dict( + { + name: 'Label("@{}//:python")'.format(name) + for name in rctx.attr.host_compatible_repo_names + }, + value_repr = str, + ), ), executable = False, ) + python_versions = rctx.attr.python_versions + if not python_versions and not rctx.attr.toolchain_python_versions: + content = rctx.read(rctx.path(Label("//python/private:runtimes_manifest.txt"))) + entries = parse_runtime_manifest(content) + python_versions_str = render.list(sorted({getattr(e, "python_version", ""): None for e in entries if getattr(e, "python_version", "")})) + + else: + python_versions_str = render.list(python_versions) if python_versions else render.list(sorted({ + v: None + for v in rctx.attr.toolchain_python_versions + })) + rctx.file( "versions.bzl", _versions_bzl_template.format( default_python_version = rctx.attr.default_python_version, minor_mapping = render.dict(rctx.attr.minor_mapping), - python_versions = rctx.attr.python_versions or render.list(sorted({ - v: None - for v in rctx.attr.toolchain_python_versions - })), + python_versions = python_versions_str, ), executable = False, ) @@ -154,8 +153,9 @@ This rule also writes out the various toolchains for the different Python versio doc = "Default Python version for the build in `X.Y` or `X.Y.Z` format.", mandatory = True, ), - "loaded_platforms": attr.string_list_dict( - doc = "The list of loaded platforms keyed by the toolchain full python version", + "host_compatible_repo_names": attr.string_list( + doc = "Names of `host_compatible_python_repo` repos.", + mandatory = True, ), "minor_mapping": attr.string_dict( doc = "The minor mapping of the `X.Y` to `X.Y.Z` format that is used in config settings.", @@ -165,20 +165,32 @@ This rule also writes out the various toolchains for the different Python versio doc = "The list of python versions to include in the `interpreters.bzl` if the toolchains are not specified. Used in `WORKSPACE` builds.", mandatory = False, ), - "toolchain_prefixes": attr.string_list( - doc = "List prefixed for the toolchains", + "toolchain_names": attr.string_list( + doc = "Names of toolchains", + mandatory = True, + ), + "toolchain_platform_keys": attr.string_dict( + doc = "The platform key in PLATFORMS for toolchains.", mandatory = True, ), - "toolchain_python_versions": attr.string_list( + "toolchain_python_versions": attr.string_dict( doc = "List of Python versions for the toolchains. In `X.Y.Z` format.", mandatory = True, ), - "toolchain_set_python_version_constraints": attr.string_list( + "toolchain_repo_names": attr.string_dict( + doc = "The repo names containing toolchain implementations.", + mandatory = True, + ), + "toolchain_set_python_version_constraints": attr.string_dict( doc = "List of version contraints for the toolchains", mandatory = True, ), - "toolchain_user_repository_names": attr.string_list( - doc = "List of the user repo names for the toolchains", + "toolchain_target_compatible_with_map": attr.string_list_dict( + doc = "The target_compatible_with settings for toolchains.", + mandatory = True, + ), + "toolchain_target_settings_map": attr.string_list_dict( + doc = "The target_settings for toolchains", mandatory = True, ), "_rules_python_workspace": attr.label(default = Label("//:does_not_matter_what_this_name_is")), diff --git a/python/private/repl.bzl b/python/private/repl.bzl new file mode 100644 index 0000000000..838166a187 --- /dev/null +++ b/python/private/repl.bzl @@ -0,0 +1,84 @@ +"""Implementation of the rules to expose a REPL.""" + +load("//python:py_binary.bzl", _py_binary = "py_binary") + +def _generate_repl_main_impl(ctx): + stub_repo = ctx.attr.stub.label.repo_name or ctx.workspace_name + stub_path = "/".join([stub_repo, ctx.file.stub.short_path]) + + out = ctx.actions.declare_file(ctx.label.name + ".py") + + # Point the generated main file at the stub. + ctx.actions.expand_template( + template = ctx.file._template, + output = out, + substitutions = { + "%stub_path%": stub_path, + }, + ) + + return [DefaultInfo(files = depset([out]))] + +_generate_repl_main = rule( + implementation = _generate_repl_main_impl, + attrs = { + "stub": attr.label( + mandatory = True, + allow_single_file = True, + doc = ("The stub responsible for actually invoking the final shell. " + + "See the \"Customizing the REPL\" docs for details."), + ), + "_template": attr.label( + default = "//python/private:repl_template.py", + allow_single_file = True, + doc = "The template to use for generating `out`.", + ), + }, + doc = """\ +Generates a "main" script for a py_binary target that starts a Python REPL. + +The template is designed to take care of the majority of the logic. The user +customizes the exact shell that will be started via the stub. The stub is a +simple shell script that imports the desired shell and then executes it. + +The target's name is used for the output filename (with a .py extension). +""", +) + +def py_repl_binary(name, stub, deps = [], data = [], **kwargs): + """A py_binary target that executes a REPL when run. + + The stub is the script that ultimately decides which shell the REPL will run. + It can be as simple as this: + + import code + code.interact() + + Or it can load something like IPython instead. + + Args: + name: Name of the generated py_binary target. + stub: The script that invokes the shell. + deps: The dependencies of the py_binary. + data: The runtime dependencies of the py_binary. + **kwargs: Forwarded to the py_binary. + """ + _generate_repl_main( + name = "%s_py" % name, + stub = stub, + ) + + _py_binary( + name = name, + srcs = [ + ":%s_py" % name, + ], + main = "%s_py.py" % name, + data = data + [ + stub, + ], + deps = deps + [ + "//python/runfiles", + ], + **kwargs + ) diff --git a/python/private/repl_template.py b/python/private/repl_template.py new file mode 100644 index 0000000000..dd8beb9784 --- /dev/null +++ b/python/private/repl_template.py @@ -0,0 +1,47 @@ +import os +import runpy +import sys +from pathlib import Path + +from python.runfiles import runfiles + +# runfiles.py will reject paths which aren't normalized, which can happen when the REPL rules are +# used from a remote module. +STUB_PATH = os.path.normpath("%stub_path%") + + +def start_repl(): + if sys.stdin.isatty(): + # Print the banner similar to how python does it on startup when running interactively. + cprt = 'Type "help", "copyright", "credits" or "license" for more information.' + sys.stderr.write("Python %s on %s\n%s\n" % (sys.version, sys.platform, cprt)) + + # If there's a PYTHONSTARTUP script, we need to capture the new variables + # that it defines. + new_globals = {} + + # Simulate Python's behavior when a valid startup script is defined by the + # PYTHONSTARTUP variable. If this file path fails to load, print the error + # and revert to the default behavior. + # + # See upstream for more information: + # https://docs.python.org/3/using/cmdline.html#envvar-PYTHONSTARTUP + if startup_file := os.getenv("PYTHONSTARTUP"): + try: + source_code = Path(startup_file).read_text() + except Exception as error: + print(f"{type(error).__name__}: {error}") + else: + compiled_code = compile(source_code, filename=startup_file, mode="exec") + eval(compiled_code, new_globals) + + bazel_runfiles = runfiles.Create() + runpy.run_path( + bazel_runfiles.Rlocation(STUB_PATH), + init_globals=new_globals, + run_name="__main__", + ) + + +if __name__ == "__main__": + start_repl() diff --git a/python/private/repo_utils.bzl b/python/private/repo_utils.bzl index 0e3f7b024b..0e83fda28c 100644 --- a/python/private/repo_utils.bzl +++ b/python/private/repo_utils.bzl @@ -29,15 +29,20 @@ def _is_repo_debug_enabled(mrctx): Returns: True if enabled, False if not. """ - return _getenv(mrctx, REPO_DEBUG_ENV_VAR) == "1" + return mrctx.getenv(REPO_DEBUG_ENV_VAR) == "1" -def _logger(mrctx, name = None): +def _logger(mrctx = None, name = None, verbosity_level = None, printer = None, mod = None): """Creates a logger instance for printing messages. Args: mrctx: repository_ctx or module_ctx object. If the attribute `_rule_name` is present, it will be included in log messages. name: name for the logger. Optional for repository_ctx usage. + verbosity_level: {type}`int | None` verbosity level. If not set, + taken from `mrctx`. + printer: a function to use for printing. Defaults to `print` or `fail` depending + on the logging method. + mod: {type}`module_ctx.module`. The module for which the logger is created. Returns: A struct with attributes logging: trace, debug, info, warn, fail. @@ -46,18 +51,30 @@ def _logger(mrctx, name = None): the logger injected into the function work as expected by terminating on the given line. """ - if _is_repo_debug_enabled(mrctx): - verbosity_level = "DEBUG" - else: - verbosity_level = "WARN" + default_verbosity_level = "WARN" + if mod: + if name: + name = "{}:{}".format(mod.name, name) + else: + name = mod.name + + if not mod.is_root: + default_verbosity_level = "ERROR" # the warnings are non actionable anyway, but we should keep them. - env_var_verbosity = _getenv(mrctx, REPO_VERBOSITY_ENV_VAR) - verbosity_level = env_var_verbosity or verbosity_level + if verbosity_level == None: + if _is_repo_debug_enabled(mrctx): + default_verbosity_level = "DEBUG" + + env_var_verbosity = mrctx.getenv(REPO_VERBOSITY_ENV_VAR) + verbosity_level = env_var_verbosity or default_verbosity_level verbosity = { "DEBUG": 2, + "ERROR": -1, + "FAIL": -1, "INFO": 1, "TRACE": 3, + "WARN": 0, }.get(verbosity_level, 0) if hasattr(mrctx, "attr"): @@ -66,10 +83,15 @@ def _logger(mrctx, name = None): elif not name: fail("The name has to be specified when using the logger with `module_ctx`") + failures = [] + def _log(enabled_on_verbosity, level, message_cb_or_str, printer = print): if verbosity < enabled_on_verbosity: return + if level == "FAIL": + failures.append(None) + if type(message_cb_or_str) == "string": message = message_cb_or_str else: @@ -82,11 +104,12 @@ def _logger(mrctx, name = None): ), message) # buildifier: disable=print return struct( - trace = lambda message_cb: _log(3, "TRACE", message_cb), - debug = lambda message_cb: _log(2, "DEBUG", message_cb), - info = lambda message_cb: _log(1, "INFO", message_cb), - warn = lambda message_cb: _log(0, "WARNING", message_cb), - fail = lambda message_cb: _log(-1, "FAIL", message_cb, fail), + trace = lambda message_cb: _log(3, "TRACE", message_cb, printer or print), + debug = lambda message_cb: _log(2, "DEBUG", message_cb, printer or print), + info = lambda message_cb: _log(1, "INFO", message_cb, printer or print), + warn = lambda message_cb: _log(0, "WARNING", message_cb, printer or print), + fail = lambda message_cb: _log(-1, "FAIL", message_cb, printer or fail), + failed = lambda: len(failures) != 0, ) def _execute_internal( @@ -97,6 +120,8 @@ def _execute_internal( arguments, environment = {}, logger = None, + log_stdout = True, + log_stderr = True, **kwargs): """Execute a subprocess with debugging instrumentation. @@ -115,6 +140,10 @@ def _execute_internal( logger: optional `Logger` to use for logging execution details. Must be specified when using module_ctx. If not specified, a default will be created. + log_stdout: If True (the default), write stdout to the logged message. Setting + to False can be useful for large stdout messages or for secrets. + log_stderr: If True (the default), write stderr to the logged message. Setting + to False can be useful for large stderr messages or for secrets. **kwargs: additional kwargs to pass onto rctx.execute Returns: @@ -159,7 +188,7 @@ def _execute_internal( cwd = _cwd_to_str(mrctx, kwargs), timeout = _timeout_to_str(kwargs), env_str = _env_to_str(environment), - output = _outputs_to_str(result), + output = _outputs_to_str(result, log_stdout = log_stdout, log_stderr = log_stderr), )) elif _is_repo_debug_enabled(mrctx): logger.debug(( @@ -170,10 +199,10 @@ def _execute_internal( op = op, status = "success" if result.return_code == 0 else "failure", return_code = result.return_code, - output = _outputs_to_str(result), + output = _outputs_to_str(result, log_stdout = log_stdout, log_stderr = log_stderr), )) - result_kwargs = {k: getattr(result, k) for k in dir(result)} + result_kwargs = {k: getattr(result, k) for k in dir(result) if k not in ["to_json", "to_proto"]} return struct( describe_failure = lambda: _execute_describe_failure( op = op, @@ -182,6 +211,8 @@ def _execute_internal( mrctx = mrctx, kwargs = kwargs, environment = environment, + log_stdout = log_stdout, + log_stderr = log_stderr, ), **result_kwargs ) @@ -219,7 +250,16 @@ def _execute_checked_stdout(*args, **kwargs): """Calls execute_checked, but only returns the stdout value.""" return _execute_checked(*args, **kwargs).stdout -def _execute_describe_failure(*, op, arguments, result, mrctx, kwargs, environment): +def _execute_describe_failure( + *, + op, + arguments, + result, + mrctx, + kwargs, + environment, + log_stdout = True, + log_stderr = True): return ( "repo.execute: {op}: failure:\n" + " command: {cmd}\n" + @@ -235,7 +275,7 @@ def _execute_describe_failure(*, op, arguments, result, mrctx, kwargs, environme cwd = _cwd_to_str(mrctx, kwargs), timeout = _timeout_to_str(kwargs), env_str = _env_to_str(environment), - output = _outputs_to_str(result), + output = _outputs_to_str(result, log_stdout = log_stdout, log_stderr = log_stderr), ) def _which_checked(mrctx, binary_name): @@ -256,7 +296,7 @@ def _which_checked(mrctx, binary_name): def _which_unchecked(mrctx, binary_name): """Tests to see if a binary exists. - This is also watch the `PATH` environment variable. + Watches the `PATH` environment variable if the binary doesn't exist. Args: binary_name: name of the binary to find. @@ -268,12 +308,12 @@ def _which_unchecked(mrctx, binary_name): * `describe_failure`: `Callable | None`; takes no args. If the binary couldn't be found, provides a detailed error description. """ - path = _getenv(mrctx, "PATH", "") binary = mrctx.which(binary_name) if binary: - _watch(mrctx, binary) + mrctx.watch(binary) describe_failure = None else: + path = mrctx.getenv("PATH", "") describe_failure = lambda: _which_describe_failure(binary_name, path) return struct( @@ -282,17 +322,80 @@ def _which_unchecked(mrctx, binary_name): ) def _which_describe_failure(binary_name, path): + if "\\" in path or ";" in path: + path_parts = path.split(";") + else: + path_parts = path.split(":") + for i, v in enumerate(path_parts): + path_parts[i] = " [{}]: {}".format(i, v) return ( "Unable to find the binary '{binary_name}' on PATH.\n" + - " PATH = {path}" + " PATH entries:\n" + + "{path_str}" ).format( binary_name = binary_name, - path = path, + path_str = "\n".join(path_parts), ) -def _getenv(mrctx, name, default = None): - # Bazel 7+ API has (repository|module)_ctx.getenv - return getattr(mrctx, "getenv", mrctx.os.environ.get)(name, default) +def _mkdir(mrctx, path): + path = mrctx.path(path) + if path.exists: + return path + + repo_root = str(mrctx.path(".")) + path_str = str(path) + + if not _is_relative_to(mrctx, path_str, repo_root): + mkdir_bin = mrctx.which("mkdir") + if not mkdir_bin: + return None + res = mrctx.execute([mkdir_bin, "-p", path_str]) + if res.return_code != 0: + return None + return path + else: + placeholder = path.get_child(".placeholder") + mrctx.file(placeholder) + mrctx.delete(placeholder) + return path + +def _norm_path(mrctx, p): + p = str(p) + + # Windows is case-insensitive + if _get_platforms_os_name(mrctx) == "windows": + return p.lower() + return p + +def _relative_to(mrctx, path, parent, fail = fail): + path_str = str(path) + parent_str = str(parent) + path_d = _norm_path(mrctx, path_str) + "/" + parent_d = _norm_path(mrctx, parent_str) + "/" + if path_d.startswith(parent_d): + return path_str[len(parent_str):].removeprefix("/") + else: + fail("{} is not relative to {}".format(path, parent)) + +def _is_relative_to(mrctx, path, parent): + """Tell if `path` is equal to or beneath `parent`.""" + path_d = _norm_path(mrctx, path) + "/" + parent_d = _norm_path(mrctx, parent) + "/" + return path_d.startswith(parent_d) + +def _repo_root_relative_path(mrctx, path): + """Takes a path object and returns a repo-relative path string. + + Args: + mrctx: module_ctx or repository_ctx + path: {type}`path` a path within `mrctx` + + Returns: + {type}`str` a repo-root-relative path string. + """ + repo_root = str(mrctx.path(".")) + path_str = str(path) + return _relative_to(mrctx, path_str, repo_root) def _args_to_str(arguments): return " ".join([_arg_repr(a) for a in arguments]) @@ -330,11 +433,11 @@ def _env_to_str(environment): def _timeout_to_str(kwargs): return kwargs.get("timeout", "") -def _outputs_to_str(result): +def _outputs_to_str(result, log_stdout = True, log_stderr = True): lines = [] items = [ - ("stdout", result.stdout), - ("stderr", result.stderr), + ("stdout", result.stdout if log_stdout else ""), + ("stderr", result.stderr if log_stderr else ""), ] for name, content in items: if content: @@ -358,7 +461,7 @@ def _get_platforms_os_name(mrctx): """Return the name in @platforms//os for the host os. Args: - mrctx: module_ctx or repository_ctx. + mrctx: {type}`module_ctx | repository_ctx` Returns: `str`. The target name. @@ -387,12 +490,15 @@ def _get_platforms_cpu_name(mrctx): `str`. The target name. """ arch = mrctx.os.arch.lower() + if arch in ["i386", "i486", "i586", "i686", "i786", "x86"]: return "x86_32" if arch in ["amd64", "x86_64", "x64"]: return "x86_64" - if arch in ["ppc", "ppc64", "ppc64le"]: + if arch in ["ppc", "ppc64"]: return "ppc" + if arch in ["ppc64le"]: + return "ppc64le" if arch in ["arm", "armv7l"]: return "arm" if arch in ["aarch64"]: @@ -405,36 +511,97 @@ def _get_platforms_cpu_name(mrctx): return "riscv64" return arch -# TODO: Remove after Bazel 6 support dropped -def _watch(mrctx, *args, **kwargs): - """Calls mrctx.watch, if available.""" - if not args and not kwargs: - fail("'watch' needs at least a single argument.") +def _extract(mrctx, *, archive, supports_whl_extraction = False, extract_needs_chmod = False, **kwargs): + """Extract an archive + + TODO: remove when the earliest supported bazel version is at least 8.3. + + Note, we are using the parameter here because there is very little ways how we can detect + whether we can support just extracting the whl. + """ + archive_original = None + if not supports_whl_extraction and archive.basename.endswith(".whl"): + archive_original = archive + archive = mrctx.path(archive.basename + ".zip") + mrctx.symlink(archive_original, archive) + + mrctx.extract( + archive = archive, + **kwargs + ) + if archive_original: + if not mrctx.delete(archive): + fail("Failed to remove the symlink after extracting") + + if extract_needs_chmod: + _maybe_fix_permissions(mrctx, whl_path = archive) + +def _maybe_fix_permissions(mrctx, *, whl_path, logger = None): + if not logger and hasattr(mrctx, "attr"): + logger = _logger(mrctx) + elif not logger: + fail("logger must be specified when using 'module_ctx'") + + os_name = _get_platforms_os_name(mrctx) + if os_name != "windows": + result = _execute_unchecked( + mrctx, + op = "Fixing wheel permissions {}".format(whl_path), + arguments = ["chmod", "-R", "a+rX", "."], + logger = logger, + ) + if result.return_code != 0: + logger.warn(lambda: "Failed to fix file permissions: {}".format(result.stderr)) + +def _rename(mrctx, src, dest): + """Rename a file or directory. - if hasattr(mrctx, "watch"): - mrctx.watch(*args, **kwargs) + TODO: remove when the earliest supported bazel version is at least 8.0. -# TODO: Remove after Bazel 6 support dropped -def _watch_tree(mrctx, *args, **kwargs): - """Calls mrctx.watch_tree, if available.""" - if not args and not kwargs: - fail("'watch_tree' needs at least a single argument.") + Args: + mrctx: module_ctx or repository_ctx object + src: {type}`path` the source path + dest: {type}`path` the destination path + """ + if hasattr(mrctx, "rename"): + mrctx.rename(src, dest) + return + + # Fallback for Bazel < 8.0 + os_name = _get_platforms_os_name(mrctx) + if os_name == "windows": + # On Windows, we use `cmd.exe /c move` to rename files/directories. + # We need to use backslashes for the paths. + res = mrctx.execute([ + "cmd.exe", + "/c", + "move", + str(src).replace("/", "\\"), + str(dest).replace("/", "\\"), + ]) + else: + res = mrctx.execute(["mv", str(src), str(dest)]) - if hasattr(mrctx, "watch_tree"): - mrctx.watch_tree(*args, **kwargs) + if res.return_code != 0: + fail("Failed to rename {} to {}: {}".format(src, dest, res.stderr)) repo_utils = struct( # keep sorted execute_checked = _execute_checked, execute_checked_stdout = _execute_checked_stdout, execute_unchecked = _execute_unchecked, + extract = _extract, get_platforms_cpu_name = _get_platforms_cpu_name, get_platforms_os_name = _get_platforms_os_name, - getenv = _getenv, is_repo_debug_enabled = _is_repo_debug_enabled, logger = _logger, - watch = _watch, - watch_tree = _watch_tree, + maybe_fix_permissions = _maybe_fix_permissions, + mkdir = _mkdir, + norm_path = _norm_path, + relative_to = _relative_to, + is_relative_to = _is_relative_to, + rename = _rename, + repo_root_relative_path = _repo_root_relative_path, which_checked = _which_checked, which_unchecked = _which_unchecked, ) diff --git a/python/private/rule_builders.bzl b/python/private/rule_builders.bzl new file mode 100644 index 0000000000..876ca2bf97 --- /dev/null +++ b/python/private/rule_builders.bzl @@ -0,0 +1,720 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Builders for creating rules, aspects et al. + +When defining rules, Bazel only allows creating *immutable* objects that can't +be introspected. This makes it difficult to perform arbitrary customizations of +how a rule is defined, which makes extending a rule implementation prone to +copy/paste issues and version skew. + +These builders are, essentially, mutable and inspectable wrappers for those +Bazel objects. This allows defining a rule where the values are mutable and +callers can customize them to derive their own variant of the rule while still +inheriting everything else about the rule. + +To that end, the builders are not strict in how they handle values. They +generally assume that the values provided are valid and provide ways to +override their logic and force particular values to be used when they are +eventually converted to the args for calling e.g. `rule()`. + +:::{important} +When using builders, most lists, dicts, et al passed into them **must** be +locally created values, otherwise they won't be mutable. This is due to Bazel's +implicit immutability rules: after evaluating a `.bzl` file, its global +variables are frozen. +::: + +:::{tip} +To aid defining reusable pieces, many APIs accept no-arg callable functions +that create a builder. For example, common attributes can be stored +in a `dict[str, lambda]`, e.g. `ATTRS = {"srcs": lambda: LabelList(...)}`. +::: + +Example usage: + +``` + +load(":rule_builders.bzl", "ruleb") +load(":attr_builders.bzl", "attrb") + +# File: foo_binary.bzl +_COMMON_ATTRS = { + "srcs": lambda: attrb.LabelList(...), +} + +def create_foo_binary_builder(): + foo = ruleb.Rule( + executable = True, + ) + foo.implementation.set(_foo_binary_impl) + foo.attrs.update(COMMON_ATTRS) + return foo + +def create_foo_test_builder(): + foo = create_foo_binary_build() + + binary_impl = foo.implementation.get() + def foo_test_impl(ctx): + binary_impl(ctx) + ... + + foo.implementation.set(foo_test_impl) + foo.executable.set(False) + foo.test.test(True) + foo.attrs.update( + _coverage = attrb.Label(default="//:coverage") + ) + return foo + +foo_binary = create_foo_binary_builder().build() +foo_test = create_foo_test_builder().build() + +# File: custom_foo_binary.bzl +load(":foo_binary.bzl", "create_foo_binary_builder") + +def create_custom_foo_binary(): + r = create_foo_binary_builder() + r.attrs["srcs"].default.append("whatever.txt") + return r.build() + +custom_foo_binary = create_custom_foo_binary() +``` + +:::{versionadded} 1.3.0 +::: +""" + +load("@bazel_skylib//lib:types.bzl", "types") +load( + ":builders_util.bzl", + "kwargs_getter", + "kwargs_getter_doc", + "kwargs_set_default_dict", + "kwargs_set_default_doc", + "kwargs_set_default_ignore_none", + "kwargs_set_default_list", + "kwargs_setter", + "kwargs_setter_doc", + "list_add_unique", + "normalize_transition_in_out_value", + "normalize_transition_in_out_values", +) + +# Various string constants for kwarg key names used across two or more +# functions, or in contexts with optional lookups (e.g. dict.dict, key in dict). +# Constants are used to reduce the chance of typos. +# NOTE: These keys are often part of function signature via `**kwargs`; they +# are not simply internal names. +_ATTRS = "attrs" +_CFG = "cfg" +_EXEC_COMPATIBLE_WITH = "exec_compatible_with" +_EXEC_GROUPS = "exec_groups" +_IMPLEMENTATION = "implementation" +_INPUTS = "inputs" +_OUTPUTS = "outputs" +_TOOLCHAINS = "toolchains" + +def _is_builder(obj): + return hasattr(obj, "build") + +def _ExecGroup_typedef(): + """Builder for {external:bzl:obj}`exec_group` + + :::{function} toolchains() -> list[ToolchainType] + ::: + + :::{function} exec_compatible_with() -> list[str | Label] + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + """ + +def _ExecGroup_new(**kwargs): + """Creates a builder for {external:bzl:obj}`exec_group`. + + Args: + **kwargs: Same as {external:bzl:obj}`exec_group` + + Returns: + {type}`ExecGroup` + """ + kwargs_set_default_list(kwargs, _TOOLCHAINS) + kwargs_set_default_list(kwargs, _EXEC_COMPATIBLE_WITH) + + for i, value in enumerate(kwargs[_TOOLCHAINS]): + kwargs[_TOOLCHAINS][i] = _ToolchainType_maybe_from(value) + + # buildifier: disable=uninitialized + self = struct( + toolchains = kwargs_getter(kwargs, _TOOLCHAINS), + exec_compatible_with = kwargs_getter(kwargs, _EXEC_COMPATIBLE_WITH), + kwargs = kwargs, + build = lambda: _ExecGroup_build(self), + ) + return self + +def _ExecGroup_maybe_from(obj): + if types.is_function(obj): + return obj() + else: + return obj + +def _ExecGroup_build(self): + kwargs = dict(self.kwargs) + if kwargs.get(_TOOLCHAINS): + kwargs[_TOOLCHAINS] = [ + v.build() if _is_builder(v) else v + for v in kwargs[_TOOLCHAINS] + ] + if kwargs.get(_EXEC_COMPATIBLE_WITH): + kwargs[_EXEC_COMPATIBLE_WITH] = [ + v.build() if _is_builder(v) else v + for v in kwargs[_EXEC_COMPATIBLE_WITH] + ] + return exec_group(**kwargs) + +# buildifier: disable=name-conventions +ExecGroup = struct( + TYPEDEF = _ExecGroup_typedef, + new = _ExecGroup_new, + build = _ExecGroup_build, +) + +def _ToolchainType_typedef(): + """Builder for {obj}`config_common.toolchain_type` + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} mandatory() -> bool + ::: + + :::{function} name() -> str | Label | None + ::: + + :::{function} set_name(v: str) + ::: + + :::{function} set_mandatory(v: bool) + ::: + """ + +def _ToolchainType_new(name = None, **kwargs): + """Creates a builder for `config_common.toolchain_type`. + + Args: + name: {type}`str | Label | None` the toolchain type target. + **kwargs: Same as {obj}`config_common.toolchain_type` + + Returns: + {type}`ToolchainType` + """ + kwargs["name"] = name + kwargs_set_default_ignore_none(kwargs, "mandatory", True) + + # buildifier: disable=uninitialized + self = struct( + # keep sorted + build = lambda: _ToolchainType_build(self), + kwargs = kwargs, + mandatory = kwargs_getter(kwargs, "mandatory"), + name = kwargs_getter(kwargs, "name"), + set_mandatory = kwargs_setter(kwargs, "mandatory"), + set_name = kwargs_setter(kwargs, "name"), + ) + return self + +def _ToolchainType_maybe_from(obj): + if types.is_string(obj) or type(obj) == "Label": + return ToolchainType.new(name = obj) + elif types.is_function(obj): + # A lambda to create a builder + return obj() + else: + # For lack of another option, return it as-is. + # Presumably it's already a builder or other valid object. + return obj + +def _ToolchainType_build(self): + """Builds a `config_common.toolchain_type` + + Args: + self: implicitly added + + Returns: + {type}`toolchain_type` + """ + kwargs = dict(self.kwargs) + name = kwargs.pop("name") # Name must be positional + return config_common.toolchain_type(name, **kwargs) + +# buildifier: disable=name-conventions +ToolchainType = struct( + TYPEDEF = _ToolchainType_typedef, + new = _ToolchainType_new, + build = _ToolchainType_build, +) + +def _RuleCfg_typedef(): + """Wrapper for `rule.cfg` arg. + + :::{function} implementation() -> str | callable | None | config.target | config.none + ::: + + ::::{function} inputs() -> list[Label] + + :::{seealso} + The {obj}`add_inputs()` and {obj}`update_inputs` methods for adding unique + values. + ::: + :::: + + :::{function} outputs() -> list[Label] + + :::{seealso} + The {obj}`add_outputs()` and {obj}`update_outputs` methods for adding unique + values. + ::: + ::: + + :::{function} set_implementation(v: str | callable | None | config.target | config.none) + + The string values "target" and "none" are supported. + ::: + """ + +def _RuleCfg_new(rule_cfg_arg): + """Creates a builder for the `rule.cfg` arg. + + Args: + rule_cfg_arg: {type}`str | dict | None` The `cfg` arg passed to Rule(). + + Returns: + {type}`RuleCfg` + """ + state = {} + if types.is_dict(rule_cfg_arg): + state.update(rule_cfg_arg) + else: + # Assume its a string, config.target, config.none, or other + # valid object. + state[_IMPLEMENTATION] = rule_cfg_arg + + kwargs_set_default_list(state, _INPUTS) + kwargs_set_default_list(state, _OUTPUTS) + + normalize_transition_in_out_values("input", state[_INPUTS]) + normalize_transition_in_out_values("output", state[_OUTPUTS]) + + # buildifier: disable=uninitialized + self = struct( + add_inputs = lambda *a, **k: _RuleCfg_add_inputs(self, *a, **k), + add_outputs = lambda *a, **k: _RuleCfg_add_outputs(self, *a, **k), + _state = state, + build = lambda: _RuleCfg_build(self), + implementation = kwargs_getter(state, _IMPLEMENTATION), + inputs = kwargs_getter(state, _INPUTS), + outputs = kwargs_getter(state, _OUTPUTS), + set_implementation = kwargs_setter(state, _IMPLEMENTATION), + update_inputs = lambda *a, **k: _RuleCfg_update_inputs(self, *a, **k), + update_outputs = lambda *a, **k: _RuleCfg_update_outputs(self, *a, **k), + ) + return self + +def _RuleCfg_add_inputs(self, *inputs): + """Adds an input to the list of inputs, if not present already. + + :::{seealso} + The {obj}`update_inputs()` method for adding a collection of + values. + ::: + + Args: + self: implicitly arg. + *inputs: {type}`Label` the inputs to add. Note that a `Label`, + not `str`, should be passed to ensure different apparent labels + can be properly de-duplicated. + """ + self.update_inputs(inputs) + +def _RuleCfg_add_outputs(self, *outputs): + """Adds an output to the list of outputs, if not present already. + + :::{seealso} + The {obj}`update_outputs()` method for adding a collection of + values. + ::: + + Args: + self: implicitly arg. + *outputs: {type}`Label` the outputs to add. Note that a `Label`, + not `str`, should be passed to ensure different apparent labels + can be properly de-duplicated. + """ + self.update_outputs(outputs) + +def _RuleCfg_build(self): + """Builds the rule cfg into the value rule.cfg arg value. + + Returns: + {type}`transition` the transition object to apply to the rule. + """ + impl = self._state[_IMPLEMENTATION] + if impl == "target" or impl == None: + # config.target is Bazel 8+ + if hasattr(config, "target"): + return config.target() + else: + return None + elif impl == "none": + return config.none() + elif types.is_function(impl): + return transition( + implementation = impl, + # Transitions only accept unique lists of strings. + inputs = {str(v): None for v in self._state[_INPUTS]}.keys(), + outputs = {str(v): None for v in self._state[_OUTPUTS]}.keys(), + ) + else: + # Assume its valid. Probably an `config.XXX` object or manually + # set transition object. + return impl + +def _RuleCfg_update_inputs(self, *others): + """Add a collection of values to inputs. + + Args: + self: implicitly added + *others: {type}`list[Label]` collection of labels to add to + inputs. Only values not already present are added. Note that a + `Label`, not `str`, should be passed to ensure different apparent + labels can be properly de-duplicated. + """ + list_add_unique( + self._state[_INPUTS], + others, + convert = lambda v: normalize_transition_in_out_value("input", v), + ) + +def _RuleCfg_update_outputs(self, *others): + """Add a collection of values to outputs. + + Args: + self: implicitly added + *others: {type}`list[Label]` collection of labels to add to + outputs. Only values not already present are added. Note that a + `Label`, not `str`, should be passed to ensure different apparent + labels can be properly de-duplicated. + """ + list_add_unique( + self._state[_OUTPUTS], + others, + convert = lambda v: normalize_transition_in_out_value("output", v), + ) + +# buildifier: disable=name-conventions +RuleCfg = struct( + TYPEDEF = _RuleCfg_typedef, + new = _RuleCfg_new, + # keep sorted + add_inputs = _RuleCfg_add_inputs, + add_outputs = _RuleCfg_add_outputs, + build = _RuleCfg_build, + update_inputs = _RuleCfg_update_inputs, + update_outputs = _RuleCfg_update_outputs, +) + +def _Rule_typedef(): + """A builder to accumulate state for constructing a `rule` object. + + :::{field} attrs + :type: AttrsDict + ::: + + :::{field} cfg + :type: RuleCfg + ::: + + :::{function} doc() -> str + ::: + + :::{function} exec_groups() -> dict[str, ExecGroup] + ::: + + :::{function} executable() -> bool + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} fragments() -> list[str] + ::: + + :::{function} implementation() -> callable | None + ::: + + :::{function} provides() -> list[provider | list[provider]] + ::: + + :::{function} set_doc(v: str) + ::: + + :::{function} set_executable(v: bool) + ::: + + :::{function} set_implementation(v: callable) + ::: + + :::{function} set_test(v: bool) + ::: + + :::{function} test() -> bool + ::: + + :::{function} toolchains() -> list[ToolchainType] + ::: + """ + +def _Rule_new(**kwargs): + """Builder for creating rules. + + Args: + **kwargs: The same as the `rule()` function, but using builders or + dicts to specify sub-objects instead of the immutable Bazel + objects. + """ + kwargs.setdefault(_IMPLEMENTATION, None) + kwargs_set_default_doc(kwargs) + kwargs_set_default_dict(kwargs, _EXEC_GROUPS) + kwargs_set_default_ignore_none(kwargs, "executable", False) + kwargs_set_default_list(kwargs, "fragments") + kwargs_set_default_list(kwargs, "provides") + kwargs_set_default_ignore_none(kwargs, "test", False) + kwargs_set_default_list(kwargs, _TOOLCHAINS) + + for name, value in kwargs[_EXEC_GROUPS].items(): + kwargs[_EXEC_GROUPS][name] = _ExecGroup_maybe_from(value) + + for i, value in enumerate(kwargs[_TOOLCHAINS]): + kwargs[_TOOLCHAINS][i] = _ToolchainType_maybe_from(value) + + # buildifier: disable=uninitialized + self = struct( + attrs = _AttrsDict_new(kwargs.pop(_ATTRS, None)), + build = lambda *a, **k: _Rule_build(self, *a, **k), + cfg = _RuleCfg_new(kwargs.pop(_CFG, None)), + doc = kwargs_getter_doc(kwargs), + exec_groups = kwargs_getter(kwargs, _EXEC_GROUPS), + executable = kwargs_getter(kwargs, "executable"), + fragments = kwargs_getter(kwargs, "fragments"), + implementation = kwargs_getter(kwargs, _IMPLEMENTATION), + kwargs = kwargs, + provides = kwargs_getter(kwargs, "provides"), + set_doc = kwargs_setter_doc(kwargs), + set_executable = kwargs_setter(kwargs, "executable"), + set_implementation = kwargs_setter(kwargs, _IMPLEMENTATION), + set_test = kwargs_setter(kwargs, "test"), + test = kwargs_getter(kwargs, "test"), + to_kwargs = lambda: _Rule_to_kwargs(self), + toolchains = kwargs_getter(kwargs, _TOOLCHAINS), + ) + return self + +def _Rule_build(self, debug = ""): + """Builds a `rule` object + + Args: + self: implicitly added + debug: {type}`str` If set, prints the args used to create the rule. + + Returns: + {type}`rule` + """ + kwargs = self.to_kwargs() + if debug: + lines = ["=" * 80, "rule kwargs: {}:".format(debug)] + for k, v in sorted(kwargs.items()): + if types.is_dict(v): + lines.append(" %s={" % k) + for k2, v2 in sorted(v.items()): + lines.append(" {}: {}".format(k2, v2)) + lines.append(" }") + elif types.is_list(v): + lines.append(" {}=[".format(k)) + for i, v2 in enumerate(v): + lines.append(" [{}] {}".format(i, v2)) + lines.append(" ]") + else: + lines.append(" {}={}".format(k, v)) + print("\n".join(lines)) # buildifier: disable=print + return rule(**kwargs) + +def _Rule_to_kwargs(self): + """Builds the arguments for calling `rule()`. + + This is added as an escape hatch to construct the final values `rule()` + kwarg values in case callers want to manually change them. + + Args: + self: implicitly added. + + Returns: + {type}`dict` + """ + kwargs = dict(self.kwargs) + if _EXEC_GROUPS in kwargs: + kwargs[_EXEC_GROUPS] = { + k: v.build() if _is_builder(v) else v + for k, v in kwargs[_EXEC_GROUPS].items() + } + if _TOOLCHAINS in kwargs: + kwargs[_TOOLCHAINS] = [ + v.build() if _is_builder(v) else v + for v in kwargs[_TOOLCHAINS] + ] + if _ATTRS not in kwargs: + kwargs[_ATTRS] = self.attrs.build() + if _CFG not in kwargs: + kwargs[_CFG] = self.cfg.build() + return kwargs + +# buildifier: disable=name-conventions +Rule = struct( + TYPEDEF = _Rule_typedef, + new = _Rule_new, + build = _Rule_build, + to_kwargs = _Rule_to_kwargs, +) + +def _AttrsDict_typedef(): + """Builder for the dictionary of rule attributes. + + :::{field} map + :type: dict[str, AttributeBuilder] + + The underlying dict of attributes. Directly accessible so that regular + dict operations (e.g. `x in y`) can be performed, if necessary. + ::: + + :::{function} get(key, default=None) + Get an entry from the dict. Convenience wrapper for `.map.get(...)` + ::: + + :::{function} items() -> list[tuple[str, object]] + Returns a list of key-value tuples. Convenience wrapper for `.map.items()` + ::: + + :::{function} pop(key, default) -> object + Removes a key from the attr dict + ::: + """ + +def _AttrsDict_new(initial): + """Creates a builder for the `rule.attrs` dict. + + Args: + initial: {type}`dict[str, callable | AttributeBuilder] | None` dict of + initial values to populate the attributes dict with. + + Returns: + {type}`AttrsDict` + """ + + # buildifier: disable=uninitialized + self = struct( + # keep sorted + build = lambda: _AttrsDict_build(self), + get = lambda *a, **k: self.map.get(*a, **k), + items = lambda: self.map.items(), + map = {}, + put = lambda key, value: _AttrsDict_put(self, key, value), + update = lambda *a, **k: _AttrsDict_update(self, *a, **k), + pop = lambda *a, **k: self.map.pop(*a, **k), + ) + if initial: + _AttrsDict_update(self, initial) + return self + +def _AttrsDict_put(self, name, value): + """Sets a value in the attrs dict. + + Args: + self: implicitly added + name: {type}`str` the attribute name to set in the dict + value: {type}`AttributeBuilder | callable` the value for the + attribute. If a callable, then it is treated as an + attribute builder factory (no-arg callable that returns an + attribute builder) and is called immediately. + """ + if types.is_function(value): + # Convert factory function to builder + value = value() + self.map[name] = value + +def _AttrsDict_update(self, other): + """Merge `other` into this object. + + Args: + self: implicitly added + other: {type}`dict[str, callable | AttributeBuilder]` the values to + merge into this object. If the value a function, it is called + with no args and expected to return an attribute builder. This + allows defining dicts of common attributes (where the values are + functions that create a builder) and merge them into the rule. + """ + for k, v in other.items(): + # Handle factory functions that create builders + if types.is_function(v): + self.map[k] = v() + else: + self.map[k] = v + +def _AttrsDict_build(self): + """Build an attribute dict for passing to `rule()`. + + Returns: + {type}`dict[str, Attribute]` where the values are `attr.XXX` objects + """ + attrs = {} + for k, v in self.map.items(): + attrs[k] = v.build() if _is_builder(v) else v + return attrs + +def _AttributeBuilder_typedef(): + """An abstract base typedef for builder for a Bazel {obj}`Attribute` + + Instances of this are a builder for a particular `Attribute` type, + e.g. `attr.label`, `attr.string`, etc. + """ + +# buildifier: disable=name-conventions +AttributeBuilder = struct( + TYPEDEF = _AttributeBuilder_typedef, +) + +# buildifier: disable=name-conventions +AttrsDict = struct( + TYPEDEF = _AttrsDict_typedef, + new = _AttrsDict_new, + update = _AttrsDict_update, + build = _AttrsDict_build, +) + +ruleb = struct( + Rule = _Rule_new, + ToolchainType = _ToolchainType_new, + ExecGroup = _ExecGroup_new, +) diff --git a/python/private/runtime_env_repo.bzl b/python/private/runtime_env_repo.bzl new file mode 100644 index 0000000000..cade1968bb --- /dev/null +++ b/python/private/runtime_env_repo.bzl @@ -0,0 +1,41 @@ +"""Internal setup to help the runtime_env toolchain.""" + +load("//python/private:repo_utils.bzl", "repo_utils") + +def _runtime_env_repo_impl(rctx): + pyenv = repo_utils.which_unchecked(rctx, "pyenv").binary + if pyenv != None: + pyenv_version_file = repo_utils.execute_checked( + rctx, + op = "GetPyenvVersionFile", + arguments = [pyenv, "version-file"], + ).stdout.strip() + + # When pyenv is used, the version file is what decided the + # version used. Watch it so we compute the correct value if the + # user changes it. + rctx.watch(pyenv_version_file) + + version = repo_utils.execute_checked( + rctx, + op = "GetPythonVersion", + arguments = [ + "python3", + "-I", + "-c", + """import sys; print(f"{sys.version_info.major}.{sys.version_info.minor}")""", + ], + environment = { + # Prevent the user's current shell from influencing the result. + # This envvar won't be present when a test is run. + # NOTE: This should be None, but Bazel 7 doesn't support None + # values. Thankfully, pyenv treats empty string the same as missing. + "PYENV_VERSION": "", + }, + ).stdout.strip() + rctx.file("info.bzl", "PYTHON_VERSION = '{}'\n".format(version)) + rctx.file("BUILD.bazel", "") + +runtime_env_repo = repository_rule( + implementation = _runtime_env_repo_impl, +) diff --git a/python/private/runtime_env_toolchain.bzl b/python/private/runtime_env_toolchain.bzl index 2116012c03..de74900750 100644 --- a/python/private/runtime_env_toolchain.bzl +++ b/python/private/runtime_env_toolchain.bzl @@ -17,6 +17,7 @@ load("@rules_cc//cc:cc_library.bzl", "cc_library") load("//python:py_runtime.bzl", "py_runtime") load("//python:py_runtime_pair.bzl", "py_runtime_pair") load("//python/cc:py_cc_toolchain.bzl", "py_cc_toolchain") +load("//python/private:config_settings.bzl", "is_python_version_at_least") load(":py_exec_tools_toolchain.bzl", "py_exec_tools_toolchain") load(":toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", "PY_CC_TOOLCHAIN_TYPE", "TARGET_TOOLCHAIN_TYPE") @@ -38,6 +39,11 @@ def define_runtime_env_toolchain(name): """ base_name = name.replace("_toolchain", "") + supports_build_time_venv = select({ + ":_is_at_least_py3.11": True, + "//conditions:default": False, + }) + py_runtime( name = "_runtime_env_py3_runtime", interpreter = "//python/private:runtime_env_toolchain_interpreter.sh", @@ -45,6 +51,7 @@ def define_runtime_env_toolchain(name): stub_shebang = "#!/usr/bin/env python3", visibility = ["//visibility:private"], tags = ["manual"], + supports_build_time_venv = supports_build_time_venv, ) # This is a dummy runtime whose interpreter_path triggers the native rule @@ -56,6 +63,7 @@ def define_runtime_env_toolchain(name): python_version = "PY3", visibility = ["//visibility:private"], tags = ["manual"], + supports_build_time_venv = supports_build_time_venv, ) py_runtime_pair( @@ -99,8 +107,9 @@ def define_runtime_env_toolchain(name): ) py_cc_toolchain( name = "_runtime_env_py_cc_toolchain_impl", - headers = ":_empty_cc_lib", - libs = ":_empty_cc_lib", + headers = "//python/private/cc:empty", + headers_abi3 = "//python/private/cc:empty", + libs = "//python/private/cc:empty", python_version = "0.0", tags = ["manual"], ) @@ -110,3 +119,7 @@ def define_runtime_env_toolchain(name): toolchain_type = PY_CC_TOOLCHAIN_TYPE, visibility = ["//visibility:public"], ) + is_python_version_at_least( + name = "_is_at_least_py3.11", + at_least = "3.11", + ) diff --git a/python/private/runtime_env_toolchain_interpreter.sh b/python/private/runtime_env_toolchain_interpreter.sh index 2cb7cc7151..c78cfe1a9b 100755 --- a/python/private/runtime_env_toolchain_interpreter.sh +++ b/python/private/runtime_env_toolchain_interpreter.sh @@ -17,16 +17,14 @@ die() { exit 1 } -# We use `which` to locate the Python interpreter command on PATH. `command -v` -# is another option, but it doesn't check whether the file it finds has the -# executable bit. +# We use `command -v` to locate the Python interpreter command on PATH. # # A tricky situation happens when this wrapper is invoked as part of running a # tool, e.g. passing a py_binary target to `ctx.actions.run()`. Bazel will unset # the PATH variable. Then the shell will see there's no PATH and initialize its -# own, sometimes without exporting it. This causes `which` to fail and this +# own, sometimes without exporting it. This causes `command -v` to fail and this # script to think there's no Python interpreter installed. To avoid this we -# explicitly pass PATH to each `which` invocation. We can't just export PATH +# explicitly pass PATH to each `command -v` invocation. We can't just export PATH # because that would modify the environment seen by the final user Python # program. # @@ -37,9 +35,9 @@ die() { # https://github.com/bazelbuild/bazel/issues/8415 # Try the "python3" command name first, then fall back on "python". -PYTHON_BIN="$(PATH="$PATH" which python3 2> /dev/null)" +PYTHON_BIN="$(PATH="$PATH" command -v python3 2> /dev/null)" if [ -z "${PYTHON_BIN:-}" ]; then - PYTHON_BIN="$(PATH="$PATH" which python 2>/dev/null)" + PYTHON_BIN="$(PATH="$PATH" command -v python 2>/dev/null)" fi if [ -z "${PYTHON_BIN:-}" ]; then die "Neither 'python3' nor 'python' were found on the target \ @@ -50,8 +48,36 @@ $PATH Please ensure an interpreter is available on this platform (and marked \ executable), or else register an appropriate Python toolchain as per the \ documentation for py_runtime_pair \ -(https://github.com/bazelbuild/rules_python/blob/master/docs/python.md#py_runtime_pair)." +(https://github.com/bazel-contrib/rules_python/blob/master/docs/python.md#py_runtime_pair)." fi -exec "$PYTHON_BIN" "$@" +# Because this is a wrapper script that invokes Python, it prevents Python from +# detecting virtualenvs like normal (i.e. using the venv symlink to find the +# real interpreter). To work around this, we have to manually detect the venv, +# then trick the interpreter into understanding we're in a virtual env. +self_dir=$(dirname "$0") +if [ -e "$self_dir/pyvenv.cfg" ] || [ -e "$self_dir/../pyvenv.cfg" ]; then + case "$0" in + /*) + venv_bin="$0" + ;; + *) + venv_bin="$PWD/$0" + ;; + esac + if [ ! -e "$PYTHON_BIN" ]; then + die "ERROR: Python interpreter does not exist: $PYTHON_BIN" + fi + # PYTHONEXECUTABLE is also used because switching argv0 doesn't fully trick + # the pyenv wrappers. + # NOTE: The PYTHONEXECUTABLE envvar only works for non-Mac starting in Python 3.11 + export PYTHONEXECUTABLE="$venv_bin" + # Python looks at argv[0] to determine sys.executable, so set that to the venv + # binary, not the actual one invoked. + # NOTE: exec -a would be simpler, but isn't posix-compatible, and dash shell + # (Ubuntu/debian default) doesn't support it; see #3009. + exec sh -c "$PYTHON_BIN \$@" "$venv_bin" "$@" +else + exec "$PYTHON_BIN" "$@" +fi diff --git a/python/private/runtimes_manifest.txt b/python/private/runtimes_manifest.txt new file mode 100755 index 0000000000..df02a3049c --- /dev/null +++ b/python/private/runtimes_manifest.txt @@ -0,0 +1,621 @@ +# Manifest of runtimes to make available +# Originally generated circa 2026-06 from the runtimes in TOOL_VERSIONS from the original python/versions.bzl +# To sort this file, execute: ./python/private/tools/sort_manifest.py python/private/runtimes_manifest.txt + +1409acd9a506e2d1d3b65c1488db4e40d8f19d09a7df099667c87a506f71c0ef 20220227/cpython-3.10.2+20220227-aarch64-apple-darwin-install_only.tar.gz +8f351a8cc348bb45c0f95b8634c8345ec6e749e483384188ad865b7428342703 20220227/cpython-3.10.2+20220227-aarch64-unknown-linux-gnu-install_only.tar.gz +8146ad4390710ec69b316a5649912df0247d35f4a42e2aa9615bffd87b3e235a 20220227/cpython-3.10.2+20220227-x86_64-apple-darwin-install_only.tar.gz +a1d9a594cd3103baa24937ad9150c1a389544b4350e859200b3e5c036ac352bd 20220227/cpython-3.10.2+20220227-x86_64-pc-windows-msvc-shared-install_only.tar.gz +9b64eca2a94f7aff9409ad70bdaa7fbbf8148692662e764401883957943620dd 20220227/cpython-3.10.2+20220227-x86_64-unknown-linux-gnu-install_only.tar.gz +2c99983d1e83e4b6e7411ed9334019f193fba626344a50c36fba6c25d4de78a2 20220502/cpython-3.10.4+20220502-aarch64-apple-darwin-install_only.tar.gz +d8098c0c54546637e7516f93b13403b11f9db285def8d7abd825c31407a13d7e 20220502/cpython-3.10.4+20220502-aarch64-unknown-linux-gnu-install_only.tar.gz +f2711eaffff3477826a401d09a013c6802f11c04c63ab3686aa72664f1216a05 20220502/cpython-3.10.4+20220502-x86_64-apple-darwin-install_only.tar.gz +bee24a3a5c83325215521d261d73a5207ab7060ef3481f76f69b4366744eb81d 20220502/cpython-3.10.4+20220502-x86_64-pc-windows-msvc-shared-install_only.tar.gz +f6f871e53a7b1469c13f9bd7920ad98c4589e549acad8e5a1e14760fff3dd5c9 20220502/cpython-3.10.4+20220502-x86_64-unknown-linux-gnu-install_only.tar.gz +efaf66acdb9a4eb33d57702607d2e667b1a319d58c167a43c96896b97419b8b7 20220802/cpython-3.10.6+20220802-aarch64-apple-darwin-install_only.tar.gz +81625f5c97f61e2e3d7e9f62c484b1aa5311f21bd6545451714b949a29da5435 20220802/cpython-3.10.6+20220802-aarch64-unknown-linux-gnu-install_only.tar.gz +7718411adf3ea1480f3f018a643eb0550282aefe39e5ecb3f363a4a566a9398c 20220802/cpython-3.10.6+20220802-x86_64-apple-darwin-install_only.tar.gz +91889a7dbdceea585ff4d3b7856a6bb8f8a4eca83a0ff52a73542c2e67220eaa 20220802/cpython-3.10.6+20220802-x86_64-pc-windows-msvc-shared-install_only.tar.gz +55aa2190d28dcfdf414d96dc5dcea9fe048fadcd583dc3981fec020869826111 20220802/cpython-3.10.6+20220802-x86_64-unknown-linux-gnu-install_only.tar.gz +d52b03817bd245d28e0a8b2f715716cd0fcd112820ccff745636932c76afa20a 20221106/cpython-3.10.8+20221106-aarch64-apple-darwin-install_only.tar.gz +33170bef18c811906b738be530f934640491b065bf16c4d276c6515321918132 20221106/cpython-3.10.8+20221106-aarch64-unknown-linux-gnu-install_only.tar.gz +525b79c7ce5de90ab66bd07b0ac1008bafa147ddc8a41bef15ffb7c9c1e9e7c5 20221106/cpython-3.10.8+20221106-x86_64-apple-darwin-install_only.tar.gz +f2b6d2f77118f06dd2ca04dae1175e44aaa5077a5ed8ddc63333c15347182bfe 20221106/cpython-3.10.8+20221106-x86_64-pc-windows-msvc-shared-install_only.tar.gz +6c8db44ae0e18e320320bbaaafd2d69cde8bfea171ae2d651b7993d1396260b7 20221106/cpython-3.10.8+20221106-x86_64-unknown-linux-gnu-install_only.tar.gz +018d05a779b2de7a476f3b3ff2d10f503d69d14efcedd0774e6dab8c22ef84ff 20230116/cpython-3.10.9+20230116-aarch64-apple-darwin-install_only.tar.gz +2003750f40cd09d4bf7a850342613992f8d9454f03b3c067989911fb37e7a4d1 20230116/cpython-3.10.9+20230116-aarch64-unknown-linux-gnu-install_only.tar.gz +0e685f98dce0e5bc8da93c7081f4e6c10219792e223e4b5886730fd73a7ba4c6 20230116/cpython-3.10.9+20230116-x86_64-apple-darwin-install_only.tar.gz +59c6970cecb357dc1d8554bd0540eb81ee7f6d16a07acf3d14ed294ece02c035 20230116/cpython-3.10.9+20230116-x86_64-pc-windows-msvc-shared-install_only.tar.gz +d196347aeb701a53fe2bb2b095abec38d27d0fa0443f8a1c2023a1bed6e18cdf 20230116/cpython-3.10.9+20230116-x86_64-unknown-linux-gnu-install_only.tar.gz +4918cdf1cab742a90f85318f88b8122aeaa2d04705803c7b6e78e81a3dd40f80 20230116/cpython-3.11.1+20230116-aarch64-apple-darwin-install_only.tar.gz +debf15783bdcb5530504f533d33fda75a7b905cec5361ae8f33da5ba6599f8b4 20230116/cpython-3.11.1+20230116-aarch64-unknown-linux-gnu-install_only.tar.gz +20a4203d069dc9b710f70b09e7da2ce6f473d6b1110f9535fb6f4c469ed54733 20230116/cpython-3.11.1+20230116-x86_64-apple-darwin-install_only.tar.gz +edc08979cb0666a597466176511529c049a6f0bba8adf70df441708f766de5bf 20230116/cpython-3.11.1+20230116-x86_64-pc-windows-msvc-shared-install_only.tar.gz +02a551fefab3750effd0e156c25446547c238688a32fabde2995c941c03a6423 20230116/cpython-3.11.1+20230116-x86_64-unknown-linux-gnu-install_only.tar.gz +8348bc3c2311f94ec63751fb71bd0108174be1c4def002773cf519ee1506f96f 20230507/cpython-3.10.11+20230507-aarch64-apple-darwin-install_only.tar.gz +c7573fdb00239f86b22ea0e8e926ca881d24fde5e5890851339911d76110bc35 20230507/cpython-3.10.11+20230507-aarch64-unknown-linux-gnu-install_only.tar.gz +73a9d4c89ed51be39dd2de4e235078281087283e9fdedef65bec02f503e906ee 20230507/cpython-3.10.11+20230507-ppc64le-unknown-linux-gnu-install_only.tar.gz +bd3fc6e4da6f4033ebf19d66704e73b0804c22641ddae10bbe347c48f82374ad 20230507/cpython-3.10.11+20230507-x86_64-apple-darwin-install_only.tar.gz +9c2d3604a06fcd422289df73015cd00e7271d90de28d2c910f0e2309a7f73a68 20230507/cpython-3.10.11+20230507-x86_64-pc-windows-msvc-shared-install_only.tar.gz +c5bcaac91bc80bfc29cf510669ecad12d506035ecb3ad85ef213416d54aecd79 20230507/cpython-3.10.11+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz +09e412506a8d63edbb6901742b54da9aa7faf120b8dbdce56c57b303fc892c86 20230507/cpython-3.11.3+20230507-aarch64-apple-darwin-install_only.tar.gz +8190accbbbbcf7620f1ff6d668e4dd090c639665d11188ce864b62554d40e5ab 20230507/cpython-3.11.3+20230507-aarch64-unknown-linux-gnu-install_only.tar.gz +767d24f3570b35fedb945f5ac66224c8983f2d556ab83c5cfaa5f3666e9c212c 20230507/cpython-3.11.3+20230507-ppc64le-unknown-linux-gnu-install_only.tar.gz +f710b8d60621308149c100d5175fec39274ed0b9c99645484fd93d1716ef4310 20230507/cpython-3.11.3+20230507-x86_64-apple-darwin-install_only.tar.gz +24741066da6f35a7ff67bee65ce82eae870d84e1181843e64a7076d1571e95af 20230507/cpython-3.11.3+20230507-x86_64-pc-windows-msvc-shared-install_only.tar.gz +da50b87d1ec42b3cb577dfd22a3655e43a53150f4f98a4bfb40757c9d7839ab5 20230507/cpython-3.11.3+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz +bc66c706ea8c5fc891635fda8f9da971a1a901d41342f6798c20ad0b2a25d1d6 20230726/cpython-3.10.12+20230726-aarch64-apple-darwin-install_only.tar.gz +fee80e221663eca5174bd794cb5047e40d3910dbeadcdf1f09d405a4c1c15fe4 20230726/cpython-3.10.12+20230726-aarch64-unknown-linux-gnu-install_only.tar.gz +bb5e8cb0d2e44241725fa9b342238245503e7849917660006b0246a9c97b1d6c 20230726/cpython-3.10.12+20230726-ppc64le-unknown-linux-gnu-install_only.tar.gz +8d33d435ae6fb93ded7fc26798cc0a1a4f546a4e527012a1e2909cc314b332df 20230726/cpython-3.10.12+20230726-s390x-unknown-linux-gnu-install_only.tar.gz +8a6e3ed973a671de468d9c691ed9cb2c3a4858c5defffcf0b08969fba9c1dd04 20230726/cpython-3.10.12+20230726-x86_64-apple-darwin-install_only.tar.gz +c1a31c353ca44de7d1b1a3b6c55a823e9c1eed0423d4f9f66e617bdb1b608685 20230726/cpython-3.10.12+20230726-x86_64-pc-windows-msvc-shared-install_only.tar.gz +a476dbca9184df9fc69fe6309cda5ebaf031d27ca9e529852437c94ec1bc43d3 20230726/cpython-3.10.12+20230726-x86_64-unknown-linux-gnu-install_only.tar.gz +cb6d2948384a857321f2aa40fa67744cd9676a330f08b6dad7070bda0b6120a4 20230726/cpython-3.11.4+20230726-aarch64-apple-darwin-install_only.tar.gz +2e84fc53f4e90e11963281c5c871f593abcb24fc796a50337fa516be99af02fb 20230726/cpython-3.11.4+20230726-aarch64-unknown-linux-gnu-install_only.tar.gz +df7b92ed9cec96b3bb658fb586be947722ecd8e420fb23cee13d2e90abcfcf25 20230726/cpython-3.11.4+20230726-ppc64le-unknown-linux-gnu-install_only.tar.gz +e477f0749161f9aa7887964f089d9460a539f6b4a8fdab5166f898210e1a87a4 20230726/cpython-3.11.4+20230726-s390x-unknown-linux-gnu-install_only.tar.gz +47e1557d93a42585972772e82661047ca5f608293158acb2778dccf120eabb00 20230726/cpython-3.11.4+20230726-x86_64-apple-darwin-install_only.tar.gz +878614c03ea38538ae2f758e36c85d2c0eb1eaaca86cd400ff8c76693ee0b3e1 20230726/cpython-3.11.4+20230726-x86_64-pc-windows-msvc-shared-install_only.tar.gz +e26247302bc8e9083a43ce9e8dd94905b40d464745b1603041f7bc9a93c65d05 20230726/cpython-3.11.4+20230726-x86_64-unknown-linux-gnu-install_only.tar.gz +dab64b3580118ad2073babd7c29fd2053b616479df5c107d31fe2af1f45e948b 20230826/cpython-3.11.5+20230826-aarch64-apple-darwin-install_only.tar.gz +bb5c5d1ea0f199fe2d3f0996fff4b48ca6ddc415a3dbd98f50bff7fce48aac80 20230826/cpython-3.11.5+20230826-aarch64-unknown-linux-gnu-install_only.tar.gz +14121b53e9c8c6d0741f911ae00102a35adbcf5c3cdf732687ef7617b7d7304d 20230826/cpython-3.11.5+20230826-ppc64le-unknown-linux-gnu-install_only.tar.gz +fe459da39874443579d6fe88c68777c6d3e331038e1fb92a0451879fb6beb16d 20230826/cpython-3.11.5+20230826-s390x-unknown-linux-gnu-install_only.tar.gz +4a4efa7378c72f1dd8ebcce1afb99b24c01b07023aa6b8fea50eaedb50bf2bfc 20230826/cpython-3.11.5+20230826-x86_64-apple-darwin-install_only.tar.gz +00f002263efc8aea896bcfaaf906b1f4dab3e5cd3db53e2b69ab9a10ba220b97 20230826/cpython-3.11.5+20230826-x86_64-pc-windows-msvc-shared-install_only.tar.gz +fbed6f7694b2faae5d7c401a856219c945397f772eea5ca50c6eb825cbc9d1e1 20230826/cpython-3.11.5+20230826-x86_64-unknown-linux-gnu-install_only.tar.gz +916c35125b5d8323a21526d7a9154ca626453f63d0878e95b9f613a95006c990 20231002/cpython-3.11.6+20231002-aarch64-apple-darwin-install_only.tar.gz +3e26a672df17708c4dc928475a5974c3fb3a34a9b45c65fb4bd1e50504cc84ec 20231002/cpython-3.11.6+20231002-aarch64-unknown-linux-gnu-install_only.tar.gz +7937035f690a624dba4d014ffd20c342e843dd46f89b0b0a1e5726b85deb8eaf 20231002/cpython-3.11.6+20231002-ppc64le-unknown-linux-gnu-install_only.tar.gz +f9f19823dba3209cedc4647b00f46ed0177242917db20fb7fb539970e384531c 20231002/cpython-3.11.6+20231002-s390x-unknown-linux-gnu-install_only.tar.gz +178cb1716c2abc25cb56ae915096c1a083e60abeba57af001996e8bc6ce1a371 20231002/cpython-3.11.6+20231002-x86_64-apple-darwin-install_only.tar.gz +3933545e6d41462dd6a47e44133ea40995bc6efeed8c2e4cbdf1a699303e95ea 20231002/cpython-3.11.6+20231002-x86_64-pc-windows-msvc-shared-install_only.tar.gz +ee37a7eae6e80148c7e3abc56e48a397c1664f044920463ad0df0fc706eacea8 20231002/cpython-3.11.6+20231002-x86_64-unknown-linux-gnu-install_only.tar.gz +4734a2be2becb813830112c780c9879ac3aff111a0b0cd590e65ec7465774d02 20231002/cpython-3.12.0+20231002-aarch64-apple-darwin-install_only.tar.gz +bccfe67cf5465a3dfb0336f053966e2613a9bc85a6588c2fcf1366ef930c4f88 20231002/cpython-3.12.0+20231002-aarch64-unknown-linux-gnu-install_only.tar.gz +b5dae075467ace32c594c7877fe6ebe0837681f814601d5d90ba4c0dfd87a1f2 20231002/cpython-3.12.0+20231002-ppc64le-unknown-linux-gnu-install_only.tar.gz +5681621349dd85d9726d1b67c84a9686ce78f72e73a6f9e4cc4119911655759e 20231002/cpython-3.12.0+20231002-s390x-unknown-linux-gnu-install_only.tar.gz +5a9e88c8aa52b609d556777b52ebde464ae4b4f77e4aac4eb693af57395c9abf 20231002/cpython-3.12.0+20231002-x86_64-apple-darwin-install_only.tar.gz +facfaa1fbc8653f95057f3c4a0f8aa833dab0e0b316e24ee8686bc761d4b4f8d 20231002/cpython-3.12.0+20231002-x86_64-pc-windows-msvc-shared-install_only.tar.gz +e51a5293f214053ddb4645b2c9f84542e2ef86870b8655704367bd4b29d39fe9 20231002/cpython-3.12.0+20231002-x86_64-unknown-linux-gnu-install_only.tar.gz +b042c966920cf8465385ca3522986b12d745151a72c060991088977ca36d3883 20240107/cpython-3.11.7+20240107-aarch64-apple-darwin-install_only.tar.gz +b102eaf865eb715aa98a8a2ef19037b6cc3ae7dfd4a632802650f29de635aa13 20240107/cpython-3.11.7+20240107-aarch64-unknown-linux-gnu-install_only.tar.gz +b44e1b74afe75c7b19143413632c4386708ae229117f8f950c2094e9681d34c7 20240107/cpython-3.11.7+20240107-ppc64le-unknown-linux-gnu-install_only.tar.gz +49520e3ff494708020f306e30b0964f079170be83e956be4504f850557378a22 20240107/cpython-3.11.7+20240107-s390x-unknown-linux-gnu-install_only.tar.gz +a0e615eef1fafdc742da0008425a9030b7ea68a4ae4e73ac557ef27b112836d4 20240107/cpython-3.11.7+20240107-x86_64-apple-darwin-install_only.tar.gz +67077e6fa918e4f4fd60ba169820b00be7c390c497bf9bc9cab2c255ea8e6f3e 20240107/cpython-3.11.7+20240107-x86_64-pc-windows-msvc-shared-install_only.tar.gz +4a51ce60007a6facf64e5495f4cf322e311ba9f39a8cd3f3e4c026eae488e140 20240107/cpython-3.11.7+20240107-x86_64-unknown-linux-gnu-install_only.tar.gz +f93f8375ca6ac0a35d58ff007043cbd3a88d9609113f1cb59cf7c8d215f064af 20240107/cpython-3.12.1+20240107-aarch64-apple-darwin-install_only.tar.gz +236533ef20e665007a111c2f36efb59c87ae195ad7dca223b6dc03fb07064f0b 20240107/cpython-3.12.1+20240107-aarch64-unknown-linux-gnu-install_only.tar.gz +78051f0d1411ee62bc2af5edfccf6e8400ac4ef82887a2affc19a7ace6a05267 20240107/cpython-3.12.1+20240107-ppc64le-unknown-linux-gnu-install_only.tar.gz +60631211c701f8d2c56e5dd7b154e68868128a019b9db1d53a264f56c0d4aee2 20240107/cpython-3.12.1+20240107-s390x-unknown-linux-gnu-install_only.tar.gz +eca96158c1568dedd9a0b3425375637a83764d1fa74446438293089a8bfac1f8 20240107/cpython-3.12.1+20240107-x86_64-apple-darwin-install_only.tar.gz +fd5a9e0f41959d0341246d3643f2b8794f638adc0cec8dd5e1b6465198eae08a 20240107/cpython-3.12.1+20240107-x86_64-pc-windows-msvc-shared-install_only.tar.gz +74e330b8212ca22fd4d9a2003b9eec14892155566738febc8e5e572f267b9472 20240107/cpython-3.12.1+20240107-x86_64-unknown-linux-gnu-install_only.tar.gz +5fdc0f6a5b5a90fd3c528e8b1da8e3aac931ea8690126c2fdb4254c84a3ff04a 20240224/cpython-3.10.13+20240224-aarch64-apple-darwin-install_only.tar.gz +a898a88705611b372297bb8fe4d23cc16b8603ce5f24494c3a8cfa65d83787f9 20240224/cpython-3.10.13+20240224-aarch64-unknown-linux-gnu-install_only.tar.gz +c23706e138a0351fc1e9def2974af7b8206bac7ecbbb98a78f5aa9e7535fee42 20240224/cpython-3.10.13+20240224-ppc64le-unknown-linux-gnu-install_only.tar.gz +09be8fb2cdfbb4a93d555f268f244dbe4d8ff1854b2658e8043aa4ec08aede3e 20240224/cpython-3.10.13+20240224-s390x-unknown-linux-gnu-install_only.tar.gz +6378dfd22f58bb553ddb02be28304d739cd730c1f95c15c74955c923a1bc3d6a 20240224/cpython-3.10.13+20240224-x86_64-apple-darwin-install_only.tar.gz +086f7fe9156b897bb401273db8359017104168ac36f60f3af4e31ac7acd6634e 20240224/cpython-3.10.13+20240224-x86_64-pc-windows-msvc-shared-install_only.tar.gz +d995d032ca702afd2fc3a689c1f84a6c64972ecd82bba76a61d525f08eb0e195 20240224/cpython-3.10.13+20240224-x86_64-unknown-linux-gnu-install_only.tar.gz +389a51139f5abe071a0d70091ca5df3e7a3dfcfcbe3e0ba6ad85fb4c5638421e 20240224/cpython-3.11.8+20240224-aarch64-apple-darwin-install_only.tar.gz +389b9005fb78dd5a6f68df5ea45ab7b30d9a4b3222af96999e94fd20d4ad0c6a 20240224/cpython-3.11.8+20240224-aarch64-unknown-linux-gnu-install_only.tar.gz +eb2b31f8e50309aae493c6a359c32b723a676f07c641f5e8fe4b6aa4dbb50946 20240224/cpython-3.11.8+20240224-ppc64le-unknown-linux-gnu-install_only.tar.gz +844f64f4c16e24965778281da61d1e0e6cd1358a581df1662da814b1eed096b9 20240224/cpython-3.11.8+20240224-s390x-unknown-linux-gnu-install_only.tar.gz +097f467b0c36706bfec13f199a2eaf924e668f70c6e2bd1f1366806962f7e86e 20240224/cpython-3.11.8+20240224-x86_64-apple-darwin-install_only.tar.gz +b618f1f047349770ee1ef11d1b05899840abd53884b820fd25c7dfe2ec1664d4 20240224/cpython-3.11.8+20240224-x86_64-pc-windows-msvc-shared-install_only.tar.gz +94e13d0e5ad417035b80580f3e893a72e094b0900d5d64e7e34ab08e95439987 20240224/cpython-3.11.8+20240224-x86_64-unknown-linux-gnu-install_only.tar.gz +01c064c00013b0175c7858b159989819ead53f4746d40580b5b0b35b6e80fba6 20240224/cpython-3.12.2+20240224-aarch64-apple-darwin-install_only.tar.gz +e52550379e7c4ac27a87de832d172658bc04150e4e27d4e858e6d8cbb96fd709 20240224/cpython-3.12.2+20240224-aarch64-unknown-linux-gnu-install_only.tar.gz +74bc02c4bbbd26245c37b29b9e12d0a9c1b7ab93477fed8b651c988b6a9a6251 20240224/cpython-3.12.2+20240224-ppc64le-unknown-linux-gnu-install_only.tar.gz +ecd6b0285e5eef94deb784b588b4b425a15a43ae671bf206556659dc141a9825 20240224/cpython-3.12.2+20240224-s390x-unknown-linux-gnu-install_only.tar.gz +a53a6670a202c96fec0b8c55ccc780ea3af5307eb89268d5b41a9775b109c094 20240224/cpython-3.12.2+20240224-x86_64-apple-darwin-install_only.tar.gz +1e5655a6ccb1a64a78460e4e3ee21036c70246800f176a6c91043a3fe3654a3b 20240224/cpython-3.12.2+20240224-x86_64-pc-windows-msvc-shared-install_only.tar.gz +57a37b57f8243caa4cdac016176189573ad7620f0b6da5941c5e40660f9468ab 20240224/cpython-3.12.2+20240224-x86_64-unknown-linux-gnu-install_only.tar.gz +ccc40e5af329ef2af81350db2a88bbd6c17b56676e82d62048c15d548401519e 20240415/cpython-3.12.3+20240415-aarch64-apple-darwin-install_only.tar.gz +ec8126de97945e629cca9aedc80a29c4ae2992c9d69f2655e27ae73906ba187d 20240415/cpython-3.12.3+20240415-aarch64-unknown-linux-gnu-install_only.tar.gz +c5dcf08b8077e617d949bda23027c49712f583120b3ed744f9b143da1d580572 20240415/cpython-3.12.3+20240415-ppc64le-unknown-linux-gnu-install_only.tar.gz +872fc321363b8cdd826fd2cb1adfd1ceb813bc1281f9d410c1c2c4e177e8df86 20240415/cpython-3.12.3+20240415-s390x-unknown-linux-gnu-install_only.tar.gz +c37a22fca8f57d4471e3708de6d13097668c5f160067f264bb2b18f524c890c8 20240415/cpython-3.12.3+20240415-x86_64-apple-darwin-install_only.tar.gz +f7cfa4ad072feb4578c8afca5ba9a54ad591d665a441dd0d63aa366edbe19279 20240415/cpython-3.12.3+20240415-x86_64-pc-windows-msvc-shared-install_only.tar.gz +a73ba777b5d55ca89edef709e6b8521e3f3d4289581f174c8699adfb608d09d6 20240415/cpython-3.12.3+20240415-x86_64-unknown-linux-gnu-install_only.tar.gz +164d89f0df2feb689981864ecc1dffb19e6aa3696c8880166de555494fe92607 20240726/cpython-3.10.14+20240726-aarch64-apple-darwin-install_only.tar.gz +39bcd46b4d70e40da177c55259be16d5c2be7a3f7f93f1e3bde47e71b4833f29 20240726/cpython-3.10.14+20240726-aarch64-unknown-linux-gnu-install_only.tar.gz +549d38b9ef59cba9ab2990025255231bfa1cb32b4bc5eac321667640fdee19d1 20240726/cpython-3.10.14+20240726-ppc64le-unknown-linux-gnu-install_only.tar.gz +de4bc878a8666c734f983db971610980870148f333bda8b0c34abfaeae88d7ec 20240726/cpython-3.10.14+20240726-s390x-unknown-linux-gnu-install_only.tar.gz +1a1455838cd1e8ed0da14a152a2d559a2fd3a6047ba7013e841db4a35a228c1d 20240726/cpython-3.10.14+20240726-x86_64-apple-darwin-install_only.tar.gz +7f68821a8b5445267eca480660364ebd06ec84632b336770c6e39de07ac0f6c3 20240726/cpython-3.10.14+20240726-x86_64-pc-windows-msvc-shared-install_only.tar.gz +32b34cd13d9d745b3db3f3b8398ab2c07de74544829915dbebd8dce39bdc405e 20240726/cpython-3.10.14+20240726-x86_64-unknown-linux-gnu-install_only.tar.gz +cbdac9462bab9671c8e84650e425d3f43b775752a930a2ef954a0d457d5c00c3 20240726/cpython-3.11.9+20240726-aarch64-apple-darwin-install_only.tar.gz +4d17cf988abe24449d649aad3ef974091ab76807904d41839907061925b4c9e3 20240726/cpython-3.11.9+20240726-aarch64-unknown-linux-gnu-install_only.tar.gz +fc4f3c9ef9bfac2ed0282126ff376e544697ad04a5408d6429d46899d7d3bf21 20240726/cpython-3.11.9+20240726-ppc64le-unknown-linux-gnu-install_only.tar.gz +e69b66e53e926460df044f44846eef3fea642f630e829719e1a4112fc370dc56 20240726/cpython-3.11.9+20240726-s390x-unknown-linux-gnu-install_only.tar.gz +dc3174666a30f4c38d04e79a80c3159b4b3aa69597c4676701c8386696811611 20240726/cpython-3.11.9+20240726-x86_64-apple-darwin-install_only.tar.gz +f694be48bdfec1dace6d69a19906b6083f4dd7c7c61f1138ba520e433e5598f8 20240726/cpython-3.11.9+20240726-x86_64-pc-windows-msvc-shared-install_only.tar.gz +f6e955dc9ddfcad74e77abe6f439dac48ebca14b101ed7c85a5bf3206ed2c53d 20240726/cpython-3.11.9+20240726-x86_64-unknown-linux-gnu-install_only.tar.gz +1801025e825c04b3907e4ef6220a13607bc0397628c9485897073110ef7fde15 20240726/cpython-3.12.4+20240726-aarch64-apple-darwin-install_only.tar.gz +a098b18b7e9fea0c66867b76c0124fce9465765017572b2e7b522154c87c78d7 20240726/cpython-3.12.4+20240726-aarch64-unknown-linux-gnu-install_only.tar.gz +04011c4c5b7fe34b0b895edf4ad8748e410686c1d69aaee11d6688d481023bcb 20240726/cpython-3.12.4+20240726-ppc64le-unknown-linux-gnu-install_only.tar.gz +8f8f3e29cf0c2facdbcfee70660939fda7667ac24fee8656d3388fc72f3acc7c 20240726/cpython-3.12.4+20240726-s390x-unknown-linux-gnu-install_only.tar.gz +4c325838c1b0ed13698506fcd515be25c73dcbe195f8522cf98f9148a97601ed 20240726/cpython-3.12.4+20240726-x86_64-apple-darwin-install_only.tar.gz +74309b0f322716409883d38c621743ea7fa0376eb00927b8ee1e1671d3aff450 20240726/cpython-3.12.4+20240726-x86_64-pc-windows-msvc-shared-install_only.tar.gz +e133dd6fc6a2d0033e2658637cc22e9c95f9d7073b80115037ee1f16417a54ac 20240726/cpython-3.12.4+20240726-x86_64-unknown-linux-gnu-install_only.tar.gz +f64776f455a44c24d50f947c813738cfb7b9ac43732c44891bc831fa7940a33c 20241016/cpython-3.10.15+20241016-aarch64-apple-darwin-install_only.tar.gz +eb58581f85fde83d1f3e8e1f8c6f5a15c7ae4fdbe3b1d1083931f9167fdd8dbc 20241016/cpython-3.10.15+20241016-aarch64-unknown-linux-gnu-install_only.tar.gz +0c45af4e7525e2db59901606db32b2896ac1e9830c6f95551402207f537c2ce4 20241016/cpython-3.10.15+20241016-ppc64le-unknown-linux-gnu-install_only.tar.gz +de205896b070e6f5259ac0f2b3379eead875ea84e6a6ef533b89886fcbb46a4c 20241016/cpython-3.10.15+20241016-s390x-unknown-linux-gnu-install_only.tar.gz +90b46dfb1abd98d45663c7a2a8c45d3047a59391d8586d71b459cec7b75f662b 20241016/cpython-3.10.15+20241016-x86_64-apple-darwin-install_only.tar.gz +e48952619796c66ec9719867b87be97edca791c2ef7fbf87d42c417c3331609e 20241016/cpython-3.10.15+20241016-x86_64-pc-windows-msvc-shared-install_only.tar.gz +3db2171e03c1a7acdc599fba583c1b92306d3788b375c9323077367af1e9d9de 20241016/cpython-3.10.15+20241016-x86_64-unknown-linux-gnu-install_only.tar.gz +ed519c47d9620eb916a6f95ec2875396e7b1a9ab993ee40b2f31b837733f318c 20241016/cpython-3.10.15+20241016-x86_64-unknown-linux-musl-install_only.tar.gz +5a69382da99c4620690643517ca1f1f53772331b347e75f536088c42a4cf6620 20241016/cpython-3.11.10+20241016-aarch64-apple-darwin-install_only.tar.gz +803e49259280af0f5466d32829cd9d65a302b0226e424b3f0b261f9daf6aee8f 20241016/cpython-3.11.10+20241016-aarch64-unknown-linux-gnu-install_only.tar.gz +92b666d103902001322f42badbd68da92adc5cebb826af9c1c906c33166e2f34 20241016/cpython-3.11.10+20241016-ppc64le-unknown-linux-gnu-install_only.tar.gz +6d584317651c1ad4a857cb32d1999707e8bb3046fcb2f156d80381814fa19fde 20241016/cpython-3.11.10+20241016-s390x-unknown-linux-gnu-install_only.tar.gz +1e23ffe5bc473e1323ab8f51464da62d77399afb423babf67f8e13c82b69c674 20241016/cpython-3.11.10+20241016-x86_64-apple-darwin-install_only.tar.gz +647b66ff4552e70aec3bf634dd470891b4a2b291e8e8715b3bdb162f577d4c55 20241016/cpython-3.11.10+20241016-x86_64-pc-windows-msvc-shared-install_only.tar.gz +8b50a442b04724a24c1eebb65a36a0c0e833d35374dbdf9c9470d8a97b164cd9 20241016/cpython-3.11.10+20241016-x86_64-unknown-linux-gnu-install_only.tar.gz +d36fc77a8dd76155a7530f6235999a693b9e7c48aa11afeb5610a091cae5aa6f 20241016/cpython-3.11.10+20241016-x86_64-unknown-linux-musl-install_only.tar.gz +4c18852bf9c1a11b56f21bcf0df1946f7e98ee43e9e4c0c5374b2b3765cf9508 20241016/cpython-3.12.7+20241016-aarch64-apple-darwin-install_only.tar.gz +bba3c6be6153f715f2941da34f3a6a69c2d0035c9c5396bc5bb68c6d2bd1065a 20241016/cpython-3.12.7+20241016-aarch64-unknown-linux-gnu-install_only.tar.gz +0a1d1d92e33a969bd2f40a80af53c97b6c0cc1060d384ceff50ff801593bf9d6 20241016/cpython-3.12.7+20241016-ppc64le-unknown-linux-gnu-install_only.tar.gz +935676a0c960b552f95e9ac2e1e385de5de4b34038ff65ffdc688838f1189c17 20241016/cpython-3.12.7+20241016-s390x-unknown-linux-gnu-install_only.tar.gz +60c5271e7edc3c2ab47440b7abf4ed50fbc693880b474f74f05768f5b657045a 20241016/cpython-3.12.7+20241016-x86_64-apple-darwin-install_only.tar.gz +f05531bff16fa77b53be0776587b97b466070e768e6d5920894de988bdcd547a 20241016/cpython-3.12.7+20241016-x86_64-pc-windows-msvc-shared-install_only.tar.gz +43576f7db1033dd57b900307f09c2e86f371152ac8a2607133afa51cbfc36064 20241016/cpython-3.12.7+20241016-x86_64-unknown-linux-gnu-install_only.tar.gz +5ed4a4078db3cbac563af66403aaa156cd6e48831d90382a1820db2b120627b5 20241016/cpython-3.12.7+20241016-x86_64-unknown-linux-musl-install_only.tar.gz +efc2e71c0e05bc5bedb7a846e05f28dd26491b1744ded35ed82f8b49ccfa684b 20241016/cpython-3.13.0+20241016-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +31397953849d275aa2506580f3fa1cb5a85b6a3d392e495f8030e8b6412f5556 20241016/cpython-3.13.0+20241016-aarch64-apple-darwin-install_only.tar.gz +59b50df9826475d24bb7eff781fa3949112b5e9c92adb29e96a09cdf1216d5bd 20241016/cpython-3.13.0+20241016-aarch64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +e8378c0162b2e0e4cc1f62b29443a3305d116d09583304dbb0149fecaff6347b 20241016/cpython-3.13.0+20241016-aarch64-unknown-linux-gnu-install_only.tar.gz +1217efa5f4ce67fcc9f7eb64165b1bd0912b2a21bc25c1a7e2cb174a21a5df7e 20241016/cpython-3.13.0+20241016-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +fc4b7f27c4e84c78f3c8e6c7f8e4023e4638d11f1b36b6b5ce457b1926cebb53 20241016/cpython-3.13.0+20241016-ppc64le-unknown-linux-gnu-install_only.tar.gz +6c3e1e4f19d2b018b65a7e3ef4cd4225c5b9adfbc490218628466e636d5c4b8c 20241016/cpython-3.13.0+20241016-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +66b19e6a07717f6cfcd3a8ca953f0a2eaa232291142f3d26a8d17c979ec0f467 20241016/cpython-3.13.0+20241016-s390x-unknown-linux-gnu-install_only.tar.gz +2e07dfea62fe2215738551a179c87dbed1cc79d1b3654f4d7559889a6d5ce4eb 20241016/cpython-3.13.0+20241016-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +cff1b7e7cd26f2d47acac1ad6590e27d29829776f77e8afa067e9419f2f6ce77 20241016/cpython-3.13.0+20241016-x86_64-apple-darwin-install_only.tar.gz +bfd89f9acf866463bc4baf01733da5e767d13f5d0112175a4f57ba91f1541310 20241016/cpython-3.13.0+20241016-x86_64-pc-windows-msvc-shared-freethreaded+pgo-full.tar.zst +b25926e8ce4164cf103bacc4f4d154894ea53e07dd3fdd5ebb16fb1a82a7b1a0 20241016/cpython-3.13.0+20241016-x86_64-pc-windows-msvc-shared-install_only.tar.gz +a73adeda301ad843cce05f31a2d3e76222b656984535a7b87696a24a098b216c 20241016/cpython-3.13.0+20241016-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +2c8cb15c6a2caadaa98af51df6fe78a8155b8471cb3dd7b9836038e0d3657fb4 20241016/cpython-3.13.0+20241016-x86_64-unknown-linux-gnu-install_only.tar.gz +2f61ee3b628a56aceea63b46c7afe2df3e22a61da706606b0c8efda57f953cf4 20241016/cpython-3.13.0+20241016-x86_64-unknown-linux-musl-install_only.tar.gz +08f05618bdcf8064a7960b25d9ba92155447c9b08e0cf2f46a981e4c6a1bb5a5 20241205/cpython-3.13.1+20241205-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +88b88b609129c12f4b3841845aca13230f61e97ba97bd0fb28ee64b0e442a34f 20241205/cpython-3.13.1+20241205-aarch64-apple-darwin-install_only.tar.gz +9f2fcb809f9ba6c7c014a8803073a88786701a98971135bce684355062e4bb35 20241205/cpython-3.13.1+20241205-aarch64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +fdfa86c2746d2ae700042c461846e6c37f70c249925b58de8cd02eb8d1423d4e 20241205/cpython-3.13.1+20241205-aarch64-unknown-linux-gnu-install_only.tar.gz +15ceea78dff78ca8ccaac8d9c54b808af30daaa126f1f561e920a6896e098634 20241205/cpython-3.13.1+20241205-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +27b20b3237c55430ca1304e687d021f88373f906249f9cd272c5ff2803d5e5c3 20241205/cpython-3.13.1+20241205-ppc64le-unknown-linux-gnu-install_only.tar.gz +ed3c6118d1d12603309c930e93421ac7a30a69045ffd43006f63ecf71d72c317 20241205/cpython-3.13.1+20241205-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +7d0187e20cb5e36c689eec27e4d3de56d8b7f1c50dc5523550fc47377801521f 20241205/cpython-3.13.1+20241205-s390x-unknown-linux-gnu-install_only.tar.gz +dc780fecd215d2cc9e573abf1e13a175fcfa8f6efd100ef888494a248a16cda8 20241205/cpython-3.13.1+20241205-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +47eef6efb8664e2d1d23a7cdaf56262d784f8ace48f3bfca1b183e95a49888d6 20241205/cpython-3.13.1+20241205-x86_64-apple-darwin-install_only.tar.gz +7537b2ab361c0eabc0eabfca9ffd9862d7f5f6576eda13b97e98aceb5eea4fd3 20241205/cpython-3.13.1+20241205-x86_64-pc-windows-msvc-shared-freethreaded+pgo-full.tar.zst +f51f0493a5f979ff0b8d8c598a8d74f2a4d86a190c2729c85e0af65c36a9cbbe 20241205/cpython-3.13.1+20241205-x86_64-pc-windows-msvc-shared-install_only.tar.gz +9ec1b81213f849d91f5ebe6a16196e85cd6ff7c05ca823ce0ab7ba5b0e9fee84 20241205/cpython-3.13.1+20241205-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +242b2727df6c1e00de6a9f0f0dcb4562e168d27f428c785b0eb41a6aeb34d69a 20241205/cpython-3.13.1+20241205-x86_64-unknown-linux-gnu-install_only.tar.gz +76b30c6373b9c0aa2ba610e07da02f384aa210ac79643da38c66d3e6171c6ef5 20241205/cpython-3.13.1+20241205-x86_64-unknown-linux-musl-install_only.tar.gz +e3c4aa607717b23903ca2650d5c3ee24f89b97543e2db2b0f463bddc7a9e92f3 20241206/cpython-3.12.8+20241206-aarch64-apple-darwin-install_only.tar.gz +ce674b55442b732973afb2932c281bb1ded4ad7e22bcf9b07071165770758c7e 20241206/cpython-3.12.8+20241206-aarch64-unknown-linux-gnu-install_only.tar.gz +b7214790b273de9ed0532420054b72ba1393d62d2fc844ec55ade193771bd90c 20241206/cpython-3.12.8+20241206-ppc64le-unknown-linux-gnu-install_only.tar.gz +73102f5dbd7d1e7e9c2f2c80aedf2893d99a7fa407f6674ec8b2f57ba07daee5 20241206/cpython-3.12.8+20241206-s390x-unknown-linux-gnu-install_only.tar.gz +3ba35c706577d755e8e52a4c161a042464577c0e695e2a605362fa469e26de10 20241206/cpython-3.12.8+20241206-x86_64-apple-darwin-install_only.tar.gz +767b4be3ddf6b99e5ade519789c1615c191d8cf99d5aff4685cc18b48931f1e6 20241206/cpython-3.12.8+20241206-x86_64-pc-windows-msvc-shared-install_only.tar.gz +b9d6ee5ddac1198e72d53112698773fc8bb597de095592eb849ca794306699ba 20241206/cpython-3.12.8+20241206-x86_64-unknown-linux-gnu-install_only.tar.gz +6f305888703691dd04cfff85284d23ea0b0146ed7c4415e472f1fb72b3f32cdf 20241206/cpython-3.12.8+20241206-x86_64-unknown-linux-musl-install_only.tar.gz +e99f8457d9c79592c036489c5cfa78df76e4762d170665e499833e045d82608f 20250317/cpython-3.10.16+20250317-aarch64-apple-darwin-install_only.tar.gz +76d0f04d2444e77200fdc70d1c57480e29cca78cb7420d713bc1c523709c198d 20250317/cpython-3.10.16+20250317-aarch64-unknown-linux-gnu-install_only.tar.gz +39c9b3486de984fe1d72d90278229c70d6b08bcf69cd55796881b2d75077b603 20250317/cpython-3.10.16+20250317-ppc64le-unknown-linux-gnu-install_only.tar.gz +ebe949ada9293581c17d9bcdaa8f645f67d95f73eac65def760a71ef9dd6600d 20250317/cpython-3.10.16+20250317-riscv64-unknown-linux-gnu-install_only.tar.gz +9b2fc0b7f1c75b48e799b6fa14f7e24f5c61f2db82e3c65d13ed25e08f7f0857 20250317/cpython-3.10.16+20250317-s390x-unknown-linux-gnu-install_only.tar.gz +e03e62dbe95afa2f56b7344ff3bd061b180a0b690ff77f9a1d7e6601935e05ca 20250317/cpython-3.10.16+20250317-x86_64-apple-darwin-install_only.tar.gz +c7e0eb0ff5b36758b7a8cacd42eb223c056b9c4d36eded9bf5b9fe0c0b9aeb08 20250317/cpython-3.10.16+20250317-x86_64-pc-windows-msvc-install_only.tar.gz +b350c7e63956ca8edb856b91316328e0fd003a840cbd63d08253af43b2c63643 20250317/cpython-3.10.16+20250317-x86_64-unknown-linux-gnu-install_only.tar.gz +6ed64923ee4fbea4c5780f1a5a66651d239191ac10bd23420db4f5e4e0bf79c4 20250317/cpython-3.10.16+20250317-x86_64-unknown-linux-musl-install_only.tar.gz +7c7fd9809da0382a601a79287b5d62d61ce0b15f5a5ee836233727a516e85381 20250317/cpython-3.12.9+20250317-aarch64-apple-darwin-install_only.tar.gz +00c6bf9acef21ac741fea24dc449d0149834d30e9113429e50a95cce4b00bb80 20250317/cpython-3.12.9+20250317-aarch64-unknown-linux-gnu-install_only.tar.gz +25d77599dfd5849f17391d92da0da99079e4e94f19a881f763f5cc62530ef7e1 20250317/cpython-3.12.9+20250317-ppc64le-unknown-linux-gnu-install_only.tar.gz +e97ab0fdf443b302c56a52b4fd08f513bf3be66aa47263f0f9df3c6e60e05f2e 20250317/cpython-3.12.9+20250317-riscv64-unknown-linux-gnu-install_only.tar.gz +7492d079ffa8425c8f6c58e43b237c37e3fb7b31e2e14635927bb4d3397ba21e 20250317/cpython-3.12.9+20250317-s390x-unknown-linux-gnu-install_only.tar.gz +1ee1b1bb9fbce5c145c4bec9a3c98d7a4fa22543e09a7c1d932bc8599283c2dc 20250317/cpython-3.12.9+20250317-x86_64-apple-darwin-install_only.tar.gz +d15361fd202dd74ae9c3eece1abdab7655f1eba90bf6255cad1d7c53d463ed4d 20250317/cpython-3.12.9+20250317-x86_64-pc-windows-msvc-install_only.tar.gz +ef382fb88cbb41a3b0801690bd716b8a1aec07a6c6471010bcc6bd14cd575226 20250317/cpython-3.12.9+20250317-x86_64-unknown-linux-gnu-install_only.tar.gz +94e3837da1adf9964aab2d6047b33f70167de3096d1f9a2d1fa9340b1bbf537d 20250317/cpython-3.12.9+20250317-x86_64-unknown-linux-musl-install_only.tar.gz +c98c9c977e6fa05c3813bd49f3553904d89d60fed27e2e36468da7afa1d6d5e2 20250317/cpython-3.13.2+20250317-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +faa44274a331eb39786362818b21b3a4e74514e8805000b20b0e55c590cecb94 20250317/cpython-3.13.2+20250317-aarch64-apple-darwin-install_only.tar.gz +b8635e59e3143fd17f19a3dfe8ccc246ee6587c87da359bd1bcab35eefbb5f19 20250317/cpython-3.13.2+20250317-aarch64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +9c67260446fee6ea706dad577a0b32936c63f449c25d66e4383d5846b2ab2e36 20250317/cpython-3.13.2+20250317-aarch64-unknown-linux-gnu-install_only.tar.gz +6ae8fa44cb2edf4ab49cff1820b53c40c10349c0f39e11b8cd76ce7f3e7e1def 20250317/cpython-3.13.2+20250317-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +345b53d2f86c9dbd7f1320657cb227ff9a42ef63ff21f129abbbc8c82a375147 20250317/cpython-3.13.2+20250317-ppc64le-unknown-linux-gnu-install_only.tar.gz +2af1b8850c52801fb6189e7a17a51e0c93d9e46ddefcca72247b76329c97d02a 20250317/cpython-3.13.2+20250317-riscv64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +172d22b2330737f3a028ea538ffe497c39a066a8d3200b22dd4d177a3332ad85 20250317/cpython-3.13.2+20250317-riscv64-unknown-linux-gnu-install_only.tar.gz +c074144cc80c2af32c420b79a9df26e8db405212619990c1fbdd308bd75afe3f 20250317/cpython-3.13.2+20250317-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +ec3b16ea8a97e3138acec72bc5ff35949950c62c8994a8ec8e213fd93f0e806b 20250317/cpython-3.13.2+20250317-s390x-unknown-linux-gnu-install_only.tar.gz +0d73e4348d8d4b5159058609d2303705190405b485dd09ad05d870d7e0f36e0f 20250317/cpython-3.13.2+20250317-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +ee4526e84b5ce5b11141c50060b385320f2773616249a741f90c96d460ce8e8f 20250317/cpython-3.13.2+20250317-x86_64-apple-darwin-install_only.tar.gz +c51b4845fda5421e044067c111192f645234081d704313f74ee77fa013a186ea 20250317/cpython-3.13.2+20250317-x86_64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +84d7b52f3558c8e35c670a4fa14080c75e3ec584adfae49fec8b51008b75b21e 20250317/cpython-3.13.2+20250317-x86_64-pc-windows-msvc-install_only.tar.gz +1aea5062614c036904b55c1cc2fb4b500b7f6f7a4cacc263f4888889d355eef8 20250317/cpython-3.13.2+20250317-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +db011f0cd29cab2291584958f4e2eb001b0e6051848d89b38a2dc23c5c54e512 20250317/cpython-3.13.2+20250317-x86_64-unknown-linux-gnu-install_only.tar.gz +00bb2d629f7eacbb5c6b44dc04af26d1f1da64cee3425b0d8eb5135a93830296 20250317/cpython-3.13.2+20250317-x86_64-unknown-linux-musl-install_only.tar.gz +278dccade56b4bbeecb9a613b77012cf5c1433a5e9b8ef99230d5e61f31d9e02 20250610/cpython-3.13.4+20250610-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +c2ce6601b2668c7bd1f799986af5ddfbff36e88795741864aba6e578cb02ed7f 20250610/cpython-3.13.4+20250610-aarch64-apple-darwin-install_only.tar.gz +b1c1bd6ab9ef95b464d92a6a911cef1a8d9f0b0f6a192f694ef18ed15d882edf 20250610/cpython-3.13.4+20250610-aarch64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +3c2596ece08ffe17e11bc1f27aeb4ce1195d2490a83d695d36ef4933d5c5ca53 20250610/cpython-3.13.4+20250610-aarch64-unknown-linux-gnu-install_only.tar.gz +ed66ae213a62b286b9b7338b816ccd2815f5248b7a28a185dc8159fe004149ae 20250610/cpython-3.13.4+20250610-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +b3cc13ee177b8db1d3e9b2eac413484e3c6a356f97d91dc59de8d3fd8cf79d6b 20250610/cpython-3.13.4+20250610-ppc64le-unknown-linux-gnu-install_only.tar.gz +913264545215236660e4178bc3e5b57a20a444a8deb5c11680c95afc960b4016 20250610/cpython-3.13.4+20250610-riscv64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +d1b989e57a9ce29f6c945eeffe0e9750c222fdd09e99d2f8d6b0d8532a523053 20250610/cpython-3.13.4+20250610-riscv64-unknown-linux-gnu-install_only.tar.gz +7556a38ab5e507c1ec22bc38f9859982bc956cab7f4de05a2faac114feb306db 20250610/cpython-3.13.4+20250610-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +d1d19fb01961ac6476712fdd6c5031f74c83666f6f11aa066207e9a158f7e3d8 20250610/cpython-3.13.4+20250610-s390x-unknown-linux-gnu-install_only.tar.gz +64ab7ac8c88002d9ba20a92f72945bfa350268e944a7922500af75d20330574d 20250610/cpython-3.13.4+20250610-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +79feb6ca68f3921d07af52d9db06cf134e6f36916941ea850ab0bc20f5ff638b 20250610/cpython-3.13.4+20250610-x86_64-apple-darwin-install_only.tar.gz +9457504547edb2e0156bf76b53c7e4941c7f61c0eff9fd5f4d816d3df51c58e3 20250610/cpython-3.13.4+20250610-x86_64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +29ac3585cc2dcfd79e3fe380c272d00e9d34351fc456e149403c86d3fea34057 20250610/cpython-3.13.4+20250610-x86_64-pc-windows-msvc-install_only.tar.gz +864df6e6819e8f8e855ce30f34410fdc5867d0616e904daeb9a40e5806e970d7 20250610/cpython-3.13.4+20250610-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +44e5477333ebca298a7a0a316985c6c3533b8645f92a83f7f73c44033832bf32 20250610/cpython-3.13.4+20250610-x86_64-unknown-linux-gnu-install_only.tar.gz +a3afbfa94b9ff4d9fc426b47eb3c8446cada535075b8d51b7bdc9d9ab9911fc2 20250610/cpython-3.13.4+20250610-x86_64-unknown-linux-musl-install_only.tar.gz +a94c02b2d597cd6b075a713fe4e9a909cc97ca6a3b2b2ce86eda21be2062d48e 20250808/cpython-3.10.18+20250808-aarch64-apple-darwin-install_only.tar.gz +ef7de3b715d519e246d98ff7856247f7f7b357068705f09c6f300b7e7b76c701 20250808/cpython-3.10.18+20250808-aarch64-unknown-linux-gnu-install_only.tar.gz +f580efed11cc54e1a221c052e8bc88bfbc12844d3ca8949da828351a1232386e 20250808/cpython-3.10.18+20250808-ppc64le-unknown-linux-gnu-install_only.tar.gz +0d7e460e30203a9225b6f417ae972f66415a1cc0e32b37ebc48d195816282669 20250808/cpython-3.10.18+20250808-riscv64-unknown-linux-gnu-install_only.tar.gz +d4ada974daadb08a0184c19232ee3b03b3137aa70609760e1a94aaf7b12989ef 20250808/cpython-3.10.18+20250808-s390x-unknown-linux-gnu-install_only.tar.gz +da96fe2ba841640215788ddb9f151f03629360e37fcb94d4f76e5095b87df0d4 20250808/cpython-3.10.18+20250808-x86_64-apple-darwin-install_only.tar.gz +a648f3c9d136985ccfe57a5507e73d9d0839f7fd09eebd7c247857f2feaecb2a 20250808/cpython-3.10.18+20250808-x86_64-pc-windows-msvc-install_only.tar.gz +0b310a73bb9e7a495dbcad5f685e508ca2e7b36ee8f29301a52285730c425789 20250808/cpython-3.10.18+20250808-x86_64-unknown-linux-gnu-install_only.tar.gz +9cecf6ea2effbe183faebcf7e1160425a4ee17a68e49f2eefe5e1c59c51fa7ee 20250808/cpython-3.10.18+20250808-x86_64-unknown-linux-musl-install_only.tar.gz +d089bfd2c7b98a0942750a195e70d3172beda76d7747097b8afd87028b6e59b6 20250808/cpython-3.11.13+20250808-aarch64-apple-darwin-install_only.tar.gz +a632857c966237e7fd38b44c47c350f6e30d8ec54dcad6c832865ad670f0f22f 20250808/cpython-3.11.13+20250808-aarch64-pc-windows-msvc-install_only.tar.gz +bc57105f8a16acd57b71d926143c7f6ecf61729b40c8b4656f1b98bebd47c710 20250808/cpython-3.11.13+20250808-aarch64-unknown-linux-gnu-install_only.tar.gz +16a0165b0744940702b8fff80b8bf973ac914f78cb6fca28d389583f675e84de 20250808/cpython-3.11.13+20250808-ppc64le-unknown-linux-gnu-install_only.tar.gz +d8e62306be8f41c46bcd62ca68f91a1467f47adff632a35ff413dc1043ed56e8 20250808/cpython-3.11.13+20250808-riscv64-unknown-linux-gnu-install_only.tar.gz +4e302a4514a73baefdd9b327062bdafeb4115a799deec91c185f6ab45a857241 20250808/cpython-3.11.13+20250808-s390x-unknown-linux-gnu-install_only.tar.gz +d946d618f8bba8308b67e460a30612a71e2ccc309f85f6628aaae24e2b816981 20250808/cpython-3.11.13+20250808-x86_64-apple-darwin-install_only.tar.gz +ed963aee33d29ad8abfbb5fe63e42f57a2638a4a11a88e11d8bb66e61f20a6e5 20250808/cpython-3.11.13+20250808-x86_64-pc-windows-msvc-install_only.tar.gz +3ad988c702cbb017fef1208d47dea4138a2e85fd0f7f01ec5e1e335e597131b9 20250808/cpython-3.11.13+20250808-x86_64-unknown-linux-gnu-install_only.tar.gz +3a5810f0696f844289aa06d5c3a1efeab66eee999c25196b7d1954192a2c2100 20250808/cpython-3.11.13+20250808-x86_64-unknown-linux-musl-install_only.tar.gz +8792c4a84c364ab975feca0c27d3157a5435b7baab325a346ae56b223893b661 20250808/cpython-3.12.11+20250808-aarch64-apple-darwin-install_only.tar.gz +00bf7d7e8bcf5d1e9c4dfca0247d8e035147777cd57ee9d4c64dedca86b0a464 20250808/cpython-3.12.11+20250808-aarch64-pc-windows-msvc-install_only.tar.gz +4d7ba5314fab02130d6538f074961ffbf61310cade9180e59026074f9a8939cb 20250808/cpython-3.12.11+20250808-aarch64-unknown-linux-gnu-install_only.tar.gz +2c862eb40a81549d9c11e6bf5a7f07c3406310b14e6a4d16dcdf1c4763ef7090 20250808/cpython-3.12.11+20250808-ppc64le-unknown-linux-gnu-install_only.tar.gz +0bb729b95fabd49c7b495f7c44a9086e3970ea57daf66365741574bd36a17e81 20250808/cpython-3.12.11+20250808-riscv64-unknown-linux-gnu-install_only.tar.gz +99e465882d217d24ac90e99fac8f32e6a644d0340ac05ee510fb5cdf53f0cfb8 20250808/cpython-3.12.11+20250808-s390x-unknown-linux-gnu-install_only.tar.gz +e0c932709dafb05f00e528a7560ef8ee559ac82b75faca60dd1245bca1c1553f 20250808/cpython-3.12.11+20250808-x86_64-apple-darwin-install_only.tar.gz +81214ef71964a40ec269a79067ca490d45298c350583bc3af0e5781451a05c3c 20250808/cpython-3.12.11+20250808-x86_64-pc-windows-msvc-install_only.tar.gz +63d78840bf209af8da8f24e335d910f88387b892ca9187be571d481c071751bb 20250808/cpython-3.12.11+20250808-x86_64-unknown-linux-gnu-install_only.tar.gz +d633d070780590aa03ac5575cd9d7b9e17682d80f14b400313c009c387cf706b 20250808/cpython-3.12.11+20250808-x86_64-unknown-linux-musl-install_only.tar.gz +f2143304012e021a603bf1807bf3e4ce163832e43ab9a9829e53cb136497f207 20250808/cpython-3.13.6+20250808-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +8a1efa6af4e80f08e2c97dda822a3d6c24d6c98e518242f802c6a43ae8401488 20250808/cpython-3.13.6+20250808-aarch64-apple-darwin-install_only.tar.gz +552cfabcc3b103f4b1c4036d2592d5f0373c9554a2c4d2b6631b04ef7e592067 20250808/cpython-3.13.6+20250808-aarch64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +8e1617bd407ec1a874499daab26ae95080d1e0267ae616d34490137a28705827 20250808/cpython-3.13.6+20250808-aarch64-pc-windows-msvc-install_only.tar.gz +d84a7d64c284be387386b9f5da273f6d05486eb6bd8f9e86e2575cb59604cb22 20250808/cpython-3.13.6+20250808-aarch64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +11fa0591ae2211c08a42ae54944260e36ddf88a1d5604ea0c49e2477be4e5388 20250808/cpython-3.13.6+20250808-aarch64-unknown-linux-gnu-install_only.tar.gz +e76fcaf1bf80a615520dbe7f85ca0bb557fad96d132d836b0ac721e7cc1e2a37 20250808/cpython-3.13.6+20250808-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +8dcf34ae1a685fe1893b52917ae04f23328edadc4acae28499d43850c2bdd26c 20250808/cpython-3.13.6+20250808-ppc64le-unknown-linux-gnu-install_only.tar.gz +24e08a39ba4fc77753e61541e52eed39cc871f4a92a80a3c5dd495056bd8eff9 20250808/cpython-3.13.6+20250808-riscv64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +f8ed75aa6cc2011a046be00b629c3c8295267f34280324feaff34c73e7afce39 20250808/cpython-3.13.6+20250808-riscv64-unknown-linux-gnu-install_only.tar.gz +1609b223fd38a4a7a4d20e7173d7d9390fe2258f7dd9a15dc9ef0fa49613735d 20250808/cpython-3.13.6+20250808-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +7707ee5d19a78bc64ef8a66751ec7f97b64ea06714c7b1b52e8b321c2923ead8 20250808/cpython-3.13.6+20250808-s390x-unknown-linux-gnu-install_only.tar.gz +4360a1278dd0a96b526d108c8fd23498a9d2028dd7791e510fd51ff5ea3f462a 20250808/cpython-3.13.6+20250808-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +27badce7201321a8363219e438a6205165e5b4884012b1046532203df2ec9379 20250808/cpython-3.13.6+20250808-x86_64-apple-darwin-install_only.tar.gz +4e727cdbe4057b16a170f887c0fa4227a825ac59bcda84ae946c77cc932af78c 20250808/cpython-3.13.6+20250808-x86_64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +af5cc733c33b9aa9f1d74c81a59351e9b27215486d8b6cdbc06d97646a58c953 20250808/cpython-3.13.6+20250808-x86_64-pc-windows-msvc-install_only.tar.gz +e48c13c59cc3c01b79f63c8bccec27d2db6e97f64213b8731e2077b6ed8ed52c 20250808/cpython-3.13.6+20250808-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +f844e8c8b6847628b472f7e97d8893a4e93acd5382a902b465776063668c4d64 20250808/cpython-3.13.6+20250808-x86_64-unknown-linux-gnu-install_only.tar.gz +70076dea0ff65b3c05aae1a97b4a556bf613cc73db30309e59134f9d318f4f7b 20250808/cpython-3.13.6+20250808-x86_64-unknown-linux-musl-install_only.tar.gz +43bda24c2fc073bc308bf631203b917a72640d59b59fdad4ba14503d84727012 20251031/cpython-3.10.19+20251031-aarch64-apple-darwin-install_only.tar.gz +f77a8a8aa77f3f943126fa9215a25309da4bf20398fc8f4b4eec54b5fc7570ef 20251031/cpython-3.10.19+20251031-aarch64-unknown-linux-gnu-install_only.tar.gz +1c55d160fc4c3b93528cd6aaa2bb4ca6018a99e5a45919d33dc761a43a69f860 20251031/cpython-3.10.19+20251031-ppc64le-unknown-linux-gnu-install_only.tar.gz +21134d35721cdad4c881f35d0957cc19df9a45d194afb38a099faded3c1cfb4d 20251031/cpython-3.10.19+20251031-riscv64-unknown-linux-gnu-install_only.tar.gz +df0db070f1eb73ab4e371eea32213ddb3500737ea5560a6f0ffd65c82af64ddc 20251031/cpython-3.10.19+20251031-s390x-unknown-linux-gnu-install_only.tar.gz +76c12e633c09c2a790f8a958a55df4495527e0718d1875310c836e757c0c7b55 20251031/cpython-3.10.19+20251031-x86_64-apple-darwin-install_only.tar.gz +cfa08a4caf2df1b43551b843c052d6a8814e2ea0c97268b021f0423646c244c3 20251031/cpython-3.10.19+20251031-x86_64-pc-windows-msvc-install_only.tar.gz +fb1caac917d7b6497bb6f5950da5f1e48d05c43a498948dd97f85760c4382d9f 20251031/cpython-3.10.19+20251031-x86_64-unknown-linux-gnu-install_only.tar.gz +ba85013ed5ac7733fc6840168cc33ed19e9959b363dc80227d54f8fd9c92c0f4 20251031/cpython-3.10.19+20251031-x86_64-unknown-linux-musl-install_only.tar.gz +6de5572b33c65af1c9b7caf00ec593fb04cffb7e14fa393a98261bb9bc464713 20251031/cpython-3.11.14+20251031-aarch64-apple-darwin-install_only.tar.gz +38d0d1466561e15965e8d2c20f5e5be649598f55c761ecab553d087fbd217337 20251031/cpython-3.11.14+20251031-aarch64-pc-windows-msvc-install_only.tar.gz +510edb027527413c4249256194cb8ad2590b52dd93f7123b4cb341aff5d05894 20251031/cpython-3.11.14+20251031-aarch64-unknown-linux-gnu-install_only.tar.gz +4e0bc6a818e0c6a9d7d3ebe1a95591fd84440520577aa837facc96a4b7a80e35 20251031/cpython-3.11.14+20251031-ppc64le-unknown-linux-gnu-install_only.tar.gz +16519e69297144f81b2421333bc9e0b6466cf3c84749b216b695cfb4c9deb32f 20251031/cpython-3.11.14+20251031-riscv64-unknown-linux-gnu-install_only.tar.gz +5f9c1b203cdf34c8bff1aef69b63bbf11309bd16ca6e429d8c3651eaa2b3d080 20251031/cpython-3.11.14+20251031-s390x-unknown-linux-gnu-install_only.tar.gz +4891cbf34e8652b7bd1054b9502395e4b7e048e2e517c040fbf6c8297cb954d6 20251031/cpython-3.11.14+20251031-x86_64-apple-darwin-install_only.tar.gz +5223b83ed9e2aa5e9e17d2ebcf767956e998876339b9cde1980a47e9d4655fb6 20251031/cpython-3.11.14+20251031-x86_64-pc-windows-msvc-install_only.tar.gz +60f0bd473d861cc45d3401d9914e47ccb9fa037f88a91879ed517a62042b8477 20251031/cpython-3.11.14+20251031-x86_64-unknown-linux-gnu-install_only.tar.gz +25e82d1e85b90a8ab724ee633a1811b1921797f5c25ee69c6595052371b91a87 20251031/cpython-3.11.14+20251031-x86_64-unknown-linux-musl-install_only.tar.gz +5e110cb821d2eb8246065d3b46faa655180c976c4e17250f7883c634a629bc63 20251031/cpython-3.12.12+20251031-aarch64-apple-darwin-install_only.tar.gz +b190fed7c2b0f6e1010f554a0d1fd191c0754c4c0718e69d9d795ae559613780 20251031/cpython-3.12.12+20251031-aarch64-pc-windows-msvc-install_only.tar.gz +81b644d166e0bfb918615af8a2363f8fcf26eccdcc60a5334b6a62c088470bac 20251031/cpython-3.12.12+20251031-aarch64-unknown-linux-gnu-install_only.tar.gz +024f5e5678c9768d45cc24d37a8e9d265aae86c4a4602352dee3d7deba367052 20251031/cpython-3.12.12+20251031-ppc64le-unknown-linux-gnu-install_only.tar.gz +b13c57fc372c131e667a99b9680f41c0b4da571cf99ed412103c2fe9ad5ed1fb 20251031/cpython-3.12.12+20251031-riscv64-unknown-linux-gnu-install_only.tar.gz +2bf05bdd56cdf5ea4fd9f2faf151ea4211be96a0d1f4230b85f5dcae620d6400 20251031/cpython-3.12.12+20251031-s390x-unknown-linux-gnu-install_only.tar.gz +687052a046d33be49dc95dd671816709067cf6176ed36c93ea61b1fe0b883b0f 20251031/cpython-3.12.12+20251031-x86_64-apple-darwin-install_only.tar.gz +cff398b3f520c442a1b085dd347126c10c1b03f01ccc0decd8c897a687e893f1 20251031/cpython-3.12.12+20251031-x86_64-pc-windows-msvc-install_only.tar.gz +80c3882f14e15cef8260ef5257d198e8f4371ca265887431d939e0d561de3253 20251031/cpython-3.12.12+20251031-x86_64-unknown-linux-gnu-install_only.tar.gz +0a461330b9b89f2ea3088dde10d7a3f96aa65897b7c5ce2404fa3b5c4b8daa14 20251031/cpython-3.12.12+20251031-x86_64-unknown-linux-musl-install_only.tar.gz +eae1272a72ccce601590a10a9ca2a58199b5fcdf022aa603a527e3e2a04de9bc 20251031/cpython-3.13.9+20251031-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +1f3568d17383426d52350c2ef7c93c1a5a043198b860cb05e5d19b35f9c25cef 20251031/cpython-3.13.9+20251031-aarch64-apple-darwin-install_only.tar.gz +743ff69935ef28834621647dab30f032dfcd80315732917531eea333210941c7 20251031/cpython-3.13.9+20251031-aarch64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +20db43873d3c4c2175d866806545e4ad4ec6bb72ca95e60082a4df6c24567e8c 20251031/cpython-3.13.9+20251031-aarch64-pc-windows-msvc-install_only.tar.gz +a6e72f9de5d9b46cf6968d6a492f2401a919f9b959f8da2d87f43484b80169ee 20251031/cpython-3.13.9+20251031-aarch64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +0a56d11b0fb1662e67f892b9d5d1717aef06f24dbb8362bc25b8f784e620d44e 20251031/cpython-3.13.9+20251031-aarch64-unknown-linux-gnu-install_only.tar.gz +0ed5c65437f875c58ba1bee2b8d261d18698d3d0347a2e66f8902fce022a2cda 20251031/cpython-3.13.9+20251031-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +99492123902bd5e9a6b1a30135061e93a2e6a11d25107a741d5a756e91054448 20251031/cpython-3.13.9+20251031-ppc64le-unknown-linux-gnu-install_only.tar.gz +584e481d9b5225ffaf02f158fb26d2818207e65fc3c6dc21a6d500277f739220 20251031/cpython-3.13.9+20251031-riscv64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +b3dce3e4ef508773521e1ee1be989fff6118f8fd1fbbd0491d7ff7dfbc98ef06 20251031/cpython-3.13.9+20251031-riscv64-unknown-linux-gnu-install_only.tar.gz +7fa7fb912ca989ceac026a332d56a2c7d6d16ab0e94d89e690de5aade26103e2 20251031/cpython-3.13.9+20251031-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +f10e34aaa856c1b8a69c2ea4a9a6723d520443d1a957bf66dc55491334ca0c1e 20251031/cpython-3.13.9+20251031-s390x-unknown-linux-gnu-install_only.tar.gz +e2bf5fa6a3ef443ade362e08b0a19bbc172f7bfe34dabe933ccaad31d53af5da 20251031/cpython-3.13.9+20251031-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +48c0f3ca5d31e90658ef99138dc21865bb62f388ab97a1ce72cac176da194ab0 20251031/cpython-3.13.9+20251031-x86_64-apple-darwin-install_only.tar.gz +318a9a1e43dd52054327de3bccc0c5b7afde7b7f2a398ccb4d38e03d28b05386 20251031/cpython-3.13.9+20251031-x86_64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +874593f641f31ea101440c70f81768c35d4d7d6df111fde63094db67465ef787 20251031/cpython-3.13.9+20251031-x86_64-pc-windows-msvc-install_only.tar.gz +dcc29b069d0588fbd4ea29c6df840c8d1207d2a3bce8cd5cd57d1b85373b6048 20251031/cpython-3.13.9+20251031-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +6f05b91ee8c7e6dd0f9c60b95bb29130e2d623961de6578b643e80ddd83f96b6 20251031/cpython-3.13.9+20251031-x86_64-unknown-linux-gnu-install_only.tar.gz +ad987197034185e628715da504a50613af213dc21ba6d5ccaeab3db2c464aa6c 20251031/cpython-3.13.9+20251031-x86_64-unknown-linux-musl-install_only.tar.gz +d9c7b430b25bd3837dbb03f945dbe6b7bc526c5940ca96f5db7cdc42f6b2b801 20251031/cpython-3.14.0+20251031-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +b4bcd3c6c24cab32ae99e1b05c89312b783b4d69431d702e5012fe1fdcad4087 20251031/cpython-3.14.0+20251031-aarch64-apple-darwin-install_only.tar.gz +40266e60f655e49cd1d5303295255909a4b593b08b88be6e6a55b2c9fe6ed13d 20251031/cpython-3.14.0+20251031-aarch64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +599a8b7e12439cd95a201dbdfe95cf363146b1ff91f379555dafd86b170caab9 20251031/cpython-3.14.0+20251031-aarch64-pc-windows-msvc-install_only.tar.gz +f383ef50d1da6ca511212e5ae601923b56636b87351fd5fc847e0ea0a19fa9b3 20251031/cpython-3.14.0+20251031-aarch64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +128a9cbfb9645d5237ec01704d9d1d2ac5f084464cc43c37a4cd96aa9c3b1ad5 20251031/cpython-3.14.0+20251031-aarch64-unknown-linux-gnu-install_only.tar.gz +cb0e4ff781b856a47f0f461ceb41c78c7eeff65effd0957857ec4702ef1e1bd3 20251031/cpython-3.14.0+20251031-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +e16ca51f018e99a609faf953bd3a3aea31f45ee84262d1a517fb3abd98f1f4af 20251031/cpython-3.14.0+20251031-ppc64le-unknown-linux-gnu-install_only.tar.gz +929223470d11a55cd75f880ac3bd4969e42407e2cdf08d4e7e38ba721cf4abec 20251031/cpython-3.14.0+20251031-riscv64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +fca340d8fb7a05cd90e216ce601b25d492ed8c1a3b6a6d77703e0f15ab3711a7 20251031/cpython-3.14.0+20251031-riscv64-unknown-linux-gnu-install_only.tar.gz +613fb1f7b249f798b52af957d181305244e936c8e5c94c84688fcdf93fe14253 20251031/cpython-3.14.0+20251031-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +c5803644970eee931bb0581b3b64511d1a8612f67bc98951a7f7ab5581a9ed04 20251031/cpython-3.14.0+20251031-s390x-unknown-linux-gnu-install_only.tar.gz +b3196f6b57bbb3dc2ee07f348f1d51117ffa376979eceafbf50c15f0f7980bf8 20251031/cpython-3.14.0+20251031-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +4e71a3ce973be377ef18637826648bb936e2f9490f64a9e4f33a49bcc431d344 20251031/cpython-3.14.0+20251031-x86_64-apple-darwin-install_only.tar.gz +b81de5fc9e783ea6dfcf1098c28a278c874999c71afbb0309f6a8b4276c769d0 20251031/cpython-3.14.0+20251031-x86_64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +39acfcb3857d83eab054a3de11756ffc16b3d49c31393b9800dd2704d1f07fdf 20251031/cpython-3.14.0+20251031-x86_64-pc-windows-msvc-install_only.tar.gz +f4acbef0fbfaf7ab31ac63986da1d93dfa1c5cb797de1dcdc1a988aa18670120 20251031/cpython-3.14.0+20251031-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +3dec1ab70758a3467ac3313bbcdabf7a9b3016db5c072c4537e3cf0a9e6290f6 20251031/cpython-3.14.0+20251031-x86_64-unknown-linux-gnu-install_only.tar.gz +d0a2a6d3b1bb00dce2105377fda8aa79675d187f8d6d7010a42f651af25018dc 20251031/cpython-3.14.0+20251031-x86_64-unknown-linux-musl-install_only.tar.gz +12f1b16be4017181ad67904caf9e59e525b9b5d62f49105017d837e27b832959 20251031/cpython-3.15.0a1+20251031-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +3acf7aa3559b746498b18929456c5cacb84bae4e09249834cbc818970d71de87 20251031/cpython-3.15.0a1+20251031-aarch64-apple-darwin-install_only.tar.gz +54ca78dae455ece6fefbd7f5f287cc55d5ce197caf51921f6d871d15069d9489 20251031/cpython-3.15.0a1+20251031-aarch64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +1508bcd7195008479ed156aad3afbb3a3793097ed530690f0304a8107f0e53e8 20251031/cpython-3.15.0a1+20251031-aarch64-pc-windows-msvc-install_only.tar.gz +981fe8dfc6e7e1d0ffefa945a18d5c4c759bbe21722acf3a5cc7e62f16aa5f3c 20251031/cpython-3.15.0a1+20251031-aarch64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +d55c2aeece827e6bec83fd18515ee281d9ea0efaa3e2d20130db8f1c7cbb71c6 20251031/cpython-3.15.0a1+20251031-aarch64-unknown-linux-gnu-install_only.tar.gz +088400dec25139f38eeecb48f090ff2ce06a96a1dd79fa8f1dfec1cd1786f5ef 20251031/cpython-3.15.0a1+20251031-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +c28beda791c499b16f06256339522f0002a3e9acba003e6b8374755d7be1def2 20251031/cpython-3.15.0a1+20251031-ppc64le-unknown-linux-gnu-install_only.tar.gz +938061a0a31a06672526885de36037ddefd8c4acdb09424691b7000a8c8f8d01 20251031/cpython-3.15.0a1+20251031-riscv64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +36619f576b8154e4b56643c5c4a85c352f152df2989c4e602cbbe9c2b7ded870 20251031/cpython-3.15.0a1+20251031-riscv64-unknown-linux-gnu-install_only.tar.gz +2003e7e40bb44b3db7bca81087bfb738fe6af40e5db61cda8e23b59bf55d409e 20251031/cpython-3.15.0a1+20251031-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +5ea47be2a3a563ddd87ff510dae26b7aa7f3855ca00c5f1056ff8114c067c4e4 20251031/cpython-3.15.0a1+20251031-s390x-unknown-linux-gnu-install_only.tar.gz +64fc29e6c7a2f02a18645d968f1b3fc1d00d12a5ef3fcbb0d077fa8c62c08904 20251031/cpython-3.15.0a1+20251031-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +0ab19d3ac25f99da438b088751e5ec2421f9f6aa4292fd2dc0f8e49eb3e16bdf 20251031/cpython-3.15.0a1+20251031-x86_64-apple-darwin-install_only.tar.gz +34abc5603e1b4131f753d29b7deac865b9277912b851cbed5a149cf3e6745d3d 20251031/cpython-3.15.0a1+20251031-x86_64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +5f5d6bec2b381cfc771c49972d2a6f7b7e7ab6a1651d8fb6ef3983f3571722b3 20251031/cpython-3.15.0a1+20251031-x86_64-pc-windows-msvc-install_only.tar.gz +0e0272186d9f5169394dbc4d4d72a3f4a5762a04c2e5ac2ab1e23aa41fc8538a 20251031/cpython-3.15.0a1+20251031-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +1f356288c2b2713619cb7a4e453d33bf8882f812af2987e21e01e7ae382fefba 20251031/cpython-3.15.0a1+20251031-x86_64-unknown-linux-gnu-install_only.tar.gz +caf5311f333eef082dd69a669ca65aceba09a08fc1e78aad602ad649106f294c 20251031/cpython-3.15.0a1+20251031-x86_64-unknown-linux-musl-install_only.tar.gz +87275619c2706affa4d1090d2ca3dad354b6d69f8b85dbfafe38785870751b9a 20251031/cpython-3.9.25+20251031-aarch64-apple-darwin-install_only.tar.gz +6112d46355857680b81849764a6cf9f38cc4cd0d1cf29d432bc12fe5aeedf9d0 20251031/cpython-3.9.25+20251031-aarch64-unknown-linux-gnu-install_only.tar.gz +828364b6f54fa45ac2dc91f8e45d5b74306372af374a9ef16eeb2ea81253ed3f 20251031/cpython-3.9.25+20251031-ppc64le-unknown-linux-gnu-install_only.tar.gz +17467e0158e5ad04453c447d6773c23b044172276441e22e23058fd3ea053e27 20251031/cpython-3.9.25+20251031-riscv64-unknown-linux-gnu-install_only.tar.gz +3e9539f83e67faa813fd06171199b2d33c89821dfa9a33bf6e27ad67f1b6932d 20251031/cpython-3.9.25+20251031-s390x-unknown-linux-gnu-install_only.tar.gz +ace63cfe27a9487c4d72e1cb518be01c1d985271da0b2158e813801f7d3e5503 20251031/cpython-3.9.25+20251031-x86_64-apple-darwin-install_only.tar.gz +4fb1b416482ce94d73cfa140317a670c596c830671d137b07c26afe8c461768a 20251031/cpython-3.9.25+20251031-x86_64-pc-windows-msvc-install_only.tar.gz +42834f61eb6df43432c3dd6ab9ca3fdf8c06d10a404ebdb53d6902e6b9570b08 20251031/cpython-3.9.25+20251031-x86_64-unknown-linux-gnu-install_only.tar.gz +76593e8c889e81e82db5fe117fe15b69466f85100ab2ec0e4035aa86242b4e93 20251031/cpython-3.9.25+20251031-x86_64-unknown-linux-musl-install_only.tar.gz +3c9fdd76447c1549a0d3bc2a70c63f1daec997ab034206ac0260a03237166dbb 20251202/cpython-3.13.10+20251202-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +37afe4e77ab62ac50f197b1cb1f3bc02c82735c6be893da0996afcde5dc41048 20251202/cpython-3.13.10+20251202-aarch64-apple-darwin-install_only.tar.gz +cdb7141327bdc244715b25752593e2c9eeb3cc2764f37dfe81cfbc92db9d6d57 20251202/cpython-3.13.10+20251202-aarch64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +9060d644bd32ac0e0af970d0b21e207e6ff416b7c4dc26ffc4f9b043fb45b463 20251202/cpython-3.13.10+20251202-aarch64-pc-windows-msvc-install_only.tar.gz +6d277221fa4b172e00b29c7158ca9661917bc8db9a0084b1a0ff5c3a0ba8b648 20251202/cpython-3.13.10+20251202-aarch64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +c68280591cda1c9515a04809fa6926020177e8e5892300206e0496ea1d10290e 20251202/cpython-3.13.10+20251202-aarch64-unknown-linux-gnu-install_only.tar.gz +d265d8d1c51e25ed70279540223589f79cf99ad00b50d28b6150c2658c973885 20251202/cpython-3.13.10+20251202-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +1507e5528bd88131dc742a2941176aceea1838bc09860c21f179285b7865133b 20251202/cpython-3.13.10+20251202-ppc64le-unknown-linux-gnu-install_only.tar.gz +ec411b4a2d167c3be0a9aeb3905e045d62c8e3c3db0caeade5d47d5f60b98dd0 20251202/cpython-3.13.10+20251202-riscv64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +70169e916860b2e5b34c37c302d699eb2b8f24f28090968881942a37aeb7ed08 20251202/cpython-3.13.10+20251202-riscv64-unknown-linux-gnu-install_only.tar.gz +4fc6443948bf5b729481ea02cc5c68e80cd0da42631f6936587a2b8fd45bc62c 20251202/cpython-3.13.10+20251202-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +c5448863b64aacae62f3a213a6e6cf94ec63f96ee4d518491cd62fd3c81d952f 20251202/cpython-3.13.10+20251202-s390x-unknown-linux-gnu-install_only.tar.gz +6ce608684df0f90350c7a1742e9685a7782d9b26ec99d1bd9d55c8cf9a405040 20251202/cpython-3.13.10+20251202-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +a02761a4f189f71c0512e88df7ca2843696d61da659e47f8a5c8a9bd2c0d16f4 20251202/cpython-3.13.10+20251202-x86_64-apple-darwin-install_only.tar.gz +6a8b0372ded655e0d55318089fbce3122a446e69bcd120c79aaadfe9b017299c 20251202/cpython-3.13.10+20251202-x86_64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +8b00014c7c35f9ad4cb1c565f067500bacc4125c8bc30e4389ee0be9fd6ffa3d 20251202/cpython-3.13.10+20251202-x86_64-pc-windows-msvc-install_only.tar.gz +e39127fbe8d2ae7d86099f18b4da0918f9b60ce73ed491774d6dcfaa42b5c9ae 20251202/cpython-3.13.10+20251202-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +0cac1495fff920219904b1d573aaec0df54d549c226cb45f5c60cb6d2c72727a 20251202/cpython-3.13.10+20251202-x86_64-unknown-linux-gnu-install_only.tar.gz +04108190972ac98e13098abd972ec3f4f8b0880f83c0bb68249ce1a6164fa041 20251202/cpython-3.13.10+20251202-x86_64-unknown-linux-musl-install_only.tar.gz +61f38e947449cf00f32f0838e813358f6bf61025d0797531e5b8b8b175c617f0 20251202/cpython-3.14.1+20251202-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +cdf1ba0789f529fa34bb5b5619c5da9757ac1067d6b8dd0ee8b78e50078fc561 20251202/cpython-3.14.1+20251202-aarch64-apple-darwin-install_only.tar.gz +ddb10b645de2b1f6f2832a80b115a9cd34a4a760249983027efe46618a8efc48 20251202/cpython-3.14.1+20251202-aarch64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +19129cf8b4d68c4e64c25bae43bca139d871267b59cf7f02b9dcf25f0bf59497 20251202/cpython-3.14.1+20251202-aarch64-pc-windows-msvc-install_only.tar.gz +1a88a1fe21eb443d280999464b1a397605a7ca950d8ab73813ca6868835439a2 20251202/cpython-3.14.1+20251202-aarch64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +5dde7dba0b8ef34c0d5cb8a721254b1e11028bfc09ff06664879c245fe8df73f 20251202/cpython-3.14.1+20251202-aarch64-unknown-linux-gnu-install_only.tar.gz +7207b736ed2569f307649ffd4b615a5346631bc244730b8702babee377cef528 20251202/cpython-3.14.1+20251202-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +d2774701d53e2ac06f8c8c8e52dfa4ff346890de9b417c9a7664195443a4c766 20251202/cpython-3.14.1+20251202-ppc64le-unknown-linux-gnu-install_only.tar.gz +d1356ccd279920edc31bf0350674d966beb9522f9503846ed7855dbb109ccc14 20251202/cpython-3.14.1+20251202-riscv64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +af840506efbcd5026d9140c0a0230e45e46bb1f339a65c10a22875930b2c0159 20251202/cpython-3.14.1+20251202-riscv64-unknown-linux-gnu-install_only.tar.gz +477758eabc06dbc7e5e5d16e97c4672478acd409f420dd2e1b84d3452c0668d1 20251202/cpython-3.14.1+20251202-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +43f8f79bf4c66689d2019f193671d1df3e5e5dbb293382036285e8ce55fc55bb 20251202/cpython-3.14.1+20251202-s390x-unknown-linux-gnu-install_only.tar.gz +c2cb2a9b44285fbc13c3c9b7eea813db6ed8d94909406b059db7afd39b32e786 20251202/cpython-3.14.1+20251202-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +f25ce050e1d370f9c05c9623b769ffa4b269a6ae17e611b435fd2b8b09972a88 20251202/cpython-3.14.1+20251202-x86_64-apple-darwin-install_only.tar.gz +8ef7048315cac6d26bdbef18512a87b1a24fffa21cec86e32f9a9425f2af9bf6 20251202/cpython-3.14.1+20251202-x86_64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +cb478a5a37eb93ce4d3c27ae64d211d6a5a42475ae53f666a8d1570e71fcf409 20251202/cpython-3.14.1+20251202-x86_64-pc-windows-msvc-install_only.tar.gz +c5d5b89aab7de683e465e36de2477a131435076badda775ef6e9ea21109c1c32 20251202/cpython-3.14.1+20251202-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +a72f313bad49846e5e9671af2be7476033a877c80831cf47f431400ccb520090 20251202/cpython-3.14.1+20251202-x86_64-unknown-linux-gnu-install_only.tar.gz +15d50b15713097c38c67b1a06a0498ad102377f9b3999e98e4eefd6bf91bd82d 20251202/cpython-3.14.1+20251202-x86_64-unknown-linux-musl-install_only.tar.gz +4213058b7fcd875596c12b58cd46a399358b0a87ecde4b349cbdd00cf87ed79a 20251209/cpython-3.13.11+20251209-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +295a9f7bc899ea1cc08baf60bbf511bdd1e4a29b2dd7e5f59b48f18bfa6bf585 20251209/cpython-3.13.11+20251209-aarch64-apple-darwin-install_only.tar.gz +6daf6d092c7294cfe68c4c7bf2698ac134235489c874b3bf796c7972b9dbba30 20251209/cpython-3.13.11+20251209-aarch64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +ba646d0c3b7dd7bdfb770d9b2ebd6cd2df02a37fda90c9c79a7cf59c7df6f165 20251209/cpython-3.13.11+20251209-aarch64-pc-windows-msvc-install_only.tar.gz +290ca3bd0007db9e551f90b08dfcb6c1b2d62c33b2fc3e9a43e77d385d94f569 20251209/cpython-3.13.11+20251209-aarch64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +ea1e678e6e82301bb32bf3917732125949b6e46d541504465972024a3f165343 20251209/cpython-3.13.11+20251209-aarch64-unknown-linux-gnu-install_only.tar.gz +09d4b50f8abb443f7e3af858c920aa61c2430b0954df465e861caa7078e55e69 20251209/cpython-3.13.11+20251209-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +7660e53aad9d35ee256913c6d98427f81f078699962035c5fa8b5c3138695109 20251209/cpython-3.13.11+20251209-ppc64le-unknown-linux-gnu-install_only.tar.gz +5406f2a7cacafbd2aac3ce2de066a0929aab55423824276c36e04cb83babc36c 20251209/cpython-3.13.11+20251209-riscv64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +763fa1548e6a432e9402916e690c74ea30f26dcd2e131893dd506f72b87c27c9 20251209/cpython-3.13.11+20251209-riscv64-unknown-linux-gnu-install_only.tar.gz +3984b67c4292892eaccdd1c094c7ec788884c4c9b3534ab6995f6be96d5ed51d 20251209/cpython-3.13.11+20251209-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +ffb6af51fbfabfc6fbc4e7379bdec70c2f51e972b1d2f45c053493b9da3a1bbe 20251209/cpython-3.13.11+20251209-s390x-unknown-linux-gnu-install_only.tar.gz +d6f489464045d6895ae68b0a04a9e16477e74fe3185a75f3a9a0af8ccd25eade 20251209/cpython-3.13.11+20251209-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +dac4a0a0a9b71f6b02a8b0886547fa22814474239bffb948e3e77185406ea136 20251209/cpython-3.13.11+20251209-x86_64-apple-darwin-install_only.tar.gz +bb9a29a7ba8f179273b79971da6aaa7be592d78c606a63f99eff3e4c12fb0fae 20251209/cpython-3.13.11+20251209-x86_64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +87822417007045a28a7eccc47fe67b8c61265b99b10dbbfa24d231a3622b1c27 20251209/cpython-3.13.11+20251209-x86_64-pc-windows-msvc-install_only.tar.gz +33f89c957d986d525529b8a980103735776f4d20cf52f55960a057c760188ac3 20251209/cpython-3.13.11+20251209-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +1ffa06d714a44aea14c0c54c30656413e5955a6c92074b4b3cb4351dcc28b63b 20251209/cpython-3.13.11+20251209-x86_64-unknown-linux-gnu-install_only.tar.gz +969fe24017380b987c4e3ce15e9edf82a4618c1e61672b2cc9b021a1c98eae78 20251209/cpython-3.13.11+20251209-x86_64-unknown-linux-musl-install_only.tar.gz +d6d17b8ef28326552cdeb2a7541c8a0cb711b378df9b93ebdb461dca065edfea 20251209/cpython-3.14.2+20251209-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +2f74bd26bd16487aca357c879d11f7b16c0521328e5148a1930ab6357bcb89fe 20251209/cpython-3.14.2+20251209-aarch64-apple-darwin-install_only.tar.gz +43aac5bb4cdba71fc6775d26f47348d573a0b1210911438be71d7d96f4b18b51 20251209/cpython-3.14.2+20251209-aarch64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +0be0d2557d73efa7f6f3f99679f05252d57fe2aad2d81cac3cad410a9b1eacbd 20251209/cpython-3.14.2+20251209-aarch64-pc-windows-msvc-install_only.tar.gz +adfcb90f3a7e1b3fbc6a99f9c8c8dce1f2e26ea72b724bbe4e9fa39e81e2b0db 20251209/cpython-3.14.2+20251209-aarch64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +869af31b2963194e8a2ecfadc36027c4c1c86a10f4960baec36dadb41b2acf02 20251209/cpython-3.14.2+20251209-aarch64-unknown-linux-gnu-install_only.tar.gz +2b1ce0c5a5f5e5add7e4f934f5bd35ac41660895a30b3098db7f7303d6952a4f 20251209/cpython-3.14.2+20251209-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +86129976403fb5d64cf576329f94148f28cf6f82834e94df81ff31e9d5f404e0 20251209/cpython-3.14.2+20251209-ppc64le-unknown-linux-gnu-install_only.tar.gz +4efb610fa07a6ee2639d14d78fc3b6ecb47431c14e1e4bda03c7f7dd60a5c1e5 20251209/cpython-3.14.2+20251209-riscv64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +318dceecf119ea903aef1fb03a552cc592ecd61c08da891b68f5755e21e13511 20251209/cpython-3.14.2+20251209-riscv64-unknown-linux-gnu-install_only.tar.gz +e62f3bb3e66dac6c459690f9e9cd8cc2f6fe1dcf8bfed452af4c3df24cd7874f 20251209/cpython-3.14.2+20251209-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +53875c849a14194344ead1d9cd1e128cadd42a4b83c35eeb212417909ef05a6a 20251209/cpython-3.14.2+20251209-s390x-unknown-linux-gnu-install_only.tar.gz +1fd76c79f7fc1753e8d2ed2f71406c0b65776c75f3e95ed99ffde8c95af2adc1 20251209/cpython-3.14.2+20251209-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +58fa3e17d13ab956fd11055fb774c98ecfddcdf3b588e5f2369bdbc14ef9d76a 20251209/cpython-3.14.2+20251209-x86_64-apple-darwin-install_only.tar.gz +9927951e3997c186d2813ca1a0f4a8f5a2f771463f7f8ad0752fd3d2be2b74e4 20251209/cpython-3.14.2+20251209-x86_64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +0d660bba9f58cb552e7e99e1f96a9c67b41618c9b8d29f9f3515fe2b5ad1966e 20251209/cpython-3.14.2+20251209-x86_64-pc-windows-msvc-install_only.tar.gz +3728872ffd74989a7b4bbf3f0c629ae8fe821cda2bd6544012c1b92b9f5d5a5b 20251209/cpython-3.14.2+20251209-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +121c3249bef497adf601df76a4d89aed6053fc5ec2f8c0ec656b86f0142e8ddd 20251209/cpython-3.14.2+20251209-x86_64-unknown-linux-gnu-install_only.tar.gz +71639cc5d1fb79840467531c5b53ca77170a58edd3f7e2d29330dd736e477469 20251209/cpython-3.14.2+20251209-x86_64-unknown-linux-musl-install_only.tar.gz +5b34488580df13df051a2e84e43cfca2ab28fdd7a61052f35988eb8b481b894a 20251209/cpython-3.15.0a2+20251209-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +5851f3744fbd39e3e323844cf4f68d7763fb25546aa5ffbb71b1b5ab69c56616 20251209/cpython-3.15.0a2+20251209-aarch64-apple-darwin-install_only.tar.gz +3d99152b4e29b947fb1cfc8d035d1d511e50aeed72886ff4a5fd0a3694bd0b51 20251209/cpython-3.15.0a2+20251209-aarch64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +39bc2fcac13aeba7d650f76badf63350a81c86167a62174cb092eab7a749f4a5 20251209/cpython-3.15.0a2+20251209-aarch64-pc-windows-msvc-install_only.tar.gz +0c2c83236f6e28c103e2660a82be94b2459ee8cfdd90f5dd82f0d503ca2aec09 20251209/cpython-3.15.0a2+20251209-aarch64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +17ba65d669be3052524e03b4d1426c072ef38df2a9065ff4525d1f4d1bc9f82c 20251209/cpython-3.15.0a2+20251209-aarch64-unknown-linux-gnu-install_only.tar.gz +216842df2377fd032f279ded7fd23d7bdbd92d4c1fa7619523bc0dbdef5bd212 20251209/cpython-3.15.0a2+20251209-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +5585bd7c5eefe28b9bf544d902cad9a2f81f33c618f2a1d3c006cbfcdec77abc 20251209/cpython-3.15.0a2+20251209-ppc64le-unknown-linux-gnu-install_only.tar.gz +2a8b56f318d2e21b01b54909554c53d81871b9bb05d23ea7808dde9acec4dc7e 20251209/cpython-3.15.0a2+20251209-riscv64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +bb7252edaffd422bd1c044a4764dfcf83a5d7159942f445abbef524e54ea79a0 20251209/cpython-3.15.0a2+20251209-riscv64-unknown-linux-gnu-install_only.tar.gz +06c4ca3983aad20723f68786e3663ab49fee1bf09326f341649205ed79d34fc6 20251209/cpython-3.15.0a2+20251209-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +03a90ffa9f92d4cf4caeefb9d15f0b39c05c1e60ade6688f32165f957db4f8f3 20251209/cpython-3.15.0a2+20251209-s390x-unknown-linux-gnu-install_only.tar.gz +4d8102b70ea9fe726ee3ae9ad9e9bc4cbe0b6ed18f7989c81aef81de578f0163 20251209/cpython-3.15.0a2+20251209-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +cee576de4919cd422dbc31eb85d3c145ee82acec84f651daaf32dc669b5149c9 20251209/cpython-3.15.0a2+20251209-x86_64-apple-darwin-install_only.tar.gz +6ff71bac78d650ce621fe6db49f06290e48bcceb61f69cccc7728584f70b6346 20251209/cpython-3.15.0a2+20251209-x86_64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +e538475ee249eacf63bfdae0e70af73e9c47360e6dd3d6825e7a35107e177de5 20251209/cpython-3.15.0a2+20251209-x86_64-pc-windows-msvc-install_only.tar.gz +70f552e213734c0e260a57603bee504dd7ed0e78a10558b591e724ea8730fef5 20251209/cpython-3.15.0a2+20251209-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +58addaabfab2de422180d32543fb3878ffc984c8a2e4005ff658a5cd83b31fc7 20251209/cpython-3.15.0a2+20251209-x86_64-unknown-linux-gnu-install_only.tar.gz +dcf844400dc2e7f5f3604e994532e4d49db45f4deefe9afdf6809ca1bc6532ee 20251209/cpython-3.15.0a2+20251209-x86_64-unknown-linux-musl-install_only.tar.gz +e7cf7bc717082bb38f5ca75988ecd8e5dbc1b0535192129371e30235d29d67b5 20260325/cpython-3.13.12+20260325-aarch64-apple-darwin-freethreaded-install_only.tar.gz +688da81bcaa6ed91792397c7d5433b13a4f02f021f940637c3972639bc516dca 20260325/cpython-3.13.12+20260325-aarch64-apple-darwin-install_only.tar.gz +54187be504ea5be2f8ed455e9377112bb04f34c9259eae263779e56b403e3e3f 20260325/cpython-3.13.12+20260325-aarch64-pc-windows-msvc-freethreaded-install_only.tar.gz +d2c8b00044cd2e4c5fc7e697e63d5e481ed44b87c2def0beb42991d59f65d930 20260325/cpython-3.13.12+20260325-aarch64-pc-windows-msvc-install_only.tar.gz +9794866e9a464f349055d791ea8f14dfa7f339ecac5aa9b1084bb2ce388fc598 20260325/cpython-3.13.12+20260325-aarch64-unknown-linux-gnu-freethreaded-install_only.tar.gz +31c6e61eed48ca4e156d0e473025a792338641109e8277a63518ded438390c96 20260325/cpython-3.13.12+20260325-aarch64-unknown-linux-gnu-install_only.tar.gz +1a8a4a97f33740a1cb9fa480321818cdc610c79c9137e511e76dc53635615494 20260325/cpython-3.13.12+20260325-ppc64le-unknown-linux-gnu-freethreaded-install_only.tar.gz +654939bc40d5f76f08eb17335bb19e9efa11eb48a0818eda2293a3f7c3570ae7 20260325/cpython-3.13.12+20260325-ppc64le-unknown-linux-gnu-install_only.tar.gz +178d20e568c25abcca9b1dbedf77e904cc3f10a79d22e31f87ddabd2d28f87dc 20260325/cpython-3.13.12+20260325-riscv64-unknown-linux-gnu-freethreaded-install_only.tar.gz +fc7e1fb553c47b831ed7fa529575145207f000f967513f7b9ea809cce006ed79 20260325/cpython-3.13.12+20260325-riscv64-unknown-linux-gnu-install_only.tar.gz +d23c93ea7502420c71e4acf02999c72ab80797d51843b1b6a315ca7bac3cb780 20260325/cpython-3.13.12+20260325-s390x-unknown-linux-gnu-freethreaded-install_only.tar.gz +7d7919358e88fcc672b061be8c2316c3a604c7074200515d7104166ed611f7f9 20260325/cpython-3.13.12+20260325-s390x-unknown-linux-gnu-install_only.tar.gz +6aff211689e30889cfe90b0b2a76b6f5a7b9e6e0bb28d6a66fd5ba35d36dc78a 20260325/cpython-3.13.12+20260325-x86_64-apple-darwin-freethreaded-install_only.tar.gz +7411e47939783708381017a90944a69641ac84d43f74fb6e2d52576c599a2717 20260325/cpython-3.13.12+20260325-x86_64-apple-darwin-install_only.tar.gz +088754e90ff22962a4ab6f7cb6bdabe5d9e7618266595df2cf7b211766e15132 20260325/cpython-3.13.12+20260325-x86_64-pc-windows-msvc-freethreaded-install_only.tar.gz +5b4093f92d9bffcb0d92aea050f3d77d5a4fc8e918b31cea000ee4b3ca751f1d 20260325/cpython-3.13.12+20260325-x86_64-pc-windows-msvc-install_only.tar.gz +6070796c894ef0a25b5a944c8c0327e155df534302e1612a5ddd57d177ddadf7 20260325/cpython-3.13.12+20260325-x86_64-unknown-linux-gnu-freethreaded-install_only.tar.gz +ebb1051ca2822b9803f46a5f10b6d51d153189ef1b1f1e142f733c0cbeaf86eb 20260325/cpython-3.13.12+20260325-x86_64-unknown-linux-gnu-install_only.tar.gz +b2e9400731c7f18069ec2804ba87a404385fe440f93b7dcb59004b9f56651202 20260325/cpython-3.13.12+20260325-x86_64-unknown-linux-musl-install_only.tar.gz +21f297bc1e0503fa077364417e2213c60951d94fd65d837ae6d9d9201ae27483 20260325/cpython-3.14.3+20260325-aarch64-apple-darwin-freethreaded-install_only.tar.gz +80c996c23aab828134821f078a8a77a6f33f3f2c14000f071718c540e20c64d4 20260325/cpython-3.14.3+20260325-aarch64-apple-darwin-install_only.tar.gz +d0e355df7362d12542108f78b3f8085b21e6824420769117c262ac86569bb2a7 20260325/cpython-3.14.3+20260325-aarch64-pc-windows-msvc-freethreaded-install_only.tar.gz +b35fe7c2fe169574f382cef125e95cbd904ddcb98fc337167356371b6d2e8c60 20260325/cpython-3.14.3+20260325-aarch64-pc-windows-msvc-install_only.tar.gz +112cf42bdf4d04f69ff4f9bf18c8ce45f494bac1645310bfdeff6f2ffb30dd9a 20260325/cpython-3.14.3+20260325-aarch64-unknown-linux-gnu-freethreaded-install_only.tar.gz +6faf5478f910741c477830f5fd842011208af0f9678faf77106c9421b325bfc1 20260325/cpython-3.14.3+20260325-aarch64-unknown-linux-gnu-install_only.tar.gz +9d7e5ba8020fd942a89a57179d9015eb0237c2d95cdbf8378639723663f11706 20260325/cpython-3.14.3+20260325-ppc64le-unknown-linux-gnu-freethreaded-install_only.tar.gz +5eafe32e12f33f98c40de920482b013170dcf97d8c7f5dc780271ccf4cded76a 20260325/cpython-3.14.3+20260325-ppc64le-unknown-linux-gnu-install_only.tar.gz +32955ad52ec7931e76f4509134a2ba5a6ba6ea0cd55e05217c1ccca3967c4a5c 20260325/cpython-3.14.3+20260325-riscv64-unknown-linux-gnu-freethreaded-install_only.tar.gz +481d3faef258964e57b7102c63de12b2bb388c7ed07cfe456f33e63b4e061202 20260325/cpython-3.14.3+20260325-riscv64-unknown-linux-gnu-install_only.tar.gz +7a1d36a1567cd747411c9c2bc7e2b5c1ac277ea7c734f74b158b94101fd5ea43 20260325/cpython-3.14.3+20260325-s390x-unknown-linux-gnu-freethreaded-install_only.tar.gz +d706eae2f4d963187b7c866603aed75d7eb3ea59590b06fb34f5fd7d0fe8e432 20260325/cpython-3.14.3+20260325-s390x-unknown-linux-gnu-install_only.tar.gz +3788781d0f9704f91ab5f7ad2a040d26b0f9b6aba0a2535db21755aebb69e620 20260325/cpython-3.14.3+20260325-x86_64-apple-darwin-freethreaded-install_only.tar.gz +847a49fea36c066f8df7a57cd8c4c02d17667e25d30b7930e8f8ba15e72d7efc 20260325/cpython-3.14.3+20260325-x86_64-apple-darwin-install_only.tar.gz +99dd7e425b3dac23e03f37787d77ee0af531e96b1c748275185342bc6642eb6b 20260325/cpython-3.14.3+20260325-x86_64-pc-windows-msvc-freethreaded-install_only.tar.gz +8b4e1329c4901ce2c0f1c20ac5d2ffa62fc13f12e26b5d1e5a1000f910f980d4 20260325/cpython-3.14.3+20260325-x86_64-pc-windows-msvc-install_only.tar.gz +20d3bcd7f175e09fa08f4cb3039e5f90fe7e4ce2476534e83f5aa21eb0d7cee9 20260325/cpython-3.14.3+20260325-x86_64-unknown-linux-gnu-freethreaded-install_only.tar.gz +18270c5a7b1a572599df5e68b497ba5254811dac43ba6f542245807d821fcb44 20260325/cpython-3.14.3+20260325-x86_64-unknown-linux-gnu-install_only.tar.gz +726a28734d2878a637b0d16ce07ce24c7d6ca1043d8e6f4a23b1b0a3478eedb9 20260325/cpython-3.14.3+20260325-x86_64-unknown-linux-musl-install_only.tar.gz +f76cc83c7db16cfc8794bf6e44d834152b57d8bab4e04e823cbc59ed23ec22f8 20260414/cpython-3.10.20+20260414-aarch64-apple-darwin-install_only.tar.gz +64932c8e8bbdf9d6b66ee85934f6f8ad1d18218b51a87ea06cefd3b84554a3e4 20260414/cpython-3.10.20+20260414-aarch64-unknown-linux-gnu-install_only.tar.gz +76b48eb26ef274045772186e63431419294c41baf6d5a372b722d4c9e711082e 20260414/cpython-3.10.20+20260414-ppc64le-unknown-linux-gnu-install_only.tar.gz +76e1ec72717d17493976fc176ec661f02412666d4f19e50908d8e4303c0511d5 20260414/cpython-3.10.20+20260414-riscv64-unknown-linux-gnu-install_only.tar.gz +2edf241199d11a3ef79a312737c1bcdb86908352c585ca14b667539080630e85 20260414/cpython-3.10.20+20260414-s390x-unknown-linux-gnu-install_only.tar.gz +95a2d794b8981723095190fa94b574ceb4272bb49d83b9e418bb90341e304d09 20260414/cpython-3.10.20+20260414-x86_64-apple-darwin-install_only.tar.gz +0d828683d30185ab9f1110ad2194ef384cef0533b8e0da7e03ce837548841788 20260414/cpython-3.10.20+20260414-x86_64-pc-windows-msvc-install_only.tar.gz +303047011b2c9f58504a930fc974d84547477cf69a3f2962f25552e2395c13af 20260414/cpython-3.10.20+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +84eb198d318f8b1b8bf59eef5d30d742e13afd97c213fa229578f8fdab0c406f 20260414/cpython-3.10.20+20260414-x86_64-unknown-linux-musl-install_only.tar.gz +a57ffd435652092d16b30e783f9826c55e9c64b0f0a72cbae0a9f39e663137fb 20260414/cpython-3.11.15+20260414-aarch64-apple-darwin-install_only.tar.gz +a882abe4876985c9dc3d433420548506fb0cc9bb9d9fe336a2d3aaf28922aa45 20260414/cpython-3.11.15+20260414-aarch64-pc-windows-msvc-install_only.tar.gz +77836944ae15b74e0b25bdc68a4703a340f2ccb684effc0f45fbd7910e1a1f39 20260414/cpython-3.11.15+20260414-aarch64-unknown-linux-gnu-install_only.tar.gz +30a2107f000dbe304820627cbe2cc257027c20f3241d96e6c7df796b69ac2062 20260414/cpython-3.11.15+20260414-ppc64le-unknown-linux-gnu-install_only.tar.gz +373b98fbf2d04099139a2f6be57593714382ed790be7e7419e358830c23ddd0f 20260414/cpython-3.11.15+20260414-riscv64-unknown-linux-gnu-install_only.tar.gz +7838efa839158c80568de35ac78d438f564f4c32272a2fe7d9e14a9b351d1a62 20260414/cpython-3.11.15+20260414-s390x-unknown-linux-gnu-install_only.tar.gz +317055d80e553764feeaef432d833dd8385c14b83465a8b3fa7c2b7819cba681 20260414/cpython-3.11.15+20260414-x86_64-apple-darwin-install_only.tar.gz +8e69ecf1d9fc194e029aafa608d483bf24ccaa8f56d456d7009f20462d62ad23 20260414/cpython-3.11.15+20260414-x86_64-pc-windows-msvc-install_only.tar.gz +8b14030dd3af9ea7f7c51b4c90feb04afd8a8f45435727e67b875270bd08f3bc 20260414/cpython-3.11.15+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +ca92d3a68a39fa330498b09714733f347bead7313ba9d9b7fbed837aa4ba7796 20260414/cpython-3.11.15+20260414-x86_64-unknown-linux-musl-install_only.tar.gz +8966b2bcd9fa03ba22c080ad15a86bc12e41a00122b16f4b3740e302261124d9 20260414/cpython-3.12.13+20260414-aarch64-apple-darwin-install_only.tar.gz +f55326c894fde76fc0faffe95d2bce60be533c88a8c44c1b88bbbc17bf6a5cd5 20260414/cpython-3.12.13+20260414-aarch64-pc-windows-msvc-install_only.tar.gz +355d981eafb9b2870af79ddc106ced7266b6f6d2101d8fbcb05620fa386642b9 20260414/cpython-3.12.13+20260414-aarch64-unknown-linux-gnu-install_only.tar.gz +4aef4cffe73c4a65ea486f14d684a9ad3f831a354174d163bb531b5baa70fc49 20260414/cpython-3.12.13+20260414-ppc64le-unknown-linux-gnu-install_only.tar.gz +c2629d69324155132343913f064be93509bd162531e08a292e50c3973ec8b5db 20260414/cpython-3.12.13+20260414-riscv64-unknown-linux-gnu-install_only.tar.gz +e5baafd64180f45165d2751b25d1bcc89254eefc7926f3ab341fc61b541d7606 20260414/cpython-3.12.13+20260414-s390x-unknown-linux-gnu-install_only.tar.gz +801b03fbe004181d55a02ebd8b4e04d74973e70d716062aebe3b3cf32e9be297 20260414/cpython-3.12.13+20260414-x86_64-apple-darwin-install_only.tar.gz +c5a9e011e284c49c48106ca177342f3e3f64e95b4c6652d4a382cc7c9bb1cc46 20260414/cpython-3.12.13+20260414-x86_64-pc-windows-msvc-install_only.tar.gz +cdcf8724d46e4857f8db5ee9f4252dc2f5da34f7940294ec6b312389dd3f41e0 20260414/cpython-3.12.13+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +d10e971238c130fdf25e577c6538a3effa5589d5fcf53665e3c711edd6a6ff2f 20260414/cpython-3.12.13+20260414-x86_64-unknown-linux-musl-install_only.tar.gz +2662b1c3f6d5ed4d02d877c07f9384acc0d18b9046d54cd2853dad3ca172784f 20260414/cpython-3.13.13+20260414-aarch64-apple-darwin-freethreaded-install_only.tar.gz +c652dad552122cd2e76968ec41c803f8222038169b11310dba0c85928265f5c1 20260414/cpython-3.13.13+20260414-aarch64-apple-darwin-install_only.tar.gz +c6c1aae3809ef585271f6f1bb3643a2c6e0c82b811b93284c6218b31f0b931d7 20260414/cpython-3.13.13+20260414-aarch64-pc-windows-msvc-freethreaded-install_only.tar.gz +586ba71c75f341e1d111399b7f719ae784dc11e8672e93e017388f28684226d0 20260414/cpython-3.13.13+20260414-aarch64-pc-windows-msvc-install_only.tar.gz +46ac7e9476b938ef19f71029a77d28ed1e201335dd0aa0237fcfed2e5ce0ee61 20260414/cpython-3.13.13+20260414-aarch64-unknown-linux-gnu-freethreaded-install_only.tar.gz +6a65f68043d7fadcd580415493d2929d1fd686013f9ae44ddbd3a81307ab256d 20260414/cpython-3.13.13+20260414-aarch64-unknown-linux-gnu-install_only.tar.gz +abe26a6cab523a5d00d75f1353cbad9c5dc04262dcb0dc4a2b47d02384e2a7d7 20260414/cpython-3.13.13+20260414-ppc64le-unknown-linux-gnu-freethreaded-install_only.tar.gz +aef73894107300264222b19e357baf5bad616b1c4bf5daa5c3b97cfee8f5ed7b 20260414/cpython-3.13.13+20260414-ppc64le-unknown-linux-gnu-install_only.tar.gz +eea71fc3625fcc2408171b17fb97e0c6286ed60ed225ca7fd6e2fc5d9cc21dce 20260414/cpython-3.13.13+20260414-riscv64-unknown-linux-gnu-freethreaded-install_only.tar.gz +f47c09f8e7f2fb0bc4afe52422705af4016c8d3ec1cf004b67bb56a86caa62cb 20260414/cpython-3.13.13+20260414-riscv64-unknown-linux-gnu-install_only.tar.gz +dd8b6161c4af3c2f5f29b3535decdcf146ce90d7a062687c9e5229b4151198b0 20260414/cpython-3.13.13+20260414-s390x-unknown-linux-gnu-freethreaded-install_only.tar.gz +4d205af9654e1f33cefd23ff798af470e565f3ac0eba18d2f98f18a2abd07166 20260414/cpython-3.13.13+20260414-s390x-unknown-linux-gnu-install_only.tar.gz +27edbaad8f0c1a8814647d24df3f87eb13c89bbc2cb90e2fc23d8fa48dd64b15 20260414/cpython-3.13.13+20260414-x86_64-apple-darwin-freethreaded-install_only.tar.gz +540337412d2c4220e99280f741dbf45c1e3da3a39edaaab20c6ba1d53e1692ef 20260414/cpython-3.13.13+20260414-x86_64-apple-darwin-install_only.tar.gz +002c07103bfbe1b889f41eb1b9fade81651a21aed35a3512e2a916c5d7903cfe 20260414/cpython-3.13.13+20260414-x86_64-pc-windows-msvc-freethreaded-install_only.tar.gz +ee0cb26453d6e025d36502d765c1639c34830355e46ab3ad31c0360bc4cd9b79 20260414/cpython-3.13.13+20260414-x86_64-pc-windows-msvc-install_only.tar.gz +a183ec7a10c38ab8c3f19968614f1e69ec697199e94525583662dfbc22b70d9a 20260414/cpython-3.13.13+20260414-x86_64-unknown-linux-gnu-freethreaded-install_only.tar.gz +e5ec3b2c5693215d153c434ac018e75511b2c4f96d2bce30468a477cb3a89d5e 20260414/cpython-3.13.13+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +24ac6bf80dd2991c8be348f777c96c6eb69b71e78d8fa28c09beb3ddca015a47 20260414/cpython-3.13.13+20260414-x86_64-unknown-linux-musl-install_only.tar.gz +a4bfd77675740a0362c137b094f3cd9995775e8e6c0a7874a095dd055fd1ea99 20260414/cpython-3.14.4+20260414-aarch64-apple-darwin-freethreaded-install_only.tar.gz +8b7865e511b17093e090449bf71eb52933c17d45ad5257ddeacaffbb2c7239df 20260414/cpython-3.14.4+20260414-aarch64-apple-darwin-install_only.tar.gz +0458cb9885c30df690cdf304a16ec335cbc7344792ef0e8a904614b24a61316d 20260414/cpython-3.14.4+20260414-aarch64-pc-windows-msvc-freethreaded-install_only.tar.gz +82613380d582d806e562d7701496c34c87753ab13c37aa0afe2039003651f389 20260414/cpython-3.14.4+20260414-aarch64-pc-windows-msvc-install_only.tar.gz +6d84fb153ccb5cb650652aadc490d99881a8d9b68cf273d44cb553e8cd087734 20260414/cpython-3.14.4+20260414-aarch64-unknown-linux-gnu-freethreaded-install_only.tar.gz +5c8db1c21023316adad827a46d917bbbd6a85ae4e39bc3a58febda712c2f963d 20260414/cpython-3.14.4+20260414-aarch64-unknown-linux-gnu-install_only.tar.gz +b5e025e340d0faa1772ef234e320401b0aa5cf6c9d16ed63a8c44be7c531bc58 20260414/cpython-3.14.4+20260414-ppc64le-unknown-linux-gnu-freethreaded-install_only.tar.gz +055977a09de092744bbb22db64144e6afef8592eaac5e2bce4cca33f2592281a 20260414/cpython-3.14.4+20260414-ppc64le-unknown-linux-gnu-install_only.tar.gz +4373553133eb4712bc10f720da29e091a23153f587fdb2c38f1fb105e70db53a 20260414/cpython-3.14.4+20260414-riscv64-unknown-linux-gnu-freethreaded-install_only.tar.gz +e959df167c502fb0bbcacc31a997e25c6b0ff6b5e496321b691955aa702d0c09 20260414/cpython-3.14.4+20260414-riscv64-unknown-linux-gnu-install_only.tar.gz +a6797ad05c7d7f74a2cea28bf012f9199f4d6c1ed6d09f7adfeb9b3c538c6258 20260414/cpython-3.14.4+20260414-s390x-unknown-linux-gnu-freethreaded-install_only.tar.gz +35f70ad05b2c4045889ee0c3d93f61b012654c1d91e10e671f0e5b4d4a6c6637 20260414/cpython-3.14.4+20260414-s390x-unknown-linux-gnu-install_only.tar.gz +1c366767d203b722efbd5b3796d16a08436e8a328afd31e551289efba9bf56d1 20260414/cpython-3.14.4+20260414-x86_64-apple-darwin-freethreaded-install_only.tar.gz +9ecb2b942e6698c04af10a63a3d73c0b2e8d8e11ce44933fbffe8651bef4577d 20260414/cpython-3.14.4+20260414-x86_64-apple-darwin-install_only.tar.gz +5ccaecdb899431f393209647182def14b36d7398bd59be4fa73dd79b48b3f290 20260414/cpython-3.14.4+20260414-x86_64-pc-windows-msvc-freethreaded-install_only.tar.gz +9647bb46d3c236e34c1c11bbb7113444d9711811f0d11c39956168807a955b1a 20260414/cpython-3.14.4+20260414-x86_64-pc-windows-msvc-install_only.tar.gz +c1a845a79da56265dc49628bc3b9e20d34f04674fd2d637ee40cbe259d2b1b95 20260414/cpython-3.14.4+20260414-x86_64-unknown-linux-gnu-freethreaded-install_only.tar.gz +e17275eaf95ceb5877aa6816e209b7733f41fee401d39c3921b88fb73fc4a4ba 20260414/cpython-3.14.4+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +12687a989a2384665577e1ef9864f33d4c074a1e69b38a8bac8d656531aefa3e 20260414/cpython-3.14.4+20260414-x86_64-unknown-linux-musl-install_only.tar.gz +5791a69a73b76b908f5bdf96da1928de8db696ab198f4ced04b77b22fe712ce0 20260414/cpython-3.15.0a8+20260414-aarch64-apple-darwin-freethreaded-install_only.tar.gz +780d46b3da0e58e15c620d9e7dfd29b54c8359c195f625858f85df9c2c7ecc32 20260414/cpython-3.15.0a8+20260414-aarch64-apple-darwin-install_only.tar.gz +95ddfe7dd52185f7e5d55524eafb48e54d1eab0b0cf013966f144a411f3ddd0f 20260414/cpython-3.15.0a8+20260414-aarch64-pc-windows-msvc-freethreaded-install_only.tar.gz +10fb470e900e65df4e37f8deaf1726397c914861ffc37b43ae3743a7eee88377 20260414/cpython-3.15.0a8+20260414-aarch64-pc-windows-msvc-install_only.tar.gz +b72908bce86036a0a1ba98ca9917ea0b99dc1e6c5d715d3d463c4f330880c09b 20260414/cpython-3.15.0a8+20260414-aarch64-unknown-linux-gnu-freethreaded-install_only.tar.gz +8f6dda4d8ff44976f1aa6a94674a09a503dc50b015297e1b62c8cdc591c90f4f 20260414/cpython-3.15.0a8+20260414-aarch64-unknown-linux-gnu-install_only.tar.gz +b3c8210674140a4c5beefa2d4afd752979222638a0fb68de672c60300b4a6642 20260414/cpython-3.15.0a8+20260414-ppc64le-unknown-linux-gnu-freethreaded-install_only.tar.gz +09f076c63fadbf675143674aa3b23229482b9a44840b8b1808a216def2a9af15 20260414/cpython-3.15.0a8+20260414-ppc64le-unknown-linux-gnu-install_only.tar.gz +1a4984207974563c6aea7dc934579d058dbac7436642081113e86011114b9fdf 20260414/cpython-3.15.0a8+20260414-riscv64-unknown-linux-gnu-freethreaded-install_only.tar.gz +9d41ce752e8b731872f0f5c9c48199e63c789d24ce3ae9e91d6c8008f36e7c51 20260414/cpython-3.15.0a8+20260414-riscv64-unknown-linux-gnu-install_only.tar.gz +f525a6244d73450e0c0a7ba125b5934894ab25ee171f7099c239d4eb7ce2f5f2 20260414/cpython-3.15.0a8+20260414-s390x-unknown-linux-gnu-freethreaded-install_only.tar.gz +1de2593c40cce2d8ea883f8c8580223bfa1478cbd9d0191ba3640aed083c2202 20260414/cpython-3.15.0a8+20260414-s390x-unknown-linux-gnu-install_only.tar.gz +3dcee23c21e4a3518947e988e115c1d824f07540f4326d93d4ea2028918e0193 20260414/cpython-3.15.0a8+20260414-x86_64-apple-darwin-freethreaded-install_only.tar.gz +a7744d34148969a2ec010da6f0a46ddeceda7c02e5cdfa2b4e1811487381491a 20260414/cpython-3.15.0a8+20260414-x86_64-apple-darwin-install_only.tar.gz +6e69670347e3a6ac1d0cd89b9506d825bd2f2690cc51ead5dec61aec6857d08d 20260414/cpython-3.15.0a8+20260414-x86_64-pc-windows-msvc-freethreaded-install_only.tar.gz +3ded476f676fdf260d56a5e49aa083d5ffd218fc3390e4480ed42bee1acfb3fb 20260414/cpython-3.15.0a8+20260414-x86_64-pc-windows-msvc-install_only.tar.gz +2d06d97e230b7f74de0fe4f661918a0ee827b08127b9372e0890e167de52a8c6 20260414/cpython-3.15.0a8+20260414-x86_64-unknown-linux-gnu-freethreaded-install_only.tar.gz +c93f4b15287ac48d7e3a475b245cb59cc51079382747e3e6213d6406c158969d 20260414/cpython-3.15.0a8+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +0568e953f837f09689eb4dd1af0043ba5e2ebae0c6395b8b9f8344a53b1f1da5 20260414/cpython-3.15.0a8+20260414-x86_64-unknown-linux-musl-freethreaded-install_only.tar.gz +9fbd6f243a424d4ae973e72aa0075122a7cfe05ac8f6cfde986e7b00d0dbc0bf 20260414/cpython-3.15.0a8+20260414-x86_64-unknown-linux-musl-install_only.tar.gz diff --git a/python/private/runtimes_manifest_workspace.bzl b/python/private/runtimes_manifest_workspace.bzl new file mode 100644 index 0000000000..c2da00ac46 --- /dev/null +++ b/python/private/runtimes_manifest_workspace.bzl @@ -0,0 +1,633 @@ +"""Manifest of runtimes for workspace mode builds. + +This is the workspace equivalent of runtimes_manifest.txt. It's a bzl file +to simplify loading of the data under workspace mode, which doesn't +support parsing a runtimes_manifest.txt file. + +NOTE: This file is automatically generated by sync_runtimes_manifest_workspace.py. +Do not edit directly! +""" + +MANIFEST_TEXT = """ +# Manifest of runtimes to make available +# Originally generated circa 2026-06 from the runtimes in TOOL_VERSIONS from the original python/versions.bzl +# To sort this file, execute: ./python/private/tools/sort_manifest.py python/private/runtimes_manifest.txt + +1409acd9a506e2d1d3b65c1488db4e40d8f19d09a7df099667c87a506f71c0ef 20220227/cpython-3.10.2+20220227-aarch64-apple-darwin-install_only.tar.gz +8f351a8cc348bb45c0f95b8634c8345ec6e749e483384188ad865b7428342703 20220227/cpython-3.10.2+20220227-aarch64-unknown-linux-gnu-install_only.tar.gz +8146ad4390710ec69b316a5649912df0247d35f4a42e2aa9615bffd87b3e235a 20220227/cpython-3.10.2+20220227-x86_64-apple-darwin-install_only.tar.gz +a1d9a594cd3103baa24937ad9150c1a389544b4350e859200b3e5c036ac352bd 20220227/cpython-3.10.2+20220227-x86_64-pc-windows-msvc-shared-install_only.tar.gz +9b64eca2a94f7aff9409ad70bdaa7fbbf8148692662e764401883957943620dd 20220227/cpython-3.10.2+20220227-x86_64-unknown-linux-gnu-install_only.tar.gz +2c99983d1e83e4b6e7411ed9334019f193fba626344a50c36fba6c25d4de78a2 20220502/cpython-3.10.4+20220502-aarch64-apple-darwin-install_only.tar.gz +d8098c0c54546637e7516f93b13403b11f9db285def8d7abd825c31407a13d7e 20220502/cpython-3.10.4+20220502-aarch64-unknown-linux-gnu-install_only.tar.gz +f2711eaffff3477826a401d09a013c6802f11c04c63ab3686aa72664f1216a05 20220502/cpython-3.10.4+20220502-x86_64-apple-darwin-install_only.tar.gz +bee24a3a5c83325215521d261d73a5207ab7060ef3481f76f69b4366744eb81d 20220502/cpython-3.10.4+20220502-x86_64-pc-windows-msvc-shared-install_only.tar.gz +f6f871e53a7b1469c13f9bd7920ad98c4589e549acad8e5a1e14760fff3dd5c9 20220502/cpython-3.10.4+20220502-x86_64-unknown-linux-gnu-install_only.tar.gz +efaf66acdb9a4eb33d57702607d2e667b1a319d58c167a43c96896b97419b8b7 20220802/cpython-3.10.6+20220802-aarch64-apple-darwin-install_only.tar.gz +81625f5c97f61e2e3d7e9f62c484b1aa5311f21bd6545451714b949a29da5435 20220802/cpython-3.10.6+20220802-aarch64-unknown-linux-gnu-install_only.tar.gz +7718411adf3ea1480f3f018a643eb0550282aefe39e5ecb3f363a4a566a9398c 20220802/cpython-3.10.6+20220802-x86_64-apple-darwin-install_only.tar.gz +91889a7dbdceea585ff4d3b7856a6bb8f8a4eca83a0ff52a73542c2e67220eaa 20220802/cpython-3.10.6+20220802-x86_64-pc-windows-msvc-shared-install_only.tar.gz +55aa2190d28dcfdf414d96dc5dcea9fe048fadcd583dc3981fec020869826111 20220802/cpython-3.10.6+20220802-x86_64-unknown-linux-gnu-install_only.tar.gz +d52b03817bd245d28e0a8b2f715716cd0fcd112820ccff745636932c76afa20a 20221106/cpython-3.10.8+20221106-aarch64-apple-darwin-install_only.tar.gz +33170bef18c811906b738be530f934640491b065bf16c4d276c6515321918132 20221106/cpython-3.10.8+20221106-aarch64-unknown-linux-gnu-install_only.tar.gz +525b79c7ce5de90ab66bd07b0ac1008bafa147ddc8a41bef15ffb7c9c1e9e7c5 20221106/cpython-3.10.8+20221106-x86_64-apple-darwin-install_only.tar.gz +f2b6d2f77118f06dd2ca04dae1175e44aaa5077a5ed8ddc63333c15347182bfe 20221106/cpython-3.10.8+20221106-x86_64-pc-windows-msvc-shared-install_only.tar.gz +6c8db44ae0e18e320320bbaaafd2d69cde8bfea171ae2d651b7993d1396260b7 20221106/cpython-3.10.8+20221106-x86_64-unknown-linux-gnu-install_only.tar.gz +018d05a779b2de7a476f3b3ff2d10f503d69d14efcedd0774e6dab8c22ef84ff 20230116/cpython-3.10.9+20230116-aarch64-apple-darwin-install_only.tar.gz +2003750f40cd09d4bf7a850342613992f8d9454f03b3c067989911fb37e7a4d1 20230116/cpython-3.10.9+20230116-aarch64-unknown-linux-gnu-install_only.tar.gz +0e685f98dce0e5bc8da93c7081f4e6c10219792e223e4b5886730fd73a7ba4c6 20230116/cpython-3.10.9+20230116-x86_64-apple-darwin-install_only.tar.gz +59c6970cecb357dc1d8554bd0540eb81ee7f6d16a07acf3d14ed294ece02c035 20230116/cpython-3.10.9+20230116-x86_64-pc-windows-msvc-shared-install_only.tar.gz +d196347aeb701a53fe2bb2b095abec38d27d0fa0443f8a1c2023a1bed6e18cdf 20230116/cpython-3.10.9+20230116-x86_64-unknown-linux-gnu-install_only.tar.gz +4918cdf1cab742a90f85318f88b8122aeaa2d04705803c7b6e78e81a3dd40f80 20230116/cpython-3.11.1+20230116-aarch64-apple-darwin-install_only.tar.gz +debf15783bdcb5530504f533d33fda75a7b905cec5361ae8f33da5ba6599f8b4 20230116/cpython-3.11.1+20230116-aarch64-unknown-linux-gnu-install_only.tar.gz +20a4203d069dc9b710f70b09e7da2ce6f473d6b1110f9535fb6f4c469ed54733 20230116/cpython-3.11.1+20230116-x86_64-apple-darwin-install_only.tar.gz +edc08979cb0666a597466176511529c049a6f0bba8adf70df441708f766de5bf 20230116/cpython-3.11.1+20230116-x86_64-pc-windows-msvc-shared-install_only.tar.gz +02a551fefab3750effd0e156c25446547c238688a32fabde2995c941c03a6423 20230116/cpython-3.11.1+20230116-x86_64-unknown-linux-gnu-install_only.tar.gz +8348bc3c2311f94ec63751fb71bd0108174be1c4def002773cf519ee1506f96f 20230507/cpython-3.10.11+20230507-aarch64-apple-darwin-install_only.tar.gz +c7573fdb00239f86b22ea0e8e926ca881d24fde5e5890851339911d76110bc35 20230507/cpython-3.10.11+20230507-aarch64-unknown-linux-gnu-install_only.tar.gz +73a9d4c89ed51be39dd2de4e235078281087283e9fdedef65bec02f503e906ee 20230507/cpython-3.10.11+20230507-ppc64le-unknown-linux-gnu-install_only.tar.gz +bd3fc6e4da6f4033ebf19d66704e73b0804c22641ddae10bbe347c48f82374ad 20230507/cpython-3.10.11+20230507-x86_64-apple-darwin-install_only.tar.gz +9c2d3604a06fcd422289df73015cd00e7271d90de28d2c910f0e2309a7f73a68 20230507/cpython-3.10.11+20230507-x86_64-pc-windows-msvc-shared-install_only.tar.gz +c5bcaac91bc80bfc29cf510669ecad12d506035ecb3ad85ef213416d54aecd79 20230507/cpython-3.10.11+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz +09e412506a8d63edbb6901742b54da9aa7faf120b8dbdce56c57b303fc892c86 20230507/cpython-3.11.3+20230507-aarch64-apple-darwin-install_only.tar.gz +8190accbbbbcf7620f1ff6d668e4dd090c639665d11188ce864b62554d40e5ab 20230507/cpython-3.11.3+20230507-aarch64-unknown-linux-gnu-install_only.tar.gz +767d24f3570b35fedb945f5ac66224c8983f2d556ab83c5cfaa5f3666e9c212c 20230507/cpython-3.11.3+20230507-ppc64le-unknown-linux-gnu-install_only.tar.gz +f710b8d60621308149c100d5175fec39274ed0b9c99645484fd93d1716ef4310 20230507/cpython-3.11.3+20230507-x86_64-apple-darwin-install_only.tar.gz +24741066da6f35a7ff67bee65ce82eae870d84e1181843e64a7076d1571e95af 20230507/cpython-3.11.3+20230507-x86_64-pc-windows-msvc-shared-install_only.tar.gz +da50b87d1ec42b3cb577dfd22a3655e43a53150f4f98a4bfb40757c9d7839ab5 20230507/cpython-3.11.3+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz +bc66c706ea8c5fc891635fda8f9da971a1a901d41342f6798c20ad0b2a25d1d6 20230726/cpython-3.10.12+20230726-aarch64-apple-darwin-install_only.tar.gz +fee80e221663eca5174bd794cb5047e40d3910dbeadcdf1f09d405a4c1c15fe4 20230726/cpython-3.10.12+20230726-aarch64-unknown-linux-gnu-install_only.tar.gz +bb5e8cb0d2e44241725fa9b342238245503e7849917660006b0246a9c97b1d6c 20230726/cpython-3.10.12+20230726-ppc64le-unknown-linux-gnu-install_only.tar.gz +8d33d435ae6fb93ded7fc26798cc0a1a4f546a4e527012a1e2909cc314b332df 20230726/cpython-3.10.12+20230726-s390x-unknown-linux-gnu-install_only.tar.gz +8a6e3ed973a671de468d9c691ed9cb2c3a4858c5defffcf0b08969fba9c1dd04 20230726/cpython-3.10.12+20230726-x86_64-apple-darwin-install_only.tar.gz +c1a31c353ca44de7d1b1a3b6c55a823e9c1eed0423d4f9f66e617bdb1b608685 20230726/cpython-3.10.12+20230726-x86_64-pc-windows-msvc-shared-install_only.tar.gz +a476dbca9184df9fc69fe6309cda5ebaf031d27ca9e529852437c94ec1bc43d3 20230726/cpython-3.10.12+20230726-x86_64-unknown-linux-gnu-install_only.tar.gz +cb6d2948384a857321f2aa40fa67744cd9676a330f08b6dad7070bda0b6120a4 20230726/cpython-3.11.4+20230726-aarch64-apple-darwin-install_only.tar.gz +2e84fc53f4e90e11963281c5c871f593abcb24fc796a50337fa516be99af02fb 20230726/cpython-3.11.4+20230726-aarch64-unknown-linux-gnu-install_only.tar.gz +df7b92ed9cec96b3bb658fb586be947722ecd8e420fb23cee13d2e90abcfcf25 20230726/cpython-3.11.4+20230726-ppc64le-unknown-linux-gnu-install_only.tar.gz +e477f0749161f9aa7887964f089d9460a539f6b4a8fdab5166f898210e1a87a4 20230726/cpython-3.11.4+20230726-s390x-unknown-linux-gnu-install_only.tar.gz +47e1557d93a42585972772e82661047ca5f608293158acb2778dccf120eabb00 20230726/cpython-3.11.4+20230726-x86_64-apple-darwin-install_only.tar.gz +878614c03ea38538ae2f758e36c85d2c0eb1eaaca86cd400ff8c76693ee0b3e1 20230726/cpython-3.11.4+20230726-x86_64-pc-windows-msvc-shared-install_only.tar.gz +e26247302bc8e9083a43ce9e8dd94905b40d464745b1603041f7bc9a93c65d05 20230726/cpython-3.11.4+20230726-x86_64-unknown-linux-gnu-install_only.tar.gz +dab64b3580118ad2073babd7c29fd2053b616479df5c107d31fe2af1f45e948b 20230826/cpython-3.11.5+20230826-aarch64-apple-darwin-install_only.tar.gz +bb5c5d1ea0f199fe2d3f0996fff4b48ca6ddc415a3dbd98f50bff7fce48aac80 20230826/cpython-3.11.5+20230826-aarch64-unknown-linux-gnu-install_only.tar.gz +14121b53e9c8c6d0741f911ae00102a35adbcf5c3cdf732687ef7617b7d7304d 20230826/cpython-3.11.5+20230826-ppc64le-unknown-linux-gnu-install_only.tar.gz +fe459da39874443579d6fe88c68777c6d3e331038e1fb92a0451879fb6beb16d 20230826/cpython-3.11.5+20230826-s390x-unknown-linux-gnu-install_only.tar.gz +4a4efa7378c72f1dd8ebcce1afb99b24c01b07023aa6b8fea50eaedb50bf2bfc 20230826/cpython-3.11.5+20230826-x86_64-apple-darwin-install_only.tar.gz +00f002263efc8aea896bcfaaf906b1f4dab3e5cd3db53e2b69ab9a10ba220b97 20230826/cpython-3.11.5+20230826-x86_64-pc-windows-msvc-shared-install_only.tar.gz +fbed6f7694b2faae5d7c401a856219c945397f772eea5ca50c6eb825cbc9d1e1 20230826/cpython-3.11.5+20230826-x86_64-unknown-linux-gnu-install_only.tar.gz +916c35125b5d8323a21526d7a9154ca626453f63d0878e95b9f613a95006c990 20231002/cpython-3.11.6+20231002-aarch64-apple-darwin-install_only.tar.gz +3e26a672df17708c4dc928475a5974c3fb3a34a9b45c65fb4bd1e50504cc84ec 20231002/cpython-3.11.6+20231002-aarch64-unknown-linux-gnu-install_only.tar.gz +7937035f690a624dba4d014ffd20c342e843dd46f89b0b0a1e5726b85deb8eaf 20231002/cpython-3.11.6+20231002-ppc64le-unknown-linux-gnu-install_only.tar.gz +f9f19823dba3209cedc4647b00f46ed0177242917db20fb7fb539970e384531c 20231002/cpython-3.11.6+20231002-s390x-unknown-linux-gnu-install_only.tar.gz +178cb1716c2abc25cb56ae915096c1a083e60abeba57af001996e8bc6ce1a371 20231002/cpython-3.11.6+20231002-x86_64-apple-darwin-install_only.tar.gz +3933545e6d41462dd6a47e44133ea40995bc6efeed8c2e4cbdf1a699303e95ea 20231002/cpython-3.11.6+20231002-x86_64-pc-windows-msvc-shared-install_only.tar.gz +ee37a7eae6e80148c7e3abc56e48a397c1664f044920463ad0df0fc706eacea8 20231002/cpython-3.11.6+20231002-x86_64-unknown-linux-gnu-install_only.tar.gz +4734a2be2becb813830112c780c9879ac3aff111a0b0cd590e65ec7465774d02 20231002/cpython-3.12.0+20231002-aarch64-apple-darwin-install_only.tar.gz +bccfe67cf5465a3dfb0336f053966e2613a9bc85a6588c2fcf1366ef930c4f88 20231002/cpython-3.12.0+20231002-aarch64-unknown-linux-gnu-install_only.tar.gz +b5dae075467ace32c594c7877fe6ebe0837681f814601d5d90ba4c0dfd87a1f2 20231002/cpython-3.12.0+20231002-ppc64le-unknown-linux-gnu-install_only.tar.gz +5681621349dd85d9726d1b67c84a9686ce78f72e73a6f9e4cc4119911655759e 20231002/cpython-3.12.0+20231002-s390x-unknown-linux-gnu-install_only.tar.gz +5a9e88c8aa52b609d556777b52ebde464ae4b4f77e4aac4eb693af57395c9abf 20231002/cpython-3.12.0+20231002-x86_64-apple-darwin-install_only.tar.gz +facfaa1fbc8653f95057f3c4a0f8aa833dab0e0b316e24ee8686bc761d4b4f8d 20231002/cpython-3.12.0+20231002-x86_64-pc-windows-msvc-shared-install_only.tar.gz +e51a5293f214053ddb4645b2c9f84542e2ef86870b8655704367bd4b29d39fe9 20231002/cpython-3.12.0+20231002-x86_64-unknown-linux-gnu-install_only.tar.gz +b042c966920cf8465385ca3522986b12d745151a72c060991088977ca36d3883 20240107/cpython-3.11.7+20240107-aarch64-apple-darwin-install_only.tar.gz +b102eaf865eb715aa98a8a2ef19037b6cc3ae7dfd4a632802650f29de635aa13 20240107/cpython-3.11.7+20240107-aarch64-unknown-linux-gnu-install_only.tar.gz +b44e1b74afe75c7b19143413632c4386708ae229117f8f950c2094e9681d34c7 20240107/cpython-3.11.7+20240107-ppc64le-unknown-linux-gnu-install_only.tar.gz +49520e3ff494708020f306e30b0964f079170be83e956be4504f850557378a22 20240107/cpython-3.11.7+20240107-s390x-unknown-linux-gnu-install_only.tar.gz +a0e615eef1fafdc742da0008425a9030b7ea68a4ae4e73ac557ef27b112836d4 20240107/cpython-3.11.7+20240107-x86_64-apple-darwin-install_only.tar.gz +67077e6fa918e4f4fd60ba169820b00be7c390c497bf9bc9cab2c255ea8e6f3e 20240107/cpython-3.11.7+20240107-x86_64-pc-windows-msvc-shared-install_only.tar.gz +4a51ce60007a6facf64e5495f4cf322e311ba9f39a8cd3f3e4c026eae488e140 20240107/cpython-3.11.7+20240107-x86_64-unknown-linux-gnu-install_only.tar.gz +f93f8375ca6ac0a35d58ff007043cbd3a88d9609113f1cb59cf7c8d215f064af 20240107/cpython-3.12.1+20240107-aarch64-apple-darwin-install_only.tar.gz +236533ef20e665007a111c2f36efb59c87ae195ad7dca223b6dc03fb07064f0b 20240107/cpython-3.12.1+20240107-aarch64-unknown-linux-gnu-install_only.tar.gz +78051f0d1411ee62bc2af5edfccf6e8400ac4ef82887a2affc19a7ace6a05267 20240107/cpython-3.12.1+20240107-ppc64le-unknown-linux-gnu-install_only.tar.gz +60631211c701f8d2c56e5dd7b154e68868128a019b9db1d53a264f56c0d4aee2 20240107/cpython-3.12.1+20240107-s390x-unknown-linux-gnu-install_only.tar.gz +eca96158c1568dedd9a0b3425375637a83764d1fa74446438293089a8bfac1f8 20240107/cpython-3.12.1+20240107-x86_64-apple-darwin-install_only.tar.gz +fd5a9e0f41959d0341246d3643f2b8794f638adc0cec8dd5e1b6465198eae08a 20240107/cpython-3.12.1+20240107-x86_64-pc-windows-msvc-shared-install_only.tar.gz +74e330b8212ca22fd4d9a2003b9eec14892155566738febc8e5e572f267b9472 20240107/cpython-3.12.1+20240107-x86_64-unknown-linux-gnu-install_only.tar.gz +5fdc0f6a5b5a90fd3c528e8b1da8e3aac931ea8690126c2fdb4254c84a3ff04a 20240224/cpython-3.10.13+20240224-aarch64-apple-darwin-install_only.tar.gz +a898a88705611b372297bb8fe4d23cc16b8603ce5f24494c3a8cfa65d83787f9 20240224/cpython-3.10.13+20240224-aarch64-unknown-linux-gnu-install_only.tar.gz +c23706e138a0351fc1e9def2974af7b8206bac7ecbbb98a78f5aa9e7535fee42 20240224/cpython-3.10.13+20240224-ppc64le-unknown-linux-gnu-install_only.tar.gz +09be8fb2cdfbb4a93d555f268f244dbe4d8ff1854b2658e8043aa4ec08aede3e 20240224/cpython-3.10.13+20240224-s390x-unknown-linux-gnu-install_only.tar.gz +6378dfd22f58bb553ddb02be28304d739cd730c1f95c15c74955c923a1bc3d6a 20240224/cpython-3.10.13+20240224-x86_64-apple-darwin-install_only.tar.gz +086f7fe9156b897bb401273db8359017104168ac36f60f3af4e31ac7acd6634e 20240224/cpython-3.10.13+20240224-x86_64-pc-windows-msvc-shared-install_only.tar.gz +d995d032ca702afd2fc3a689c1f84a6c64972ecd82bba76a61d525f08eb0e195 20240224/cpython-3.10.13+20240224-x86_64-unknown-linux-gnu-install_only.tar.gz +389a51139f5abe071a0d70091ca5df3e7a3dfcfcbe3e0ba6ad85fb4c5638421e 20240224/cpython-3.11.8+20240224-aarch64-apple-darwin-install_only.tar.gz +389b9005fb78dd5a6f68df5ea45ab7b30d9a4b3222af96999e94fd20d4ad0c6a 20240224/cpython-3.11.8+20240224-aarch64-unknown-linux-gnu-install_only.tar.gz +eb2b31f8e50309aae493c6a359c32b723a676f07c641f5e8fe4b6aa4dbb50946 20240224/cpython-3.11.8+20240224-ppc64le-unknown-linux-gnu-install_only.tar.gz +844f64f4c16e24965778281da61d1e0e6cd1358a581df1662da814b1eed096b9 20240224/cpython-3.11.8+20240224-s390x-unknown-linux-gnu-install_only.tar.gz +097f467b0c36706bfec13f199a2eaf924e668f70c6e2bd1f1366806962f7e86e 20240224/cpython-3.11.8+20240224-x86_64-apple-darwin-install_only.tar.gz +b618f1f047349770ee1ef11d1b05899840abd53884b820fd25c7dfe2ec1664d4 20240224/cpython-3.11.8+20240224-x86_64-pc-windows-msvc-shared-install_only.tar.gz +94e13d0e5ad417035b80580f3e893a72e094b0900d5d64e7e34ab08e95439987 20240224/cpython-3.11.8+20240224-x86_64-unknown-linux-gnu-install_only.tar.gz +01c064c00013b0175c7858b159989819ead53f4746d40580b5b0b35b6e80fba6 20240224/cpython-3.12.2+20240224-aarch64-apple-darwin-install_only.tar.gz +e52550379e7c4ac27a87de832d172658bc04150e4e27d4e858e6d8cbb96fd709 20240224/cpython-3.12.2+20240224-aarch64-unknown-linux-gnu-install_only.tar.gz +74bc02c4bbbd26245c37b29b9e12d0a9c1b7ab93477fed8b651c988b6a9a6251 20240224/cpython-3.12.2+20240224-ppc64le-unknown-linux-gnu-install_only.tar.gz +ecd6b0285e5eef94deb784b588b4b425a15a43ae671bf206556659dc141a9825 20240224/cpython-3.12.2+20240224-s390x-unknown-linux-gnu-install_only.tar.gz +a53a6670a202c96fec0b8c55ccc780ea3af5307eb89268d5b41a9775b109c094 20240224/cpython-3.12.2+20240224-x86_64-apple-darwin-install_only.tar.gz +1e5655a6ccb1a64a78460e4e3ee21036c70246800f176a6c91043a3fe3654a3b 20240224/cpython-3.12.2+20240224-x86_64-pc-windows-msvc-shared-install_only.tar.gz +57a37b57f8243caa4cdac016176189573ad7620f0b6da5941c5e40660f9468ab 20240224/cpython-3.12.2+20240224-x86_64-unknown-linux-gnu-install_only.tar.gz +ccc40e5af329ef2af81350db2a88bbd6c17b56676e82d62048c15d548401519e 20240415/cpython-3.12.3+20240415-aarch64-apple-darwin-install_only.tar.gz +ec8126de97945e629cca9aedc80a29c4ae2992c9d69f2655e27ae73906ba187d 20240415/cpython-3.12.3+20240415-aarch64-unknown-linux-gnu-install_only.tar.gz +c5dcf08b8077e617d949bda23027c49712f583120b3ed744f9b143da1d580572 20240415/cpython-3.12.3+20240415-ppc64le-unknown-linux-gnu-install_only.tar.gz +872fc321363b8cdd826fd2cb1adfd1ceb813bc1281f9d410c1c2c4e177e8df86 20240415/cpython-3.12.3+20240415-s390x-unknown-linux-gnu-install_only.tar.gz +c37a22fca8f57d4471e3708de6d13097668c5f160067f264bb2b18f524c890c8 20240415/cpython-3.12.3+20240415-x86_64-apple-darwin-install_only.tar.gz +f7cfa4ad072feb4578c8afca5ba9a54ad591d665a441dd0d63aa366edbe19279 20240415/cpython-3.12.3+20240415-x86_64-pc-windows-msvc-shared-install_only.tar.gz +a73ba777b5d55ca89edef709e6b8521e3f3d4289581f174c8699adfb608d09d6 20240415/cpython-3.12.3+20240415-x86_64-unknown-linux-gnu-install_only.tar.gz +164d89f0df2feb689981864ecc1dffb19e6aa3696c8880166de555494fe92607 20240726/cpython-3.10.14+20240726-aarch64-apple-darwin-install_only.tar.gz +39bcd46b4d70e40da177c55259be16d5c2be7a3f7f93f1e3bde47e71b4833f29 20240726/cpython-3.10.14+20240726-aarch64-unknown-linux-gnu-install_only.tar.gz +549d38b9ef59cba9ab2990025255231bfa1cb32b4bc5eac321667640fdee19d1 20240726/cpython-3.10.14+20240726-ppc64le-unknown-linux-gnu-install_only.tar.gz +de4bc878a8666c734f983db971610980870148f333bda8b0c34abfaeae88d7ec 20240726/cpython-3.10.14+20240726-s390x-unknown-linux-gnu-install_only.tar.gz +1a1455838cd1e8ed0da14a152a2d559a2fd3a6047ba7013e841db4a35a228c1d 20240726/cpython-3.10.14+20240726-x86_64-apple-darwin-install_only.tar.gz +7f68821a8b5445267eca480660364ebd06ec84632b336770c6e39de07ac0f6c3 20240726/cpython-3.10.14+20240726-x86_64-pc-windows-msvc-shared-install_only.tar.gz +32b34cd13d9d745b3db3f3b8398ab2c07de74544829915dbebd8dce39bdc405e 20240726/cpython-3.10.14+20240726-x86_64-unknown-linux-gnu-install_only.tar.gz +cbdac9462bab9671c8e84650e425d3f43b775752a930a2ef954a0d457d5c00c3 20240726/cpython-3.11.9+20240726-aarch64-apple-darwin-install_only.tar.gz +4d17cf988abe24449d649aad3ef974091ab76807904d41839907061925b4c9e3 20240726/cpython-3.11.9+20240726-aarch64-unknown-linux-gnu-install_only.tar.gz +fc4f3c9ef9bfac2ed0282126ff376e544697ad04a5408d6429d46899d7d3bf21 20240726/cpython-3.11.9+20240726-ppc64le-unknown-linux-gnu-install_only.tar.gz +e69b66e53e926460df044f44846eef3fea642f630e829719e1a4112fc370dc56 20240726/cpython-3.11.9+20240726-s390x-unknown-linux-gnu-install_only.tar.gz +dc3174666a30f4c38d04e79a80c3159b4b3aa69597c4676701c8386696811611 20240726/cpython-3.11.9+20240726-x86_64-apple-darwin-install_only.tar.gz +f694be48bdfec1dace6d69a19906b6083f4dd7c7c61f1138ba520e433e5598f8 20240726/cpython-3.11.9+20240726-x86_64-pc-windows-msvc-shared-install_only.tar.gz +f6e955dc9ddfcad74e77abe6f439dac48ebca14b101ed7c85a5bf3206ed2c53d 20240726/cpython-3.11.9+20240726-x86_64-unknown-linux-gnu-install_only.tar.gz +1801025e825c04b3907e4ef6220a13607bc0397628c9485897073110ef7fde15 20240726/cpython-3.12.4+20240726-aarch64-apple-darwin-install_only.tar.gz +a098b18b7e9fea0c66867b76c0124fce9465765017572b2e7b522154c87c78d7 20240726/cpython-3.12.4+20240726-aarch64-unknown-linux-gnu-install_only.tar.gz +04011c4c5b7fe34b0b895edf4ad8748e410686c1d69aaee11d6688d481023bcb 20240726/cpython-3.12.4+20240726-ppc64le-unknown-linux-gnu-install_only.tar.gz +8f8f3e29cf0c2facdbcfee70660939fda7667ac24fee8656d3388fc72f3acc7c 20240726/cpython-3.12.4+20240726-s390x-unknown-linux-gnu-install_only.tar.gz +4c325838c1b0ed13698506fcd515be25c73dcbe195f8522cf98f9148a97601ed 20240726/cpython-3.12.4+20240726-x86_64-apple-darwin-install_only.tar.gz +74309b0f322716409883d38c621743ea7fa0376eb00927b8ee1e1671d3aff450 20240726/cpython-3.12.4+20240726-x86_64-pc-windows-msvc-shared-install_only.tar.gz +e133dd6fc6a2d0033e2658637cc22e9c95f9d7073b80115037ee1f16417a54ac 20240726/cpython-3.12.4+20240726-x86_64-unknown-linux-gnu-install_only.tar.gz +f64776f455a44c24d50f947c813738cfb7b9ac43732c44891bc831fa7940a33c 20241016/cpython-3.10.15+20241016-aarch64-apple-darwin-install_only.tar.gz +eb58581f85fde83d1f3e8e1f8c6f5a15c7ae4fdbe3b1d1083931f9167fdd8dbc 20241016/cpython-3.10.15+20241016-aarch64-unknown-linux-gnu-install_only.tar.gz +0c45af4e7525e2db59901606db32b2896ac1e9830c6f95551402207f537c2ce4 20241016/cpython-3.10.15+20241016-ppc64le-unknown-linux-gnu-install_only.tar.gz +de205896b070e6f5259ac0f2b3379eead875ea84e6a6ef533b89886fcbb46a4c 20241016/cpython-3.10.15+20241016-s390x-unknown-linux-gnu-install_only.tar.gz +90b46dfb1abd98d45663c7a2a8c45d3047a59391d8586d71b459cec7b75f662b 20241016/cpython-3.10.15+20241016-x86_64-apple-darwin-install_only.tar.gz +e48952619796c66ec9719867b87be97edca791c2ef7fbf87d42c417c3331609e 20241016/cpython-3.10.15+20241016-x86_64-pc-windows-msvc-shared-install_only.tar.gz +3db2171e03c1a7acdc599fba583c1b92306d3788b375c9323077367af1e9d9de 20241016/cpython-3.10.15+20241016-x86_64-unknown-linux-gnu-install_only.tar.gz +ed519c47d9620eb916a6f95ec2875396e7b1a9ab993ee40b2f31b837733f318c 20241016/cpython-3.10.15+20241016-x86_64-unknown-linux-musl-install_only.tar.gz +5a69382da99c4620690643517ca1f1f53772331b347e75f536088c42a4cf6620 20241016/cpython-3.11.10+20241016-aarch64-apple-darwin-install_only.tar.gz +803e49259280af0f5466d32829cd9d65a302b0226e424b3f0b261f9daf6aee8f 20241016/cpython-3.11.10+20241016-aarch64-unknown-linux-gnu-install_only.tar.gz +92b666d103902001322f42badbd68da92adc5cebb826af9c1c906c33166e2f34 20241016/cpython-3.11.10+20241016-ppc64le-unknown-linux-gnu-install_only.tar.gz +6d584317651c1ad4a857cb32d1999707e8bb3046fcb2f156d80381814fa19fde 20241016/cpython-3.11.10+20241016-s390x-unknown-linux-gnu-install_only.tar.gz +1e23ffe5bc473e1323ab8f51464da62d77399afb423babf67f8e13c82b69c674 20241016/cpython-3.11.10+20241016-x86_64-apple-darwin-install_only.tar.gz +647b66ff4552e70aec3bf634dd470891b4a2b291e8e8715b3bdb162f577d4c55 20241016/cpython-3.11.10+20241016-x86_64-pc-windows-msvc-shared-install_only.tar.gz +8b50a442b04724a24c1eebb65a36a0c0e833d35374dbdf9c9470d8a97b164cd9 20241016/cpython-3.11.10+20241016-x86_64-unknown-linux-gnu-install_only.tar.gz +d36fc77a8dd76155a7530f6235999a693b9e7c48aa11afeb5610a091cae5aa6f 20241016/cpython-3.11.10+20241016-x86_64-unknown-linux-musl-install_only.tar.gz +4c18852bf9c1a11b56f21bcf0df1946f7e98ee43e9e4c0c5374b2b3765cf9508 20241016/cpython-3.12.7+20241016-aarch64-apple-darwin-install_only.tar.gz +bba3c6be6153f715f2941da34f3a6a69c2d0035c9c5396bc5bb68c6d2bd1065a 20241016/cpython-3.12.7+20241016-aarch64-unknown-linux-gnu-install_only.tar.gz +0a1d1d92e33a969bd2f40a80af53c97b6c0cc1060d384ceff50ff801593bf9d6 20241016/cpython-3.12.7+20241016-ppc64le-unknown-linux-gnu-install_only.tar.gz +935676a0c960b552f95e9ac2e1e385de5de4b34038ff65ffdc688838f1189c17 20241016/cpython-3.12.7+20241016-s390x-unknown-linux-gnu-install_only.tar.gz +60c5271e7edc3c2ab47440b7abf4ed50fbc693880b474f74f05768f5b657045a 20241016/cpython-3.12.7+20241016-x86_64-apple-darwin-install_only.tar.gz +f05531bff16fa77b53be0776587b97b466070e768e6d5920894de988bdcd547a 20241016/cpython-3.12.7+20241016-x86_64-pc-windows-msvc-shared-install_only.tar.gz +43576f7db1033dd57b900307f09c2e86f371152ac8a2607133afa51cbfc36064 20241016/cpython-3.12.7+20241016-x86_64-unknown-linux-gnu-install_only.tar.gz +5ed4a4078db3cbac563af66403aaa156cd6e48831d90382a1820db2b120627b5 20241016/cpython-3.12.7+20241016-x86_64-unknown-linux-musl-install_only.tar.gz +efc2e71c0e05bc5bedb7a846e05f28dd26491b1744ded35ed82f8b49ccfa684b 20241016/cpython-3.13.0+20241016-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +31397953849d275aa2506580f3fa1cb5a85b6a3d392e495f8030e8b6412f5556 20241016/cpython-3.13.0+20241016-aarch64-apple-darwin-install_only.tar.gz +59b50df9826475d24bb7eff781fa3949112b5e9c92adb29e96a09cdf1216d5bd 20241016/cpython-3.13.0+20241016-aarch64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +e8378c0162b2e0e4cc1f62b29443a3305d116d09583304dbb0149fecaff6347b 20241016/cpython-3.13.0+20241016-aarch64-unknown-linux-gnu-install_only.tar.gz +1217efa5f4ce67fcc9f7eb64165b1bd0912b2a21bc25c1a7e2cb174a21a5df7e 20241016/cpython-3.13.0+20241016-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +fc4b7f27c4e84c78f3c8e6c7f8e4023e4638d11f1b36b6b5ce457b1926cebb53 20241016/cpython-3.13.0+20241016-ppc64le-unknown-linux-gnu-install_only.tar.gz +6c3e1e4f19d2b018b65a7e3ef4cd4225c5b9adfbc490218628466e636d5c4b8c 20241016/cpython-3.13.0+20241016-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +66b19e6a07717f6cfcd3a8ca953f0a2eaa232291142f3d26a8d17c979ec0f467 20241016/cpython-3.13.0+20241016-s390x-unknown-linux-gnu-install_only.tar.gz +2e07dfea62fe2215738551a179c87dbed1cc79d1b3654f4d7559889a6d5ce4eb 20241016/cpython-3.13.0+20241016-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +cff1b7e7cd26f2d47acac1ad6590e27d29829776f77e8afa067e9419f2f6ce77 20241016/cpython-3.13.0+20241016-x86_64-apple-darwin-install_only.tar.gz +bfd89f9acf866463bc4baf01733da5e767d13f5d0112175a4f57ba91f1541310 20241016/cpython-3.13.0+20241016-x86_64-pc-windows-msvc-shared-freethreaded+pgo-full.tar.zst +b25926e8ce4164cf103bacc4f4d154894ea53e07dd3fdd5ebb16fb1a82a7b1a0 20241016/cpython-3.13.0+20241016-x86_64-pc-windows-msvc-shared-install_only.tar.gz +a73adeda301ad843cce05f31a2d3e76222b656984535a7b87696a24a098b216c 20241016/cpython-3.13.0+20241016-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +2c8cb15c6a2caadaa98af51df6fe78a8155b8471cb3dd7b9836038e0d3657fb4 20241016/cpython-3.13.0+20241016-x86_64-unknown-linux-gnu-install_only.tar.gz +2f61ee3b628a56aceea63b46c7afe2df3e22a61da706606b0c8efda57f953cf4 20241016/cpython-3.13.0+20241016-x86_64-unknown-linux-musl-install_only.tar.gz +08f05618bdcf8064a7960b25d9ba92155447c9b08e0cf2f46a981e4c6a1bb5a5 20241205/cpython-3.13.1+20241205-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +88b88b609129c12f4b3841845aca13230f61e97ba97bd0fb28ee64b0e442a34f 20241205/cpython-3.13.1+20241205-aarch64-apple-darwin-install_only.tar.gz +9f2fcb809f9ba6c7c014a8803073a88786701a98971135bce684355062e4bb35 20241205/cpython-3.13.1+20241205-aarch64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +fdfa86c2746d2ae700042c461846e6c37f70c249925b58de8cd02eb8d1423d4e 20241205/cpython-3.13.1+20241205-aarch64-unknown-linux-gnu-install_only.tar.gz +15ceea78dff78ca8ccaac8d9c54b808af30daaa126f1f561e920a6896e098634 20241205/cpython-3.13.1+20241205-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +27b20b3237c55430ca1304e687d021f88373f906249f9cd272c5ff2803d5e5c3 20241205/cpython-3.13.1+20241205-ppc64le-unknown-linux-gnu-install_only.tar.gz +ed3c6118d1d12603309c930e93421ac7a30a69045ffd43006f63ecf71d72c317 20241205/cpython-3.13.1+20241205-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +7d0187e20cb5e36c689eec27e4d3de56d8b7f1c50dc5523550fc47377801521f 20241205/cpython-3.13.1+20241205-s390x-unknown-linux-gnu-install_only.tar.gz +dc780fecd215d2cc9e573abf1e13a175fcfa8f6efd100ef888494a248a16cda8 20241205/cpython-3.13.1+20241205-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +47eef6efb8664e2d1d23a7cdaf56262d784f8ace48f3bfca1b183e95a49888d6 20241205/cpython-3.13.1+20241205-x86_64-apple-darwin-install_only.tar.gz +7537b2ab361c0eabc0eabfca9ffd9862d7f5f6576eda13b97e98aceb5eea4fd3 20241205/cpython-3.13.1+20241205-x86_64-pc-windows-msvc-shared-freethreaded+pgo-full.tar.zst +f51f0493a5f979ff0b8d8c598a8d74f2a4d86a190c2729c85e0af65c36a9cbbe 20241205/cpython-3.13.1+20241205-x86_64-pc-windows-msvc-shared-install_only.tar.gz +9ec1b81213f849d91f5ebe6a16196e85cd6ff7c05ca823ce0ab7ba5b0e9fee84 20241205/cpython-3.13.1+20241205-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +242b2727df6c1e00de6a9f0f0dcb4562e168d27f428c785b0eb41a6aeb34d69a 20241205/cpython-3.13.1+20241205-x86_64-unknown-linux-gnu-install_only.tar.gz +76b30c6373b9c0aa2ba610e07da02f384aa210ac79643da38c66d3e6171c6ef5 20241205/cpython-3.13.1+20241205-x86_64-unknown-linux-musl-install_only.tar.gz +e3c4aa607717b23903ca2650d5c3ee24f89b97543e2db2b0f463bddc7a9e92f3 20241206/cpython-3.12.8+20241206-aarch64-apple-darwin-install_only.tar.gz +ce674b55442b732973afb2932c281bb1ded4ad7e22bcf9b07071165770758c7e 20241206/cpython-3.12.8+20241206-aarch64-unknown-linux-gnu-install_only.tar.gz +b7214790b273de9ed0532420054b72ba1393d62d2fc844ec55ade193771bd90c 20241206/cpython-3.12.8+20241206-ppc64le-unknown-linux-gnu-install_only.tar.gz +73102f5dbd7d1e7e9c2f2c80aedf2893d99a7fa407f6674ec8b2f57ba07daee5 20241206/cpython-3.12.8+20241206-s390x-unknown-linux-gnu-install_only.tar.gz +3ba35c706577d755e8e52a4c161a042464577c0e695e2a605362fa469e26de10 20241206/cpython-3.12.8+20241206-x86_64-apple-darwin-install_only.tar.gz +767b4be3ddf6b99e5ade519789c1615c191d8cf99d5aff4685cc18b48931f1e6 20241206/cpython-3.12.8+20241206-x86_64-pc-windows-msvc-shared-install_only.tar.gz +b9d6ee5ddac1198e72d53112698773fc8bb597de095592eb849ca794306699ba 20241206/cpython-3.12.8+20241206-x86_64-unknown-linux-gnu-install_only.tar.gz +6f305888703691dd04cfff85284d23ea0b0146ed7c4415e472f1fb72b3f32cdf 20241206/cpython-3.12.8+20241206-x86_64-unknown-linux-musl-install_only.tar.gz +e99f8457d9c79592c036489c5cfa78df76e4762d170665e499833e045d82608f 20250317/cpython-3.10.16+20250317-aarch64-apple-darwin-install_only.tar.gz +76d0f04d2444e77200fdc70d1c57480e29cca78cb7420d713bc1c523709c198d 20250317/cpython-3.10.16+20250317-aarch64-unknown-linux-gnu-install_only.tar.gz +39c9b3486de984fe1d72d90278229c70d6b08bcf69cd55796881b2d75077b603 20250317/cpython-3.10.16+20250317-ppc64le-unknown-linux-gnu-install_only.tar.gz +ebe949ada9293581c17d9bcdaa8f645f67d95f73eac65def760a71ef9dd6600d 20250317/cpython-3.10.16+20250317-riscv64-unknown-linux-gnu-install_only.tar.gz +9b2fc0b7f1c75b48e799b6fa14f7e24f5c61f2db82e3c65d13ed25e08f7f0857 20250317/cpython-3.10.16+20250317-s390x-unknown-linux-gnu-install_only.tar.gz +e03e62dbe95afa2f56b7344ff3bd061b180a0b690ff77f9a1d7e6601935e05ca 20250317/cpython-3.10.16+20250317-x86_64-apple-darwin-install_only.tar.gz +c7e0eb0ff5b36758b7a8cacd42eb223c056b9c4d36eded9bf5b9fe0c0b9aeb08 20250317/cpython-3.10.16+20250317-x86_64-pc-windows-msvc-install_only.tar.gz +b350c7e63956ca8edb856b91316328e0fd003a840cbd63d08253af43b2c63643 20250317/cpython-3.10.16+20250317-x86_64-unknown-linux-gnu-install_only.tar.gz +6ed64923ee4fbea4c5780f1a5a66651d239191ac10bd23420db4f5e4e0bf79c4 20250317/cpython-3.10.16+20250317-x86_64-unknown-linux-musl-install_only.tar.gz +7c7fd9809da0382a601a79287b5d62d61ce0b15f5a5ee836233727a516e85381 20250317/cpython-3.12.9+20250317-aarch64-apple-darwin-install_only.tar.gz +00c6bf9acef21ac741fea24dc449d0149834d30e9113429e50a95cce4b00bb80 20250317/cpython-3.12.9+20250317-aarch64-unknown-linux-gnu-install_only.tar.gz +25d77599dfd5849f17391d92da0da99079e4e94f19a881f763f5cc62530ef7e1 20250317/cpython-3.12.9+20250317-ppc64le-unknown-linux-gnu-install_only.tar.gz +e97ab0fdf443b302c56a52b4fd08f513bf3be66aa47263f0f9df3c6e60e05f2e 20250317/cpython-3.12.9+20250317-riscv64-unknown-linux-gnu-install_only.tar.gz +7492d079ffa8425c8f6c58e43b237c37e3fb7b31e2e14635927bb4d3397ba21e 20250317/cpython-3.12.9+20250317-s390x-unknown-linux-gnu-install_only.tar.gz +1ee1b1bb9fbce5c145c4bec9a3c98d7a4fa22543e09a7c1d932bc8599283c2dc 20250317/cpython-3.12.9+20250317-x86_64-apple-darwin-install_only.tar.gz +d15361fd202dd74ae9c3eece1abdab7655f1eba90bf6255cad1d7c53d463ed4d 20250317/cpython-3.12.9+20250317-x86_64-pc-windows-msvc-install_only.tar.gz +ef382fb88cbb41a3b0801690bd716b8a1aec07a6c6471010bcc6bd14cd575226 20250317/cpython-3.12.9+20250317-x86_64-unknown-linux-gnu-install_only.tar.gz +94e3837da1adf9964aab2d6047b33f70167de3096d1f9a2d1fa9340b1bbf537d 20250317/cpython-3.12.9+20250317-x86_64-unknown-linux-musl-install_only.tar.gz +c98c9c977e6fa05c3813bd49f3553904d89d60fed27e2e36468da7afa1d6d5e2 20250317/cpython-3.13.2+20250317-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +faa44274a331eb39786362818b21b3a4e74514e8805000b20b0e55c590cecb94 20250317/cpython-3.13.2+20250317-aarch64-apple-darwin-install_only.tar.gz +b8635e59e3143fd17f19a3dfe8ccc246ee6587c87da359bd1bcab35eefbb5f19 20250317/cpython-3.13.2+20250317-aarch64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +9c67260446fee6ea706dad577a0b32936c63f449c25d66e4383d5846b2ab2e36 20250317/cpython-3.13.2+20250317-aarch64-unknown-linux-gnu-install_only.tar.gz +6ae8fa44cb2edf4ab49cff1820b53c40c10349c0f39e11b8cd76ce7f3e7e1def 20250317/cpython-3.13.2+20250317-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +345b53d2f86c9dbd7f1320657cb227ff9a42ef63ff21f129abbbc8c82a375147 20250317/cpython-3.13.2+20250317-ppc64le-unknown-linux-gnu-install_only.tar.gz +2af1b8850c52801fb6189e7a17a51e0c93d9e46ddefcca72247b76329c97d02a 20250317/cpython-3.13.2+20250317-riscv64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +172d22b2330737f3a028ea538ffe497c39a066a8d3200b22dd4d177a3332ad85 20250317/cpython-3.13.2+20250317-riscv64-unknown-linux-gnu-install_only.tar.gz +c074144cc80c2af32c420b79a9df26e8db405212619990c1fbdd308bd75afe3f 20250317/cpython-3.13.2+20250317-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +ec3b16ea8a97e3138acec72bc5ff35949950c62c8994a8ec8e213fd93f0e806b 20250317/cpython-3.13.2+20250317-s390x-unknown-linux-gnu-install_only.tar.gz +0d73e4348d8d4b5159058609d2303705190405b485dd09ad05d870d7e0f36e0f 20250317/cpython-3.13.2+20250317-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +ee4526e84b5ce5b11141c50060b385320f2773616249a741f90c96d460ce8e8f 20250317/cpython-3.13.2+20250317-x86_64-apple-darwin-install_only.tar.gz +c51b4845fda5421e044067c111192f645234081d704313f74ee77fa013a186ea 20250317/cpython-3.13.2+20250317-x86_64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +84d7b52f3558c8e35c670a4fa14080c75e3ec584adfae49fec8b51008b75b21e 20250317/cpython-3.13.2+20250317-x86_64-pc-windows-msvc-install_only.tar.gz +1aea5062614c036904b55c1cc2fb4b500b7f6f7a4cacc263f4888889d355eef8 20250317/cpython-3.13.2+20250317-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +db011f0cd29cab2291584958f4e2eb001b0e6051848d89b38a2dc23c5c54e512 20250317/cpython-3.13.2+20250317-x86_64-unknown-linux-gnu-install_only.tar.gz +00bb2d629f7eacbb5c6b44dc04af26d1f1da64cee3425b0d8eb5135a93830296 20250317/cpython-3.13.2+20250317-x86_64-unknown-linux-musl-install_only.tar.gz +278dccade56b4bbeecb9a613b77012cf5c1433a5e9b8ef99230d5e61f31d9e02 20250610/cpython-3.13.4+20250610-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +c2ce6601b2668c7bd1f799986af5ddfbff36e88795741864aba6e578cb02ed7f 20250610/cpython-3.13.4+20250610-aarch64-apple-darwin-install_only.tar.gz +b1c1bd6ab9ef95b464d92a6a911cef1a8d9f0b0f6a192f694ef18ed15d882edf 20250610/cpython-3.13.4+20250610-aarch64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +3c2596ece08ffe17e11bc1f27aeb4ce1195d2490a83d695d36ef4933d5c5ca53 20250610/cpython-3.13.4+20250610-aarch64-unknown-linux-gnu-install_only.tar.gz +ed66ae213a62b286b9b7338b816ccd2815f5248b7a28a185dc8159fe004149ae 20250610/cpython-3.13.4+20250610-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +b3cc13ee177b8db1d3e9b2eac413484e3c6a356f97d91dc59de8d3fd8cf79d6b 20250610/cpython-3.13.4+20250610-ppc64le-unknown-linux-gnu-install_only.tar.gz +913264545215236660e4178bc3e5b57a20a444a8deb5c11680c95afc960b4016 20250610/cpython-3.13.4+20250610-riscv64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +d1b989e57a9ce29f6c945eeffe0e9750c222fdd09e99d2f8d6b0d8532a523053 20250610/cpython-3.13.4+20250610-riscv64-unknown-linux-gnu-install_only.tar.gz +7556a38ab5e507c1ec22bc38f9859982bc956cab7f4de05a2faac114feb306db 20250610/cpython-3.13.4+20250610-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +d1d19fb01961ac6476712fdd6c5031f74c83666f6f11aa066207e9a158f7e3d8 20250610/cpython-3.13.4+20250610-s390x-unknown-linux-gnu-install_only.tar.gz +64ab7ac8c88002d9ba20a92f72945bfa350268e944a7922500af75d20330574d 20250610/cpython-3.13.4+20250610-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +79feb6ca68f3921d07af52d9db06cf134e6f36916941ea850ab0bc20f5ff638b 20250610/cpython-3.13.4+20250610-x86_64-apple-darwin-install_only.tar.gz +9457504547edb2e0156bf76b53c7e4941c7f61c0eff9fd5f4d816d3df51c58e3 20250610/cpython-3.13.4+20250610-x86_64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +29ac3585cc2dcfd79e3fe380c272d00e9d34351fc456e149403c86d3fea34057 20250610/cpython-3.13.4+20250610-x86_64-pc-windows-msvc-install_only.tar.gz +864df6e6819e8f8e855ce30f34410fdc5867d0616e904daeb9a40e5806e970d7 20250610/cpython-3.13.4+20250610-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +44e5477333ebca298a7a0a316985c6c3533b8645f92a83f7f73c44033832bf32 20250610/cpython-3.13.4+20250610-x86_64-unknown-linux-gnu-install_only.tar.gz +a3afbfa94b9ff4d9fc426b47eb3c8446cada535075b8d51b7bdc9d9ab9911fc2 20250610/cpython-3.13.4+20250610-x86_64-unknown-linux-musl-install_only.tar.gz +a94c02b2d597cd6b075a713fe4e9a909cc97ca6a3b2b2ce86eda21be2062d48e 20250808/cpython-3.10.18+20250808-aarch64-apple-darwin-install_only.tar.gz +ef7de3b715d519e246d98ff7856247f7f7b357068705f09c6f300b7e7b76c701 20250808/cpython-3.10.18+20250808-aarch64-unknown-linux-gnu-install_only.tar.gz +f580efed11cc54e1a221c052e8bc88bfbc12844d3ca8949da828351a1232386e 20250808/cpython-3.10.18+20250808-ppc64le-unknown-linux-gnu-install_only.tar.gz +0d7e460e30203a9225b6f417ae972f66415a1cc0e32b37ebc48d195816282669 20250808/cpython-3.10.18+20250808-riscv64-unknown-linux-gnu-install_only.tar.gz +d4ada974daadb08a0184c19232ee3b03b3137aa70609760e1a94aaf7b12989ef 20250808/cpython-3.10.18+20250808-s390x-unknown-linux-gnu-install_only.tar.gz +da96fe2ba841640215788ddb9f151f03629360e37fcb94d4f76e5095b87df0d4 20250808/cpython-3.10.18+20250808-x86_64-apple-darwin-install_only.tar.gz +a648f3c9d136985ccfe57a5507e73d9d0839f7fd09eebd7c247857f2feaecb2a 20250808/cpython-3.10.18+20250808-x86_64-pc-windows-msvc-install_only.tar.gz +0b310a73bb9e7a495dbcad5f685e508ca2e7b36ee8f29301a52285730c425789 20250808/cpython-3.10.18+20250808-x86_64-unknown-linux-gnu-install_only.tar.gz +9cecf6ea2effbe183faebcf7e1160425a4ee17a68e49f2eefe5e1c59c51fa7ee 20250808/cpython-3.10.18+20250808-x86_64-unknown-linux-musl-install_only.tar.gz +d089bfd2c7b98a0942750a195e70d3172beda76d7747097b8afd87028b6e59b6 20250808/cpython-3.11.13+20250808-aarch64-apple-darwin-install_only.tar.gz +a632857c966237e7fd38b44c47c350f6e30d8ec54dcad6c832865ad670f0f22f 20250808/cpython-3.11.13+20250808-aarch64-pc-windows-msvc-install_only.tar.gz +bc57105f8a16acd57b71d926143c7f6ecf61729b40c8b4656f1b98bebd47c710 20250808/cpython-3.11.13+20250808-aarch64-unknown-linux-gnu-install_only.tar.gz +16a0165b0744940702b8fff80b8bf973ac914f78cb6fca28d389583f675e84de 20250808/cpython-3.11.13+20250808-ppc64le-unknown-linux-gnu-install_only.tar.gz +d8e62306be8f41c46bcd62ca68f91a1467f47adff632a35ff413dc1043ed56e8 20250808/cpython-3.11.13+20250808-riscv64-unknown-linux-gnu-install_only.tar.gz +4e302a4514a73baefdd9b327062bdafeb4115a799deec91c185f6ab45a857241 20250808/cpython-3.11.13+20250808-s390x-unknown-linux-gnu-install_only.tar.gz +d946d618f8bba8308b67e460a30612a71e2ccc309f85f6628aaae24e2b816981 20250808/cpython-3.11.13+20250808-x86_64-apple-darwin-install_only.tar.gz +ed963aee33d29ad8abfbb5fe63e42f57a2638a4a11a88e11d8bb66e61f20a6e5 20250808/cpython-3.11.13+20250808-x86_64-pc-windows-msvc-install_only.tar.gz +3ad988c702cbb017fef1208d47dea4138a2e85fd0f7f01ec5e1e335e597131b9 20250808/cpython-3.11.13+20250808-x86_64-unknown-linux-gnu-install_only.tar.gz +3a5810f0696f844289aa06d5c3a1efeab66eee999c25196b7d1954192a2c2100 20250808/cpython-3.11.13+20250808-x86_64-unknown-linux-musl-install_only.tar.gz +8792c4a84c364ab975feca0c27d3157a5435b7baab325a346ae56b223893b661 20250808/cpython-3.12.11+20250808-aarch64-apple-darwin-install_only.tar.gz +00bf7d7e8bcf5d1e9c4dfca0247d8e035147777cd57ee9d4c64dedca86b0a464 20250808/cpython-3.12.11+20250808-aarch64-pc-windows-msvc-install_only.tar.gz +4d7ba5314fab02130d6538f074961ffbf61310cade9180e59026074f9a8939cb 20250808/cpython-3.12.11+20250808-aarch64-unknown-linux-gnu-install_only.tar.gz +2c862eb40a81549d9c11e6bf5a7f07c3406310b14e6a4d16dcdf1c4763ef7090 20250808/cpython-3.12.11+20250808-ppc64le-unknown-linux-gnu-install_only.tar.gz +0bb729b95fabd49c7b495f7c44a9086e3970ea57daf66365741574bd36a17e81 20250808/cpython-3.12.11+20250808-riscv64-unknown-linux-gnu-install_only.tar.gz +99e465882d217d24ac90e99fac8f32e6a644d0340ac05ee510fb5cdf53f0cfb8 20250808/cpython-3.12.11+20250808-s390x-unknown-linux-gnu-install_only.tar.gz +e0c932709dafb05f00e528a7560ef8ee559ac82b75faca60dd1245bca1c1553f 20250808/cpython-3.12.11+20250808-x86_64-apple-darwin-install_only.tar.gz +81214ef71964a40ec269a79067ca490d45298c350583bc3af0e5781451a05c3c 20250808/cpython-3.12.11+20250808-x86_64-pc-windows-msvc-install_only.tar.gz +63d78840bf209af8da8f24e335d910f88387b892ca9187be571d481c071751bb 20250808/cpython-3.12.11+20250808-x86_64-unknown-linux-gnu-install_only.tar.gz +d633d070780590aa03ac5575cd9d7b9e17682d80f14b400313c009c387cf706b 20250808/cpython-3.12.11+20250808-x86_64-unknown-linux-musl-install_only.tar.gz +f2143304012e021a603bf1807bf3e4ce163832e43ab9a9829e53cb136497f207 20250808/cpython-3.13.6+20250808-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +8a1efa6af4e80f08e2c97dda822a3d6c24d6c98e518242f802c6a43ae8401488 20250808/cpython-3.13.6+20250808-aarch64-apple-darwin-install_only.tar.gz +552cfabcc3b103f4b1c4036d2592d5f0373c9554a2c4d2b6631b04ef7e592067 20250808/cpython-3.13.6+20250808-aarch64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +8e1617bd407ec1a874499daab26ae95080d1e0267ae616d34490137a28705827 20250808/cpython-3.13.6+20250808-aarch64-pc-windows-msvc-install_only.tar.gz +d84a7d64c284be387386b9f5da273f6d05486eb6bd8f9e86e2575cb59604cb22 20250808/cpython-3.13.6+20250808-aarch64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +11fa0591ae2211c08a42ae54944260e36ddf88a1d5604ea0c49e2477be4e5388 20250808/cpython-3.13.6+20250808-aarch64-unknown-linux-gnu-install_only.tar.gz +e76fcaf1bf80a615520dbe7f85ca0bb557fad96d132d836b0ac721e7cc1e2a37 20250808/cpython-3.13.6+20250808-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +8dcf34ae1a685fe1893b52917ae04f23328edadc4acae28499d43850c2bdd26c 20250808/cpython-3.13.6+20250808-ppc64le-unknown-linux-gnu-install_only.tar.gz +24e08a39ba4fc77753e61541e52eed39cc871f4a92a80a3c5dd495056bd8eff9 20250808/cpython-3.13.6+20250808-riscv64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +f8ed75aa6cc2011a046be00b629c3c8295267f34280324feaff34c73e7afce39 20250808/cpython-3.13.6+20250808-riscv64-unknown-linux-gnu-install_only.tar.gz +1609b223fd38a4a7a4d20e7173d7d9390fe2258f7dd9a15dc9ef0fa49613735d 20250808/cpython-3.13.6+20250808-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +7707ee5d19a78bc64ef8a66751ec7f97b64ea06714c7b1b52e8b321c2923ead8 20250808/cpython-3.13.6+20250808-s390x-unknown-linux-gnu-install_only.tar.gz +4360a1278dd0a96b526d108c8fd23498a9d2028dd7791e510fd51ff5ea3f462a 20250808/cpython-3.13.6+20250808-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +27badce7201321a8363219e438a6205165e5b4884012b1046532203df2ec9379 20250808/cpython-3.13.6+20250808-x86_64-apple-darwin-install_only.tar.gz +4e727cdbe4057b16a170f887c0fa4227a825ac59bcda84ae946c77cc932af78c 20250808/cpython-3.13.6+20250808-x86_64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +af5cc733c33b9aa9f1d74c81a59351e9b27215486d8b6cdbc06d97646a58c953 20250808/cpython-3.13.6+20250808-x86_64-pc-windows-msvc-install_only.tar.gz +e48c13c59cc3c01b79f63c8bccec27d2db6e97f64213b8731e2077b6ed8ed52c 20250808/cpython-3.13.6+20250808-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +f844e8c8b6847628b472f7e97d8893a4e93acd5382a902b465776063668c4d64 20250808/cpython-3.13.6+20250808-x86_64-unknown-linux-gnu-install_only.tar.gz +70076dea0ff65b3c05aae1a97b4a556bf613cc73db30309e59134f9d318f4f7b 20250808/cpython-3.13.6+20250808-x86_64-unknown-linux-musl-install_only.tar.gz +43bda24c2fc073bc308bf631203b917a72640d59b59fdad4ba14503d84727012 20251031/cpython-3.10.19+20251031-aarch64-apple-darwin-install_only.tar.gz +f77a8a8aa77f3f943126fa9215a25309da4bf20398fc8f4b4eec54b5fc7570ef 20251031/cpython-3.10.19+20251031-aarch64-unknown-linux-gnu-install_only.tar.gz +1c55d160fc4c3b93528cd6aaa2bb4ca6018a99e5a45919d33dc761a43a69f860 20251031/cpython-3.10.19+20251031-ppc64le-unknown-linux-gnu-install_only.tar.gz +21134d35721cdad4c881f35d0957cc19df9a45d194afb38a099faded3c1cfb4d 20251031/cpython-3.10.19+20251031-riscv64-unknown-linux-gnu-install_only.tar.gz +df0db070f1eb73ab4e371eea32213ddb3500737ea5560a6f0ffd65c82af64ddc 20251031/cpython-3.10.19+20251031-s390x-unknown-linux-gnu-install_only.tar.gz +76c12e633c09c2a790f8a958a55df4495527e0718d1875310c836e757c0c7b55 20251031/cpython-3.10.19+20251031-x86_64-apple-darwin-install_only.tar.gz +cfa08a4caf2df1b43551b843c052d6a8814e2ea0c97268b021f0423646c244c3 20251031/cpython-3.10.19+20251031-x86_64-pc-windows-msvc-install_only.tar.gz +fb1caac917d7b6497bb6f5950da5f1e48d05c43a498948dd97f85760c4382d9f 20251031/cpython-3.10.19+20251031-x86_64-unknown-linux-gnu-install_only.tar.gz +ba85013ed5ac7733fc6840168cc33ed19e9959b363dc80227d54f8fd9c92c0f4 20251031/cpython-3.10.19+20251031-x86_64-unknown-linux-musl-install_only.tar.gz +6de5572b33c65af1c9b7caf00ec593fb04cffb7e14fa393a98261bb9bc464713 20251031/cpython-3.11.14+20251031-aarch64-apple-darwin-install_only.tar.gz +38d0d1466561e15965e8d2c20f5e5be649598f55c761ecab553d087fbd217337 20251031/cpython-3.11.14+20251031-aarch64-pc-windows-msvc-install_only.tar.gz +510edb027527413c4249256194cb8ad2590b52dd93f7123b4cb341aff5d05894 20251031/cpython-3.11.14+20251031-aarch64-unknown-linux-gnu-install_only.tar.gz +4e0bc6a818e0c6a9d7d3ebe1a95591fd84440520577aa837facc96a4b7a80e35 20251031/cpython-3.11.14+20251031-ppc64le-unknown-linux-gnu-install_only.tar.gz +16519e69297144f81b2421333bc9e0b6466cf3c84749b216b695cfb4c9deb32f 20251031/cpython-3.11.14+20251031-riscv64-unknown-linux-gnu-install_only.tar.gz +5f9c1b203cdf34c8bff1aef69b63bbf11309bd16ca6e429d8c3651eaa2b3d080 20251031/cpython-3.11.14+20251031-s390x-unknown-linux-gnu-install_only.tar.gz +4891cbf34e8652b7bd1054b9502395e4b7e048e2e517c040fbf6c8297cb954d6 20251031/cpython-3.11.14+20251031-x86_64-apple-darwin-install_only.tar.gz +5223b83ed9e2aa5e9e17d2ebcf767956e998876339b9cde1980a47e9d4655fb6 20251031/cpython-3.11.14+20251031-x86_64-pc-windows-msvc-install_only.tar.gz +60f0bd473d861cc45d3401d9914e47ccb9fa037f88a91879ed517a62042b8477 20251031/cpython-3.11.14+20251031-x86_64-unknown-linux-gnu-install_only.tar.gz +25e82d1e85b90a8ab724ee633a1811b1921797f5c25ee69c6595052371b91a87 20251031/cpython-3.11.14+20251031-x86_64-unknown-linux-musl-install_only.tar.gz +5e110cb821d2eb8246065d3b46faa655180c976c4e17250f7883c634a629bc63 20251031/cpython-3.12.12+20251031-aarch64-apple-darwin-install_only.tar.gz +b190fed7c2b0f6e1010f554a0d1fd191c0754c4c0718e69d9d795ae559613780 20251031/cpython-3.12.12+20251031-aarch64-pc-windows-msvc-install_only.tar.gz +81b644d166e0bfb918615af8a2363f8fcf26eccdcc60a5334b6a62c088470bac 20251031/cpython-3.12.12+20251031-aarch64-unknown-linux-gnu-install_only.tar.gz +024f5e5678c9768d45cc24d37a8e9d265aae86c4a4602352dee3d7deba367052 20251031/cpython-3.12.12+20251031-ppc64le-unknown-linux-gnu-install_only.tar.gz +b13c57fc372c131e667a99b9680f41c0b4da571cf99ed412103c2fe9ad5ed1fb 20251031/cpython-3.12.12+20251031-riscv64-unknown-linux-gnu-install_only.tar.gz +2bf05bdd56cdf5ea4fd9f2faf151ea4211be96a0d1f4230b85f5dcae620d6400 20251031/cpython-3.12.12+20251031-s390x-unknown-linux-gnu-install_only.tar.gz +687052a046d33be49dc95dd671816709067cf6176ed36c93ea61b1fe0b883b0f 20251031/cpython-3.12.12+20251031-x86_64-apple-darwin-install_only.tar.gz +cff398b3f520c442a1b085dd347126c10c1b03f01ccc0decd8c897a687e893f1 20251031/cpython-3.12.12+20251031-x86_64-pc-windows-msvc-install_only.tar.gz +80c3882f14e15cef8260ef5257d198e8f4371ca265887431d939e0d561de3253 20251031/cpython-3.12.12+20251031-x86_64-unknown-linux-gnu-install_only.tar.gz +0a461330b9b89f2ea3088dde10d7a3f96aa65897b7c5ce2404fa3b5c4b8daa14 20251031/cpython-3.12.12+20251031-x86_64-unknown-linux-musl-install_only.tar.gz +eae1272a72ccce601590a10a9ca2a58199b5fcdf022aa603a527e3e2a04de9bc 20251031/cpython-3.13.9+20251031-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +1f3568d17383426d52350c2ef7c93c1a5a043198b860cb05e5d19b35f9c25cef 20251031/cpython-3.13.9+20251031-aarch64-apple-darwin-install_only.tar.gz +743ff69935ef28834621647dab30f032dfcd80315732917531eea333210941c7 20251031/cpython-3.13.9+20251031-aarch64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +20db43873d3c4c2175d866806545e4ad4ec6bb72ca95e60082a4df6c24567e8c 20251031/cpython-3.13.9+20251031-aarch64-pc-windows-msvc-install_only.tar.gz +a6e72f9de5d9b46cf6968d6a492f2401a919f9b959f8da2d87f43484b80169ee 20251031/cpython-3.13.9+20251031-aarch64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +0a56d11b0fb1662e67f892b9d5d1717aef06f24dbb8362bc25b8f784e620d44e 20251031/cpython-3.13.9+20251031-aarch64-unknown-linux-gnu-install_only.tar.gz +0ed5c65437f875c58ba1bee2b8d261d18698d3d0347a2e66f8902fce022a2cda 20251031/cpython-3.13.9+20251031-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +99492123902bd5e9a6b1a30135061e93a2e6a11d25107a741d5a756e91054448 20251031/cpython-3.13.9+20251031-ppc64le-unknown-linux-gnu-install_only.tar.gz +584e481d9b5225ffaf02f158fb26d2818207e65fc3c6dc21a6d500277f739220 20251031/cpython-3.13.9+20251031-riscv64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +b3dce3e4ef508773521e1ee1be989fff6118f8fd1fbbd0491d7ff7dfbc98ef06 20251031/cpython-3.13.9+20251031-riscv64-unknown-linux-gnu-install_only.tar.gz +7fa7fb912ca989ceac026a332d56a2c7d6d16ab0e94d89e690de5aade26103e2 20251031/cpython-3.13.9+20251031-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +f10e34aaa856c1b8a69c2ea4a9a6723d520443d1a957bf66dc55491334ca0c1e 20251031/cpython-3.13.9+20251031-s390x-unknown-linux-gnu-install_only.tar.gz +e2bf5fa6a3ef443ade362e08b0a19bbc172f7bfe34dabe933ccaad31d53af5da 20251031/cpython-3.13.9+20251031-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +48c0f3ca5d31e90658ef99138dc21865bb62f388ab97a1ce72cac176da194ab0 20251031/cpython-3.13.9+20251031-x86_64-apple-darwin-install_only.tar.gz +318a9a1e43dd52054327de3bccc0c5b7afde7b7f2a398ccb4d38e03d28b05386 20251031/cpython-3.13.9+20251031-x86_64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +874593f641f31ea101440c70f81768c35d4d7d6df111fde63094db67465ef787 20251031/cpython-3.13.9+20251031-x86_64-pc-windows-msvc-install_only.tar.gz +dcc29b069d0588fbd4ea29c6df840c8d1207d2a3bce8cd5cd57d1b85373b6048 20251031/cpython-3.13.9+20251031-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +6f05b91ee8c7e6dd0f9c60b95bb29130e2d623961de6578b643e80ddd83f96b6 20251031/cpython-3.13.9+20251031-x86_64-unknown-linux-gnu-install_only.tar.gz +ad987197034185e628715da504a50613af213dc21ba6d5ccaeab3db2c464aa6c 20251031/cpython-3.13.9+20251031-x86_64-unknown-linux-musl-install_only.tar.gz +d9c7b430b25bd3837dbb03f945dbe6b7bc526c5940ca96f5db7cdc42f6b2b801 20251031/cpython-3.14.0+20251031-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +b4bcd3c6c24cab32ae99e1b05c89312b783b4d69431d702e5012fe1fdcad4087 20251031/cpython-3.14.0+20251031-aarch64-apple-darwin-install_only.tar.gz +40266e60f655e49cd1d5303295255909a4b593b08b88be6e6a55b2c9fe6ed13d 20251031/cpython-3.14.0+20251031-aarch64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +599a8b7e12439cd95a201dbdfe95cf363146b1ff91f379555dafd86b170caab9 20251031/cpython-3.14.0+20251031-aarch64-pc-windows-msvc-install_only.tar.gz +f383ef50d1da6ca511212e5ae601923b56636b87351fd5fc847e0ea0a19fa9b3 20251031/cpython-3.14.0+20251031-aarch64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +128a9cbfb9645d5237ec01704d9d1d2ac5f084464cc43c37a4cd96aa9c3b1ad5 20251031/cpython-3.14.0+20251031-aarch64-unknown-linux-gnu-install_only.tar.gz +cb0e4ff781b856a47f0f461ceb41c78c7eeff65effd0957857ec4702ef1e1bd3 20251031/cpython-3.14.0+20251031-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +e16ca51f018e99a609faf953bd3a3aea31f45ee84262d1a517fb3abd98f1f4af 20251031/cpython-3.14.0+20251031-ppc64le-unknown-linux-gnu-install_only.tar.gz +929223470d11a55cd75f880ac3bd4969e42407e2cdf08d4e7e38ba721cf4abec 20251031/cpython-3.14.0+20251031-riscv64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +fca340d8fb7a05cd90e216ce601b25d492ed8c1a3b6a6d77703e0f15ab3711a7 20251031/cpython-3.14.0+20251031-riscv64-unknown-linux-gnu-install_only.tar.gz +613fb1f7b249f798b52af957d181305244e936c8e5c94c84688fcdf93fe14253 20251031/cpython-3.14.0+20251031-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +c5803644970eee931bb0581b3b64511d1a8612f67bc98951a7f7ab5581a9ed04 20251031/cpython-3.14.0+20251031-s390x-unknown-linux-gnu-install_only.tar.gz +b3196f6b57bbb3dc2ee07f348f1d51117ffa376979eceafbf50c15f0f7980bf8 20251031/cpython-3.14.0+20251031-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +4e71a3ce973be377ef18637826648bb936e2f9490f64a9e4f33a49bcc431d344 20251031/cpython-3.14.0+20251031-x86_64-apple-darwin-install_only.tar.gz +b81de5fc9e783ea6dfcf1098c28a278c874999c71afbb0309f6a8b4276c769d0 20251031/cpython-3.14.0+20251031-x86_64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +39acfcb3857d83eab054a3de11756ffc16b3d49c31393b9800dd2704d1f07fdf 20251031/cpython-3.14.0+20251031-x86_64-pc-windows-msvc-install_only.tar.gz +f4acbef0fbfaf7ab31ac63986da1d93dfa1c5cb797de1dcdc1a988aa18670120 20251031/cpython-3.14.0+20251031-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +3dec1ab70758a3467ac3313bbcdabf7a9b3016db5c072c4537e3cf0a9e6290f6 20251031/cpython-3.14.0+20251031-x86_64-unknown-linux-gnu-install_only.tar.gz +d0a2a6d3b1bb00dce2105377fda8aa79675d187f8d6d7010a42f651af25018dc 20251031/cpython-3.14.0+20251031-x86_64-unknown-linux-musl-install_only.tar.gz +12f1b16be4017181ad67904caf9e59e525b9b5d62f49105017d837e27b832959 20251031/cpython-3.15.0a1+20251031-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +3acf7aa3559b746498b18929456c5cacb84bae4e09249834cbc818970d71de87 20251031/cpython-3.15.0a1+20251031-aarch64-apple-darwin-install_only.tar.gz +54ca78dae455ece6fefbd7f5f287cc55d5ce197caf51921f6d871d15069d9489 20251031/cpython-3.15.0a1+20251031-aarch64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +1508bcd7195008479ed156aad3afbb3a3793097ed530690f0304a8107f0e53e8 20251031/cpython-3.15.0a1+20251031-aarch64-pc-windows-msvc-install_only.tar.gz +981fe8dfc6e7e1d0ffefa945a18d5c4c759bbe21722acf3a5cc7e62f16aa5f3c 20251031/cpython-3.15.0a1+20251031-aarch64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +d55c2aeece827e6bec83fd18515ee281d9ea0efaa3e2d20130db8f1c7cbb71c6 20251031/cpython-3.15.0a1+20251031-aarch64-unknown-linux-gnu-install_only.tar.gz +088400dec25139f38eeecb48f090ff2ce06a96a1dd79fa8f1dfec1cd1786f5ef 20251031/cpython-3.15.0a1+20251031-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +c28beda791c499b16f06256339522f0002a3e9acba003e6b8374755d7be1def2 20251031/cpython-3.15.0a1+20251031-ppc64le-unknown-linux-gnu-install_only.tar.gz +938061a0a31a06672526885de36037ddefd8c4acdb09424691b7000a8c8f8d01 20251031/cpython-3.15.0a1+20251031-riscv64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +36619f576b8154e4b56643c5c4a85c352f152df2989c4e602cbbe9c2b7ded870 20251031/cpython-3.15.0a1+20251031-riscv64-unknown-linux-gnu-install_only.tar.gz +2003e7e40bb44b3db7bca81087bfb738fe6af40e5db61cda8e23b59bf55d409e 20251031/cpython-3.15.0a1+20251031-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +5ea47be2a3a563ddd87ff510dae26b7aa7f3855ca00c5f1056ff8114c067c4e4 20251031/cpython-3.15.0a1+20251031-s390x-unknown-linux-gnu-install_only.tar.gz +64fc29e6c7a2f02a18645d968f1b3fc1d00d12a5ef3fcbb0d077fa8c62c08904 20251031/cpython-3.15.0a1+20251031-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +0ab19d3ac25f99da438b088751e5ec2421f9f6aa4292fd2dc0f8e49eb3e16bdf 20251031/cpython-3.15.0a1+20251031-x86_64-apple-darwin-install_only.tar.gz +34abc5603e1b4131f753d29b7deac865b9277912b851cbed5a149cf3e6745d3d 20251031/cpython-3.15.0a1+20251031-x86_64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +5f5d6bec2b381cfc771c49972d2a6f7b7e7ab6a1651d8fb6ef3983f3571722b3 20251031/cpython-3.15.0a1+20251031-x86_64-pc-windows-msvc-install_only.tar.gz +0e0272186d9f5169394dbc4d4d72a3f4a5762a04c2e5ac2ab1e23aa41fc8538a 20251031/cpython-3.15.0a1+20251031-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +1f356288c2b2713619cb7a4e453d33bf8882f812af2987e21e01e7ae382fefba 20251031/cpython-3.15.0a1+20251031-x86_64-unknown-linux-gnu-install_only.tar.gz +caf5311f333eef082dd69a669ca65aceba09a08fc1e78aad602ad649106f294c 20251031/cpython-3.15.0a1+20251031-x86_64-unknown-linux-musl-install_only.tar.gz +87275619c2706affa4d1090d2ca3dad354b6d69f8b85dbfafe38785870751b9a 20251031/cpython-3.9.25+20251031-aarch64-apple-darwin-install_only.tar.gz +6112d46355857680b81849764a6cf9f38cc4cd0d1cf29d432bc12fe5aeedf9d0 20251031/cpython-3.9.25+20251031-aarch64-unknown-linux-gnu-install_only.tar.gz +828364b6f54fa45ac2dc91f8e45d5b74306372af374a9ef16eeb2ea81253ed3f 20251031/cpython-3.9.25+20251031-ppc64le-unknown-linux-gnu-install_only.tar.gz +17467e0158e5ad04453c447d6773c23b044172276441e22e23058fd3ea053e27 20251031/cpython-3.9.25+20251031-riscv64-unknown-linux-gnu-install_only.tar.gz +3e9539f83e67faa813fd06171199b2d33c89821dfa9a33bf6e27ad67f1b6932d 20251031/cpython-3.9.25+20251031-s390x-unknown-linux-gnu-install_only.tar.gz +ace63cfe27a9487c4d72e1cb518be01c1d985271da0b2158e813801f7d3e5503 20251031/cpython-3.9.25+20251031-x86_64-apple-darwin-install_only.tar.gz +4fb1b416482ce94d73cfa140317a670c596c830671d137b07c26afe8c461768a 20251031/cpython-3.9.25+20251031-x86_64-pc-windows-msvc-install_only.tar.gz +42834f61eb6df43432c3dd6ab9ca3fdf8c06d10a404ebdb53d6902e6b9570b08 20251031/cpython-3.9.25+20251031-x86_64-unknown-linux-gnu-install_only.tar.gz +76593e8c889e81e82db5fe117fe15b69466f85100ab2ec0e4035aa86242b4e93 20251031/cpython-3.9.25+20251031-x86_64-unknown-linux-musl-install_only.tar.gz +3c9fdd76447c1549a0d3bc2a70c63f1daec997ab034206ac0260a03237166dbb 20251202/cpython-3.13.10+20251202-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +37afe4e77ab62ac50f197b1cb1f3bc02c82735c6be893da0996afcde5dc41048 20251202/cpython-3.13.10+20251202-aarch64-apple-darwin-install_only.tar.gz +cdb7141327bdc244715b25752593e2c9eeb3cc2764f37dfe81cfbc92db9d6d57 20251202/cpython-3.13.10+20251202-aarch64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +9060d644bd32ac0e0af970d0b21e207e6ff416b7c4dc26ffc4f9b043fb45b463 20251202/cpython-3.13.10+20251202-aarch64-pc-windows-msvc-install_only.tar.gz +6d277221fa4b172e00b29c7158ca9661917bc8db9a0084b1a0ff5c3a0ba8b648 20251202/cpython-3.13.10+20251202-aarch64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +c68280591cda1c9515a04809fa6926020177e8e5892300206e0496ea1d10290e 20251202/cpython-3.13.10+20251202-aarch64-unknown-linux-gnu-install_only.tar.gz +d265d8d1c51e25ed70279540223589f79cf99ad00b50d28b6150c2658c973885 20251202/cpython-3.13.10+20251202-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +1507e5528bd88131dc742a2941176aceea1838bc09860c21f179285b7865133b 20251202/cpython-3.13.10+20251202-ppc64le-unknown-linux-gnu-install_only.tar.gz +ec411b4a2d167c3be0a9aeb3905e045d62c8e3c3db0caeade5d47d5f60b98dd0 20251202/cpython-3.13.10+20251202-riscv64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +70169e916860b2e5b34c37c302d699eb2b8f24f28090968881942a37aeb7ed08 20251202/cpython-3.13.10+20251202-riscv64-unknown-linux-gnu-install_only.tar.gz +4fc6443948bf5b729481ea02cc5c68e80cd0da42631f6936587a2b8fd45bc62c 20251202/cpython-3.13.10+20251202-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +c5448863b64aacae62f3a213a6e6cf94ec63f96ee4d518491cd62fd3c81d952f 20251202/cpython-3.13.10+20251202-s390x-unknown-linux-gnu-install_only.tar.gz +6ce608684df0f90350c7a1742e9685a7782d9b26ec99d1bd9d55c8cf9a405040 20251202/cpython-3.13.10+20251202-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +a02761a4f189f71c0512e88df7ca2843696d61da659e47f8a5c8a9bd2c0d16f4 20251202/cpython-3.13.10+20251202-x86_64-apple-darwin-install_only.tar.gz +6a8b0372ded655e0d55318089fbce3122a446e69bcd120c79aaadfe9b017299c 20251202/cpython-3.13.10+20251202-x86_64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +8b00014c7c35f9ad4cb1c565f067500bacc4125c8bc30e4389ee0be9fd6ffa3d 20251202/cpython-3.13.10+20251202-x86_64-pc-windows-msvc-install_only.tar.gz +e39127fbe8d2ae7d86099f18b4da0918f9b60ce73ed491774d6dcfaa42b5c9ae 20251202/cpython-3.13.10+20251202-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +0cac1495fff920219904b1d573aaec0df54d549c226cb45f5c60cb6d2c72727a 20251202/cpython-3.13.10+20251202-x86_64-unknown-linux-gnu-install_only.tar.gz +04108190972ac98e13098abd972ec3f4f8b0880f83c0bb68249ce1a6164fa041 20251202/cpython-3.13.10+20251202-x86_64-unknown-linux-musl-install_only.tar.gz +61f38e947449cf00f32f0838e813358f6bf61025d0797531e5b8b8b175c617f0 20251202/cpython-3.14.1+20251202-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +cdf1ba0789f529fa34bb5b5619c5da9757ac1067d6b8dd0ee8b78e50078fc561 20251202/cpython-3.14.1+20251202-aarch64-apple-darwin-install_only.tar.gz +ddb10b645de2b1f6f2832a80b115a9cd34a4a760249983027efe46618a8efc48 20251202/cpython-3.14.1+20251202-aarch64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +19129cf8b4d68c4e64c25bae43bca139d871267b59cf7f02b9dcf25f0bf59497 20251202/cpython-3.14.1+20251202-aarch64-pc-windows-msvc-install_only.tar.gz +1a88a1fe21eb443d280999464b1a397605a7ca950d8ab73813ca6868835439a2 20251202/cpython-3.14.1+20251202-aarch64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +5dde7dba0b8ef34c0d5cb8a721254b1e11028bfc09ff06664879c245fe8df73f 20251202/cpython-3.14.1+20251202-aarch64-unknown-linux-gnu-install_only.tar.gz +7207b736ed2569f307649ffd4b615a5346631bc244730b8702babee377cef528 20251202/cpython-3.14.1+20251202-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +d2774701d53e2ac06f8c8c8e52dfa4ff346890de9b417c9a7664195443a4c766 20251202/cpython-3.14.1+20251202-ppc64le-unknown-linux-gnu-install_only.tar.gz +d1356ccd279920edc31bf0350674d966beb9522f9503846ed7855dbb109ccc14 20251202/cpython-3.14.1+20251202-riscv64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +af840506efbcd5026d9140c0a0230e45e46bb1f339a65c10a22875930b2c0159 20251202/cpython-3.14.1+20251202-riscv64-unknown-linux-gnu-install_only.tar.gz +477758eabc06dbc7e5e5d16e97c4672478acd409f420dd2e1b84d3452c0668d1 20251202/cpython-3.14.1+20251202-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +43f8f79bf4c66689d2019f193671d1df3e5e5dbb293382036285e8ce55fc55bb 20251202/cpython-3.14.1+20251202-s390x-unknown-linux-gnu-install_only.tar.gz +c2cb2a9b44285fbc13c3c9b7eea813db6ed8d94909406b059db7afd39b32e786 20251202/cpython-3.14.1+20251202-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +f25ce050e1d370f9c05c9623b769ffa4b269a6ae17e611b435fd2b8b09972a88 20251202/cpython-3.14.1+20251202-x86_64-apple-darwin-install_only.tar.gz +8ef7048315cac6d26bdbef18512a87b1a24fffa21cec86e32f9a9425f2af9bf6 20251202/cpython-3.14.1+20251202-x86_64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +cb478a5a37eb93ce4d3c27ae64d211d6a5a42475ae53f666a8d1570e71fcf409 20251202/cpython-3.14.1+20251202-x86_64-pc-windows-msvc-install_only.tar.gz +c5d5b89aab7de683e465e36de2477a131435076badda775ef6e9ea21109c1c32 20251202/cpython-3.14.1+20251202-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +a72f313bad49846e5e9671af2be7476033a877c80831cf47f431400ccb520090 20251202/cpython-3.14.1+20251202-x86_64-unknown-linux-gnu-install_only.tar.gz +15d50b15713097c38c67b1a06a0498ad102377f9b3999e98e4eefd6bf91bd82d 20251202/cpython-3.14.1+20251202-x86_64-unknown-linux-musl-install_only.tar.gz +4213058b7fcd875596c12b58cd46a399358b0a87ecde4b349cbdd00cf87ed79a 20251209/cpython-3.13.11+20251209-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +295a9f7bc899ea1cc08baf60bbf511bdd1e4a29b2dd7e5f59b48f18bfa6bf585 20251209/cpython-3.13.11+20251209-aarch64-apple-darwin-install_only.tar.gz +6daf6d092c7294cfe68c4c7bf2698ac134235489c874b3bf796c7972b9dbba30 20251209/cpython-3.13.11+20251209-aarch64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +ba646d0c3b7dd7bdfb770d9b2ebd6cd2df02a37fda90c9c79a7cf59c7df6f165 20251209/cpython-3.13.11+20251209-aarch64-pc-windows-msvc-install_only.tar.gz +290ca3bd0007db9e551f90b08dfcb6c1b2d62c33b2fc3e9a43e77d385d94f569 20251209/cpython-3.13.11+20251209-aarch64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +ea1e678e6e82301bb32bf3917732125949b6e46d541504465972024a3f165343 20251209/cpython-3.13.11+20251209-aarch64-unknown-linux-gnu-install_only.tar.gz +09d4b50f8abb443f7e3af858c920aa61c2430b0954df465e861caa7078e55e69 20251209/cpython-3.13.11+20251209-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +7660e53aad9d35ee256913c6d98427f81f078699962035c5fa8b5c3138695109 20251209/cpython-3.13.11+20251209-ppc64le-unknown-linux-gnu-install_only.tar.gz +5406f2a7cacafbd2aac3ce2de066a0929aab55423824276c36e04cb83babc36c 20251209/cpython-3.13.11+20251209-riscv64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +763fa1548e6a432e9402916e690c74ea30f26dcd2e131893dd506f72b87c27c9 20251209/cpython-3.13.11+20251209-riscv64-unknown-linux-gnu-install_only.tar.gz +3984b67c4292892eaccdd1c094c7ec788884c4c9b3534ab6995f6be96d5ed51d 20251209/cpython-3.13.11+20251209-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +ffb6af51fbfabfc6fbc4e7379bdec70c2f51e972b1d2f45c053493b9da3a1bbe 20251209/cpython-3.13.11+20251209-s390x-unknown-linux-gnu-install_only.tar.gz +d6f489464045d6895ae68b0a04a9e16477e74fe3185a75f3a9a0af8ccd25eade 20251209/cpython-3.13.11+20251209-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +dac4a0a0a9b71f6b02a8b0886547fa22814474239bffb948e3e77185406ea136 20251209/cpython-3.13.11+20251209-x86_64-apple-darwin-install_only.tar.gz +bb9a29a7ba8f179273b79971da6aaa7be592d78c606a63f99eff3e4c12fb0fae 20251209/cpython-3.13.11+20251209-x86_64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +87822417007045a28a7eccc47fe67b8c61265b99b10dbbfa24d231a3622b1c27 20251209/cpython-3.13.11+20251209-x86_64-pc-windows-msvc-install_only.tar.gz +33f89c957d986d525529b8a980103735776f4d20cf52f55960a057c760188ac3 20251209/cpython-3.13.11+20251209-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +1ffa06d714a44aea14c0c54c30656413e5955a6c92074b4b3cb4351dcc28b63b 20251209/cpython-3.13.11+20251209-x86_64-unknown-linux-gnu-install_only.tar.gz +969fe24017380b987c4e3ce15e9edf82a4618c1e61672b2cc9b021a1c98eae78 20251209/cpython-3.13.11+20251209-x86_64-unknown-linux-musl-install_only.tar.gz +d6d17b8ef28326552cdeb2a7541c8a0cb711b378df9b93ebdb461dca065edfea 20251209/cpython-3.14.2+20251209-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +2f74bd26bd16487aca357c879d11f7b16c0521328e5148a1930ab6357bcb89fe 20251209/cpython-3.14.2+20251209-aarch64-apple-darwin-install_only.tar.gz +43aac5bb4cdba71fc6775d26f47348d573a0b1210911438be71d7d96f4b18b51 20251209/cpython-3.14.2+20251209-aarch64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +0be0d2557d73efa7f6f3f99679f05252d57fe2aad2d81cac3cad410a9b1eacbd 20251209/cpython-3.14.2+20251209-aarch64-pc-windows-msvc-install_only.tar.gz +adfcb90f3a7e1b3fbc6a99f9c8c8dce1f2e26ea72b724bbe4e9fa39e81e2b0db 20251209/cpython-3.14.2+20251209-aarch64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +869af31b2963194e8a2ecfadc36027c4c1c86a10f4960baec36dadb41b2acf02 20251209/cpython-3.14.2+20251209-aarch64-unknown-linux-gnu-install_only.tar.gz +2b1ce0c5a5f5e5add7e4f934f5bd35ac41660895a30b3098db7f7303d6952a4f 20251209/cpython-3.14.2+20251209-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +86129976403fb5d64cf576329f94148f28cf6f82834e94df81ff31e9d5f404e0 20251209/cpython-3.14.2+20251209-ppc64le-unknown-linux-gnu-install_only.tar.gz +4efb610fa07a6ee2639d14d78fc3b6ecb47431c14e1e4bda03c7f7dd60a5c1e5 20251209/cpython-3.14.2+20251209-riscv64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +318dceecf119ea903aef1fb03a552cc592ecd61c08da891b68f5755e21e13511 20251209/cpython-3.14.2+20251209-riscv64-unknown-linux-gnu-install_only.tar.gz +e62f3bb3e66dac6c459690f9e9cd8cc2f6fe1dcf8bfed452af4c3df24cd7874f 20251209/cpython-3.14.2+20251209-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +53875c849a14194344ead1d9cd1e128cadd42a4b83c35eeb212417909ef05a6a 20251209/cpython-3.14.2+20251209-s390x-unknown-linux-gnu-install_only.tar.gz +1fd76c79f7fc1753e8d2ed2f71406c0b65776c75f3e95ed99ffde8c95af2adc1 20251209/cpython-3.14.2+20251209-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +58fa3e17d13ab956fd11055fb774c98ecfddcdf3b588e5f2369bdbc14ef9d76a 20251209/cpython-3.14.2+20251209-x86_64-apple-darwin-install_only.tar.gz +9927951e3997c186d2813ca1a0f4a8f5a2f771463f7f8ad0752fd3d2be2b74e4 20251209/cpython-3.14.2+20251209-x86_64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +0d660bba9f58cb552e7e99e1f96a9c67b41618c9b8d29f9f3515fe2b5ad1966e 20251209/cpython-3.14.2+20251209-x86_64-pc-windows-msvc-install_only.tar.gz +3728872ffd74989a7b4bbf3f0c629ae8fe821cda2bd6544012c1b92b9f5d5a5b 20251209/cpython-3.14.2+20251209-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +121c3249bef497adf601df76a4d89aed6053fc5ec2f8c0ec656b86f0142e8ddd 20251209/cpython-3.14.2+20251209-x86_64-unknown-linux-gnu-install_only.tar.gz +71639cc5d1fb79840467531c5b53ca77170a58edd3f7e2d29330dd736e477469 20251209/cpython-3.14.2+20251209-x86_64-unknown-linux-musl-install_only.tar.gz +5b34488580df13df051a2e84e43cfca2ab28fdd7a61052f35988eb8b481b894a 20251209/cpython-3.15.0a2+20251209-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +5851f3744fbd39e3e323844cf4f68d7763fb25546aa5ffbb71b1b5ab69c56616 20251209/cpython-3.15.0a2+20251209-aarch64-apple-darwin-install_only.tar.gz +3d99152b4e29b947fb1cfc8d035d1d511e50aeed72886ff4a5fd0a3694bd0b51 20251209/cpython-3.15.0a2+20251209-aarch64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +39bc2fcac13aeba7d650f76badf63350a81c86167a62174cb092eab7a749f4a5 20251209/cpython-3.15.0a2+20251209-aarch64-pc-windows-msvc-install_only.tar.gz +0c2c83236f6e28c103e2660a82be94b2459ee8cfdd90f5dd82f0d503ca2aec09 20251209/cpython-3.15.0a2+20251209-aarch64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +17ba65d669be3052524e03b4d1426c072ef38df2a9065ff4525d1f4d1bc9f82c 20251209/cpython-3.15.0a2+20251209-aarch64-unknown-linux-gnu-install_only.tar.gz +216842df2377fd032f279ded7fd23d7bdbd92d4c1fa7619523bc0dbdef5bd212 20251209/cpython-3.15.0a2+20251209-ppc64le-unknown-linux-gnu-freethreaded+lto-full.tar.zst +5585bd7c5eefe28b9bf544d902cad9a2f81f33c618f2a1d3c006cbfcdec77abc 20251209/cpython-3.15.0a2+20251209-ppc64le-unknown-linux-gnu-install_only.tar.gz +2a8b56f318d2e21b01b54909554c53d81871b9bb05d23ea7808dde9acec4dc7e 20251209/cpython-3.15.0a2+20251209-riscv64-unknown-linux-gnu-freethreaded+lto-full.tar.zst +bb7252edaffd422bd1c044a4764dfcf83a5d7159942f445abbef524e54ea79a0 20251209/cpython-3.15.0a2+20251209-riscv64-unknown-linux-gnu-install_only.tar.gz +06c4ca3983aad20723f68786e3663ab49fee1bf09326f341649205ed79d34fc6 20251209/cpython-3.15.0a2+20251209-s390x-unknown-linux-gnu-freethreaded+lto-full.tar.zst +03a90ffa9f92d4cf4caeefb9d15f0b39c05c1e60ade6688f32165f957db4f8f3 20251209/cpython-3.15.0a2+20251209-s390x-unknown-linux-gnu-install_only.tar.gz +4d8102b70ea9fe726ee3ae9ad9e9bc4cbe0b6ed18f7989c81aef81de578f0163 20251209/cpython-3.15.0a2+20251209-x86_64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +cee576de4919cd422dbc31eb85d3c145ee82acec84f651daaf32dc669b5149c9 20251209/cpython-3.15.0a2+20251209-x86_64-apple-darwin-install_only.tar.gz +6ff71bac78d650ce621fe6db49f06290e48bcceb61f69cccc7728584f70b6346 20251209/cpython-3.15.0a2+20251209-x86_64-pc-windows-msvc-freethreaded+pgo-full.tar.zst +e538475ee249eacf63bfdae0e70af73e9c47360e6dd3d6825e7a35107e177de5 20251209/cpython-3.15.0a2+20251209-x86_64-pc-windows-msvc-install_only.tar.gz +70f552e213734c0e260a57603bee504dd7ed0e78a10558b591e724ea8730fef5 20251209/cpython-3.15.0a2+20251209-x86_64-unknown-linux-gnu-freethreaded+pgo+lto-full.tar.zst +58addaabfab2de422180d32543fb3878ffc984c8a2e4005ff658a5cd83b31fc7 20251209/cpython-3.15.0a2+20251209-x86_64-unknown-linux-gnu-install_only.tar.gz +dcf844400dc2e7f5f3604e994532e4d49db45f4deefe9afdf6809ca1bc6532ee 20251209/cpython-3.15.0a2+20251209-x86_64-unknown-linux-musl-install_only.tar.gz +e7cf7bc717082bb38f5ca75988ecd8e5dbc1b0535192129371e30235d29d67b5 20260325/cpython-3.13.12+20260325-aarch64-apple-darwin-freethreaded-install_only.tar.gz +688da81bcaa6ed91792397c7d5433b13a4f02f021f940637c3972639bc516dca 20260325/cpython-3.13.12+20260325-aarch64-apple-darwin-install_only.tar.gz +54187be504ea5be2f8ed455e9377112bb04f34c9259eae263779e56b403e3e3f 20260325/cpython-3.13.12+20260325-aarch64-pc-windows-msvc-freethreaded-install_only.tar.gz +d2c8b00044cd2e4c5fc7e697e63d5e481ed44b87c2def0beb42991d59f65d930 20260325/cpython-3.13.12+20260325-aarch64-pc-windows-msvc-install_only.tar.gz +9794866e9a464f349055d791ea8f14dfa7f339ecac5aa9b1084bb2ce388fc598 20260325/cpython-3.13.12+20260325-aarch64-unknown-linux-gnu-freethreaded-install_only.tar.gz +31c6e61eed48ca4e156d0e473025a792338641109e8277a63518ded438390c96 20260325/cpython-3.13.12+20260325-aarch64-unknown-linux-gnu-install_only.tar.gz +1a8a4a97f33740a1cb9fa480321818cdc610c79c9137e511e76dc53635615494 20260325/cpython-3.13.12+20260325-ppc64le-unknown-linux-gnu-freethreaded-install_only.tar.gz +654939bc40d5f76f08eb17335bb19e9efa11eb48a0818eda2293a3f7c3570ae7 20260325/cpython-3.13.12+20260325-ppc64le-unknown-linux-gnu-install_only.tar.gz +178d20e568c25abcca9b1dbedf77e904cc3f10a79d22e31f87ddabd2d28f87dc 20260325/cpython-3.13.12+20260325-riscv64-unknown-linux-gnu-freethreaded-install_only.tar.gz +fc7e1fb553c47b831ed7fa529575145207f000f967513f7b9ea809cce006ed79 20260325/cpython-3.13.12+20260325-riscv64-unknown-linux-gnu-install_only.tar.gz +d23c93ea7502420c71e4acf02999c72ab80797d51843b1b6a315ca7bac3cb780 20260325/cpython-3.13.12+20260325-s390x-unknown-linux-gnu-freethreaded-install_only.tar.gz +7d7919358e88fcc672b061be8c2316c3a604c7074200515d7104166ed611f7f9 20260325/cpython-3.13.12+20260325-s390x-unknown-linux-gnu-install_only.tar.gz +6aff211689e30889cfe90b0b2a76b6f5a7b9e6e0bb28d6a66fd5ba35d36dc78a 20260325/cpython-3.13.12+20260325-x86_64-apple-darwin-freethreaded-install_only.tar.gz +7411e47939783708381017a90944a69641ac84d43f74fb6e2d52576c599a2717 20260325/cpython-3.13.12+20260325-x86_64-apple-darwin-install_only.tar.gz +088754e90ff22962a4ab6f7cb6bdabe5d9e7618266595df2cf7b211766e15132 20260325/cpython-3.13.12+20260325-x86_64-pc-windows-msvc-freethreaded-install_only.tar.gz +5b4093f92d9bffcb0d92aea050f3d77d5a4fc8e918b31cea000ee4b3ca751f1d 20260325/cpython-3.13.12+20260325-x86_64-pc-windows-msvc-install_only.tar.gz +6070796c894ef0a25b5a944c8c0327e155df534302e1612a5ddd57d177ddadf7 20260325/cpython-3.13.12+20260325-x86_64-unknown-linux-gnu-freethreaded-install_only.tar.gz +ebb1051ca2822b9803f46a5f10b6d51d153189ef1b1f1e142f733c0cbeaf86eb 20260325/cpython-3.13.12+20260325-x86_64-unknown-linux-gnu-install_only.tar.gz +b2e9400731c7f18069ec2804ba87a404385fe440f93b7dcb59004b9f56651202 20260325/cpython-3.13.12+20260325-x86_64-unknown-linux-musl-install_only.tar.gz +21f297bc1e0503fa077364417e2213c60951d94fd65d837ae6d9d9201ae27483 20260325/cpython-3.14.3+20260325-aarch64-apple-darwin-freethreaded-install_only.tar.gz +80c996c23aab828134821f078a8a77a6f33f3f2c14000f071718c540e20c64d4 20260325/cpython-3.14.3+20260325-aarch64-apple-darwin-install_only.tar.gz +d0e355df7362d12542108f78b3f8085b21e6824420769117c262ac86569bb2a7 20260325/cpython-3.14.3+20260325-aarch64-pc-windows-msvc-freethreaded-install_only.tar.gz +b35fe7c2fe169574f382cef125e95cbd904ddcb98fc337167356371b6d2e8c60 20260325/cpython-3.14.3+20260325-aarch64-pc-windows-msvc-install_only.tar.gz +112cf42bdf4d04f69ff4f9bf18c8ce45f494bac1645310bfdeff6f2ffb30dd9a 20260325/cpython-3.14.3+20260325-aarch64-unknown-linux-gnu-freethreaded-install_only.tar.gz +6faf5478f910741c477830f5fd842011208af0f9678faf77106c9421b325bfc1 20260325/cpython-3.14.3+20260325-aarch64-unknown-linux-gnu-install_only.tar.gz +9d7e5ba8020fd942a89a57179d9015eb0237c2d95cdbf8378639723663f11706 20260325/cpython-3.14.3+20260325-ppc64le-unknown-linux-gnu-freethreaded-install_only.tar.gz +5eafe32e12f33f98c40de920482b013170dcf97d8c7f5dc780271ccf4cded76a 20260325/cpython-3.14.3+20260325-ppc64le-unknown-linux-gnu-install_only.tar.gz +32955ad52ec7931e76f4509134a2ba5a6ba6ea0cd55e05217c1ccca3967c4a5c 20260325/cpython-3.14.3+20260325-riscv64-unknown-linux-gnu-freethreaded-install_only.tar.gz +481d3faef258964e57b7102c63de12b2bb388c7ed07cfe456f33e63b4e061202 20260325/cpython-3.14.3+20260325-riscv64-unknown-linux-gnu-install_only.tar.gz +7a1d36a1567cd747411c9c2bc7e2b5c1ac277ea7c734f74b158b94101fd5ea43 20260325/cpython-3.14.3+20260325-s390x-unknown-linux-gnu-freethreaded-install_only.tar.gz +d706eae2f4d963187b7c866603aed75d7eb3ea59590b06fb34f5fd7d0fe8e432 20260325/cpython-3.14.3+20260325-s390x-unknown-linux-gnu-install_only.tar.gz +3788781d0f9704f91ab5f7ad2a040d26b0f9b6aba0a2535db21755aebb69e620 20260325/cpython-3.14.3+20260325-x86_64-apple-darwin-freethreaded-install_only.tar.gz +847a49fea36c066f8df7a57cd8c4c02d17667e25d30b7930e8f8ba15e72d7efc 20260325/cpython-3.14.3+20260325-x86_64-apple-darwin-install_only.tar.gz +99dd7e425b3dac23e03f37787d77ee0af531e96b1c748275185342bc6642eb6b 20260325/cpython-3.14.3+20260325-x86_64-pc-windows-msvc-freethreaded-install_only.tar.gz +8b4e1329c4901ce2c0f1c20ac5d2ffa62fc13f12e26b5d1e5a1000f910f980d4 20260325/cpython-3.14.3+20260325-x86_64-pc-windows-msvc-install_only.tar.gz +20d3bcd7f175e09fa08f4cb3039e5f90fe7e4ce2476534e83f5aa21eb0d7cee9 20260325/cpython-3.14.3+20260325-x86_64-unknown-linux-gnu-freethreaded-install_only.tar.gz +18270c5a7b1a572599df5e68b497ba5254811dac43ba6f542245807d821fcb44 20260325/cpython-3.14.3+20260325-x86_64-unknown-linux-gnu-install_only.tar.gz +726a28734d2878a637b0d16ce07ce24c7d6ca1043d8e6f4a23b1b0a3478eedb9 20260325/cpython-3.14.3+20260325-x86_64-unknown-linux-musl-install_only.tar.gz +f76cc83c7db16cfc8794bf6e44d834152b57d8bab4e04e823cbc59ed23ec22f8 20260414/cpython-3.10.20+20260414-aarch64-apple-darwin-install_only.tar.gz +64932c8e8bbdf9d6b66ee85934f6f8ad1d18218b51a87ea06cefd3b84554a3e4 20260414/cpython-3.10.20+20260414-aarch64-unknown-linux-gnu-install_only.tar.gz +76b48eb26ef274045772186e63431419294c41baf6d5a372b722d4c9e711082e 20260414/cpython-3.10.20+20260414-ppc64le-unknown-linux-gnu-install_only.tar.gz +76e1ec72717d17493976fc176ec661f02412666d4f19e50908d8e4303c0511d5 20260414/cpython-3.10.20+20260414-riscv64-unknown-linux-gnu-install_only.tar.gz +2edf241199d11a3ef79a312737c1bcdb86908352c585ca14b667539080630e85 20260414/cpython-3.10.20+20260414-s390x-unknown-linux-gnu-install_only.tar.gz +95a2d794b8981723095190fa94b574ceb4272bb49d83b9e418bb90341e304d09 20260414/cpython-3.10.20+20260414-x86_64-apple-darwin-install_only.tar.gz +0d828683d30185ab9f1110ad2194ef384cef0533b8e0da7e03ce837548841788 20260414/cpython-3.10.20+20260414-x86_64-pc-windows-msvc-install_only.tar.gz +303047011b2c9f58504a930fc974d84547477cf69a3f2962f25552e2395c13af 20260414/cpython-3.10.20+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +84eb198d318f8b1b8bf59eef5d30d742e13afd97c213fa229578f8fdab0c406f 20260414/cpython-3.10.20+20260414-x86_64-unknown-linux-musl-install_only.tar.gz +a57ffd435652092d16b30e783f9826c55e9c64b0f0a72cbae0a9f39e663137fb 20260414/cpython-3.11.15+20260414-aarch64-apple-darwin-install_only.tar.gz +a882abe4876985c9dc3d433420548506fb0cc9bb9d9fe336a2d3aaf28922aa45 20260414/cpython-3.11.15+20260414-aarch64-pc-windows-msvc-install_only.tar.gz +77836944ae15b74e0b25bdc68a4703a340f2ccb684effc0f45fbd7910e1a1f39 20260414/cpython-3.11.15+20260414-aarch64-unknown-linux-gnu-install_only.tar.gz +30a2107f000dbe304820627cbe2cc257027c20f3241d96e6c7df796b69ac2062 20260414/cpython-3.11.15+20260414-ppc64le-unknown-linux-gnu-install_only.tar.gz +373b98fbf2d04099139a2f6be57593714382ed790be7e7419e358830c23ddd0f 20260414/cpython-3.11.15+20260414-riscv64-unknown-linux-gnu-install_only.tar.gz +7838efa839158c80568de35ac78d438f564f4c32272a2fe7d9e14a9b351d1a62 20260414/cpython-3.11.15+20260414-s390x-unknown-linux-gnu-install_only.tar.gz +317055d80e553764feeaef432d833dd8385c14b83465a8b3fa7c2b7819cba681 20260414/cpython-3.11.15+20260414-x86_64-apple-darwin-install_only.tar.gz +8e69ecf1d9fc194e029aafa608d483bf24ccaa8f56d456d7009f20462d62ad23 20260414/cpython-3.11.15+20260414-x86_64-pc-windows-msvc-install_only.tar.gz +8b14030dd3af9ea7f7c51b4c90feb04afd8a8f45435727e67b875270bd08f3bc 20260414/cpython-3.11.15+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +ca92d3a68a39fa330498b09714733f347bead7313ba9d9b7fbed837aa4ba7796 20260414/cpython-3.11.15+20260414-x86_64-unknown-linux-musl-install_only.tar.gz +8966b2bcd9fa03ba22c080ad15a86bc12e41a00122b16f4b3740e302261124d9 20260414/cpython-3.12.13+20260414-aarch64-apple-darwin-install_only.tar.gz +f55326c894fde76fc0faffe95d2bce60be533c88a8c44c1b88bbbc17bf6a5cd5 20260414/cpython-3.12.13+20260414-aarch64-pc-windows-msvc-install_only.tar.gz +355d981eafb9b2870af79ddc106ced7266b6f6d2101d8fbcb05620fa386642b9 20260414/cpython-3.12.13+20260414-aarch64-unknown-linux-gnu-install_only.tar.gz +4aef4cffe73c4a65ea486f14d684a9ad3f831a354174d163bb531b5baa70fc49 20260414/cpython-3.12.13+20260414-ppc64le-unknown-linux-gnu-install_only.tar.gz +c2629d69324155132343913f064be93509bd162531e08a292e50c3973ec8b5db 20260414/cpython-3.12.13+20260414-riscv64-unknown-linux-gnu-install_only.tar.gz +e5baafd64180f45165d2751b25d1bcc89254eefc7926f3ab341fc61b541d7606 20260414/cpython-3.12.13+20260414-s390x-unknown-linux-gnu-install_only.tar.gz +801b03fbe004181d55a02ebd8b4e04d74973e70d716062aebe3b3cf32e9be297 20260414/cpython-3.12.13+20260414-x86_64-apple-darwin-install_only.tar.gz +c5a9e011e284c49c48106ca177342f3e3f64e95b4c6652d4a382cc7c9bb1cc46 20260414/cpython-3.12.13+20260414-x86_64-pc-windows-msvc-install_only.tar.gz +cdcf8724d46e4857f8db5ee9f4252dc2f5da34f7940294ec6b312389dd3f41e0 20260414/cpython-3.12.13+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +d10e971238c130fdf25e577c6538a3effa5589d5fcf53665e3c711edd6a6ff2f 20260414/cpython-3.12.13+20260414-x86_64-unknown-linux-musl-install_only.tar.gz +2662b1c3f6d5ed4d02d877c07f9384acc0d18b9046d54cd2853dad3ca172784f 20260414/cpython-3.13.13+20260414-aarch64-apple-darwin-freethreaded-install_only.tar.gz +c652dad552122cd2e76968ec41c803f8222038169b11310dba0c85928265f5c1 20260414/cpython-3.13.13+20260414-aarch64-apple-darwin-install_only.tar.gz +c6c1aae3809ef585271f6f1bb3643a2c6e0c82b811b93284c6218b31f0b931d7 20260414/cpython-3.13.13+20260414-aarch64-pc-windows-msvc-freethreaded-install_only.tar.gz +586ba71c75f341e1d111399b7f719ae784dc11e8672e93e017388f28684226d0 20260414/cpython-3.13.13+20260414-aarch64-pc-windows-msvc-install_only.tar.gz +46ac7e9476b938ef19f71029a77d28ed1e201335dd0aa0237fcfed2e5ce0ee61 20260414/cpython-3.13.13+20260414-aarch64-unknown-linux-gnu-freethreaded-install_only.tar.gz +6a65f68043d7fadcd580415493d2929d1fd686013f9ae44ddbd3a81307ab256d 20260414/cpython-3.13.13+20260414-aarch64-unknown-linux-gnu-install_only.tar.gz +abe26a6cab523a5d00d75f1353cbad9c5dc04262dcb0dc4a2b47d02384e2a7d7 20260414/cpython-3.13.13+20260414-ppc64le-unknown-linux-gnu-freethreaded-install_only.tar.gz +aef73894107300264222b19e357baf5bad616b1c4bf5daa5c3b97cfee8f5ed7b 20260414/cpython-3.13.13+20260414-ppc64le-unknown-linux-gnu-install_only.tar.gz +eea71fc3625fcc2408171b17fb97e0c6286ed60ed225ca7fd6e2fc5d9cc21dce 20260414/cpython-3.13.13+20260414-riscv64-unknown-linux-gnu-freethreaded-install_only.tar.gz +f47c09f8e7f2fb0bc4afe52422705af4016c8d3ec1cf004b67bb56a86caa62cb 20260414/cpython-3.13.13+20260414-riscv64-unknown-linux-gnu-install_only.tar.gz +dd8b6161c4af3c2f5f29b3535decdcf146ce90d7a062687c9e5229b4151198b0 20260414/cpython-3.13.13+20260414-s390x-unknown-linux-gnu-freethreaded-install_only.tar.gz +4d205af9654e1f33cefd23ff798af470e565f3ac0eba18d2f98f18a2abd07166 20260414/cpython-3.13.13+20260414-s390x-unknown-linux-gnu-install_only.tar.gz +27edbaad8f0c1a8814647d24df3f87eb13c89bbc2cb90e2fc23d8fa48dd64b15 20260414/cpython-3.13.13+20260414-x86_64-apple-darwin-freethreaded-install_only.tar.gz +540337412d2c4220e99280f741dbf45c1e3da3a39edaaab20c6ba1d53e1692ef 20260414/cpython-3.13.13+20260414-x86_64-apple-darwin-install_only.tar.gz +002c07103bfbe1b889f41eb1b9fade81651a21aed35a3512e2a916c5d7903cfe 20260414/cpython-3.13.13+20260414-x86_64-pc-windows-msvc-freethreaded-install_only.tar.gz +ee0cb26453d6e025d36502d765c1639c34830355e46ab3ad31c0360bc4cd9b79 20260414/cpython-3.13.13+20260414-x86_64-pc-windows-msvc-install_only.tar.gz +a183ec7a10c38ab8c3f19968614f1e69ec697199e94525583662dfbc22b70d9a 20260414/cpython-3.13.13+20260414-x86_64-unknown-linux-gnu-freethreaded-install_only.tar.gz +e5ec3b2c5693215d153c434ac018e75511b2c4f96d2bce30468a477cb3a89d5e 20260414/cpython-3.13.13+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +24ac6bf80dd2991c8be348f777c96c6eb69b71e78d8fa28c09beb3ddca015a47 20260414/cpython-3.13.13+20260414-x86_64-unknown-linux-musl-install_only.tar.gz +a4bfd77675740a0362c137b094f3cd9995775e8e6c0a7874a095dd055fd1ea99 20260414/cpython-3.14.4+20260414-aarch64-apple-darwin-freethreaded-install_only.tar.gz +8b7865e511b17093e090449bf71eb52933c17d45ad5257ddeacaffbb2c7239df 20260414/cpython-3.14.4+20260414-aarch64-apple-darwin-install_only.tar.gz +0458cb9885c30df690cdf304a16ec335cbc7344792ef0e8a904614b24a61316d 20260414/cpython-3.14.4+20260414-aarch64-pc-windows-msvc-freethreaded-install_only.tar.gz +82613380d582d806e562d7701496c34c87753ab13c37aa0afe2039003651f389 20260414/cpython-3.14.4+20260414-aarch64-pc-windows-msvc-install_only.tar.gz +6d84fb153ccb5cb650652aadc490d99881a8d9b68cf273d44cb553e8cd087734 20260414/cpython-3.14.4+20260414-aarch64-unknown-linux-gnu-freethreaded-install_only.tar.gz +5c8db1c21023316adad827a46d917bbbd6a85ae4e39bc3a58febda712c2f963d 20260414/cpython-3.14.4+20260414-aarch64-unknown-linux-gnu-install_only.tar.gz +b5e025e340d0faa1772ef234e320401b0aa5cf6c9d16ed63a8c44be7c531bc58 20260414/cpython-3.14.4+20260414-ppc64le-unknown-linux-gnu-freethreaded-install_only.tar.gz +055977a09de092744bbb22db64144e6afef8592eaac5e2bce4cca33f2592281a 20260414/cpython-3.14.4+20260414-ppc64le-unknown-linux-gnu-install_only.tar.gz +4373553133eb4712bc10f720da29e091a23153f587fdb2c38f1fb105e70db53a 20260414/cpython-3.14.4+20260414-riscv64-unknown-linux-gnu-freethreaded-install_only.tar.gz +e959df167c502fb0bbcacc31a997e25c6b0ff6b5e496321b691955aa702d0c09 20260414/cpython-3.14.4+20260414-riscv64-unknown-linux-gnu-install_only.tar.gz +a6797ad05c7d7f74a2cea28bf012f9199f4d6c1ed6d09f7adfeb9b3c538c6258 20260414/cpython-3.14.4+20260414-s390x-unknown-linux-gnu-freethreaded-install_only.tar.gz +35f70ad05b2c4045889ee0c3d93f61b012654c1d91e10e671f0e5b4d4a6c6637 20260414/cpython-3.14.4+20260414-s390x-unknown-linux-gnu-install_only.tar.gz +1c366767d203b722efbd5b3796d16a08436e8a328afd31e551289efba9bf56d1 20260414/cpython-3.14.4+20260414-x86_64-apple-darwin-freethreaded-install_only.tar.gz +9ecb2b942e6698c04af10a63a3d73c0b2e8d8e11ce44933fbffe8651bef4577d 20260414/cpython-3.14.4+20260414-x86_64-apple-darwin-install_only.tar.gz +5ccaecdb899431f393209647182def14b36d7398bd59be4fa73dd79b48b3f290 20260414/cpython-3.14.4+20260414-x86_64-pc-windows-msvc-freethreaded-install_only.tar.gz +9647bb46d3c236e34c1c11bbb7113444d9711811f0d11c39956168807a955b1a 20260414/cpython-3.14.4+20260414-x86_64-pc-windows-msvc-install_only.tar.gz +c1a845a79da56265dc49628bc3b9e20d34f04674fd2d637ee40cbe259d2b1b95 20260414/cpython-3.14.4+20260414-x86_64-unknown-linux-gnu-freethreaded-install_only.tar.gz +e17275eaf95ceb5877aa6816e209b7733f41fee401d39c3921b88fb73fc4a4ba 20260414/cpython-3.14.4+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +12687a989a2384665577e1ef9864f33d4c074a1e69b38a8bac8d656531aefa3e 20260414/cpython-3.14.4+20260414-x86_64-unknown-linux-musl-install_only.tar.gz +5791a69a73b76b908f5bdf96da1928de8db696ab198f4ced04b77b22fe712ce0 20260414/cpython-3.15.0a8+20260414-aarch64-apple-darwin-freethreaded-install_only.tar.gz +780d46b3da0e58e15c620d9e7dfd29b54c8359c195f625858f85df9c2c7ecc32 20260414/cpython-3.15.0a8+20260414-aarch64-apple-darwin-install_only.tar.gz +95ddfe7dd52185f7e5d55524eafb48e54d1eab0b0cf013966f144a411f3ddd0f 20260414/cpython-3.15.0a8+20260414-aarch64-pc-windows-msvc-freethreaded-install_only.tar.gz +10fb470e900e65df4e37f8deaf1726397c914861ffc37b43ae3743a7eee88377 20260414/cpython-3.15.0a8+20260414-aarch64-pc-windows-msvc-install_only.tar.gz +b72908bce86036a0a1ba98ca9917ea0b99dc1e6c5d715d3d463c4f330880c09b 20260414/cpython-3.15.0a8+20260414-aarch64-unknown-linux-gnu-freethreaded-install_only.tar.gz +8f6dda4d8ff44976f1aa6a94674a09a503dc50b015297e1b62c8cdc591c90f4f 20260414/cpython-3.15.0a8+20260414-aarch64-unknown-linux-gnu-install_only.tar.gz +b3c8210674140a4c5beefa2d4afd752979222638a0fb68de672c60300b4a6642 20260414/cpython-3.15.0a8+20260414-ppc64le-unknown-linux-gnu-freethreaded-install_only.tar.gz +09f076c63fadbf675143674aa3b23229482b9a44840b8b1808a216def2a9af15 20260414/cpython-3.15.0a8+20260414-ppc64le-unknown-linux-gnu-install_only.tar.gz +1a4984207974563c6aea7dc934579d058dbac7436642081113e86011114b9fdf 20260414/cpython-3.15.0a8+20260414-riscv64-unknown-linux-gnu-freethreaded-install_only.tar.gz +9d41ce752e8b731872f0f5c9c48199e63c789d24ce3ae9e91d6c8008f36e7c51 20260414/cpython-3.15.0a8+20260414-riscv64-unknown-linux-gnu-install_only.tar.gz +f525a6244d73450e0c0a7ba125b5934894ab25ee171f7099c239d4eb7ce2f5f2 20260414/cpython-3.15.0a8+20260414-s390x-unknown-linux-gnu-freethreaded-install_only.tar.gz +1de2593c40cce2d8ea883f8c8580223bfa1478cbd9d0191ba3640aed083c2202 20260414/cpython-3.15.0a8+20260414-s390x-unknown-linux-gnu-install_only.tar.gz +3dcee23c21e4a3518947e988e115c1d824f07540f4326d93d4ea2028918e0193 20260414/cpython-3.15.0a8+20260414-x86_64-apple-darwin-freethreaded-install_only.tar.gz +a7744d34148969a2ec010da6f0a46ddeceda7c02e5cdfa2b4e1811487381491a 20260414/cpython-3.15.0a8+20260414-x86_64-apple-darwin-install_only.tar.gz +6e69670347e3a6ac1d0cd89b9506d825bd2f2690cc51ead5dec61aec6857d08d 20260414/cpython-3.15.0a8+20260414-x86_64-pc-windows-msvc-freethreaded-install_only.tar.gz +3ded476f676fdf260d56a5e49aa083d5ffd218fc3390e4480ed42bee1acfb3fb 20260414/cpython-3.15.0a8+20260414-x86_64-pc-windows-msvc-install_only.tar.gz +2d06d97e230b7f74de0fe4f661918a0ee827b08127b9372e0890e167de52a8c6 20260414/cpython-3.15.0a8+20260414-x86_64-unknown-linux-gnu-freethreaded-install_only.tar.gz +c93f4b15287ac48d7e3a475b245cb59cc51079382747e3e6213d6406c158969d 20260414/cpython-3.15.0a8+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +0568e953f837f09689eb4dd1af0043ba5e2ebae0c6395b8b9f8344a53b1f1da5 20260414/cpython-3.15.0a8+20260414-x86_64-unknown-linux-musl-freethreaded-install_only.tar.gz +9fbd6f243a424d4ae973e72aa0075122a7cfe05ac8f6cfde986e7b00d0dbc0bf 20260414/cpython-3.15.0a8+20260414-x86_64-unknown-linux-musl-install_only.tar.gz +""" diff --git a/python/private/semantics.bzl b/python/private/semantics.bzl deleted file mode 100644 index 3811b17414..0000000000 --- a/python/private/semantics.bzl +++ /dev/null @@ -1,31 +0,0 @@ -# Copyright 2022 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -"""Contains constants that vary between Bazel and Google-internal""" - -IMPORTS_ATTR_SUPPORTED = True - -SRCS_ATTR_ALLOW_FILES = [".py", ".py3"] - -DEPS_ATTR_ALLOW_RULES = None - -PY_RUNTIME_ATTR_NAME = "_py_interpreter" - -BUILD_DATA_SYMLINK_PATH = None - -IS_BAZEL = True - -NATIVE_RULES_MIGRATION_HELP_URL = "https://github.com/bazelbuild/bazel/issues/17773" -NATIVE_RULES_MIGRATION_FIX_CMD = "add_python_loads" - -ALLOWED_MAIN_EXTENSIONS = [".py"] diff --git a/python/private/semver.bzl b/python/private/semver.bzl deleted file mode 100644 index 73d6b130ae..0000000000 --- a/python/private/semver.bzl +++ /dev/null @@ -1,75 +0,0 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"A semver version parser" - -def _key(version): - return ( - version.major, - version.minor or 0, - version.patch or 0, - # non pre-release versions are higher - version.pre_release == "", - # then we compare each element of the pre_release tag separately - tuple([ - ( - i if not i.isdigit() else "", - # digit values take precedence - int(i) if i.isdigit() else 0, - ) - for i in version.pre_release.split(".") - ]) if version.pre_release else None, - # And build info is just alphabetic - version.build, - ) - -def _to_dict(self): - return { - "build": self.build, - "major": self.major, - "minor": self.minor, - "patch": self.patch, - "pre_release": self.pre_release, - } - -def semver(version): - """Parse the semver version and return the values as a struct. - - Args: - version: {type}`str` the version string. - - Returns: - A {type}`struct` with `major`, `minor`, `patch` and `build` attributes. - """ - - # Implement the https://semver.org/ spec - major, _, tail = version.partition(".") - minor, _, tail = tail.partition(".") - patch, _, build = tail.partition("+") - patch, _, pre_release = patch.partition("-") - - # buildifier: disable=uninitialized - self = struct( - major = int(major), - minor = int(minor) if minor.isdigit() else None, - # NOTE: this is called `micro` in the Python interpreter versioning scheme - patch = int(patch) if patch.isdigit() else None, - pre_release = pre_release, - build = build, - # buildifier: disable=uninitialized - key = lambda: _key(self), - str = lambda: version, - to_dict = lambda: _to_dict(self), - ) - return self diff --git a/python/private/sentinel.bzl b/python/private/sentinel_impl.bzl similarity index 86% rename from python/private/sentinel.bzl rename to python/private/sentinel_impl.bzl index 6d753e1983..8b69682b49 100644 --- a/python/private/sentinel.bzl +++ b/python/private/sentinel_impl.bzl @@ -25,6 +25,10 @@ SentinelInfo = provider( def _sentinel_impl(ctx): _ = ctx # @unused - return [SentinelInfo()] + return [ + SentinelInfo(), + # Also output ToolchainInfo to allow it to be used for noop toolchains + platform_common.ToolchainInfo(), + ] sentinel = rule(implementation = _sentinel_impl) diff --git a/python/private/site_init_template.py b/python/private/site_init_template.py index 7a32210bff..12be98eb57 100644 --- a/python/private/site_init_template.py +++ b/python/private/site_init_template.py @@ -12,6 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. """site initialization logic for Bazel-built py_binary targets.""" + import os import os.path import sys @@ -26,6 +27,8 @@ _SELF_RUNFILES_RELATIVE_PATH = "%site_init_runfiles_path%" # Runfiles-relative path to the coverage tool entry point, if any. _COVERAGE_TOOL = "%coverage_tool%" +# True if the runfiles root should be added to sys.path +_ADD_RUNFILES_ROOT_TO_SYS_PATH = "%add_runfiles_root_to_sys_path%" == "1" def _is_verbose(): @@ -95,7 +98,15 @@ def _get_windows_path_with_unc_prefix(path): # Related doc: https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=cmd#enable-long-paths-in-windows-10-version-1607-and-later import platform - if platform.win32_ver()[1] >= "10.0.14393": + win32_version = None + # Windows 2022 with Python 3.12.8 gives flakey errors, so try a couple times. + for _ in range(3): + try: + win32_version = platform.win32_ver()[1] + break + except (ValueError, KeyError): + pass + if win32_version and win32_version >= "10.0.14393": return path # import sysconfig only now to maintain python 2.6 compatibility @@ -125,23 +136,35 @@ def _search_path(name): def _setup_sys_path(): + """Perform Bazel/binary specific sys.path setup.""" + _print_verbose("site init: initial sys.path:\n", "\n".join(sys.path)) seen = set(sys.path) - python_path_entries = [] - def _maybe_add_path(path): + def _maybe_add_path(path, reason): if path in seen: return path = _get_windows_path_with_unc_prefix(path) if _is_windows(): path = path.replace("/", os.sep) - _print_verbose("append sys.path:", path) + _print_verbose("append sys.path:", reason, ":", path) sys.path.append(path) seen.add(path) + # Adding the runfiles root to sys.path is a legacy behavior that will be + # removed. We don't want to add it to sys.path for two reasons: + # 1. Under workspace, it makes every external repository importable. If a Bazel + # repository matches a Python import name, they conflict. + # 2. Under bzlmod, the repo names in the runfiles directory aren't importable + # Python names, so there's no point in adding the runfiles root to sys.path. + # For temporary compatibility with the original system_python bootstrap + # behavior, it is conditionally added for that boostrap mode. + if _ADD_RUNFILES_ROOT_TO_SYS_PATH: + _maybe_add_path(_RUNFILES_ROOT, "runfiles-root") + for rel_path in _IMPORTS_STR.split(":"): abs_path = os.path.join(_RUNFILES_ROOT, rel_path) - _maybe_add_path(abs_path) + _maybe_add_path(abs_path, "imports-strs") if _IMPORT_ALL: repo_dirs = sorted( @@ -149,9 +172,9 @@ def _maybe_add_path(path): ) for d in repo_dirs: if os.path.isdir(d): - _maybe_add_path(d) + _maybe_add_path(d, "import-all") else: - _maybe_add_path(os.path.join(_RUNFILES_ROOT, _WORKSPACE_NAME)) + _maybe_add_path(os.path.join(_RUNFILES_ROOT, _WORKSPACE_NAME), "workspace-root") # COVERAGE_DIR is set if coverage is enabled and instrumentation is configured # for something, though it could be another program executing this one or @@ -163,7 +186,9 @@ def _maybe_add_path(path): if cov_tool: _print_verbose_coverage(f"Using toolchain coverage_tool {cov_tool}") elif cov_tool := os.environ.get("PYTHON_COVERAGE"): - _print_verbose_coverage(f"PYTHON_COVERAGE: {cov_tool}") + _print_verbose_coverage( + f"Using env var coverage: PYTHON_COVERAGE={cov_tool}" + ) if cov_tool: if os.path.isabs(cov_tool): @@ -181,11 +206,11 @@ def _maybe_add_path(path): # it with the directory of the program it starts. Our actual sys.path[0] is # the runfiles directory, which must not be replaced. # CoverageScript.do_execute() undoes this sys.path[0] setting. - _maybe_add_path(coverage_dir) + _maybe_add_path(coverage_dir, "coverage-dir") coverage_setup = True else: _print_verbose_coverage( - "Coverage was enabled, but python coverage tool was not configured." + "Coverage was enabled, but the coverage tool was not found or valid. " + "To enable coverage, consult the docs at " + "https://rules-python.readthedocs.io/en/latest/coverage.html" ) @@ -193,4 +218,27 @@ def _maybe_add_path(path): return coverage_setup +def _fixup_sys_base_executable(): + """Fixup sys._base_executable to account for Bazel-specific pyvenv.cfg + + The pyvenv.cfg created for py_binary leaves the `home` key unset. A + side-effect of this is `sys._base_executable` points to the venv executable, + not the actual executable. This mostly doesn't matter, but does affect + using the venv module to create venvs (they point to the venv executable, not + the actual executable). + """ + # Must have been set correctly? + if sys.executable != sys._base_executable: + return + # Not in a venv, so don't touch anything. + if sys.prefix == sys.base_prefix: + return + exe = os.path.realpath(sys.executable) + _print_verbose("setting sys._base_executable:", exe) + sys._base_executable = exe + + +_fixup_sys_base_executable() + COVERAGE_SETUP = _setup_sys_path() +_print_verbose("DONE") diff --git a/python/private/stage1_bootstrap_template.sh b/python/private/stage1_bootstrap_template.sh index b05b4a54cd..4374ff95b0 100644 --- a/python/private/stage1_bootstrap_template.sh +++ b/python/private/stage1_bootstrap_template.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash set -e @@ -6,18 +6,59 @@ if [[ -n "${RULES_PYTHON_BOOTSTRAP_VERBOSE:-}" ]]; then set -x fi -# runfiles-relative path +# Creates a symlink. If the symlink already exists, it is tolerated to avoid +# race conditions during startup. +function _symlink() { + local target="$1" + local link="$2" + if ln -s "$target" "$link" 2>/dev/null; then + return 0 + fi + # If it failed, maybe it already exists because of a race. + if [[ -L "$link" || -e "$link" ]]; then + return 0 + fi + # If it doesn't exist, maybe we don't have write permission in the directory. + local dir + dir=$(dirname "$link") + if [[ ! -w "$dir" ]]; then + echo >&2 "ERROR: Cannot create symlink $link: Directory $dir is not writable" + else + echo >&2 "ERROR: Failed to create symlink $link -> $target" + fi + return 1 +} + + +# runfiles-root-relative path STAGE2_BOOTSTRAP="%stage2_bootstrap%" -# runfiles-relative path +# runfiles-root-relative path to python interpreter to use. +# This is the `bin/python3` path in the binary's venv. PYTHON_BINARY='%python_binary%' # The path that PYTHON_BINARY should symlink to. -# runfiles-relative path, absolute path, or single word. -# Only applicable for zip files. +# runfiles-root-relative path, absolute path, or single word. +# Only applicable for zip files or when venv is recreated at runtime. PYTHON_BINARY_ACTUAL="%python_binary_actual%" # 0 or 1 IS_ZIPFILE="%is_zipfile%" +# 0 or 1. +# If 1, then a venv will be created at runtime that replicates what would have +# been the build-time structure. +RECREATE_VENV_AT_RUNTIME="%recreate_venv_at_runtime%" +# 0 or 1 +# If 1, then the path to python will be resolved by running +# PYTHON_BINARY_ACTUAL to determine the actual underlying interpreter. +RESOLVE_PYTHON_BINARY_AT_RUNTIME="%resolve_python_binary_at_runtime%" +# venv-relative path to the site-packages +# e.g. lib/python3.12t/site-packages +VENV_REL_SITE_PACKAGES="%venv_rel_site_packages%" + +# array of strings +declare -a INTERPRETER_ARGS_FROM_TARGET=( +%interpreter_args% +) if [[ "$IS_ZIPFILE" == "1" ]]; then # NOTE: Macs have an old version of mktemp, so we must use only the @@ -44,14 +85,20 @@ if [[ "$IS_ZIPFILE" == "1" ]]; then else function find_runfiles_root() { + local maybe_root="" if [[ -n "${RUNFILES_DIR:-}" ]]; then - echo "$RUNFILES_DIR" - return 0 + maybe_root="$RUNFILES_DIR" elif [[ "${RUNFILES_MANIFEST_FILE:-}" = *".runfiles_manifest" ]]; then - echo "${RUNFILES_MANIFEST_FILE%%.runfiles_manifest}.runfiles" - return 0 + maybe_root="${RUNFILES_MANIFEST_FILE%%.runfiles_manifest}.runfiles" elif [[ "${RUNFILES_MANIFEST_FILE:-}" = *".runfiles/MANIFEST" ]]; then - echo "${RUNFILES_MANIFEST_FILE%%.runfiles/MANIFEST}.runfiles" + maybe_root="${RUNFILES_MANIFEST_FILE%%.runfiles/MANIFEST}.runfiles" + fi + + # The RUNFILES_DIR et al variables may misreport the runfiles directory + # if an outer binary invokes this binary when it isn't a data dependency. + # e.g. a genrule calls `bazel-bin/outer --inner=bazel-bin/inner` + if [[ -n "$maybe_root" && -e "$maybe_root/$STAGE2_BOOTSTRAP" ]]; then + echo "$maybe_root" return 0 fi @@ -62,9 +109,9 @@ else stub_filename="$PWD/$stub_filename" fi while true; do - module_space="${stub_filename}.runfiles" - if [[ -d "$module_space" ]]; then - echo "$module_space" + runfiles_root="${stub_filename}.runfiles" + if [[ -d "$runfiles_root" ]]; then + echo "$runfiles_root" return 0 fi if [[ "$stub_filename" == *.runfiles/* ]]; then @@ -74,8 +121,7 @@ else if [[ ! -L "$stub_filename" ]]; then break fi - target=$(realpath $maybe_runfiles_root) - stub_filename="$target" + stub_filename=$(readlink $stub_filename) done echo >&2 "Unable to find runfiles directory for $1" exit 1 @@ -83,6 +129,9 @@ else RUNFILES_DIR=$(find_runfiles_root $0) fi +if [[ -n "$RULES_PYTHON_TESTING_TELL_RUNFILES_ROOT" ]]; then + export RULES_PYTHON_TESTING_RUNFILES_ROOT="$RUNFILES_DIR" +fi function find_python_interpreter() { runfiles_root="$1" @@ -104,6 +153,7 @@ python_exe=$(find_python_interpreter $RUNFILES_DIR $PYTHON_BINARY) # Zip files have to re-create the venv bin/python3 symlink because they # don't contain it already. if [[ "$IS_ZIPFILE" == "1" ]]; then + use_exec=0 # It should always be under runfiles, but double check this. We don't # want to accidentally create symlinks elsewhere. if [[ "$python_exe" != $RUNFILES_DIR/* ]]; then @@ -121,13 +171,98 @@ if [[ "$IS_ZIPFILE" == "1" ]]; then symlink_to=$(which $PYTHON_BINARY_ACTUAL) # Guard against trying to symlink to an empty value if [[ $? -ne 0 ]]; then - echo >&2 "ERROR: Python to use found on PATH: $PYTHON_BINARY_ACTUAL" + echo >&2 "ERROR: Python to use not found on PATH: $PYTHON_BINARY_ACTUAL" exit 1 fi fi # The bin/ directory may not exist if it is empty. mkdir -p "$(dirname $python_exe)" - ln -s "$symlink_to" "$python_exe" + _symlink "$symlink_to" "$python_exe" +elif [[ "$RECREATE_VENV_AT_RUNTIME" == "1" ]]; then + if [[ -n "$RULES_PYTHON_EXTRACT_ROOT" ]]; then + use_exec=1 + # Use our runfiles path as a unique, reusable, location for the + # binary-specific venv being created. + venv="$RULES_PYTHON_EXTRACT_ROOT/$(dirname $(dirname $PYTHON_BINARY))" + mkdir -p $RULES_PYTHON_EXTRACT_ROOT + else + # Re-exec'ing can't be used because we have to clean up the temporary + # venv directory that is created. + use_exec=0 + venv=$(mktemp -d) + if [[ -n "$venv" && -z "${RULES_PYTHON_BOOTSTRAP_VERBOSE:-}" ]]; then + trap 'rm -fr "$venv"' EXIT + fi + fi + + # Match the basename; some tools, e.g. pyvenv key off the executable name + python_exe="$venv/bin/$(basename $PYTHON_BINARY_ACTUAL)" + + if [[ ! -e "$python_exe" ]]; then + if [[ "$PYTHON_BINARY_ACTUAL" == /* ]]; then + # An absolute path, i.e. platform runtime, e.g. /usr/bin/python3 + python_exe_actual=$PYTHON_BINARY_ACTUAL + elif [[ "$PYTHON_BINARY_ACTUAL" == */* ]]; then + # A runfiles-relative path + python_exe_actual="$RUNFILES_DIR/$PYTHON_BINARY_ACTUAL" + else + # A plain word, e.g. "python3". Symlink to where PATH leads + python_exe_actual=$(which $PYTHON_BINARY_ACTUAL) + # Guard against trying to symlink to an empty value + if [[ $? -ne 0 ]]; then + echo >&2 "ERROR: Python to use not found on PATH: $PYTHON_BINARY_ACTUAL" + exit 1 + fi + fi + + runfiles_venv="$RUNFILES_DIR/$(dirname $(dirname $PYTHON_BINARY))" + # When RESOLVE_PYTHON_BINARY_AT_RUNTIME is true, it means the toolchain + # has thrown two complications at us: + # 1. The build-time assumption of the Python version may not match the + # runtime Python version. The site-packages directory path includes the + # Python version, so when the versions don't match, the runtime won't + # find it. + # 2. The interpreter might be a wrapper script, which interferes with Python's + # ability to detect when it's within a venv. Starting in Python 3.11, + # the PYTHONEXECUTABLE environment variable can fix this, but due to (1), + # we don't know if that is supported without running Python. + # To fix (1), we symlink the desired site-packages path to the build-time + # directory. Hopefully the version mismatch is OK :D. + # To fix (2), we determine the actual underlying interpreter and symlink + # to that. + if [[ "$RESOLVE_PYTHON_BINARY_AT_RUNTIME" == "1" ]]; then + { + read -r resolved_py_exe + read -r resolved_site_packages + } < <("$python_exe_actual" -I </dev/null 2>&1; then + ENV_CMD="env" +else + ENV_CMD="/usr/bin/env" +fi + command=( - env + "$ENV_CMD" "${interpreter_env[@]}" "$python_exe" "${interpreter_args[@]}" + "${INTERPRETER_ARGS_FROM_TARGET[@]}" "$stage2_bootstrap" "$@" ) @@ -181,12 +329,13 @@ command=( # using `kill`) to this process (the PID seen by the calling process) are # received by the Python process. Otherwise, this process receives the signal # and would have to manually propagate it. -# See https://github.com/bazelbuild/rules_python/issues/2043#issuecomment-2215469971 +# See https://github.com/bazel-contrib/rules_python/issues/2043#issuecomment-2215469971 # for more information. # -# However, when running a zip file, we need to clean up the workspace after the -# process finishes so control must return here. -if [[ "$IS_ZIPFILE" == "1" ]]; then +# However, we can't use exec when there is cleanup to do afterwards. Control +# must return to this process so it can run the trap handlers. Such cases +# occur when zip mode or recreate_venv_at_runtime creates temporary files. +if [[ "$use_exec" == "0" ]]; then "${command[@]}" exit $? else diff --git a/python/private/stage2_bootstrap_template.py b/python/private/stage2_bootstrap_template.py index d2c7497795..f445ad2b6a 100644 --- a/python/private/stage2_bootstrap_template.py +++ b/python/private/stage2_bootstrap_template.py @@ -4,39 +4,113 @@ import sys -# The Python interpreter unconditionally prepends the directory containing this +# By default the Python interpreter prepends the directory containing this # script (following symlinks) to the import path. This is the cause of #9239, -# and is a special case of #7091. We therefore explicitly delete that entry. -# TODO(#7091): Remove this hack when no longer necessary. -# TODO: Use sys.flags.safe_path to determine whether this removal should be -# performed -del sys.path[0] +# and is a special case of #7091. +# +# Python 3.11 introduced an PYTHONSAFEPATH (-P) option that disables this +# behaviour, which we set in the stage 1 bootstrap. Isolated mode (-I) also +# disables it, including on older interpreters without safe_path. +# So the prepended entry needs to be removed only if the above option is either +# unset or not supported by the interpreter. +# NOTE: This can be removed when Python 3.10 and below is no longer supported +if ( + not getattr(sys.flags, "safe_path", False) + and not getattr(sys.flags, "isolated", False) + and sys.path +): + del sys.path[0] import contextlib import os import re import runpy +import types import uuid +from functools import cache # ===== Template substitutions start ===== # We just put them in one place so its easy to tell which are used. -# Runfiles-relative path to the main Python source file. -MAIN = "%main%" +# Runfiles-root-relative path to the main Python source file. +# Empty if MAIN_MODULE is used +MAIN_PATH = "%main%" + +# Module name to execute. Empty if MAIN is used. +MAIN_MODULE = "%main_module%" + +# runfiles-root relative path to the root of the venv +VENV_ROOT = "%venv_root%" + +# venv-relative path to the expected location of the binary's site-packages +# directory. +# Only set when the toolchain doesn't support the build-time venv. Empty +# string otherwise. +VENV_SITE_PACKAGES = "%venv_rel_site_packages%" + +# Whether we should generate coverage data. +# string, 1 or 0 +COVERAGE_INSTRUMENTED = "%coverage_instrumented%" == "1" + +# runfiles-root-relative path to a file with binary-specific build information +# It uses forward slashes, so must be converted for proper usage on Windows. +BUILD_DATA_FILE = "%build_data_file%" # ===== Template substitutions end ===== +IS_WINDOWS = os.name == "nt" +IS_VERBOSE = bool(os.environ.get("RULES_PYTHON_BOOTSTRAP_VERBOSE")) + +# Windows APIs can be picky about slashes depending on the context, +# so convert to backslashes to avoid any issues. +# Related: some logic checks path strings, which needs uniform separators. +if IS_WINDOWS: + + def norm_slashes(s): + return s.replace("/", "\\") + + MAIN_PATH = norm_slashes(MAIN_PATH) + VENV_ROOT = norm_slashes(VENV_ROOT) + VENV_SITE_PACKAGES = norm_slashes(VENV_SITE_PACKAGES) + BUILD_DATA_FILE = norm_slashes(BUILD_DATA_FILE) -# Return True if running on Windows -def is_windows(): - return os.name == "nt" + +class BazelBinaryInfoModule(types.ModuleType): + BUILD_DATA_FILE = BUILD_DATA_FILE + + @cache + def get_build_data(self): + """Returns a string of the raw build data.""" + try: + # Prefer dep via pypi + import runfiles + except ImportError: + from python.runfiles import runfiles + rlocation_path = self.BUILD_DATA_FILE + path = runfiles.Create().Rlocation(rlocation_path) + if IS_WINDOWS: + path = os.path.normpath(path) + try: + # Use utf-8-sig to handle Windows BOM + with open(path, encoding="utf-8-sig") as fp: + return fp.read() + except Exception as exc: + if hasattr(exc, "add_note"): + exc.add_note(f"runfiles lookup path: {rlocation_path}") + exc.add_note(f"exists: {os.path.exists(path)}") + can_read = os.access(path, os.R_OK) + exc.add_note(f"readable: {can_read}") + raise + + +sys.modules["bazel_binary_info"] = BazelBinaryInfoModule("bazel_binary_info") def get_windows_path_with_unc_prefix(path): path = path.strip() # No need to add prefix for non-Windows platforms. - if not is_windows() or sys.version_info[0] < 3: + if not IS_WINDOWS or sys.version_info[0] < 3: return path # Starting in Windows 10, version 1607(OS build 14393), MAX_PATH limitations have been @@ -44,7 +118,15 @@ def get_windows_path_with_unc_prefix(path): # Related doc: https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=cmd#enable-long-paths-in-windows-10-version-1607-and-later import platform - if platform.win32_ver()[1] >= "10.0.14393": + win32_version = None + # Windows 2022 with Python 3.12.8 gives flakey errors, so try a couple times. + for _ in range(3): + try: + win32_version = platform.win32_ver()[1] + break + except (ValueError, KeyError): + pass + if win32_version and win32_version >= "10.0.14393": return path # import sysconfig only now to maintain python 2.6 compatibility @@ -54,61 +136,47 @@ def get_windows_path_with_unc_prefix(path): return path # Lets start the unicode fun - if path.startswith(unicode_prefix): + if path.startswith(unicode_prefix): # noqa: F821 return path # os.path.abspath returns a normalized absolute path - return unicode_prefix + os.path.abspath(path) - - -def search_path(name): - """Finds a file in a given search path.""" - search_path = os.getenv("PATH", os.defpath).split(os.pathsep) - for directory in search_path: - if directory: - path = os.path.join(directory, name) - if os.path.isfile(path) and os.access(path, os.X_OK): - return path - return None - - -def is_verbose(): - return bool(os.environ.get("RULES_PYTHON_BOOTSTRAP_VERBOSE")) + return unicode_prefix + os.path.abspath(path) # noqa: F821 def print_verbose(*args, mapping=None, values=None): - if is_verbose(): - if mapping is not None: - for key, value in sorted((mapping or {}).items()): - print( - "bootstrap: stage 2:", - *args, - f"{key}={value!r}", - file=sys.stderr, - flush=True, - ) - elif values is not None: - for i, v in enumerate(values): - print( - "bootstrap: stage 2:", - *args, - f"[{i}] {v!r}", - file=sys.stderr, - flush=True, - ) - else: - print("bootstrap: stage 2:", *args, file=sys.stderr, flush=True) + if not IS_VERBOSE: + return + if mapping is not None: + for key, value in sorted((mapping or {}).items()): + print( + "bootstrap: stage 2:", + *args, + f"{key}={value!r}", + file=sys.stderr, + flush=True, + ) + elif values is not None: + for i, v in enumerate(values): + print( + "bootstrap: stage 2:", + *args, + f"[{i}] {v!r}", + file=sys.stderr, + flush=True, + ) + else: + print("bootstrap: stage 2:", *args, file=sys.stderr, flush=True) def print_verbose_coverage(*args): """Print output if VERBOSE_COVERAGE is non-empty in the environment.""" - if os.environ.get("VERBOSE_COVERAGE"): - print(*args, file=sys.stderr, flush=True) + if is_verbose_coverage(): + print("bootstrap: stage 2: coverage:", *args, file=sys.stderr, flush=True) def is_verbose_coverage(): """Returns True if VERBOSE_COVERAGE is non-empty in the environment.""" - return os.environ.get("VERBOSE_COVERAGE") or is_verbose() + return os.environ.get("VERBOSE_COVERAGE") or IS_VERBOSE def find_runfiles_root(main_rel_path): @@ -130,16 +198,23 @@ def find_runfiles_root(main_rel_path): if runfiles_dir and os.path.exists(os.path.join(runfiles_dir, main_rel_path)): return runfiles_dir + # Clear RUNFILES_DIR & RUNFILES_MANIFEST_FILE since the runfiles dir was + # not found. These can be correctly set for a parent Python process, but + # inherited by the child, and not correct for it. Later bootstrap code + # assumes they're are correct if set. + os.environ.pop("RUNFILES_DIR", None) + os.environ.pop("RUNFILES_MANIFEST_FILE", None) + stub_filename = sys.argv[0] if not os.path.isabs(stub_filename): stub_filename = os.path.join(os.getcwd(), stub_filename) while True: - module_space = stub_filename + (".exe" if is_windows() else "") + ".runfiles" + module_space = stub_filename + (".exe" if IS_WINDOWS else "") + ".runfiles" if os.path.isdir(module_space): return module_space - runfiles_pattern = r"(.*\.runfiles)" + (r"\\" if is_windows() else "/") + ".*" + runfiles_pattern = r"(.*\.runfiles)" + (r"\\" if IS_WINDOWS else "/") + ".*" matchobj = re.match(runfiles_pattern, stub_filename) if matchobj: return matchobj.group(1) @@ -152,6 +227,8 @@ def find_runfiles_root(main_rel_path): else: stub_filename = os.path.join(os.path.dirname(stub_filename), target) + # The `--enable_runfiles=false` flag is likely set, which isn't fully + # supported. raise AssertionError("Cannot find .runfiles directory for %s" % sys.argv[0]) @@ -245,7 +322,7 @@ def unresolve_symlinks(output_filename): os.unlink(unfixed_file) -def _run_py(main_filename, *, args, cwd=None): +def _run_py_path(main_filename, *, args, cwd=None): # type: (str, str, list[str], dict[str, str]) -> ... """Executes the given Python file using the various environment settings.""" @@ -265,25 +342,47 @@ def _run_py(main_filename, *, args, cwd=None): sys.argv = orig_argv +def _run_py_module(module_name): + # Match `python -m` behavior, so modify sys.argv and the run name + runpy.run_module(module_name, alter_sys=True, run_name="__main__") + + @contextlib.contextmanager def _maybe_collect_coverage(enable): + print_verbose_coverage("enabled:", enable) if not enable: yield return - import uuid + instrumented_files = [abs_path for abs_path, _ in instrumented_file_paths()] + unique_dirs = {os.path.dirname(file) for file in instrumented_files} + source = "\n\t".join(unique_dirs) + + print_verbose_coverage("Instrumented Files:\n" + "\n".join(instrumented_files)) + print_verbose_coverage("Sources:\n" + "\n".join(unique_dirs)) import coverage + from coverage.exceptions import NoDataError coverage_dir = os.environ["COVERAGE_DIR"] unique_id = uuid.uuid4() # We need for coveragepy to use relative paths. This can only be configured + # using an rc file. rcfile_name = os.path.join(coverage_dir, ".coveragerc_{}".format(unique_id)) + disable_warnings = ( + "disable_warnings = module-not-imported, no-data-collected" + if COVERAGE_INSTRUMENTED + else "" + ) + print_verbose_coverage("coveragerc file:", rcfile_name) with open(rcfile_name, "w") as rcfile: rcfile.write( - """[run] + f"""[run] relative_files = True +{disable_warnings} +source = +\t{source} """ ) try: @@ -313,16 +412,27 @@ def _maybe_collect_coverage(enable): yield finally: cov.stop() - lcov_path = os.path.join(coverage_dir, "pylcov.dat") - cov.lcov_report( - outfile=lcov_path, - # Ignore errors because sometimes instrumented files aren't - # readable afterwards. e.g. if they come from /dev/fd or if - # they were transient code-under-test in /tmp - ignore_errors=True, - ) - if os.path.isfile(lcov_path): - unresolve_symlinks(lcov_path) + lcov_path = os.path.join(coverage_dir, "pylcov_{}.dat".format(unique_id)) + print_verbose_coverage("generating lcov from:", lcov_path) + try: + cov.lcov_report( + outfile=lcov_path, + # Ignore errors because sometimes instrumented files aren't + # readable afterwards. e.g. if they come from /dev/fd or if + # they were transient code-under-test in /tmp + ignore_errors=True, + ) + except NoDataError: + # coverage.py raises NoDataError if no instrumented Python code ran + # (e.g. tests not running Python, or subprocess-only binaries). + # Skip the report to avoid failing otherwise passing tests. + # See https://github.com/bazel-contrib/rules_python/issues/2762. + print_verbose_coverage( + "no coverage data collected; skipping lcov report:", lcov_path + ) + else: + if os.path.isfile(lcov_path): + unresolve_symlinks(lcov_path) finally: try: os.unlink(rcfile_name) @@ -333,70 +443,145 @@ def _maybe_collect_coverage(enable): print_verbose_coverage("Error removing temporary coverage rc file:", err) +def _add_site_packages(site_packages): + if sys.prefix != sys.base_prefix: + venv_root = sys.prefix + os.sep + saw_venv_site_packages = False + else: + venv_root = None + saw_venv_site_packages = True + first_global_offset = len(sys.path) + for i, p in enumerate(sys.path): + # Handle the Windows venv case: when a temporary directory is created + # for the venv, we want the build-time venv in runfiles to come after + # the venv site-packages directory. + if venv_root: + is_venv_path = (p + os.sep).startswith(venv_root) + if is_venv_path: + if p.endswith("site-packages"): + saw_venv_site_packages = True + is_after_venv_site_packages = False + else: + is_after_venv_site_packages = saw_venv_site_packages + else: + is_after_venv_site_packages = True + + # We assume the first *-packages is the runtime's. + # *-packages is matched because Debian may use dist-packages + # instead of site-packages. + if p.endswith("-packages") and is_after_venv_site_packages: + first_global_offset = i + break + prev_len = len(sys.path) + import site + + site.addsitedir(site_packages) + added_dirs = sys.path[prev_len:] + del sys.path[prev_len:] + # Re-insert the binary specific paths so the order is + # (stdlib, binary specific, runtime site) + # This matches what a venv's ordering is like. + sys.path[first_global_offset:0] = added_dirs + + def main(): print_verbose("initial argv:", values=sys.argv) print_verbose("initial cwd:", os.getcwd()) print_verbose("initial environ:", mapping=os.environ) print_verbose("initial sys.path:", values=sys.path) - main_rel_path = MAIN - if is_windows(): - main_rel_path = main_rel_path.replace("/", os.sep) - - module_space = find_runfiles_root(main_rel_path) - print_verbose("runfiles root:", module_space) - - # Recreate the "add main's dir to sys.path[0]" behavior to match the - # system-python bootstrap / typical Python behavior. - # - # Without safe path enabled, when `python foo/bar.py` is run, python will - # resolve the foo/bar.py symlink to its real path, then add the directory - # of that path to sys.path. But, the resolved directory for the symlink - # depends on if the file is generated or not. - # - # When foo/bar.py is a source file, then it's a symlink pointing - # back to the client source directory. This means anything from that source - # directory becomes importable, i.e. most code is importable. - # - # When foo/bar.py is a generated file, then it's a symlink pointing to - # somewhere under bazel-out/.../bin, i.e. where generated files are. This - # means only other generated files are importable (not source files). - # - # To replicate this behavior, we add main's directory within the runfiles - # when safe path isn't enabled. - if not getattr(sys.flags, "safe_path", False): - prepend_path_entries = [ - os.path.join(module_space, os.path.dirname(main_rel_path)) - ] + main_rel_path = None + # todo: things happen to work because find_runfiles_root + # ends up using stage2_bootstrap, and ends up computing the proper + # runfiles root + if MAIN_PATH: + main_rel_path = MAIN_PATH + if IS_WINDOWS: + main_rel_path = main_rel_path.replace("/", os.sep) + + runfiles_root = find_runfiles_root(main_rel_path) else: - prepend_path_entries = [] + runfiles_root = find_runfiles_root("") + + site_packages = os.path.join(runfiles_root, VENV_ROOT, VENV_SITE_PACKAGES) + if site_packages not in sys.path and os.path.exists(site_packages): + # This can happen in a few situations: + # 1. We're running with a different Python version than was built with. + # Things may or may not work. Such a situation is likely due to the + # runtime_env toolchain, or some toolchain configuration. In any + # case, this better matches how the previous bootstrap=system_python + # bootstrap worked (using PYTHONPATH, which isn't version-specific). + # 2. If site is disabled (`-S` interpreter arg). Some users do this to + # prevent interference from the system. + # 3. If running without a venv configured. This occurs with the + # system_python bootstrap. + print_verbose( + f"sys.path missing expected site-packages: adding {site_packages}" + ) + _add_site_packages(site_packages) + + print_verbose("runfiles root:", runfiles_root) - runfiles_envkey, runfiles_envvalue = runfiles_envvar(module_space) + runfiles_envkey, runfiles_envvalue = runfiles_envvar(runfiles_root) if runfiles_envkey: os.environ[runfiles_envkey] = runfiles_envvalue - main_filename = os.path.join(module_space, main_rel_path) - main_filename = get_windows_path_with_unc_prefix(main_filename) - assert os.path.exists(main_filename), ( - "Cannot exec() %r: file not found." % main_filename - ) - assert os.access(main_filename, os.R_OK), ( - "Cannot exec() %r: file not readable." % main_filename - ) + if MAIN_PATH: + # Recreate the "add main's dir to sys.path[0]" behavior to match the + # system-python bootstrap / typical Python behavior. + # + # Without safe path enabled, when `python foo/bar.py` is run, python will + # resolve the foo/bar.py symlink to its real path, then add the directory + # of that path to sys.path. But, the resolved directory for the symlink + # depends on if the file is generated or not. + # + # When foo/bar.py is a source file, then it's a symlink pointing + # back to the client source directory. This means anything from that source + # directory becomes importable, i.e. most code is importable. + # + # When foo/bar.py is a generated file, then it's a symlink pointing to + # somewhere under bazel-out/.../bin, i.e. where generated files are. This + # means only other generated files are importable (not source files). + # + # To replicate this behavior, we add main's directory within the runfiles + # when safe path or isolated mode isn't enabled. + if not getattr(sys.flags, "safe_path", False) and not getattr( + sys.flags, "isolated", False + ): + prepend_path_entries = [ + os.path.join(runfiles_root, os.path.dirname(main_rel_path)) + ] + else: + prepend_path_entries = [] - sys.stdout.flush() + main_filename = os.path.join(runfiles_root, main_rel_path) + main_filename = get_windows_path_with_unc_prefix(main_filename) + assert os.path.exists(main_filename), ( + "Cannot exec() %r: file not found." % main_filename + ) + assert os.access(main_filename, os.R_OK), ( + "Cannot exec() %r: file not readable." % main_filename + ) - sys.path[0:0] = prepend_path_entries + sys.stdout.flush() + + sys.path[0:0] = prepend_path_entries + else: + main_filename = None if os.environ.get("COVERAGE_DIR"): import _bazel_site_init + coverage_enabled = _bazel_site_init.COVERAGE_SETUP else: coverage_enabled = False with _maybe_collect_coverage(enable=coverage_enabled): - # The first arg is this bootstrap, so drop that for the re-invocation. - _run_py(main_filename, args=sys.argv[1:]) + if MAIN_PATH: + # The first arg is this bootstrap, so drop that for the re-invocation. + _run_py_path(main_filename, args=sys.argv[1:]) + else: + _run_py_module(MAIN_MODULE) sys.exit(0) diff --git a/python/private/stamp.bzl b/python/private/stamp_impl.bzl similarity index 96% rename from python/private/stamp.bzl rename to python/private/stamp_impl.bzl index 6bc0cd9d23..77e3da1f72 100644 --- a/python/private/stamp.bzl +++ b/python/private/stamp_impl.bzl @@ -18,6 +18,8 @@ This module can be removed likely after the following PRs ar addressed: - https://github.com/bazelbuild/bazel/issues/11164 """ +load(":visibility.bzl", "NOT_ACTUALLY_PUBLIC") + StampSettingInfo = provider( doc = "Information about the `--stamp` command line flag", fields = { @@ -50,7 +52,7 @@ Stamped binaries are not rebuilt unless their dependencies change. }, ) -def stamp_build_setting(name, visibility = ["//visibility:public"]): +def stamp_build_setting(name, visibility = NOT_ACTUALLY_PUBLIC): native.config_setting( name = "stamp_detect", values = {"stamp": "1"}, diff --git a/python/private/text_util.bzl b/python/private/text_util.bzl index a64b5d6243..eedf66009d 100644 --- a/python/private/text_util.bzl +++ b/python/private/text_util.bzl @@ -76,13 +76,14 @@ def _render_select(selects, *, no_match_error = None, key_repr = repr, value_rep return "{}({})".format(name, args) -def _render_list(items, *, hanging_indent = ""): +def _render_list(items, *, hanging_indent = "", value_repr = repr): """Convert a list to formatted text. Args: items: list of items. hanging_indent: str, indent to apply to second and following lines of the formatted text. + value_repr: callable, function to represent each item. Returns: The list pretty formatted as a string. @@ -91,12 +92,12 @@ def _render_list(items, *, hanging_indent = ""): return "[]" if len(items) == 1: - return "[{}]".format(repr(items[0])) + return "[{}]".format(value_repr(items[0])) text = "\n".join([ "[", _indent("\n".join([ - "{},".format(repr(item)) + "{},".format(value_repr(item)) for item in items ])), "]", @@ -106,8 +107,24 @@ def _render_list(items, *, hanging_indent = ""): return text def _render_str(value): + """Render a string value. + + If value is None, it is automatically rendered as the Starlark literal `None`. + + Args: + value: str or None. + + Returns: + The value represented as Starlark source text. + """ + if value == None: + return "None" return repr(value) +def _render_string_list_dict(value): + """Render an attr.string_list_dict value (`dict[str, list[str]`)""" + return _render_dict(value, value_repr = _render_list) + def _render_tuple(items, *, value_repr = repr): if not items: return "tuple()" @@ -153,9 +170,38 @@ def _left_pad_zero(index, length): fail("index must be non-negative") return ("0" * length + str(index))[-length:] +def _render_dict_dict(d): + """Render a dict[str, dict] value without recursive function calls.""" + if not d: + return "{}" + + lines = ["{"] + for k, v in d.items(): + if not v: + v_str = "{}" + else: + inner_lines = ["{"] + for ik, iv in v.items(): + inner_lines.append(_indent("{}: {},".format(repr(ik), repr(iv)))) + inner_lines.append("}") + v_str = "\n".join(inner_lines) + + # We need to correctly indent the multi-line string v_str + # but _indent acts on every line except the first if not carefully handled. + # It's easier to just do: + lines.append(_indent("{}: {},".format(repr(k), v_str))) + lines.append("}") + return "\n".join(lines) + +def _render_struct(value): + """Render a struct value.""" + fields = {k: repr(getattr(value, k)) for k in sorted(dir(value)) if k not in ["to_json", "to_proto"]} + return _render_call("struct", **fields) + render = struct( alias = _render_alias, dict = _render_dict, + dict_dict = _render_dict_dict, call = _render_call, hanging_indent = _hanging_indent, indent = _indent, @@ -164,6 +210,8 @@ render = struct( list = _render_list, select = _render_select, str = _render_str, + struct = _render_struct, toolchain_prefix = _toolchain_prefix, tuple = _render_tuple, + string_list_dict = _render_string_list_dict, ) diff --git a/python/private/toolchain_aliases.bzl b/python/private/toolchain_aliases.bzl index 31ac4a8fdf..092863260c 100644 --- a/python/private/toolchain_aliases.bzl +++ b/python/private/toolchain_aliases.bzl @@ -14,7 +14,8 @@ """Create toolchain alias targets.""" -load("@rules_python//python:versions.bzl", "PLATFORMS") +load("@bazel_skylib//lib:selects.bzl", "selects") +load("//python:versions.bzl", "PLATFORMS") def toolchain_aliases(*, name, platforms, visibility = None, native = native): """Create toolchain aliases for the python toolchains. @@ -30,12 +31,17 @@ def toolchain_aliases(*, name, platforms, visibility = None, native = native): if platform not in platforms: continue + _platform = "_" + platform native.config_setting( - name = platform, - flag_values = PLATFORMS[platform].flag_values, + name = _platform, constraint_values = PLATFORMS[platform].compatible_with, visibility = ["//visibility:private"], ) + selects.config_setting_group( + name = platform, + match_all = PLATFORMS[platform].target_settings + [_platform], + visibility = ["//visibility:private"], + ) prefix = name for name in [ diff --git a/python/private/toolchain_types.bzl b/python/private/toolchain_types.bzl index ef81bf3bd4..a3a52409a6 100644 --- a/python/private/toolchain_types.bzl +++ b/python/private/toolchain_types.bzl @@ -21,3 +21,5 @@ implementation of the toolchain. TARGET_TOOLCHAIN_TYPE = Label("//python:toolchain_type") EXEC_TOOLS_TOOLCHAIN_TYPE = Label("//python:exec_tools_toolchain_type") PY_CC_TOOLCHAIN_TYPE = Label("//python/cc:toolchain_type") +LAUNCHER_MAKER_TOOLCHAIN_TYPE = Label("@bazel_tools//tools/launcher:launcher_maker_toolchain_type") +CC_TOOLCHAIN_TYPE = Label("@bazel_tools//tools/cpp:toolchain_type") diff --git a/python/private/toolchains_repo.bzl b/python/private/toolchains_repo.bzl index 7e9a0c7ff9..1338e9c569 100644 --- a/python/private/toolchains_repo.bzl +++ b/python/private/toolchains_repo.bzl @@ -25,15 +25,163 @@ platform-specific repositories. load( "//python:versions.bzl", + "FREETHREADED", + "MUSL", "PLATFORMS", "WINDOWS_NAME", ) load(":repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "repo_utils") load(":text_util.bzl", "render") -def get_repository_name(repository_workspace): - dummy_label = "//:_" - return str(repository_workspace.relative(dummy_label))[:-len(dummy_label)] or "@" +_SUITE_TEMPLATE = """ +py_toolchain_suite( + flag_values = {flag_values}, + target_settings = {target_settings}, + prefix = {prefix}, + python_version = {python_version}, + set_python_version_constraint = {set_python_version_constraint}, + target_compatible_with = {target_compatible_with}, + user_repository_name = {user_repository_name}, +) +""".lstrip() + +_WORKSPACE_TOOLCHAINS_BUILD_TEMPLATE = """ +# Generated by python/private/toolchains_repo.bzl +# +# These can be registered in the workspace file or passed to --extra_toolchains +# flag. By default all these toolchains are registered by the +# python_register_toolchains macro so you don't normally need to interact with +# these targets. + +load("@@{rules_python}//python/private:py_toolchain_suite.bzl", "py_toolchain_suite") + +""".lstrip() + +_TOOLCHAIN_ALIASES_BUILD_TEMPLATE = """ +# Generated by python/private/toolchains_repo.bzl +load("@rules_python//python/private:toolchain_aliases.bzl", "toolchain_aliases") + +package(default_visibility = ["//visibility:public"]) + +exports_files(["defs.bzl"]) + +PLATFORMS = [ +{loaded_platforms} +] +toolchain_aliases( + name = "{py_repository}", + platforms = PLATFORMS, +) +""".lstrip() + +_TOOLCHAIN_ALIASES_DEFS_TEMPLATE = """ +# Generated by python/private/toolchains_repo.bzl + +load("@@{rules_python}//python:pip.bzl", _compile_pip_requirements = "compile_pip_requirements") +load("@@{rules_python}//python/private:deprecation.bzl", "with_deprecation") +load("@@{rules_python}//python/private:text_util.bzl", "render") +load("@@{rules_python}//python:py_binary.bzl", _py_binary = "py_binary") +load("@@{rules_python}//python:py_test.bzl", _py_test = "py_test") +load( + "@@{rules_python}//python/entry_points:py_console_script_binary.bzl", + _py_console_script_binary = "py_console_script_binary", +) + +def _with_deprecation(kwargs, *, name): + kwargs["python_version"] = "{python_version}" + return with_deprecation.symbol( + kwargs, + symbol_name = name, + old_load = "@{name}//:defs.bzl", + new_load = "@rules_python//python:{{}}.bzl".format(name), + snippet = render.call(name, **{{k: repr(v) for k,v in kwargs.items()}}) + ) + +def py_binary(**kwargs): + return _py_binary(**_with_deprecation(kwargs, name = "py_binary")) + +def py_console_script_binary(**kwargs): + return _py_console_script_binary(**_with_deprecation(kwargs, name = "py_console_script_binary")) + +def py_test(**kwargs): + return _py_test(**_with_deprecation(kwargs, name = "py_test")) + +def compile_pip_requirements(**kwargs): + return _compile_pip_requirements(**_with_deprecation(kwargs, name = "compile_pip_requirements")) +""".lstrip() + +_HOST_TOOLCHAIN_BUILD_CONTENT = """ +# Generated by python/private/toolchains_repo.bzl + +exports_files(["python"], visibility = ["//visibility:public"]) +""".lstrip() + +_HOST_PYTHON_TESTER_TEMPLATE = """ +from pathlib import Path +import sys + +python = Path(sys.executable) +want_python = str(Path("{python}").resolve()) +got_python = str(Path(sys.executable).resolve()) + +assert want_python == got_python, \ + "Expected to use a different interpreter:\\nwant: '{{}}'\\n got: '{{}}'".format( + want_python, + got_python, + ) +""".lstrip() + +_MULTI_TOOLCHAIN_ALIASES_DEFS_TEMPLATE = """ +# Generated by python/private/toolchains_repo.bzl + +load("@@{rules_python}//python:pip.bzl", _compile_pip_requirements = "compile_pip_requirements") +load("@@{rules_python}//python/private:deprecation.bzl", "with_deprecation") +load("@@{rules_python}//python/private:text_util.bzl", "render") +load("@@{rules_python}//python:py_binary.bzl", _py_binary = "py_binary") +load("@@{rules_python}//python:py_test.bzl", _py_test = "py_test") +load( + "@@{rules_python}//python/entry_points:py_console_script_binary.bzl", + _py_console_script_binary = "py_console_script_binary", +) + +def _with_deprecation(kwargs, *, name): + kwargs["python_version"] = "{python_version}" + return with_deprecation.symbol( + kwargs, + symbol_name = name, + old_load = "@{name}//{python_version}:defs.bzl", + new_load = "@rules_python//python:{{}}.bzl".format(name), + snippet = render.call(name, **{{k: repr(v) for k,v in kwargs.items()}}) + ) + +def py_binary(**kwargs): + return _py_binary(**_with_deprecation(kwargs, name = "py_binary")) + +def py_console_script_binary(**kwargs): + return _py_console_script_binary(**_with_deprecation(kwargs, name = "py_console_script_binary")) + +def py_test(**kwargs): + return _py_test(**_with_deprecation(kwargs, name = "py_test")) + +def compile_pip_requirements(**kwargs): + return _compile_pip_requirements(**_with_deprecation(kwargs, name = "compile_pip_requirements")) +""".lstrip() + +_MULTI_TOOLCHAIN_ALIASES_PIP_TEMPLATE = """ +# Generated by python/private/toolchains_repo.bzl + +load("@@{rules_python}//python:pip.bzl", "pip_parse", _multi_pip_parse = "multi_pip_parse") + +def multi_pip_parse(name, requirements_lock, **kwargs): + return _multi_pip_parse( + name = name, + python_versions = {python_versions}, + requirements_lock = requirements_lock, + minor_mapping = {minor_mapping}, + **kwargs + ) + +""".lstrip() def python_toolchain_build_file_content( prefix, @@ -57,43 +205,44 @@ def python_toolchain_build_file_content( build_content: Text containing toolchain definitions """ - return "\n\n".join([ - """\ -py_toolchain_suite( - user_repository_name = "{user_repository_name}_{platform}", - prefix = "{prefix}{platform}", - target_compatible_with = {compatible_with}, - flag_values = {flag_values}, - python_version = "{python_version}", - set_python_version_constraint = "{set_python_version_constraint}", -)""".format( - compatible_with = render.indent(render.list(meta.compatible_with)).lstrip(), - flag_values = render.indent(render.dict( - meta.flag_values, - key_repr = lambda x: repr(str(x)), # this is to correctly display labels - )).lstrip(), - platform = platform, - set_python_version_constraint = set_python_version_constraint, - user_repository_name = user_repository_name, - prefix = prefix, + entries = [] + for platform, meta in loaded_platforms.items(): + entries.append(toolchain_suite_content( + target_compatible_with = meta.compatible_with, + flag_values = meta.flag_values, + prefix = "{}{}".format(prefix, platform), + user_repository_name = "{}_{}".format(user_repository_name, platform), python_version = python_version, - ) - for platform, meta in loaded_platforms.items() - ]) - -def _toolchains_repo_impl(rctx): - build_content = """\ -# Generated by python/private/toolchains_repo.bzl -# -# These can be registered in the workspace file or passed to --extra_toolchains -# flag. By default all these toolchains are registered by the -# python_register_toolchains macro so you don't normally need to interact with -# these targets. + set_python_version_constraint = set_python_version_constraint, + target_settings = meta.target_settings, + )) + return "\n\n".join(entries) -load("@{rules_python}//python/private:py_toolchain_suite.bzl", "py_toolchain_suite") +def toolchain_suite_content( + *, + flag_values, + prefix, + python_version, + set_python_version_constraint, + target_compatible_with, + target_settings, + user_repository_name): + return _SUITE_TEMPLATE.format( + prefix = render.str(prefix), + user_repository_name = render.str(user_repository_name), + target_compatible_with = render.indent(render.list(target_compatible_with)).lstrip(), + flag_values = render.indent(render.dict( + flag_values, + key_repr = lambda x: repr(str(x)), # this is to correctly display labels + )).lstrip(), + target_settings = render.list(target_settings, hanging_indent = " "), + set_python_version_constraint = render.str(set_python_version_constraint), + python_version = render.str(python_version), + ) -""".format( - rules_python = rctx.attr._rules_python_workspace.workspace_name, +def _toolchains_repo_impl(rctx): + build_content = _WORKSPACE_TOOLCHAINS_BUILD_TEMPLATE.format( + rules_python = rctx.attr._rules_python_workspace.repo_name, ) toolchains = python_toolchain_build_file_content( @@ -125,22 +274,7 @@ toolchains_repo = repository_rule( def _toolchain_aliases_impl(rctx): # Base BUILD file for this repository. - build_contents = """\ -# Generated by python/private/toolchains_repo.bzl -load("@rules_python//python/private:toolchain_aliases.bzl", "toolchain_aliases") - -package(default_visibility = ["//visibility:public"]) - -exports_files(["defs.bzl"]) - -PLATFORMS = [ -{loaded_platforms} -] -toolchain_aliases( - name = "{py_repository}", - platforms = PLATFORMS, -) -""".format( + build_contents = _TOOLCHAIN_ALIASES_BUILD_TEMPLATE.format( py_repository = rctx.attr.user_repository_name, loaded_platforms = "\n".join([" \"{}\",".format(p) for p in rctx.attr.platforms]), ) @@ -148,52 +282,10 @@ toolchain_aliases( # Expose a Starlark file so rules can know what host platform we used and where to find an interpreter # when using repository_ctx.path, which doesn't understand aliases. - rctx.file("defs.bzl", content = """\ -# Generated by python/private/toolchains_repo.bzl - -load( - "{rules_python}//python/config_settings:transition.bzl", - _py_binary = "py_binary", - _py_test = "py_test", -) -load( - "{rules_python}//python/entry_points:py_console_script_binary.bzl", - _py_console_script_binary = "py_console_script_binary", -) -load("{rules_python}//python:pip.bzl", _compile_pip_requirements = "compile_pip_requirements") - -def py_binary(name, **kwargs): - return _py_binary( - name = name, - python_version = "{python_version}", - **kwargs - ) - -def py_console_script_binary(name, **kwargs): - return _py_console_script_binary( - name = name, - binary_rule = py_binary, - **kwargs - ) - -def py_test(name, **kwargs): - return _py_test( - name = name, - python_version = "{python_version}", - **kwargs - ) - -def compile_pip_requirements(name, **kwargs): - return _compile_pip_requirements( - name = name, - py_binary = py_binary, - py_test = py_test, - **kwargs - ) - -""".format( + rctx.file("defs.bzl", content = _TOOLCHAIN_ALIASES_DEFS_TEMPLATE.format( + name = rctx.attr.name, python_version = rctx.attr.python_version, - rules_python = get_repository_name(rctx.attr._rules_python_workspace), + rules_python = rctx.attr._rules_python_workspace.repo_name, )) toolchain_aliases = repository_rule( @@ -217,15 +309,11 @@ actions.""", environ = [REPO_DEBUG_ENV_VAR], ) -def _host_toolchain_impl(rctx): - rctx.file("BUILD.bazel", """\ -# Generated by python/private/toolchains_repo.bzl - -exports_files(["python"], visibility = ["//visibility:public"]) -""") +def _host_compatible_python_repo_impl(rctx): + rctx.file("BUILD.bazel", _HOST_TOOLCHAIN_BUILD_CONTENT) os_name = repo_utils.get_platforms_os_name(rctx) - host_platform = _get_host_platform( + impl_repo_name = _get_host_impl_repo_name( rctx = rctx, logger = repo_utils.logger(rctx), python_version = rctx.attr.python_version, @@ -233,10 +321,11 @@ exports_files(["python"], visibility = ["//visibility:public"]) cpu_name = repo_utils.get_platforms_cpu_name(rctx), platforms = rctx.attr.platforms, ) - repo = "@@{py_repository}_{host_platform}".format( - py_repository = rctx.attr.name[:-len("_host")], - host_platform = host_platform, - ) + + # Bzlmod quirk: A repository rule can't, in its **implemention function**, + # resolve an apparent repo name referring to a repo created by the same + # bzlmod extension. To work around this, we use a canonical label. + repo = "@@{}".format(impl_repo_name) rctx.report_progress("Symlinking interpreter files to the target platform") host_python_repo = rctx.path(Label("{repo}//:BUILD.bazel".format(repo = repo))) @@ -268,86 +357,119 @@ exports_files(["python"], visibility = ["//visibility:public"]) # Ensure that we can run the interpreter and check that we are not # using the host interpreter. - python_tester_contents = """\ -from pathlib import Path -import sys - -python = Path(sys.executable) -want_python = str(Path("{python}").resolve()) -got_python = str(Path(sys.executable).resolve()) - -assert want_python == got_python, \ - "Expected to use a different interpreter:\\nwant: '{{}}'\\n got: '{{}}'".format( - want_python, - got_python, + python_tester_contents = _HOST_PYTHON_TESTER_TEMPLATE.format( + repo = repo.strip("@"), + python = python_binary, ) -""".format(repo = repo.strip("@"), python = python_binary) python_tester = rctx.path("python_tester.py") rctx.file(python_tester, python_tester_contents) repo_utils.execute_checked( rctx, op = "CheckHostInterpreter", - arguments = [rctx.path(python_binary), python_tester], + arguments = [ + rctx.path(python_binary), + # Run the interpreter in isolated mode, this options implies -E, -P and -s. + # This ensures that environment variables are ignored that are set in userspace, such as PYTHONPATH, + # which may interfere with this invocation. + "-I", + python_tester, + ], ) if not rctx.delete(python_tester): fail("Failed to delete the python tester") -host_toolchain = repository_rule( - _host_toolchain_impl, +# NOTE: The term "toolchain" is a misnomer for this rule. This doesn't define +# a repo with toolchains or toolchain implementations. +host_compatible_python_repo = repository_rule( + implementation = _host_compatible_python_repo_impl, doc = """\ Creates a repository with a shorter name meant to be used in the repository_ctx, which needs to have `symlinks` for the interpreter. This is separate from the toolchain_aliases repo because referencing the `python` interpreter target from this repo causes an eager fetch of the toolchain for the host platform. - """, + +This repo has two ways in which is it called: + +1. Workspace. The `platforms` attribute is set, which are keys into the + PLATFORMS global. It assumes `name` + is a + valid repo name which it can use as the backing repo. + +2. Bzlmod. All platform and backing repo information is passed in via the + arch_names, impl_repo_names, os_names, python_versions attributes. +""", attrs = { - "platforms": attr.string_list(mandatory = True), - "python_version": attr.string(mandatory = True), - "_rule_name": attr.string(default = "host_toolchain"), + "arch_names": attr.string_dict( + doc = """ +Arch (cpu) names. Only set in bzlmod. Keyed by index in `platforms` +""", + ), + "base_name": attr.string( + doc = """ +The name arg, but without bzlmod canonicalization applied. Only set in bzlmod. +""", + ), + "impl_repo_names": attr.string_dict( + doc = """ +The names of backing runtime repos. Only set in bzlmod. The names must be repos +in the same extension as creates the host repo. Keyed by index in `platforms`. +""", + ), + "os_names": attr.string_dict( + doc = """ +If set, overrides the platform metadata. Only set in bzlmod. Keyed by +index in `platforms` +""", + ), + "platforms": attr.string_list( + mandatory = True, + doc = """ +Platform names (workspace) or platform name-like keys (bzlmod) + +NOTE: The order of this list matters. The first platform that is compatible +with the host will be selected; this can be customized by using the +`RULES_PYTHON_REPO_TOOLCHAIN_*` env vars. + +The values passed vary depending on workspace vs bzlmod. + +Workspace: the values are keys into the `PLATFORMS` dict and are the suffix +to append to `name` to point to the backing repo name. + +Bzlmod: The values are arbitrary keys to create the platform map from the +other attributes (os_name, arch_names, et al). +""", + ), + "python_version": attr.string( + doc = """ +Full python version, Major.Minor.Micro. + +Only set in workspace calls. +""", + ), + "python_versions": attr.string_dict( + doc = """ +If set, the Python version for the corresponding selected platform. Values in +Major.Minor.Micro format. Keyed by index in `platforms`. +""", + ), + "_rule_name": attr.string(default = "host_compatible_python_repo"), "_rules_python_workspace": attr.label(default = Label("//:WORKSPACE")), }, ) def _multi_toolchain_aliases_impl(rctx): - rules_python = rctx.attr._rules_python_workspace.workspace_name + rules_python = rctx.attr._rules_python_workspace.repo_name for python_version, repository_name in rctx.attr.python_versions.items(): file = "{}/defs.bzl".format(python_version) - rctx.file(file, content = """\ -# Generated by python/private/toolchains_repo.bzl - -load( - "@{repository_name}//:defs.bzl", - _compile_pip_requirements = "compile_pip_requirements", - _py_binary = "py_binary", - _py_console_script_binary = "py_console_script_binary", - _py_test = "py_test", -) - -compile_pip_requirements = _compile_pip_requirements -py_binary = _py_binary -py_console_script_binary = _py_console_script_binary -py_test = _py_test -""".format( + rctx.file(file, content = _MULTI_TOOLCHAIN_ALIASES_DEFS_TEMPLATE.format( repository_name = repository_name, + name = rctx.attr.name, + python_version = python_version, + rules_python = rules_python, )) rctx.file("{}/BUILD.bazel".format(python_version), "") - pip_bzl = """\ -# Generated by python/private/toolchains_repo.bzl - -load("@{rules_python}//python:pip.bzl", "pip_parse", _multi_pip_parse = "multi_pip_parse") - -def multi_pip_parse(name, requirements_lock, **kwargs): - return _multi_pip_parse( - name = name, - python_versions = {python_versions}, - requirements_lock = requirements_lock, - minor_mapping = {minor_mapping}, - **kwargs - ) - -""".format( + pip_bzl = _MULTI_TOOLCHAIN_ALIASES_PIP_TEMPLATE.format( python_versions = rctx.attr.python_versions.keys(), minor_mapping = render.indent(render.dict(rctx.attr.minor_mapping), indent = " " * 8).lstrip(), rules_python = rules_python, @@ -364,10 +486,57 @@ multi_toolchain_aliases = repository_rule( }, ) -def sanitize_platform_name(platform): - return platform.replace("-", "_") +def sorted_host_platform_names(platform_names): + """Sort platform names to give correct precedence. + + The order of keys in the platform mapping matters for the host toolchain + selection. When multiple runtimes are compatible with the host, we take the + first that is compatible (usually; there's also the + `RULES_PYTHON_REPO_TOOLCHAIN_*` environment variables). The historical + behavior carefully constructed the ordering of platform keys such that + the ordering was: + * Regular platforms + * The "-freethreaded" suffix + * The "-musl" suffix + + Here, we formalize that so it isn't subtly encoded in the ordering of keys + in a dict that autoformatters like to clobber and whose only documentation + is an innocous looking formatter disable directive. + + Args: + platform_names: a list of platform names + + Returns: + list[str] the same values, but in the desired order. + """ + + def platform_keyer(name): + # Ascending sort: lower is higher precedence + return ( + 1 if MUSL in name else 0, + 1 if FREETHREADED in name else 0, + ) + + return sorted(platform_names, key = platform_keyer) + +def sorted_host_platforms(platform_map): + """Sort the keys in the platform map to give correct precedence. + + See sorted_host_platform_names for explanation. -def _get_host_platform(*, rctx, logger, python_version, os_name, cpu_name, platforms): + Args: + platform_map: a mapping of platforms and their metadata. + + Returns: + dict; the same values, but with the keys inserted in the desired + order so that iteration happens in the desired order. + """ + return { + key: platform_map[key] + for key in sorted_host_platform_names(platform_map.keys()) + } + +def _get_host_impl_repo_name(*, rctx, logger, python_version, os_name, cpu_name, platforms): """Gets the host platform. Args: @@ -380,15 +549,42 @@ def _get_host_platform(*, rctx, logger, python_version, os_name, cpu_name, platf Returns: The host platform. """ + if rctx.attr.os_names: + platform_map = {} + base_name = rctx.attr.base_name + if not base_name: + fail("The `base_name` attribute must be set under bzlmod") + for i, platform_name in enumerate(platforms): + key = str(i) + impl_repo_name = rctx.attr.impl_repo_names[key] + impl_repo_name = rctx.name.replace(base_name, impl_repo_name) + platform_map[platform_name] = struct( + os_name = rctx.attr.os_names[key], + arch = rctx.attr.arch_names[key], + python_version = rctx.attr.python_versions[key], + impl_repo_name = impl_repo_name, + ) + else: + base_name = rctx.name.removesuffix("_host") + platform_map = {} + for platform_name, info in sorted_host_platforms(PLATFORMS).items(): + platform_map[platform_name] = struct( + os_name = info.os_name, + arch = info.arch, + python_version = python_version, + impl_repo_name = "{}_{}".format(base_name, platform_name), + ) + candidates = [] for platform in platforms: - meta = PLATFORMS[platform] + meta = platform_map[platform] if meta.os_name == os_name and meta.arch == cpu_name: - candidates.append(platform) + candidates.append((platform, meta)) if len(candidates) == 1: - return candidates[0] + platform_name, meta = candidates[0] + return meta.impl_repo_name if candidates: env_var = "RULES_PYTHON_REPO_TOOLCHAIN_{}_{}_{}".format( @@ -396,7 +592,7 @@ def _get_host_platform(*, rctx, logger, python_version, os_name, cpu_name, platf os_name.upper(), cpu_name.upper(), ) - preference = repo_utils.getenv(rctx, env_var) + preference = rctx.getenv(env_var) if preference == None: logger.info("Consider using '{}' to select from one of the platforms: {}".format( env_var, @@ -408,7 +604,11 @@ def _get_host_platform(*, rctx, logger, python_version, os_name, cpu_name, platf candidates = [preference] if candidates: - return candidates[0] + platform_name, meta = candidates[0] + suffix = meta.impl_repo_name + if not suffix: + suffix = platform_name + return suffix return logger.fail("Could not find a compatible 'host' python for '{os_name}', '{cpu_name}' from the loaded platforms: {platforms}".format( os_name = os_name, diff --git a/python/private/tools/sort_manifest.py b/python/private/tools/sort_manifest.py new file mode 100755 index 0000000000..41bf4181e5 --- /dev/null +++ b/python/private/tools/sort_manifest.py @@ -0,0 +1,123 @@ +#!/usr/bin/env python3 + +"""Sorts python-build-standalone manifest files by filename.""" + +import argparse +import sys +from pathlib import Path + + +def sort_manifest(manifest_path: Path) -> bool: + """Sorts a manifest file in place by filename. Returns True if modified.""" + # Read using pathlib.Path + lines = manifest_path.read_text(encoding="utf-8").splitlines(keepends=True) + + if not lines: + return False + + first_entry_idx = -1 + last_entry_idx = -1 + for idx, line in enumerate(lines): + stripped = line.strip() + if stripped and not stripped.startswith("#"): + if first_entry_idx == -1: + first_entry_idx = idx + last_entry_idx = idx + + if first_entry_idx == -1: + return False + + # Extract top-level comments (comments at the top followed by a blank newline) + top_level_comments = [] + pre_entry_lines = lines[:first_entry_idx] + + last_blank_idx = -1 + for idx, line in enumerate(pre_entry_lines): + if not line.strip(): + last_blank_idx = idx + + if last_blank_idx != -1: + top_level_comments = pre_entry_lines[: last_blank_idx + 1] + remaining_pre = pre_entry_lines[last_blank_idx + 1 :] + else: + remaining_pre = pre_entry_lines + + # Extract bottom-level comments + bottom_level_comments = lines[last_entry_idx + 1 :] + + # Group middle lines into actual catalog entries with their attached comments/blank lines + middle_lines = remaining_pre + lines[first_entry_idx : last_entry_idx + 1] + + entries = [] + current_attached = [] + + for line in middle_lines: + stripped = line.strip() + if stripped and not stripped.startswith("#"): + parts = [p for p in stripped.split(" ") if p] + if len(parts) == 2: + sha256, filename = parts[0], parts[1] + normalized_line = f"{sha256} {filename}\n" + else: + filename = parts[0] if parts else "" + normalized_line = line + + block = current_attached + [normalized_line] + entries.append((filename, block)) + current_attached = [] + else: + current_attached.append(line) + + if current_attached: + bottom_level_comments = current_attached + bottom_level_comments + + # Sort entries lexicographically by filename + entries.sort(key=lambda e: e[0]) + + new_lines = top_level_comments + for _, block in entries: + new_lines.extend(block) + new_lines.extend(bottom_level_comments) + + if new_lines == lines: + return False + + manifest_path.write_text("".join(new_lines), encoding="utf-8") + return True + + +def main(): + parser = argparse.ArgumentParser(description="Sort manifest files by filename.") + parser.add_argument( + "manifests", + nargs="*", + type=Path, + help="Path to manifest files to sort.", + ) + args = parser.parse_args() + + manifests = args.manifests + if not manifests: + repo_root = Path(__file__).resolve().parent.parent.parent.parent + default_manifest = repo_root / "python" / "private" / "runtimes_manifest.txt" + if default_manifest.exists(): + manifests = [default_manifest] + else: + print("No manifests provided.", file=sys.stderr) + sys.exit(1) + + changed = False + for m in manifests: + if m.exists(): + if sort_manifest(m): + print(f"Sorted {m}") + changed = True + else: + print(f"Warning: Manifest not found: {m}", file=sys.stderr) + + if changed: + sys.exit(1) + + +if __name__ == "__main__": + main() diff --git a/python/private/tools/sync_runtimes_manifest_workspace.py b/python/private/tools/sync_runtimes_manifest_workspace.py new file mode 100755 index 0000000000..450358542a --- /dev/null +++ b/python/private/tools/sync_runtimes_manifest_workspace.py @@ -0,0 +1,80 @@ +#!/usr/bin/env python3 + +"""Synchronizes runtimes_manifest_workspace.bzl with runtimes_manifest.txt.""" + +import argparse +import sys +from pathlib import Path + + +def sync_workspace_manifest(txt_path: Path, bzl_path: Path) -> bool: + with open(txt_path, "r", encoding="utf-8") as f: + txt_content = f.read() + header = '''"""Manifest of runtimes for workspace mode builds. + +This is the workspace equivalent of runtimes_manifest.txt. It's a bzl file +to simplify loading of the data under workspace mode, which doesn't +support parsing a runtimes_manifest.txt file. + +NOTE: This file is automatically generated by sync_runtimes_manifest_workspace.py. +Do not edit directly! +""" + +MANIFEST_TEXT = """ +''' + + new_content = header + txt_content + '"""\n' + + if bzl_path.exists(): + with open(bzl_path, "r", encoding="utf-8") as f: + old_content = f.read() + else: + old_content = "" + + if new_content != old_content: + with open(bzl_path, "w", encoding="utf-8", newline="\n") as f: + f.write(new_content) + return True + + return False + + +def main(): + parser = argparse.ArgumentParser( + description="Sync runtimes workspace bzl file with runtimes manifest text file." + ) + parser.add_argument( + "txt_path", + nargs="?", + type=Path, + help="Path to runtimes_manifest.txt", + ) + parser.add_argument( + "bzl_path", + nargs="?", + type=Path, + help="Path to runtimes_manifest_workspace.bzl", + ) + args = parser.parse_args() + + txt_path = args.txt_path + bzl_path = args.bzl_path + + if not txt_path or not bzl_path: + repo_root = Path(__file__).resolve().parent.parent.parent.parent + txt_path = repo_root / "python" / "private" / "runtimes_manifest.txt" + bzl_path = repo_root / "python" / "private" / "runtimes_manifest_workspace.bzl" + + if not txt_path.exists(): + print(f"Error: Manifest not found: {txt_path}", file=sys.stderr) + sys.exit(1) + + if sync_workspace_manifest(txt_path, bzl_path): + print(f"Updated {bzl_path}") + if not args.bzl_path: + # Exit 1 for pre-commit mode (in-place modification) + sys.exit(1) + + +if __name__ == "__main__": + main() diff --git a/python/private/transition_labels.bzl b/python/private/transition_labels.bzl new file mode 100644 index 0000000000..d337044cb5 --- /dev/null +++ b/python/private/transition_labels.bzl @@ -0,0 +1,31 @@ +"""Flags that terminal rules should allow transitioning on by default. + +Terminal rules are e.g. py_binary, py_test, or packaging rules. +""" + +load("@bazel_skylib//lib:collections.bzl", "collections") +load("@rules_python_internal//:extra_transition_settings.bzl", "EXTRA_TRANSITION_SETTINGS") +load(":common_labels.bzl", "labels") + +_BASE_TRANSITION_LABELS = [ + labels.ADD_SRCS_TO_RUNFILES, + labels.BOOTSTRAP_IMPL, + labels.DEBUGGER, + labels.EXEC_TOOLS_TOOLCHAIN, + "//command_line_option:extra_toolchains", + labels.PIP_ENV_MARKER_CONFIG, + labels.PIP_WHL_OSX_VERSION, + labels.PRECOMPILE, + labels.PRECOMPILE_SOURCE_RETENTION, + labels.PYTHON_SRC, + labels.PYTHON_VERSION, + labels.PY_FREETHREADED, + labels.PY_LINUX_LIBC, + labels.VENV, + labels.VENVS_SITE_PACKAGES, + labels.VENVS_USE_DECLARE_SYMLINK, +] + +TRANSITION_LABELS = collections.uniq( + _BASE_TRANSITION_LABELS + EXTRA_TRANSITION_SETTINGS, +) diff --git a/python/private/uncachable_version_file.bzl b/python/private/uncachable_version_file.bzl new file mode 100644 index 0000000000..0f58df8043 --- /dev/null +++ b/python/private/uncachable_version_file.bzl @@ -0,0 +1,40 @@ +"""Implementation of uncachable_version_file.""" + +load(":py_internal.bzl", "py_internal") + +def _uncachable_version_file_impl(ctx): + version_file = ctx.actions.declare_file("uncachable_version_file.txt") + py_internal.copy_without_caching( + ctx = ctx, + # NOTE: ctx.version_file is undocumented; see + # https://github.com/bazelbuild/bazel/issues/9363 + # NOTE: Even though the version file changes every build (it contains + # the build timestamp), it is ignored when computing what inputs + # changed. See https://bazel.build/docs/user-manual#workspace-status + read_from = ctx.version_file, + write_to = version_file, + ) + return [DefaultInfo( + files = depset([version_file]), + )] + +uncachable_version_file = rule( + doc = """ +Creates a copy of `ctx.version_file`, except it isn't ignored by +Bazel's change-detecting logic. In fact, it's the opposite: +caching is disabled for the action generating this file, so any +actions depending on this file will always re-run. +""", + implementation = _uncachable_version_file_impl, +) + +def define_uncachable_version_file(name, visibility = None): + native.alias( + name = name, + actual = select({ + ":stamp_detect": ":uncachable_version_file_impl", + "//conditions:default": ":sentinel", + }), + visibility = visibility, + ) + uncachable_version_file(name = "uncachable_version_file_impl") diff --git a/python/private/util.bzl b/python/private/util.bzl index 33261befaf..31f317fedf 100644 --- a/python/private/util.bzl +++ b/python/private/util.bzl @@ -15,7 +15,7 @@ """Functionality shared by multiple pieces of code.""" load("@bazel_skylib//lib:types.bzl", "types") -load("@rules_python_internal//:rules_python_config.bzl", "config") +load("//python/private:py_internal.bzl", "py_internal") def copy_propagating_kwargs(from_kwargs, into_kwargs = None): """Copies args that must be compatible between two targets with a dependency relationship. @@ -42,7 +42,7 @@ def copy_propagating_kwargs(from_kwargs, into_kwargs = None): into_kwargs = {} # Include tags because people generally expect tags to propagate. - for attr in ("testonly", "tags", "compatible_with", "restricted_to"): + for attr in ("testonly", "tags", "compatible_with", "restricted_to", "target_compatible_with"): if attr in from_kwargs and attr not in into_kwargs: into_kwargs[attr] = from_kwargs[attr] return into_kwargs @@ -50,21 +50,6 @@ def copy_propagating_kwargs(from_kwargs, into_kwargs = None): # The implementation of the macros and tagging mechanism follows the example # set by rules_cc and rules_java. -_MIGRATION_TAG = "__PYTHON_RULES_MIGRATION_DO_NOT_USE_WILL_BREAK__" - -def add_migration_tag(attrs): - """Add a special tag to `attrs` to aid migration off native rles. - - Args: - attrs: dict of keyword args. The `tags` key will be modified in-place. - - Returns: - The same `attrs` object, but modified. - """ - if not config.enable_pystar: - add_tag(attrs, _MIGRATION_TAG) - return attrs - def add_tag(attrs, tag): """Adds `tag` to `attrs["tags"]`. @@ -86,36 +71,17 @@ def add_tag(attrs, tag): else: attrs["tags"] = [tag] -# Helper to make the provider definitions not crash under Bazel 5.4: -# Bazel 5.4 doesn't support the `init` arg of `provider()`, so we have to -# not pass that when using Bazel 5.4. But, not passing the `init` arg -# changes the return value from a two-tuple to a single value, which then -# breaks Bazel 6+ code. -# This isn't actually used under Bazel 5.4, so just stub out the values -# to get past the loading phase. -def define_bazel_6_provider(doc, fields, **kwargs): - """Define a provider, or a stub for pre-Bazel 7.""" - if not IS_BAZEL_6_OR_HIGHER: - return provider("Stub, not used", fields = []), None - return provider(doc = doc, fields = fields, **kwargs) - -IS_BAZEL_7_4_OR_HIGHER = hasattr(native, "legacy_globals") - -IS_BAZEL_7_OR_HIGHER = hasattr(native, "starlark_doc_extract") - -# Bazel 5.4 has a bug where every access of testing.ExecutionInfo is a -# different object that isn't equal to any other. This is fixed in bazel 6+. -IS_BAZEL_6_OR_HIGHER = testing.ExecutionInfo == testing.ExecutionInfo - -_marker_rule_to_detect_bazel_6_4_or_higher = rule(implementation = lambda ctx: None) - -# Bazel 6.4 and higher have a bug fix where rule names show up in the str() -# of a rule. See -# https://github.com/bazelbuild/bazel/commit/002490b9a2376f0b2ea4a37102c5e94fc50a65ba -# https://github.com/bazelbuild/bazel/commit/443cbcb641e17f7337ccfdecdfa5e69bc16cae55 -# This technique is done instead of using native.bazel_version because, -# under stardoc, the native.bazel_version attribute is entirely missing, which -# prevents doc generation from being able to correctly generate docs. -IS_BAZEL_6_4_OR_HIGHER = "_marker_rule_to_detect_bazel_6_4_or_higher" in str( - _marker_rule_to_detect_bazel_6_4_or_higher, -) +def is_importable_name(name): + # Requires Bazel 8+ + if hasattr(py_internal, "regex_match"): + # ?U means activates unicode matching (Python allows most unicode + # in module names / identifiers). + # \w matches alphanumeric and underscore. + # NOTE: regex_match has an implicit ^ and $ + return py_internal.regex_match(name, "(?U)\\w+") + else: + # Otherwise, use a rough hueristic that should catch most cases. + return ( + "." not in name and + "-" not in name + ) diff --git a/python/private/venv_runfiles.bzl b/python/private/venv_runfiles.bzl new file mode 100644 index 0000000000..45d4d24848 --- /dev/null +++ b/python/private/venv_runfiles.bzl @@ -0,0 +1,550 @@ +"""Code for constructing venvs.""" + +load("@bazel_skylib//lib:paths.bzl", "paths") +load( + ":common.bzl", + "ExplicitSymlink", + "is_file", + "is_windows_platform", + "relative_path", + "runfiles_root_path", +) +load( + ":py_info.bzl", + "PyInfo", + "VenvSymlinkEntry", + "VenvSymlinkKind", +) +load(":util.bzl", "is_importable_name") + +# List of top-level package names that are known to be namespace +# packages, but cannot be detected as such automatically. +_WELL_KNOWN_NAMESPACE_PACKAGES = [ + # nvidia wheels incorrectly use an empty `__init__.py` file, even + # though multiple distributions install into the directory. + "nvidia", +] + +def create_venv_app_files(ctx, deps, venv_dir_map): + """Creates the tree of app-specific files for a venv for a binary. + + App specific files are the files that come from dependencies. + + Args: + ctx: {type}`ctx` current ctx. + deps: {type}`list[Target]` the targets whose venv information + to put into the returned venv files. + venv_dir_map: mapping of VenvSymlinkKind constants to the + venv path. This tells the directory name of + platform/configuration-dependent directories. The values are + paths within the current ctx's venv (e.g. `_foo.venv/bin`). + + Returns: + {type}`struct` with the following attributes: + * {type}`list[File]` `venv_files` additional files created for + the venv. + * {type}`dict[str, File]` `runfiles_symlinks` map intended for + the `runfiles.symlinks` argument. A map of main-repo + relative paths to File. + """ + + # maps venv-relative path to the runfiles path it should point to + entries = depset( + transitive = [ + dep[PyInfo].venv_symlinks + for dep in deps + if PyInfo in dep + ], + ).to_list() + + link_map = build_link_map(ctx, entries) + venv_files = [] + runfiles_symlinks = {} + explicit_symlinks = [] + + is_windows = is_windows_platform(ctx) + + ctx_rf_path = paths.join( + ctx.label.repo_name or ctx.workspace_name, + ctx.label.package, + ) + + seen_bin_venv_paths = {} + + for kind, kind_map in link_map.items(): + base = venv_dir_map[kind] + for venv_path, link_to in kind_map.items(): + bin_venv_path = paths.join(base, venv_path) + if bin_venv_path in seen_bin_venv_paths: + continue + seen_bin_venv_paths[bin_venv_path] = True + + if is_file(link_to): + # use paths.join to handle ctx.label.package = "" + # runfile_prefix should be prepended as we use runfiles.root_symlinks + runfile_prefix = ctx.label.repo_name or ctx.workspace_name + symlink_from = paths.join(runfile_prefix, ctx.label.package, bin_venv_path) + + runfiles_symlinks[symlink_from] = link_to + + # On Windows, we need to explicitly create the symlink in the venv + # because the bootstrap script won't otherwise know about it. + if is_windows: + rf_path = paths.join(ctx_rf_path, bin_venv_path) + _, _, venv_path = bin_venv_path.partition(".venv/") + explicit_symlinks.append(ExplicitSymlink( + runfiles_path = rf_path, + venv_path = venv_path, + link_to_path = runfiles_root_path(ctx, link_to.short_path), + files = depset([link_to]), + )) + elif not is_windows: + venv_link = ctx.actions.declare_symlink(bin_venv_path) + venv_link_rf_path = runfiles_root_path(ctx, venv_link.short_path) + rel_path = relative_path( + # dirname is necessary because a relative symlink is relative to + # the directory the symlink resides within. + from_ = paths.dirname(venv_link_rf_path), + to = link_to, + ) + ctx.actions.symlink(output = venv_link, target_path = rel_path) + venv_files.append(venv_link) + else: + rf_path = paths.join(ctx_rf_path, bin_venv_path) + _, _, venv_path = bin_venv_path.partition(".venv/") + explicit_symlinks.append(ExplicitSymlink( + runfiles_path = rf_path, + venv_path = venv_path, + link_to_path = link_to, + files = depset(), + )) + + return struct( + venv_files = venv_files, + runfiles_symlinks = runfiles_symlinks, + explicit_symlinks = depset(explicit_symlinks), + ) + +# Visible for testing +def build_link_map(ctx, entries, return_conflicts = False): + """Compute the mapping of venv paths to their backing objects. + + Args: + ctx: {type}`ctx` current ctx. + entries: {type}`list[VenvSymlinkEntry]` the entries that describe the + venv-relative + return_conflicts: {type}`bool`. Only present for testing. If True, + also return a list of the groups that had overlapping paths and had + to be resolved and merged. + + Returns: + {type}`dict[str, dict[str, str|File]]` Mappings of venv paths to their + backing files. The first key is a `VenvSymlinkKind` value. + The inner dict keys are venv paths relative to the kind's directory. The + inner dict values are strings or Files to link to. + """ + + version_by_pkg = {} # dict[str pkg, str version] + entries_by_kind = {} # dict[str kind, list[entry]] + + # Group by path kind and reduce to a single package's version of entries + for entry in entries: + entries_by_kind.setdefault(entry.kind, []) + if not entry.package: + entries_by_kind[entry.kind].append(entry) + continue + if entry.package not in version_by_pkg: + version_by_pkg[entry.package] = entry.version + entries_by_kind[entry.kind].append(entry) + continue + if entry.version == version_by_pkg[entry.package]: + entries_by_kind[entry.kind].append(entry) + continue + + # else: ignore it; not the selected version + + # final paths to keep, grouped by kind + keep_link_map = {} # dict[str kind, dict[path, str|File]] + conflicts = [] if return_conflicts else None + for kind, entries in entries_by_kind.items(): + # dict[str kind-relative path, str|File link_to] + keep_kind_link_map = {} + + groups = _group_venv_path_entries(entries) + + for group in groups: + # If there's just one group, we can symlink to the directory + if len(group) == 1: + entry = group[0] + if entry.link_to_file: + keep_kind_link_map[entry.venv_path] = entry.link_to_file + else: + keep_kind_link_map[entry.venv_path] = entry.link_to_path + else: + if return_conflicts: + conflicts.append(group) + + # Merge a group of overlapping prefixes + _merge_venv_path_group(ctx, group, keep_kind_link_map) + + keep_link_map[kind] = keep_kind_link_map + if return_conflicts: + return keep_link_map, conflicts + else: + return keep_link_map + +def _group_venv_path_entries(entries): + """Group entries by VenvSymlinkEntry.venv_path overlap. + + This does an initial grouping by the top-level venv path an entry wants. + Entries that are underneath another entry are put into the same group. + + Returns: + {type}`list[list[VenvSymlinkEntry]]` The inner list is the entries under + a common venv path. The inner list is ordered from shortest to longest + path. + """ + + # Sort so order is top-down, ensuring grouping by short common prefix + # Split it into path components so `foo foo-bar foo/bar` sorts as + # `foo foo/bar foo-bar` + entries = sorted(entries, key = lambda e: tuple(e.venv_path.split("/"))) + + groups = [] + current_group = None + current_group_prefix = None + for entry in entries: + # NOTE: When a file is being directly linked, the anchored prefix can look + # odd, e.g. "foo/__init__.py/". This is OK; it's just used to prevent + # incorrect prefix substring matching. + anchored_prefix = entry.venv_path + "/" + if (current_group_prefix == None or + not anchored_prefix.startswith(current_group_prefix)): + current_group_prefix = anchored_prefix + current_group = [entry] + groups.append(current_group) + else: + current_group.append(entry) + + return groups + +def _merge_venv_path_group(ctx, group, keep_map): + """Merges a group of overlapping prefixes. + + Args: + ctx: {type}`ctx` current ctx. + group: {type}`list[VenvSymlinkEntry]` a group of entries with overlapping + `venv_path` prefixes, ordered from shortest to longest path. + keep_map: {type}`dict[str, str|File]` files kept after merging are + populated into this map. + """ + + # TODO: Compute the minimum number of entries to create. This can't avoid + # flattening the files depset, but can lower the number of materialized + # files significantly. Usually overlaps are limited to a small number + # of directories. Note that, when doing so, shared libraries need to + # be symlinked directly, not the directory containing them, due to + # dynamic linker symlink resolution semantics on Linux. + for entry in group: + root_venv_path = entry.venv_path + anchored_link_to_path = entry.link_to_path + "/" + for file in entry.files.to_list(): + rf_root_path = runfiles_root_path(ctx, file.short_path) + + # It's a file (or directory) being directly linked and + # must be directly linked. + if rf_root_path == entry.link_to_path: + # For lack of a better option, first added wins. + if entry.venv_path not in keep_map: + keep_map[entry.venv_path] = file + + # Skip anything remaining: anything left is either + # the same path (first set wins), a suffix (violates + # preconditions and can't link anyways), or not under + # the prefix (violates preconditions). + break + else: + # Compute the file-specific venv path. i.e. the relative + # path of the file under entry.venv_path, joined with + # entry.venv_path + head, match, rel_venv_path = rf_root_path.partition(anchored_link_to_path) + if not match or head: + # If link_to_path didn't match, then obviously skip. + # If head is non-empty, it means link_to_path wasn't + # found at the start + # This shouldn't occur in practice, but guard against it + # just in case + continue + + venv_path = paths.join(root_venv_path, rel_venv_path) + + # For lack of a better option, first added wins. + if venv_path not in keep_map: + keep_map[venv_path] = file + +def _get_file_venv_path(ctx, f, site_packages_root): + """Computes a file's venv_path if it's under the site_packages_root. + + Args: + ctx: The current ctx. + f: The file to compute the venv_path for. + site_packages_root: The site packages root path; repo-relative + path. + + Returns: + A tuple `(venv_path, rf_root_path)` if the file is under + `site_packages_root` or data/, bin/, include/ otherwise `(None, None)`. + """ + rf_root_path = runfiles_root_path(ctx, f.short_path) + _, _, repo_rel_path = rf_root_path.partition("/") + + # Check for wheel data/bin/include folders first + for prefix in ["data/", "bin/", "include/"]: + if repo_rel_path.startswith(prefix): + return (repo_rel_path, rf_root_path) + + head, found_sp_root, venv_path = repo_rel_path.partition(site_packages_root) + if head or not found_sp_root: + # If head is set, then the path didn't start with site_packages_root + # if found_sp_root is empty, then it means it wasn't found at all. + return (None, None) + return (venv_path, rf_root_path) + +def get_venv_symlinks( + ctx, + files, + package, + version_str, + site_packages_root, + namespace_package_files = []): + """Compute the VenvSymlinkEntry objects for a library. + + Args: + ctx: {type}`ctx` the current ctx. + files: {type}`list[File]` the underlying files that are under + `site_packages_root` and intended to be part of the venv + contents. + package: {type}`str` the Python distribution name. + version_str: {type}`str` the distribution's version. + site_packages_root: {type}`str` prefix under which files are + considered to be part of the installed files. + namespace_package_files: {type}`list[File]` a list of files + that are pkgutil-style namespace packages and cannot be + directly linked. + + Returns: + {type}`list[VenvSymlinkEntry]` the entries that describe how + to map the files into a venv. + """ + if site_packages_root.endswith("/"): + fail("The `site_packages_root` value cannot end in " + + "slash, got {}".format(site_packages_root)) + if site_packages_root.startswith("/"): + fail("The `site_packages_root` cannot start with " + + "slash, got {}".format(site_packages_root)) + + # Append slash to prevent incorrect prefix-string matches + site_packages_root += "/" + + all_files = sorted(files, key = lambda f: f.short_path) + + cannot_be_linked_directly = {} + for dirname in [ + # The venv directories that bin, include, and data get put into are + # shared across wheels, so we cannot link them directly + "bin", + "include", + "data", + # The data scheme is overlaid on the venv root, so the files under it + # could, in theory, get installed into e.g. bin/ or similar. Explicitly + # mark them as non-directly linkable to avoid issues. + "data/bin", + "data/include", + "data/lib", + "data/Scripts", + "data/Include", + "data/Lib", + ]: + cannot_be_linked_directly[dirname] = True + + # dict[str venv-relative dirname, bool is_namespace_package] + namespace_package_dirs = { + ns: True + for ns in _WELL_KNOWN_NAMESPACE_PACKAGES + } + + # venv paths that cannot be directly linked. Dict acting as set. + cannot_be_linked_directly.update({ + dirname: True + for dirname in namespace_package_dirs.keys() + }) + for f in namespace_package_files: + venv_path, _ = _get_file_venv_path(ctx, f, site_packages_root) + if venv_path == None: + continue + ns_dir = paths.dirname(venv_path) + namespace_package_dirs[ns_dir] = True + cannot_be_linked_directly[ns_dir] = True + + # dict[str path, VenvSymlinkEntry] + # Where path is the venv path (i.e. relative to site_packages_prefix) + venv_symlinks = {} + + # List of (File, str venv_path) tuples + files_left_to_link = [] + + # We want to minimize the number of files symlinked. Ideally, only the + # top-level directories are symlinked. Unfortunately, shared libraries + # complicate matters: if a shared library's directory is linked, then the + # dynamic linker computes the wrong search path. + # + # To fix, we have to directly link shared libraries. This then means that + # all the parent directories of the shared library can't be linked + # directly. + for src in all_files: + venv_path, rf_root_path = _get_file_venv_path(ctx, src, site_packages_root) + if venv_path == None: + continue + + filename = paths.basename(venv_path) + if _is_linker_loaded_library(filename): + venv_symlinks[venv_path] = VenvSymlinkEntry( + kind = VenvSymlinkKind.LIB, + link_to_path = rf_root_path, + link_to_file = src, + package = package, + version = version_str, + files = depset([src]), + venv_path = venv_path, + ) + parent = paths.dirname(venv_path) + for _ in range(len(venv_path) + 1): # Iterate enough times to traverse up + if not parent: + break + if cannot_be_linked_directly.get(parent, False): + # Already seen + break + cannot_be_linked_directly[parent] = True + parent = paths.dirname(parent) + else: + files_left_to_link.append((src, venv_path)) + + top_level_dirname, _, tail = venv_path.partition("/") + if ( + # If it's already not directly linkable, nothing to do + not cannot_be_linked_directly.get(top_level_dirname, False) and + # If its already known to be non-implicit namespace, then skip + namespace_package_dirs.get(top_level_dirname, True) and + # It must be an importable name to be an implicit namespace package + is_importable_name(top_level_dirname) + ): + namespace_package_dirs.setdefault(top_level_dirname, True) + + # Looking for `__init__.` isn't 100% correct, as it'll match e.g. + # `__init__.pyi`, but it's close enough. + if "/" not in tail and tail.startswith("__init__."): + namespace_package_dirs[top_level_dirname] = False + + # We treat namespace packages as a hint that other distributions may + # install into the same directory. As such, we avoid linking them directly + # to avoid conflict merging later. + for dirname, is_namespace_package in namespace_package_dirs.items(): + if is_namespace_package: + # If it's already in cannot_be_linked_directly due to pkgutil_namespace_packages + # then we should not unset it. + if not cannot_be_linked_directly.get(dirname, False): + cannot_be_linked_directly[dirname] = True + + # At this point, venv_symlinks has entries for the shared libraries + # and cannot_be_linked_directly has the directories that cannot be + # directly linked. Next, we loop over the remaining files and group + # them into the highest level directory that can be linked. + + # dict[str venv_path, list[File]] + optimized_groups = {} + + for src, venv_path in files_left_to_link: + parent = paths.dirname(venv_path) + if not parent: + # File in root, must be linked directly + optimized_groups.setdefault(venv_path, []) + optimized_groups[venv_path].append(src) + continue + + if parent in cannot_be_linked_directly: + # File in a directory that cannot be directly linked, + # so link the file directly + optimized_groups.setdefault(venv_path, []) + optimized_groups[venv_path].append(src) + else: + # This path can be grouped. Find the highest-level directory to link. + venv_path = parent + next_parent = paths.dirname(parent) + for _ in range(len(venv_path) + 1): # Iterate enough times + if next_parent: + if next_parent not in cannot_be_linked_directly: + venv_path = next_parent + next_parent = paths.dirname(next_parent) + else: + break + else: + break + + optimized_groups.setdefault(venv_path, []) + optimized_groups[venv_path].append(src) + + # Finally, for each group, we create the VenvSymlinkEntry objects + for venv_path, files in optimized_groups.items(): + if venv_path.startswith("data/"): + out_venv_path = venv_path[len("data/"):] + kind = VenvSymlinkKind.DATA + prefix = "" + elif venv_path.startswith("include/"): + out_venv_path = venv_path[len("include/"):] + kind = VenvSymlinkKind.INCLUDE + prefix = "" + elif venv_path.startswith("bin/"): + out_venv_path = venv_path[len("bin/"):] + kind = VenvSymlinkKind.BIN + prefix = "" + else: + out_venv_path = venv_path + kind = VenvSymlinkKind.LIB + prefix = site_packages_root + + link_to_path = ( + _get_label_runfiles_repo(ctx, files[0].owner) + + "/" + + prefix + + venv_path + ) + venv_symlinks[venv_path] = VenvSymlinkEntry( + kind = kind, + link_to_path = link_to_path, + link_to_file = files[0] if kind == VenvSymlinkKind.BIN and len(files) == 1 else None, + package = package, + version = version_str, + venv_path = out_venv_path, + files = depset(files), + ) + + return venv_symlinks.values() + +def _is_linker_loaded_library(filename): + """Tells if a filename is one that `dlopen()` or the runtime linker handles. + + C libraries: *.so (linux), *.so.* (linux), *.dylib (mac), .dll (windows) + """ + if filename.endswith(".dll"): + return True + if filename.endswith((".so", ".dylib")) or ".so." in filename: + return True + return False + +def _get_label_runfiles_repo(ctx, label): + repo = label.repo_name + if repo: + return repo + else: + # For files, empty repo means the main repo + return ctx.workspace_name diff --git a/python/private/version.bzl b/python/private/version.bzl new file mode 100644 index 0000000000..b13327841b --- /dev/null +++ b/python/private/version.bzl @@ -0,0 +1,908 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"Implementation of PEP440 version string normalization" + +def mkmethod(self, method): + """Bind a struct as the first arg to a function. + + This is loosely equivalent to creating a bound method of a class. + """ + return lambda *args, **kwargs: method(self, *args, **kwargs) + +def _isdigit(token): + return token.isdigit() + +def _isalnum(token): + return token.isalnum() + +def _lower(token): + # PEP 440: Case sensitivity + return token.lower() + +def _is(reference): + """Predicate testing a token for equality with `reference`.""" + return lambda token: token == reference + +def _is_not(reference): + """Predicate testing a token for inequality with `reference`.""" + return lambda token: token != reference + +def _in(reference): + """Predicate testing if a token is in the list `reference`.""" + return lambda token: token in reference + +def _ctx(start): + """Creates a context, which is state for parsing (or sub-parsing).""" + return { + # The result value from parsing + "norm": "", + # Where in the parser's input string this context starts. + "start": start, + } + +def _open_context(self): + """Open an new parsing ctx. + + If the current parsing step succeeds, call self.accept(). + If the current parsing step fails, call self.discard() to + go back to how it was before we opened a new ctx. + + Args: + self: The normalizer. + """ + self.contexts.append(_ctx(_context(self)["start"])) + return self.contexts[-1] + +def _accept(self, key = None): + """Close the current ctx successfully and merge the results. + + Args: + self: {type}`Parser} + key: {type}`str | None` the key to store the result in + the most recent context. If not set, the key is "norm". + + Returns: + {type}`bool` always True + """ + finished = self.contexts.pop() + self.contexts[-1]["norm"] += finished["norm"] + if key: + self.contexts[-1][key] = finished["norm"] + + self.contexts[-1]["start"] = finished["start"] + return True + +def _context(self): + return self.contexts[-1] + +def _discard(self, key = None): + self.contexts.pop() + if key: + self.contexts[-1][key] = "" + return False + +def _new(input): + """Create a new parser + + Args: + input: {type}`str` input to parse + + Returns: + {type}`Parser` a struct for a parser object. + """ + self = struct( + input = input, + contexts = [_ctx(0)], + ) + + public = struct( + # methods: keep sorted + accept = mkmethod(self, _accept), + context = mkmethod(self, _context), + discard = mkmethod(self, _discard), + open_context = mkmethod(self, _open_context), + + # attributes: keep sorted + input = self.input, + ) + return public + +def accept(parser, predicate, value): + """If `predicate` matches the next token, accept the token. + + Accepting the token means adding it (according to `value`) to + the running results maintained in ctx["norm"] and + advancing the cursor in ctx["start"] to the next token in + `version`. + + Args: + parser: The normalizer. + predicate: function taking a token and returning a boolean + saying if we want to accept the token. + value: the string to add if there's a match, or, if `value` + is a function, the function to apply to the current token + to get the string to add. + + Returns: + whether a token was accepted. + """ + + ctx = parser.context() + + if ctx["start"] >= len(parser.input): + return False + + token = parser.input[ctx["start"]] + + if predicate(token): + if type(value) in ["function", "builtin_function_or_method"]: + value = value(token) + + ctx["norm"] += value + ctx["start"] += 1 + return True + + return False + +def accept_placeholder(parser): + """Accept a Bazel placeholder. + + Placeholders aren't actually part of PEP 440, but are used for + stamping purposes. A placeholder might be + ``{BUILD_TIMESTAMP}``, for instance. We'll accept these as + they are, assuming they will expand to something that makes + sense where they appear. Before the stamping has happened, a + resulting wheel file name containing a placeholder will not + actually be valid. + + Args: + parser: The normalizer. + + Returns: + whether a placeholder was accepted. + """ + ctx = parser.open_context() + + if not accept(parser, _is("{"), str): + return parser.discard() + + start = ctx["start"] + for _ in range(start, len(parser.input) + 1): + if not accept(parser, _is_not("}"), str): + break + + if not accept(parser, _is("}"), str): + return parser.discard() + + return parser.accept() + +def accept_digits(parser): + """Accept multiple digits (or placeholders), up to a non-digit/placeholder. + + Args: + parser: The normalizer. + + Returns: + whether some digits (or placeholders) were accepted. + """ + + ctx = parser.open_context() + start = ctx["start"] + + for i in range(start, len(parser.input) + 1): + if not accept(parser, _isdigit, str) and not accept_placeholder(parser): + if i - start >= 1: + if ctx["norm"].isdigit(): + # PEP 440: Integer Normalization + ctx["norm"] = str(int(ctx["norm"])) + return parser.accept() + break + + return parser.discard() + +def accept_string(parser, string, replacement): + """Accept a `string` in the input. Output `replacement`. + + Args: + parser: The normalizer. + string: The string to search for in the parser input. + replacement: The normalized string to use if the string was found. + + Returns: + whether the string was accepted. + """ + ctx = parser.open_context() + + for character in string.elems(): + if not accept(parser, _in([character, character.upper()]), ""): + return parser.discard() + + ctx["norm"] = replacement + + return parser.accept() + +def accept_alnum(parser): + """Accept an alphanumeric sequence. + + Args: + parser: The normalizer. + + Returns: + whether an alphanumeric sequence was accepted. + """ + + ctx = parser.open_context() + start = ctx["start"] + + for i in range(start, len(parser.input) + 1): + if not accept(parser, _isalnum, _lower) and not accept_placeholder(parser): + if i - start >= 1: + return parser.accept() + break + + return parser.discard() + +def accept_dot_number(parser): + """Accept a dot followed by digits. + + Args: + parser: The normalizer. + + Returns: + whether a dot+digits pair was accepted. + """ + parser.open_context() + + if accept(parser, _is("."), ".") and accept_digits(parser): + return parser.accept() + else: + return parser.discard() + +def accept_dot_number_sequence(parser): + """Accept a sequence of dot+digits. + + Args: + parser: The normalizer. + + Returns: + whether a sequence of dot+digits pairs was accepted. + """ + ctx = parser.context() + start = ctx["start"] + i = start + + for i in range(start, len(parser.input) + 1): + if not accept_dot_number(parser): + break + return i - start >= 1 + +def accept_separator_alnum(parser): + """Accept a separator followed by an alphanumeric string. + + Args: + parser: The normalizer. + + Returns: + whether a separator and an alphanumeric string were accepted. + """ + ctx = parser.open_context() + + # PEP 440: Local version segments + if not accept(parser, _in([".", "-", "_"]), "."): + return parser.discard() + + if accept_alnum(parser): + # First character is separator; skip it. + value = ctx["norm"][1:] + + # PEP 440: Integer Normalization + if value.isdigit(): + value = str(int(value)) + ctx["norm"] = ctx["norm"][0] + value + return parser.accept() + + return parser.discard() + +def accept_separator_alnum_sequence(parser): + """Accept a sequence of separator+alphanumeric. + + Args: + parser: The normalizer. + + Returns: + whether a sequence of separator+alphanumerics was accepted. + """ + ctx = parser.context() + start = ctx["start"] + i = start + + for i in range(start, len(parser.input) + 1): + if not accept_separator_alnum(parser): + break + + return i - start >= 1 + +def accept_epoch(parser): + """PEP 440: Version epochs. + + Args: + parser: The normalizer. + + Returns: + whether a PEP 440 epoch identifier was accepted. + """ + ctx = parser.open_context() + if accept_digits(parser) and accept(parser, _is("!"), "!"): + if ctx["norm"] == "0!": + ctx["norm"] = "" + return parser.accept("epoch") + else: + return parser.discard("epoch") + +def accept_release(parser): + """Accept the release segment, numbers separated by dots. + + Args: + parser: The normalizer. + + Returns: + whether a release segment was accepted. + """ + parser.open_context() + + if not accept_digits(parser): + return parser.discard("release") + + accept_dot_number_sequence(parser) + return parser.accept("release") + +def accept_pre_l(parser): + """PEP 440: Pre-release spelling. + + Args: + parser: The normalizer. + + Returns: + whether a prerelease keyword was accepted. + """ + parser.open_context() + + if ( + accept_string(parser, "alpha", "a") or + accept_string(parser, "a", "a") or + accept_string(parser, "beta", "b") or + accept_string(parser, "b", "b") or + accept_string(parser, "c", "rc") or + accept_string(parser, "preview", "rc") or + accept_string(parser, "pre", "rc") or + accept_string(parser, "rc", "rc") + ): + return parser.accept() + else: + return parser.discard() + +def accept_prerelease(parser): + """PEP 440: Pre-releases. + + Args: + parser: The normalizer. + + Returns: + whether a prerelease identifier was accepted. + """ + ctx = parser.open_context() + + # PEP 440: Pre-release separators + accept(parser, _in(["-", "_", "."]), "") + + if not accept_pre_l(parser): + return parser.discard("pre") + + accept(parser, _in(["-", "_", "."]), "") + + if not accept_digits(parser): + # PEP 440: Implicit pre-release number + ctx["norm"] += "0" + + return parser.accept("pre") + +def accept_implicit_postrelease(parser): + """PEP 440: Implicit post releases. + + Args: + parser: The normalizer. + + Returns: + whether an implicit postrelease identifier was accepted. + """ + ctx = parser.open_context() + + if accept(parser, _is("-"), "") and accept_digits(parser): + ctx["norm"] = ".post" + ctx["norm"] + return parser.accept() + + return parser.discard() + +def accept_explicit_postrelease(parser): + """PEP 440: Post-releases. + + Args: + parser: The normalizer. + + Returns: + whether an explicit postrelease identifier was accepted. + """ + ctx = parser.open_context() + + # PEP 440: Post release separators + if not accept(parser, _in(["-", "_", "."]), "."): + ctx["norm"] += "." + + # PEP 440: Post release spelling + if ( + accept_string(parser, "post", "post") or + accept_string(parser, "rev", "post") or + accept_string(parser, "r", "post") + ): + accept(parser, _in(["-", "_", "."]), "") + + if not accept_digits(parser): + # PEP 440: Implicit post release number + ctx["norm"] += "0" + + return parser.accept() + + return parser.discard() + +def accept_postrelease(parser): + """PEP 440: Post-releases. + + Args: + parser: The normalizer. + + Returns: + whether a postrelease identifier was accepted. + """ + parser.open_context() + + if accept_implicit_postrelease(parser) or accept_explicit_postrelease(parser): + return parser.accept("post") + + return parser.discard("post") + +def accept_devrelease(parser): + """PEP 440: Developmental releases. + + Args: + parser: The normalizer. + + Returns: + whether a developmental release identifier was accepted. + """ + ctx = parser.open_context() + + # PEP 440: Development release separators + if not accept(parser, _in(["-", "_", "."]), "."): + ctx["norm"] += "." + + if accept_string(parser, "dev", "dev"): + accept(parser, _in(["-", "_", "."]), "") + + if not accept_digits(parser): + # PEP 440: Implicit development release number + ctx["norm"] += "0" + + return parser.accept("dev") + + return parser.discard("dev") + +def accept_local(parser): + """PEP 440: Local version identifiers. + + Args: + parser: The normalizer. + + Returns: + whether a local version identifier was accepted. + """ + parser.open_context() + + if accept(parser, _is("+"), "+") and accept_alnum(parser): + accept_separator_alnum_sequence(parser) + return parser.accept("local") + + return parser.discard("local") + +def normalize_pep440(version): + """Escape the version component of a filename. + + See https://packaging.python.org/en/latest/specifications/binary-distribution-format/#escaping-and-unicode + and https://peps.python.org/pep-0440/ + + Args: + version: version string to be normalized according to PEP 440. + + Returns: + string containing the normalized version. + """ + return _parse(version, strict = True)["norm"] + +def _parse(version_str, strict = True, _fail = fail): + """Escape the version component of a filename. + + See https://packaging.python.org/en/latest/specifications/binary-distribution-format/#escaping-and-unicode + and https://peps.python.org/pep-0440/ + + Args: + version_str: version string to be normalized according to PEP 440. + strict: fail if the version is invalid, defaults to True. + _fail: Used for tests + + Returns: + string containing the normalized version. + """ + + # https://packaging.python.org/en/latest/specifications/version-specifiers/#leading-and-trailing-whitespace + version = version_str.strip() + is_prefix = False + + if not strict: + is_prefix = version.endswith(".*") + version = version.strip(" .*") # PEP 440: Leading and Trailing Whitespace and ".*" + + parser = _new(version) + accept(parser, _is("v"), "") # PEP 440: Preceding v character + accept_epoch(parser) + accept_release(parser) + accept_prerelease(parser) + accept_postrelease(parser) + accept_devrelease(parser) + accept_local(parser) + + parser_ctx = parser.context() + if parser.input[parser_ctx["start"]:]: + if strict: + _fail( + "Failed to parse PEP 440 version identifier '%s'." % parser.input, + "Parse error at '%s'" % parser.input[parser_ctx["start"]:], + ) + + return None + + parser_ctx["is_prefix"] = is_prefix + return parser_ctx + +def parse(version_str, strict = False, _fail = fail): + """Parse a PEP4408 compliant version. + + This is similar to `normalize_pep440`, but it parses individual components to + comparable types. + + Args: + version_str: version string to be normalized according to PEP 440. + strict: fail if the version is invalid. + _fail: used for tests + + Returns: + a struct with individual components of a version: + * `epoch` {type}`int`, defaults to `0` + * `release` {type}`tuple[int]` an n-tuple of ints + * `pre` {type}`tuple[str, int] | None` a tuple of a string and an int, + e.g. ("a", 1) + * `post` {type}`tuple[str, int] | None` a tuple of a string and an int, + e.g. ("~", 1) + * `dev` {type}`tuple[str, int] | None` a tuple of a string and an int, + e.g. ("", 1) + * `local` {type}`tuple[str, int] | None` a tuple of components in the local + version, e.g. ("abc", 123). + * `is_prefix` {type}`bool` whether the version_str ends with `.*`. + * `string` {type}`str` normalized value of the input. + """ + + parts = _parse(version_str, strict = strict, _fail = _fail) + if not parts: + return None + + if parts["is_prefix"] and (parts["local"] or parts["post"] or parts["dev"] or parts["pre"]): + if strict: + _fail("local version part has been obtained, but only public segments can have prefix matches") + + # https://peps.python.org/pep-0440/#public-version-identifiers + return None + + # buildifier: disable=uninitialized + self = struct( + epoch = _parse_epoch(parts["epoch"], _fail), + release = _parse_release(parts["release"]), + pre = _parse_pre(parts["pre"]), + post = _parse_post(parts["post"], _fail), + dev = _parse_dev(parts["dev"], _fail), + local = _parse_local(parts["local"], _fail), + string = parts["norm"], + is_prefix = parts["is_prefix"], + key = lambda *a, **k: _version_key(self, *a, **k), + ) + return self + +def _parse_epoch(value, fail): + if not value: + return 0 + + if not value.endswith("!"): + fail("epoch string segment needs to end with '!', got: {}".format(value)) + + return int(value[:-1]) + +def _parse_release(value): + return tuple([int(d) for d in value.split(".")]) + +def _parse_local(value, fail): + if not value: + return None + + if not value.startswith("+"): + fail("local release identifier must start with '+', got: {}".format(value)) + + # If the part is numerical, handle it as a number + return tuple([int(part) if part.isdigit() else part for part in value[1:].split(".")]) + +def _parse_dev(value, fail): + if not value: + return None + + if not value.startswith(".dev"): + fail("dev release identifier must start with '.dev', got: {}".format(value)) + dev = int(value[len(".dev"):]) + + # Empty string goes first when comparing + return ("", dev) + +def _parse_pre(value): + if not value: + return None + + if value.startswith("rc"): + prefix = "rc" + else: + prefix = value[0] + + return (prefix, int(value[len(prefix):])) + +def _parse_post(value, fail): + if not value: + return None + + if not value.startswith(".post"): + fail("post release identifier must start with '.post', got: {}".format(value)) + post = int(value[len(".post"):]) + + # We choose `~` since almost all of the ASCII characters will be before + # it. Use `ord` and `chr` functions to find a good value. + return ("~", post) + +def _pad_zeros(release, n): + padding = n - len(release) + if padding <= 0: + return release + + release = list(release) + [0] * padding + return tuple(release) + +def _prefix_err(left, op, right): + if left.is_prefix or right.is_prefix: + fail("PEP440: only '==' and '!=' operators can use prefix matching: {} {} {}".format( + left.string, + op, + right.string, + )) + +def _version_eeq(left, right): + """=== operator""" + if left.is_prefix or right.is_prefix: + fail(_prefix_err(left, "===", right)) + + # https://peps.python.org/pep-0440/#arbitrary-equality + # > simple string equality operations + return left.string == right.string + +def _version_eq(left, right): + """== operator""" + is_prefix = _is_prefix(left, right) + if is_prefix != None: + return is_prefix + + if left.epoch != right.epoch: + return False + + release_len = max(len(left.release), len(right.release)) + left_release = _pad_zeros(left.release, release_len) + right_release = _pad_zeros(right.release, release_len) + + if left_release != right_release: + return False + + return ( + left.pre == right.pre and + left.post == right.post and + left.dev == right.dev + # local is ignored for == checks + ) + +def _is_prefix(left, right): + if left.is_prefix and right.is_prefix: + fail("Invalid comparison: both versions cannot be prefix matching") + if left.is_prefix: + return _left_is_prefix(left, right) + if right.is_prefix: + return _left_is_prefix(right, left) + + return None + +def _left_is_prefix(left, right): + if right.string.startswith("{}.".format(left.string)): + return True + + # There is a chance that we are comparing 1.3 ~= 1.3.0 + # + # In that case the logic would be to normalize 1.3 to 1.3.0 and then + # the above would work, but we end up comparing 1.3 with 1.3. above + # and it fails. + return _version_key(left) == _version_key(right) + +def _version_compatible(left, right): + """~= operator""" + if left.is_prefix or right.is_prefix: + fail(_prefix_err(left, "~=", right)) + + # https://peps.python.org/pep-0440/#compatible-release + # Note, the ~= operator can be also expressed as: + # >= V.N, == V.* + + right_star = ".".join([str(d) for d in right.release[:-1]]) + if right.epoch: + right_star = "{}!{}.*".format(right.epoch, right_star) + else: + right_star = "{}.*".format(right_star) + + return _version_ge(left, right) and _is_prefix(left, parse(right_star, strict = False)) + +def _version_ne(left, right): + """!= operator""" + return not _version_eq(left, right) + +def _version_lt(left, right): + """< operator""" + if left.is_prefix or right.is_prefix: + fail(_prefix_err(left, "<", right)) + + if left.epoch > right.epoch: + return False + elif left.epoch < right.epoch: + return True + + release_len = max(len(left.release), len(right.release)) + left_release = _pad_zeros(left.release, release_len) + right_release = _pad_zeros(right.release, release_len) + + if left_release > right_release: + return False + elif left_release < right_release: + return True + + # From PEP440, this is not a simple ordering check and we need to check the version + # semantically: + # * The exclusive ordered comparison operator""" + if left.is_prefix or right.is_prefix: + fail(_prefix_err(left, ">", right)) + + if left.epoch > right.epoch: + return True + elif left.epoch < right.epoch: + return False + + release_len = max(len(left.release), len(right.release)) + left_release = _pad_zeros(left.release, release_len) + right_release = _pad_zeros(right.release, release_len) + + if left_release > right_release: + return True + elif left_release < right_release: + return False + + # From PEP440, this is not a simple ordering check and we need to check the version + # semantically: + # * The exclusive ordered comparison >V MUST NOT allow a post-release of the given version + # unless V itself is a post release. + # + # * The exclusive ordered comparison >V MUST NOT match a local version of the specified + # version. + + if left.post and right.post: + return left.post > right.post + else: + # ignore the left.post if right is not a post if right is a post, then this evaluates to + # False anyway. + return False + +def _version_le(left, right): + """<= operator""" + if left.is_prefix or right.is_prefix: + fail(_prefix_err(left, "<=", right)) + + # PEP440: simple order check + # https://peps.python.org/pep-0440/#inclusive-ordered-comparison + _left = _version_key(left, local = False) + _right = _version_key(right, local = False) + return _left < _right or _version_eq(left, right) + +def _version_ge(left, right): + """>= operator""" + if left.is_prefix or right.is_prefix: + fail(_prefix_err(left, ">=", right)) + + # PEP440: simple order check + # https://peps.python.org/pep-0440/#inclusive-ordered-comparison + _left = _version_key(left, local = False) + _right = _version_key(right, local = False) + return _left > _right or _version_eq(left, right) + +def _version_key(self, *, local = True): + """This function returns a tuple that can be used in 'sorted' calls. + + This implements the PEP440 version sorting. + """ + release_key = ("z",) + local = self.local if local else [] + local = local or [] + + return ( + self.epoch, + self.release, + # PEP440 Within a pre-release, post-release or development release segment with + # a shared prefix, ordering MUST be by the value of the numeric component. + # PEP440 release ordering: .devN, aN, bN, rcN, , .postN + # We choose to first match the pre-release, then post release, then dev and + # then stable + self.pre or self.post or self.dev or release_key, + # PEP440 local versions go before post versions + tuple([(type(item) == "int", item) for item in local]), + # PEP440 - pre-release ordering: .devN, , .postN + self.post or self.dev or release_key, + # PEP440 - post release ordering: .devN, + self.dev or release_key, + ) + +version = struct( + normalize = normalize_pep440, + parse = parse, + # methods, keep sorted + key = _version_key, + is_compatible = _version_compatible, + is_eq = _version_eq, + is_eeq = _version_eeq, + is_ge = _version_ge, + is_gt = _version_gt, + is_le = _version_le, + is_lt = _version_lt, + is_ne = _version_ne, +) diff --git a/python/private/visibility.bzl b/python/private/visibility.bzl new file mode 100644 index 0000000000..3883e23638 --- /dev/null +++ b/python/private/visibility.bzl @@ -0,0 +1,7 @@ +"""Shared code for use with visibility specs.""" + +# Use when a target isn't actually public, but needs public +# visibility to keep Bazel happy. +# Such cases are typically for defaults of rule attributes or macro args that +# get used outside of rules_python itself. +NOT_ACTUALLY_PUBLIC = ["//visibility:public"] diff --git a/python/private/which.bzl b/python/private/which.bzl deleted file mode 100644 index b0cbddb0e8..0000000000 --- a/python/private/which.bzl +++ /dev/null @@ -1,32 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""Wrapper for repository which call""" - -_binary_not_found_msg = "Unable to find the binary '{binary_name}'. Please update your PATH to include '{binary_name}'." - -def which_with_fail(binary_name, rctx): - """Tests to see if a binary exists, and otherwise fails with a message. - - Args: - binary_name: name of the binary to find. - rctx: repository context. - - Returns: - rctx.Path for the binary. - """ - binary = rctx.which(binary_name) - if binary == None: - fail(_binary_not_found_msg.format(binary_name = binary_name)) - return binary diff --git a/python/private/whl_filegroup/BUILD.bazel b/python/private/whl_filegroup/BUILD.bazel index 398b9af0d8..4d0a096311 100644 --- a/python/private/whl_filegroup/BUILD.bazel +++ b/python/private/whl_filegroup/BUILD.bazel @@ -1,5 +1,5 @@ load("@bazel_skylib//:bzl_library.bzl", "bzl_library") -load("//python:defs.bzl", "py_binary") +load("//python:py_binary.bzl", "py_binary") filegroup( name = "distribution", @@ -7,14 +7,14 @@ filegroup( visibility = ["//python/private:__pkg__"], ) -bzl_library( - name = "whl_filegroup_bzl", - srcs = ["whl_filegroup.bzl"], - visibility = ["//:__subpackages__"], -) - py_binary( name = "extract_wheel_files", srcs = ["extract_wheel_files.py"], visibility = ["//visibility:public"], ) + +bzl_library( + name = "whl_filegroup", + srcs = ["whl_filegroup.bzl"], + visibility = ["//python:__subpackages__"], +) diff --git a/python/private/whl_filegroup/whl_filegroup.bzl b/python/private/whl_filegroup/whl_filegroup.bzl index d2e6e43b91..c52211bfbc 100644 --- a/python/private/whl_filegroup/whl_filegroup.bzl +++ b/python/private/whl_filegroup/whl_filegroup.bzl @@ -42,7 +42,14 @@ cc_library( includes = ["numpy_includes/numpy/core/include"], deps = ["@rules_python//python/cc:current_py_cc_headers"], ) + ``` + +:::{seealso} + +The `:extracted_whl_files` target, which is a filegroup of all the files +from the already extracted whl file. +::: """, attrs = { "pattern": attr.string(default = "", doc = "Only file paths matching this regex pattern will be extracted."), diff --git a/python/private/zip_main_template.py b/python/private/zip_main_template.py deleted file mode 100644 index b4c9d279a6..0000000000 --- a/python/private/zip_main_template.py +++ /dev/null @@ -1,323 +0,0 @@ -# Template for the __main__.py file inserted into zip files -# -# NOTE: This file is a "stage 1" bootstrap, so it's responsible for locating the -# desired runtime and having it run the stage 2 bootstrap. This means it can't -# assume much about the current runtime and environment. e.g., the current -# runtime may not be the correct one, the zip may not have been extract, the -# runfiles env vars may not be set, etc. -# -# NOTE: This program must retain compatibility with a wide variety of Python -# versions since it is run by an unknown Python interpreter. - -import sys - -# The Python interpreter unconditionally prepends the directory containing this -# script (following symlinks) to the import path. This is the cause of #9239, -# and is a special case of #7091. We therefore explicitly delete that entry. -# TODO(#7091): Remove this hack when no longer necessary. -del sys.path[0] - -import os -import shutil -import subprocess -import tempfile -import zipfile - -# runfiles-relative path -_STAGE2_BOOTSTRAP = "%stage2_bootstrap%" -# runfiles-relative path -_PYTHON_BINARY = "%python_binary%" -# runfiles-relative path, absolute path, or single word -_PYTHON_BINARY_ACTUAL = "%python_binary_actual%" -_WORKSPACE_NAME = "%workspace_name%" - - -# Return True if running on Windows -def is_windows(): - return os.name == "nt" - - -def get_windows_path_with_unc_prefix(path): - """Adds UNC prefix after getting a normalized absolute Windows path. - - No-op for non-Windows platforms or if running under python2. - """ - path = path.strip() - - # No need to add prefix for non-Windows platforms. - # And \\?\ doesn't work in python 2 or on mingw - if not is_windows() or sys.version_info[0] < 3: - return path - - # Starting in Windows 10, version 1607(OS build 14393), MAX_PATH limitations have been - # removed from common Win32 file and directory functions. - # Related doc: https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=cmd#enable-long-paths-in-windows-10-version-1607-and-later - import platform - - if platform.win32_ver()[1] >= "10.0.14393": - return path - - # import sysconfig only now to maintain python 2.6 compatibility - import sysconfig - - if sysconfig.get_platform() == "mingw": - return path - - # Lets start the unicode fun - unicode_prefix = "\\\\?\\" - if path.startswith(unicode_prefix): - return path - - # os.path.abspath returns a normalized absolute path - return unicode_prefix + os.path.abspath(path) - - -def has_windows_executable_extension(path): - return path.endswith(".exe") or path.endswith(".com") or path.endswith(".bat") - - -if is_windows() and not has_windows_executable_extension(_PYTHON_BINARY): - _PYTHON_BINARY = _PYTHON_BINARY + ".exe" - - -def search_path(name): - """Finds a file in a given search path.""" - search_path = os.getenv("PATH", os.defpath).split(os.pathsep) - for directory in search_path: - if directory: - path = os.path.join(directory, name) - if os.path.isfile(path) and os.access(path, os.X_OK): - return path - return None - - -def find_python_binary(module_space): - """Finds the real Python binary if it's not a normal absolute path.""" - return find_binary(module_space, _PYTHON_BINARY) - - -def print_verbose(*args, mapping=None, values=None): - if bool(os.environ.get("RULES_PYTHON_BOOTSTRAP_VERBOSE")): - if mapping is not None: - for key, value in sorted((mapping or {}).items()): - print( - "bootstrap: stage 1:", - *args, - f"{key}={value!r}", - file=sys.stderr, - flush=True, - ) - elif values is not None: - for i, v in enumerate(values): - print( - "bootstrap: stage 1:", - *args, - f"[{i}] {v!r}", - file=sys.stderr, - flush=True, - ) - else: - print("bootstrap: stage 1:", *args, file=sys.stderr, flush=True) - - -def find_binary(module_space, bin_name): - """Finds the real binary if it's not a normal absolute path.""" - if not bin_name: - return None - if bin_name.startswith("//"): - # Case 1: Path is a label. Not supported yet. - raise AssertionError( - "Bazel does not support execution of Python interpreters via labels yet" - ) - elif os.path.isabs(bin_name): - # Case 2: Absolute path. - return bin_name - # Use normpath() to convert slashes to os.sep on Windows. - elif os.sep in os.path.normpath(bin_name): - # Case 3: Path is relative to the repo root. - return os.path.join(module_space, bin_name) - else: - # Case 4: Path has to be looked up in the search path. - return search_path(bin_name) - - -def extract_zip(zip_path, dest_dir): - """Extracts the contents of a zip file, preserving the unix file mode bits. - - These include the permission bits, and in particular, the executable bit. - - Ideally the zipfile module should set these bits, but it doesn't. See: - https://bugs.python.org/issue15795. - - Args: - zip_path: The path to the zip file to extract - dest_dir: The path to the destination directory - """ - zip_path = get_windows_path_with_unc_prefix(zip_path) - dest_dir = get_windows_path_with_unc_prefix(dest_dir) - with zipfile.ZipFile(zip_path) as zf: - for info in zf.infolist(): - zf.extract(info, dest_dir) - # UNC-prefixed paths must be absolute/normalized. See - # https://docs.microsoft.com/en-us/windows/desktop/fileio/naming-a-file#maximum-path-length-limitation - file_path = os.path.abspath(os.path.join(dest_dir, info.filename)) - # The Unix st_mode bits (see "man 7 inode") are stored in the upper 16 - # bits of external_attr. Of those, we set the lower 12 bits, which are the - # file mode bits (since the file type bits can't be set by chmod anyway). - attrs = info.external_attr >> 16 - if attrs != 0: # Rumor has it these can be 0 for zips created on Windows. - os.chmod(file_path, attrs & 0o7777) - - -# Create the runfiles tree by extracting the zip file -def create_module_space(): - temp_dir = tempfile.mkdtemp("", "Bazel.runfiles_") - extract_zip(os.path.dirname(__file__), temp_dir) - # IMPORTANT: Later code does `rm -fr` on dirname(module_space) -- it's - # important that deletion code be in sync with this directory structure - return os.path.join(temp_dir, "runfiles") - - -def execute_file( - python_program, - main_filename, - args, - env, - module_space, - workspace, -): - # type: (str, str, list[str], dict[str, str], str, str|None, str|None) -> ... - """Executes the given Python file using the various environment settings. - - This will not return, and acts much like os.execv, except is much - more restricted, and handles Bazel-related edge cases. - - Args: - python_program: (str) Path to the Python binary to use for execution - main_filename: (str) The Python file to execute - args: (list[str]) Additional args to pass to the Python file - env: (dict[str, str]) A dict of environment variables to set for the execution - module_space: (str) Path to the module space/runfiles tree directory - workspace: (str|None) Name of the workspace to execute in. This is expected to be a - directory under the runfiles tree. - """ - # We want to use os.execv instead of subprocess.call, which causes - # problems with signal passing (making it difficult to kill - # Bazel). However, these conditions force us to run via - # subprocess.call instead: - # - # - On Windows, os.execv doesn't handle arguments with spaces - # correctly, and it actually starts a subprocess just like - # subprocess.call. - # - When running in a workspace or zip file, we need to clean up the - # workspace after the process finishes so control must return here. - try: - subprocess_argv = [python_program, main_filename] + args - print_verbose("subprocess argv:", values=subprocess_argv) - print_verbose("subprocess env:", mapping=env) - print_verbose("subprocess cwd:", workspace) - ret_code = subprocess.call(subprocess_argv, env=env, cwd=workspace) - sys.exit(ret_code) - finally: - # NOTE: dirname() is called because create_module_space() creates a - # sub-directory within a temporary directory, and we want to remove the - # whole temporary directory. - shutil.rmtree(os.path.dirname(module_space), True) - - -def main(): - print_verbose("running zip main bootstrap") - print_verbose("initial argv:", values=sys.argv) - print_verbose("initial environ:", mapping=os.environ) - print_verbose("initial sys.executable", sys.executable) - print_verbose("initial sys.version", sys.version) - - args = sys.argv[1:] - - new_env = {} - - # The main Python source file. - # The magic string percent-main-percent is replaced with the runfiles-relative - # filename of the main file of the Python binary in BazelPythonSemantics.java. - main_rel_path = _STAGE2_BOOTSTRAP - if is_windows(): - main_rel_path = main_rel_path.replace("/", os.sep) - - module_space = create_module_space() - print_verbose("extracted runfiles to:", module_space) - - new_env["RUNFILES_DIR"] = module_space - - # Don't prepend a potentially unsafe path to sys.path - # See: https://docs.python.org/3.11/using/cmdline.html#envvar-PYTHONSAFEPATH - new_env["PYTHONSAFEPATH"] = "1" - - main_filename = os.path.join(module_space, main_rel_path) - main_filename = get_windows_path_with_unc_prefix(main_filename) - assert os.path.exists(main_filename), ( - "Cannot exec() %r: file not found." % main_filename - ) - assert os.access(main_filename, os.R_OK), ( - "Cannot exec() %r: file not readable." % main_filename - ) - - python_program = find_python_binary(module_space) - if python_program is None: - raise AssertionError("Could not find python binary: " + _PYTHON_BINARY) - - # The python interpreter should always be under runfiles, but double check. - # We don't want to accidentally create symlinks elsewhere. - if not python_program.startswith(module_space): - raise AssertionError( - "Program's venv binary not under runfiles: {python_program}" - ) - - if os.path.isabs(_PYTHON_BINARY_ACTUAL): - symlink_to = _PYTHON_BINARY_ACTUAL - elif "/" in _PYTHON_BINARY_ACTUAL: - symlink_to = os.path.join(module_space, _PYTHON_BINARY_ACTUAL) - else: - symlink_to = search_path(_PYTHON_BINARY_ACTUAL) - if not symlink_to: - raise AssertionError( - f"Python interpreter to use not found on PATH: {_PYTHON_BINARY_ACTUAL}" - ) - - # The bin/ directory may not exist if it is empty. - os.makedirs(os.path.dirname(python_program), exist_ok=True) - try: - os.symlink(_PYTHON_BINARY_ACTUAL, python_program) - except OSError as e: - raise Exception( - f"Unable to create venv python interpreter symlink: {python_program} -> {PYTHON_BINARY_ACTUAL}" - ) from e - - # Some older Python versions on macOS (namely Python 3.7) may unintentionally - # leave this environment variable set after starting the interpreter, which - # causes problems with Python subprocesses correctly locating sys.executable, - # which subsequently causes failure to launch on Python 3.11 and later. - if "__PYVENV_LAUNCHER__" in os.environ: - del os.environ["__PYVENV_LAUNCHER__"] - - new_env.update((key, val) for key, val in os.environ.items() if key not in new_env) - - workspace = None - # If RUN_UNDER_RUNFILES equals 1, it means we need to - # change directory to the right runfiles directory. - # (So that the data files are accessible) - if os.environ.get("RUN_UNDER_RUNFILES") == "1": - workspace = os.path.join(module_space, _WORKSPACE_NAME) - - sys.stdout.flush() - execute_file( - python_program, - main_filename, - args, - new_env, - module_space, - workspace, - ) - - -if __name__ == "__main__": - main() diff --git a/python/private/zipapp/BUILD.bazel b/python/private/zipapp/BUILD.bazel new file mode 100644 index 0000000000..66f0b4f667 --- /dev/null +++ b/python/private/zipapp/BUILD.bazel @@ -0,0 +1,48 @@ +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") + +package( + default_visibility = ["//:__subpackages__"], +) + +licenses(["notice"]) + +filegroup( + name = "distribution", + srcs = glob(["**"]), +) + +filegroup( + name = "zip_main_template", + srcs = ["zip_main_template.py"], + visibility = ["//visibility:public"], +) + +filegroup( + name = "zip_shell_template", + srcs = ["zip_shell_template.sh"], + visibility = ["//visibility:public"], +) + +filegroup( + name = "zipapp_stage2_bootstrap_template", + srcs = ["zipapp_stage2_bootstrap_template.py"], + visibility = ["//visibility:public"], +) + +bzl_library( + name = "py_zipapp_rule", + srcs = ["py_zipapp_rule.bzl"], + deps = [ + "//python/private:attributes", + "//python/private:builders", + "//python/private:common", + "//python/private:common_labels", + "//python/private:py_executable_info", + "//python/private:py_internal", + "//python/private:py_runtime_info", + "//python/private:toolchain_types", + "//python/private:transition_labels", + "@bazel_skylib//lib:paths", + "@rules_python_internal//:rules_python_config", + ], +) diff --git a/python/private/zipapp/py_zipapp_rule.bzl b/python/private/zipapp/py_zipapp_rule.bzl new file mode 100644 index 0000000000..b1a9399e90 --- /dev/null +++ b/python/private/zipapp/py_zipapp_rule.bzl @@ -0,0 +1,443 @@ +"""Implementation of the zipapp rules.""" + +load("@bazel_skylib//lib:paths.bzl", "paths") +load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") +load("//python/private:attributes.bzl", "apply_config_settings_attr") +load("//python/private:builders.bzl", "builders") +load( + "//python/private:common.bzl", + "BUILTIN_BUILD_PYTHON_ZIP", + "actions_run", + "create_windows_exe_launcher", + "is_windows_platform", + "maybe_builtin_build_python_zip", + "maybe_create_repo_mapping", + "runfiles_root_path", +) +load("//python/private:common_labels.bzl", "labels") +load("//python/private:py_executable_info.bzl", "PyExecutableInfo") +load("//python/private:py_internal.bzl", "py_internal") +load("//python/private:py_runtime_info.bzl", "PyRuntimeInfo") +load("//python/private:toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", "LAUNCHER_MAKER_TOOLCHAIN_TYPE") +load("//python/private:transition_labels.bzl", "TRANSITION_LABELS") + +def _is_symlink(f): + if hasattr(f, "is_symlink"): + return str(int(f.is_symlink)) + else: + return "-1" + +def _create_zipapp_main_py(ctx, py_runtime, py_executable, stage2_bootstrap, runfiles, explicit_symlinks): + venv_python_exe = py_executable.venv_python_exe + if venv_python_exe: + venv_python_exe_path = runfiles_root_path(ctx, venv_python_exe.short_path) + else: + venv_python_exe_path = "" + + if py_runtime.interpreter: + python_binary_actual_path = runfiles_root_path(ctx, py_runtime.interpreter.short_path) + else: + python_binary_actual_path = py_runtime.interpreter_path + + zip_main_py = ctx.actions.declare_file(ctx.label.name + ".zip_main.py") + + args = ctx.actions.args() + args.add(py_runtime.zip_main_template, format = "--template=%s") + args.add(zip_main_py, format = "--output=%s") + + args.add( + "%EXTRACT_DIR%=" + paths.join( + (ctx.label.repo_name or "_main"), + ctx.label.package, + ctx.label.name, + ), + format = "--substitution=%s", + ) + args.add("%python_binary%=" + venv_python_exe_path, format = "--substitution=%s") + args.add("%python_binary_actual%=" + python_binary_actual_path, format = "--substitution=%s") + args.add("%stage2_bootstrap%=" + runfiles_root_path(ctx, stage2_bootstrap.short_path), format = "--substitution=%s") + args.add("%workspace_name%=" + ctx.workspace_name, format = "--substitution=%s") + + hash_files_manifest = ctx.actions.args() + hash_files_manifest.use_param_file("--hash_files_manifest=%s", use_always = True) + hash_files_manifest.set_param_file_format("multiline") + + inputs = builders.DepsetBuilder() + inputs.add(py_runtime.zip_main_template) + _build_manifest(ctx, hash_files_manifest, runfiles, explicit_symlinks, inputs) + + actions_run( + ctx, + executable = ctx.attr._zip_main_maker, + arguments = [args, hash_files_manifest], + inputs = inputs.build(), + outputs = [zip_main_py], + mnemonic = "PyZipAppCreateMainPy", + progress_message = "Generating zipapp __main__.py: %{label}", + ) + return zip_main_py + +def _map_zip_empty_filenames(list_paths_cb): + return ["rf-empty|" + path for path in list_paths_cb().to_list()] + +def _map_zip_runfiles(file): + return "rf-file|" + _is_symlink(file) + "|" + file.short_path + "|" + file.path + +def _map_zip_symlinks(entry): + return "rf-symlink|" + _is_symlink(entry.target_file) + "|" + entry.path + "|" + entry.target_file.path + +def _map_zip_root_symlinks(entry): + return "rf-root-symlink|" + _is_symlink(entry.target_file) + "|" + entry.path + "|" + entry.target_file.path + +def _map_explicit_symlinks(entry): + return "symlink|" + entry.runfiles_path + "|" + entry.link_to_path + +def _build_manifest(ctx, manifest, runfiles, explicit_symlinks, inputs): + manifest.add_all( + # NOTE: Accessing runfiles.empty_filenames materializes them. A lambda + # is used to defer that. + [lambda: runfiles.empty_filenames], + map_each = _map_zip_empty_filenames, + allow_closure = True, + ) + + manifest.add_all(runfiles.files, map_each = _map_zip_runfiles) + manifest.add_all(runfiles.symlinks, map_each = _map_zip_symlinks) + manifest.add_all(runfiles.root_symlinks, map_each = _map_zip_root_symlinks) + manifest.add_all(explicit_symlinks, map_each = _map_explicit_symlinks) + + inputs.add(runfiles.files) + inputs.add([entry.target_file for entry in runfiles.symlinks.to_list()]) + inputs.add([entry.target_file for entry in runfiles.root_symlinks.to_list()]) + for entry in explicit_symlinks.to_list(): + inputs.add(entry.files) + + zip_repo_mapping_manifest = maybe_create_repo_mapping( + ctx = ctx, + runfiles = runfiles, + ) + if zip_repo_mapping_manifest: + # NOTE: rf-root-symlink is used to make it show up under the runfiles + # subdirectory within the zip. + manifest.add( + zip_repo_mapping_manifest.path, + format = "rf-root-symlink|0|_repo_mapping|%s", + ) + inputs.add(zip_repo_mapping_manifest) + +def _create_zip(ctx, py_runtime, py_executable, stage2_bootstrap): + output = ctx.actions.declare_file(ctx.label.name + ".zip") + manifest = ctx.actions.args() + manifest.use_param_file("%s", use_always = True) + manifest.set_param_file_format("multiline") + + runfiles = builders.RunfilesBuilder() + + runfiles.add(py_runtime.files) + if py_executable.venv_python_exe: + runfiles.add(py_executable.venv_python_exe) + + if py_executable.venv_interpreter_runfiles: + runfiles.add(py_executable.venv_interpreter_runfiles) + runfiles.add(py_executable.app_runfiles) + runfiles.add(stage2_bootstrap) + + runfiles = runfiles.build(ctx) + + explicit_symlinks = depset(transitive = [ + py_executable.venv_interpreter_symlinks, + py_executable.venv_app_symlinks, + ]) + zip_main = _create_zipapp_main_py( + ctx, + py_runtime, + py_executable, + stage2_bootstrap, + runfiles, + explicit_symlinks, + ) + inputs = builders.DepsetBuilder() + manifest.add("regular|0|__main__.py|{}".format(zip_main.path)) + inputs.add(zip_main) + _build_manifest(ctx, manifest, runfiles, explicit_symlinks, inputs) + + zipper_args = ctx.actions.args() + zipper_args.add(output) + zipper_args.add(ctx.workspace_name, format = "--workspace-name=%s") + zipper_args.add( + str(int(py_internal.get_legacy_external_runfiles(ctx))), + format = "--legacy-external-runfiles=%s", + ) + if ctx.attr.compression: + zipper_args.add(ctx.attr.compression, format = "--compression=%s") + zipper_args.add("--runfiles-dir=runfiles") + + is_windows = is_windows_platform(ctx) + zipper_args.add("\\" if is_windows else "/", format = "--target-platform-pathsep=%s") + + actions_run( + ctx, + executable = ctx.attr._zipper, + arguments = [manifest, zipper_args], + inputs = inputs.build(), + outputs = [output], + mnemonic = "PyZipAppCreateZip", + progress_message = "Reticulating zipapp archive: %{label} into %{output}", + ) + return output + +def _create_shell_bootstrap(ctx, py_runtime, py_executable, stage2_bootstrap): + preamble = ctx.actions.declare_file(ctx.label.name + ".preamble.sh") + + bundled_pyexe_path = "" + external_pyexe_path = "" + if py_runtime.interpreter_path: + external_pyexe_path = py_runtime.interpreter_path + else: + bundled_pyexe_path = runfiles_root_path(ctx, py_runtime.interpreter.short_path) + + substitutions = { + "%BUNDLED_PYEXE_PATH%": bundled_pyexe_path, + "%EXTERNAL_PYEXE_PATH%": external_pyexe_path, + "%EXTRACT_DIR%": paths.join( + (ctx.label.repo_name or "_main"), + ctx.label.package, + ctx.label.name, + ), + "%INTERPRETER_ARGS%": "\n".join([ + '"{}"'.format(v) + for v in py_executable.interpreter_args + ]), + "%STAGE2_BOOTSTRAP%": runfiles_root_path(ctx, stage2_bootstrap.short_path), + } + ctx.actions.expand_template( + template = ctx.file._zip_shell_template, + output = preamble, + substitutions = substitutions, + is_executable = True, + ) + return preamble + +def _create_self_executable_zip(ctx, preamble, zip_file): + pyz = ctx.actions.declare_file(ctx.label.name + ".pyz") + args = ctx.actions.args() + args.add(preamble) + args.add(zip_file) + args.add(pyz) + actions_run( + ctx, + executable = ctx.attr._exe_zip_maker, + arguments = [args], + inputs = depset([preamble, zip_file]), + outputs = [pyz], + mnemonic = "PyZipAppCreateExecutableZip", + progress_message = "Reticulating zipapp executable: %{label} into %{output}", + ) + return pyz + +def _py_zipapp_executable_impl(ctx): + py_executable = ctx.attr.binary[PyExecutableInfo] + py_runtime = ctx.attr.binary[PyRuntimeInfo] + + stage2_bootstrap = py_executable.stage2_bootstrap + + zip_file = _create_zip(ctx, py_runtime, py_executable, stage2_bootstrap) + if ctx.attr.executable: + if is_windows_platform(ctx): + executable = ctx.actions.declare_file(ctx.label.name + ".exe") + + # The zipapp is an opaque zip file, so the Bazel Python launcher doesn't + # know how to look inside it to find the Python interpreter. This means + # we can only use system paths or programs on PATH to bootstrap. + if py_runtime.interpreter_path: + bootstrap_python_path = py_runtime.interpreter_path + else: + # A special value the Bazel Python launcher recognized to skip + # lookup in the runfiles and uses `python.exe` from PATH. + bootstrap_python_path = "python" + + create_windows_exe_launcher( + ctx, + output = executable, + # The path to a python to use to invoke e.g. `python.exe foo.zip` + python_binary_path = bootstrap_python_path, + # Tell the launcher to invoke `python_binary_path` on itself + # after removing its file extension and appending `.zip`. + use_zip_file = True, + ) + default_outputs = [executable, zip_file] + else: + preamble = _create_shell_bootstrap(ctx, py_runtime, py_executable, stage2_bootstrap) + executable = _create_self_executable_zip(ctx, preamble, zip_file) + default_outputs = [executable] + else: + # Bazel requires executable=True rules to have an executable given, so give + # a fake one to satisfy that. + default_outputs = [zip_file] + executable = ctx.actions.declare_file(ctx.label.name + "-not-executable") + ctx.actions.write(executable, "echo 'ERROR: Non executable zip file'; exit 1") + + return [ + DefaultInfo( + files = depset(default_outputs), + runfiles = ctx.runfiles(files = default_outputs), + executable = executable, + ), + OutputGroupInfo( + python_zip_file = depset([zip_file]), + ), + ] + +def _transition_zipapp_impl(settings, attr): + settings = apply_config_settings_attr(dict(settings), attr) + + # Force this to false, otherwise the binary is already a zipapp + settings[labels.BUILD_PYTHON_ZIP] = False + maybe_builtin_build_python_zip("false", settings) + return settings + +_zipapp_transition = transition( + implementation = _transition_zipapp_impl, + inputs = TRANSITION_LABELS, + outputs = TRANSITION_LABELS + [ + labels.BUILD_PYTHON_ZIP, + ] + BUILTIN_BUILD_PYTHON_ZIP, +) + +_ATTRS = { + "binary": attr.label( + doc = """ +A `py_binary` or `py_test` (or equivalent) target to package. +""", + providers = [PyExecutableInfo, PyRuntimeInfo], + mandatory = True, + ), + "compression": attr.string( + doc = """ +The compression level to use. + +Typically 0 to 9, with higher numbers being to compress more. +""", + default = "", + ), + "config_settings": attr.label_keyed_string_dict( + doc = """ +Config settings to change for this target. + +The keys are labels for settings, and the values are strings for the new value +to use. Pass `Label` objects or canonical label strings for the keys to ensure +they resolve as expected (canonical labels start with `@@` and can be +obtained by calling `str(Label(...))`). + +Most `@rules_python//python/config_setting` settings can be used here, which +allows, for example, making only a certain `py_binary` use +{obj}`--bootstrap_impl=script`. + +Additional or custom config settings can be registered using the +{obj}`add_transition_setting` API. This allows, for example, forcing a +particular CPU, or defining a custom setting that `select()` uses elsewhere +to pick between `pip.parse` hubs. See the [How to guide on multiple +versions of a library] for a more concrete example. + +:::{note} +These values are transitioned on, so will affect the analysis graph and the +associated memory overhead. The more unique configurations in your overall +build, the more memory and (often unnecessary) re-analysis and re-building +can occur. See + for +more information about risks and considerations. +::: +""", + ), + "executable": attr.bool( + doc = """ +Whether the output should be an executable zip file. +""", + default = True, + ), + # Required to opt-in to the transition feature. + "_allowlist_function_transition": attr.label( + default = "@bazel_tools//tools/allowlists/function_transition_allowlist", + ), + "_exe_zip_maker": attr.label( + cfg = "exec", + default = "//tools/private/zipapp:exe_zip_maker", + ), + "_launcher": attr.label( + cfg = "target", + # NOTE: This is an executable, but is only used for Windows. It + # can't have executable=True because the backing target is an + # empty target for other platforms. + default = "//tools/launcher:launcher", + ), + "_windows_constraints": attr.label_list( + default = [ + "@platforms//os:windows", + ], + ), + "_zip_main_maker": attr.label( + cfg = "exec", + default = "//tools/private/zipapp:zip_main_maker", + ), + "_zip_shell_template": attr.label( + default = ":zip_shell_template", + allow_single_file = True, + ), + "_zipper": attr.label( + cfg = "exec", + default = "//tools/private/zipapp:zipper", + ), +} | ({ + "_windows_launcher_maker": attr.label( + default = "@bazel_tools//tools/launcher:launcher_maker", + cfg = "exec", + executable = True, + ), +} if not rp_config.bazel_9_or_later else {}) + +_TOOLCHAINS = [EXEC_TOOLS_TOOLCHAIN_TYPE] + ([LAUNCHER_MAKER_TOOLCHAIN_TYPE] if rp_config.bazel_9_or_later else []) + +_COMMON_RULE_DOC = """ + +Output groups: + +* `python_zip_file`: (*deprecated*) The plain, non-self-executable zipapp zipfile. + *This output group is deprecated and retained for compatibility with + the previous implicit zipapp functionality. Set `executable=False` + and use the default output of the target instead.* + +:::{versionadded} 1.9.0 +::: +""".lstrip() + +py_zipapp_binary = rule( + doc = """ +Packages a `py_binary` as a Python zipapp. + +{} +""".format(_COMMON_RULE_DOC), + implementation = _py_zipapp_executable_impl, + attrs = _ATTRS, + # NOTE: While this is marked executable, it is conditionally executable + # based on the `executable` attribute. + executable = True, + toolchains = _TOOLCHAINS, + cfg = _zipapp_transition, +) + +py_zipapp_test = rule( + doc = """ +Packages a `py_test` as a Python zipapp. + +This target is also a valid test target to run. + +{} +""".format(_COMMON_RULE_DOC), + implementation = _py_zipapp_executable_impl, + attrs = _ATTRS, + # NOTE: While this is marked as a test, it is conditionally executable + # based on the `executable` attribute. + test = True, + toolchains = _TOOLCHAINS, + cfg = _zipapp_transition, +) diff --git a/python/private/zipapp/zip_main_template.py b/python/private/zipapp/zip_main_template.py new file mode 100644 index 0000000000..709e08815c --- /dev/null +++ b/python/private/zipapp/zip_main_template.py @@ -0,0 +1,404 @@ +# Template for the __main__.py file inserted into zip files +# +# Generated file from @rules_python//python/private/zipapp:zip_main_template.py +# +# NOTE: This file is a "stage 1" bootstrap, so it's responsible for locating the +# desired runtime and having it run the stage 2 bootstrap. This means it can't +# assume much about the current runtime and environment. e.g., the current +# runtime may not be the correct one, the zip may not have been extracted, the +# runfiles env vars may not be set, etc. +# +# NOTE: This program must retain compatibility with a wide variety of Python +# versions since it is run by an unknown Python interpreter. +# +# NOTE: For a self-executable zip, this file may not be the entry point +# for the program and may be skipped entirely; the self-executable zip +# preamble may jump directly to the stage2 bootstrap. + +import sys + +# The Python interpreter unconditionally prepends the directory containing this +# script (following symlinks) to the import path. This is the cause of #9239, +# and is a special case of #7091. We therefore explicitly delete that entry. +# TODO(#7091): Remove this hack when no longer necessary. +del sys.path[0] + +import os # noqa: E402 +import shutil # noqa: E402 +import stat # noqa: E402 +import subprocess # noqa: E402 +import tempfile # noqa: E402 +import zipfile # noqa: E402 +from os.path import basename, dirname, join, normpath # noqa: E402 + +# runfiles-root-relative path +_STAGE2_BOOTSTRAP = "%stage2_bootstrap%" +# runfiles-root-relative path to venv's bin/python3. Empty if venv not being used. +_PYTHON_BINARY_VENV = "%python_binary%" +# runfiles-root-relative path, absolute path, or single word. The actual Python +# executable to use. +_PYTHON_BINARY_ACTUAL = "%python_binary_actual%" +_WORKSPACE_NAME = "%workspace_name%" +# relative path under EXTRACT_ROOT to extract to. +EXTRACT_DIR = "%EXTRACT_DIR%" +APP_HASH = "%APP_HASH%" + +EXTRACT_ROOT = os.environ.get("RULES_PYTHON_EXTRACT_ROOT") +IS_WINDOWS = os.name == "nt" + + +# Change the paths with Unix-style forward slashes to backslashes for Windows. +# Windows usually transparently rewrites them, but e.g. `\\?\` paths require +# backslashes to be properly understood by Windows APIs. +if IS_WINDOWS: + + def norm_slashes(s): + if not s: + return s + return s.replace("/", "\\") + + _STAGE2_BOOTSTRAP = norm_slashes(_STAGE2_BOOTSTRAP) + _PYTHON_BINARY_VENV = norm_slashes(_PYTHON_BINARY_VENV) + _PYTHON_BINARY_ACTUAL = norm_slashes(_PYTHON_BINARY_ACTUAL) + EXTRACT_DIR = norm_slashes(EXTRACT_DIR) + EXTRACT_ROOT = norm_slashes(EXTRACT_ROOT) + +IS_VERBOSE = bool(os.environ.get("RULES_PYTHON_BOOTSTRAP_VERBOSE")) + + +def print_verbose(*args, mapping=None, values=None): + if not IS_VERBOSE: + return + if mapping is not None: + for key, value in sorted((mapping or {}).items()): + print( + "bootstrap: stage 1:", + *args, + f"{key}={value!r}", + file=sys.stderr, + flush=True, + ) + elif values is not None: + for i, v in enumerate(values): + print( + "bootstrap: stage 1:", + *args, + f"[{i}] {v!r}", + file=sys.stderr, + flush=True, + ) + else: + print("bootstrap: stage 1:", *args, file=sys.stderr, flush=True) + + +def get_windows_path_with_unc_prefix(path): + """Adds UNC prefix after getting a normalized absolute Windows path. + + No-op for non-Windows platforms or if running under python2. + """ + path = path.strip() + + # No need to add prefix for non-Windows platforms. + # And \\?\ doesn't work in python 2 or on mingw + if not IS_WINDOWS or sys.version_info[0] < 3: + return path + + # Starting in Windows 10, version 1607(OS build 14393), MAX_PATH limitations have been + # removed from common Win32 file and directory functions. + # Related doc: https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=cmd#enable-long-paths-in-windows-10-version-1607-and-later + import platform + + if platform.win32_ver()[1] >= "10.0.14393": + return path + + # import sysconfig only now to maintain python 2.6 compatibility + import sysconfig + + if sysconfig.get_platform() == "mingw": + return path + + # Lets start the unicode fun + unicode_prefix = "\\\\?\\" + if path.startswith(unicode_prefix): + return path + + # os.path.abspath returns a normalized absolute path + return unicode_prefix + os.path.abspath(path) + + +def search_path(name): + """Finds a file in a given search path.""" + search_path = os.getenv("PATH", os.defpath).split(os.pathsep) + for directory in search_path: + if directory: + path = join(directory, name) + if os.path.isfile(path) and os.access(path, os.X_OK): + return path + return None + + +def find_binary(runfiles_root, bin_name): + """Finds the real binary if it's not a normal absolute path.""" + if not bin_name: + return None + if bin_name.startswith("//"): + # Case 1: Path is a label. Not supported yet. + raise AssertionError( + "Bazel does not support execution of Python interpreters via labels" + ) + elif os.path.isabs(bin_name): + # Case 2: Absolute path. + return bin_name + # Use normpath() to convert slashes to os.sep on Windows. + elif os.sep in normpath(bin_name): + # Case 3: Path is relative to the repo root. + return join(runfiles_root, bin_name) + else: + # Case 4: Path has to be looked up in the search path. + return search_path(bin_name) + + +def extract_zip(zip_path, dest_dir): + """Extracts the contents of a zip file, preserving the unix file mode bits. + + These include the permission bits, and in particular, the executable bit. + + Ideally the zipfile module should set these bits, but it doesn't. See: + https://bugs.python.org/issue15795. + + Args: + zip_path: The path to the zip file to extract + dest_dir: The path to the destination directory + """ + zip_path = get_windows_path_with_unc_prefix(zip_path) + dest_dir = get_windows_path_with_unc_prefix(dest_dir) + with zipfile.ZipFile(zip_path) as zf: + for info in zf.infolist(): + file_path = os.path.abspath(join(dest_dir, info.filename)) + # If the file exists, it might be a symlink or read-only file from a previous extraction. + # Unlink it first so zipfile.extract doesn't corrupt the symlink target or fail on read-only files. + if os.path.lexists(file_path) and not os.path.isdir(file_path): + try: + os.unlink(file_path) + except OSError: + # On Windows, unlinking a read-only file fails. + os.chmod(file_path, stat.S_IWRITE) + os.unlink(file_path) + + zf.extract(info, dest_dir) + # The Unix st_mode bits (see "man 7 inode") are stored in the upper 16 + # bits of external_attr. + attrs = info.external_attr >> 16 + # Symlink bit in st_mode is 0o120000. + if (attrs & 0o170000) == 0o120000: + with open(file_path, "r") as f: + target = f.read() + os.remove(file_path) + if IS_WINDOWS: + entry_path = normpath(join(dirname(info.filename), target)) + # Zip lookup uses forward slashes, target has backslashes. + entry_path = entry_path.replace("\\", "/") + try: + target_is_directory = zf.getinfo(entry_path).is_dir() + except KeyError: + # Directories aren't stored in zips, so a missing + # target means it points to a directory. + target_is_directory = True + else: + target_is_directory = False + os.symlink(target, file_path, target_is_directory=target_is_directory) + # Of those, we set the lower 12 bits, which are the + # file mode bits (since the file type bits can't be set by chmod anyway). + elif attrs != 0: # Rumor has it these can be 0 for zips created on Windows. + os.chmod(file_path, attrs & 0o7777) + + +# Create the runfiles tree by extracting the zip file +def create_runfiles_root(): + if EXTRACT_ROOT: + # Shorten the path for Windows in case long path support is disabled + if IS_WINDOWS: + hash_dir = APP_HASH[0:32] + extract_dir = basename(EXTRACT_DIR) + extract_root = join(EXTRACT_ROOT, extract_dir, hash_dir) + else: + extract_root = join(EXTRACT_ROOT, EXTRACT_DIR, APP_HASH) + extract_root = get_windows_path_with_unc_prefix(extract_root) + else: + extract_root = tempfile.mkdtemp("", "Bazel.runfiles_") + + extract_zip(dirname(__file__), extract_root) + print_verbose("extracted to:", extract_root) + # IMPORTANT: Later code does `rm -fr` on dirname(runfiles_root) -- it's + # important that deletion code be in sync with this directory structure + return join(extract_root, "runfiles") + + +def execute_file( + python_program, + main_filename, + args, + env, + runfiles_root, + workspace, +): + # type: (str, str, list[str], dict[str, str], str, str|None, str|None) -> ... + """Executes the given Python file using the various environment settings. + + This will not return, and acts much like os.execv, except is much + more restricted, and handles Bazel-related edge cases. + + Args: + python_program: (str) Path to the Python binary to use for execution + main_filename: (str) The Python file to execute + args: (list[str]) Additional args to pass to the Python file + env: (dict[str, str]) A dict of environment variables to set for the execution + runfiles_root: (str) Path to the runfiles tree directory + workspace: (str|None) Name of the workspace to execute in. This is expected to be a + directory under the runfiles tree. + """ + # We want to use os.execv instead of subprocess.call, which causes + # problems with signal passing (making it difficult to kill + # Bazel). However, these conditions force us to run via + # subprocess.call instead: + # + # - On Windows, os.execv doesn't handle arguments with spaces + # correctly, and it actually starts a subprocess just like + # subprocess.call. + # - When running in a zip file, we need to clean up the + # workspace after the process finishes so control must return here. + try: + subprocess_argv = [python_program] + if not EXTRACT_ROOT: + subprocess_argv.append(f"-XRULES_PYTHON_ZIP_DIR={dirname(runfiles_root)}") + subprocess_argv.append(main_filename) + subprocess_argv += args + print_verbose("subprocess env:", mapping=env) + print_verbose("subprocess cwd:", workspace) + print_verbose("subprocess argv:", values=subprocess_argv) + ret_code = subprocess.call(subprocess_argv, env=env, cwd=workspace) + print_verbose("subprocess exit code:", ret_code) + sys.exit(ret_code) + finally: + if not EXTRACT_ROOT: + # NOTE: dirname() is called because create_runfiles_root() creates a + # sub-directory within a temporary directory, and we want to remove the + # whole temporary directory. + extract_root = dirname(runfiles_root) + print_verbose("cleanup: rmtree: ", extract_root) + shutil.rmtree(extract_root, True) + + +def finish_venv_setup(runfiles_root): + python_program = os.path.join(runfiles_root, _PYTHON_BINARY_VENV) + # When a venv is used, the `bin/python3` symlink may need to be created. + # This case occurs when "create venv at runtime" or "resolve python at + # runtime" modes are enabled. + if not os.path.exists(python_program): + # The venv bin/python3 interpreter should always be under runfiles, but + # double check. We don't want to accidentally create symlinks elsewhere + if not python_program.startswith(runfiles_root): + raise AssertionError( + "Program's venv binary not under runfiles: {python_program}" + ) + symlink_to = find_binary(runfiles_root, _PYTHON_BINARY_ACTUAL) + os.makedirs(dirname(python_program), exist_ok=True) + if os.path.lexists(python_program): + os.remove(python_program) + try: + os.symlink(symlink_to, python_program) + except OSError as e: + raise Exception( + f"Unable to create venv python interpreter symlink: {python_program} -> {symlink_to}" + ) from e + venv_root = dirname(dirname(python_program)) + pyvenv_cfg = join(venv_root, "pyvenv.cfg") + if not os.path.exists(pyvenv_cfg): + print_verbose("finish_venv_setup: create pyvenv.cfg:", pyvenv_cfg) + python_home = join(runfiles_root, dirname(_PYTHON_BINARY_ACTUAL)) + print_verbose("finish_venv_setup: pyvenv.cfg home:", python_home) + with open(pyvenv_cfg, "w") as fp: + # Until Windows supports a build-time generated venv using symlinks + # to directories, we have to write the full, absolute, path to PYTHONHOME + # so that support directories (e.g. DLLs, libs) can be found. + fp.write("home = {}\n".format(python_home)) + + return python_program + + +def main(): + print_verbose("running zip main bootstrap") + print_verbose("initial sys.version:", sys.version) + print_verbose("initial sys.executable:", sys.executable) + print_verbose("initial argv:", values=sys.argv) + print_verbose("initial environ:", mapping=os.environ) + print_verbose("stage2_bootstrap:", _STAGE2_BOOTSTRAP) + print_verbose("python_binary_venv:", _PYTHON_BINARY_VENV) + print_verbose("python_binary_actual:", _PYTHON_BINARY_ACTUAL) + print_verbose("workspace_name:", _WORKSPACE_NAME) + + args = sys.argv[1:] + + new_env = {} + + # The main Python source file. + main_rel_path = _STAGE2_BOOTSTRAP + if IS_WINDOWS: + main_rel_path = main_rel_path.replace("/", os.sep) + + runfiles_root = create_runfiles_root() + print_verbose("extracted runfiles to:", runfiles_root) + + new_env["RUNFILES_DIR"] = runfiles_root + + # Don't prepend a potentially unsafe path to sys.path + # See: https://docs.python.org/3.11/using/cmdline.html#envvar-PYTHONSAFEPATH + new_env["PYTHONSAFEPATH"] = "1" + + main_filename = join(runfiles_root, main_rel_path) + main_filename = get_windows_path_with_unc_prefix(main_filename) + assert os.path.exists(main_filename), ( + "Cannot exec() %r: file not found." % main_filename + ) + assert os.access(main_filename, os.R_OK), ( + "Cannot exec() %r: file not readable." % main_filename + ) + + if _PYTHON_BINARY_VENV: + python_program = finish_venv_setup(runfiles_root) + else: + python_program = find_binary(runfiles_root, _PYTHON_BINARY_ACTUAL) + if python_program is None: + raise AssertionError( + "Could not find python binary: " + _PYTHON_BINARY_ACTUAL + ) + + # Some older Python versions on macOS (namely Python 3.7) may unintentionally + # leave this environment variable set after starting the interpreter, which + # causes problems with Python subprocesses correctly locating sys.executable, + # which subsequently causes failure to launch on Python 3.11 and later. + if "__PYVENV_LAUNCHER__" in os.environ: + del os.environ["__PYVENV_LAUNCHER__"] + + new_env.update((key, val) for key, val in os.environ.items() if key not in new_env) + + workspace = None + # If RUN_UNDER_RUNFILES equals 1, it means we need to + # change directory to the right runfiles directory. + # (So that the data files are accessible) + if os.environ.get("RUN_UNDER_RUNFILES") == "1": + workspace = join(runfiles_root, _WORKSPACE_NAME) + + sys.stdout.flush() + execute_file( + python_program, + main_filename, + args, + new_env, + runfiles_root, + workspace, + ) + + +if __name__ == "__main__": + main() diff --git a/python/private/zipapp/zip_shell_template.sh b/python/private/zipapp/zip_shell_template.sh new file mode 100644 index 0000000000..d79331444a --- /dev/null +++ b/python/private/zipapp/zip_shell_template.sh @@ -0,0 +1,88 @@ +#!/usr/bin/env bash + +set -e + +if [[ -n "${RULES_PYTHON_BOOTSTRAP_VERBOSE:-}" ]]; then + set -x +fi + +# runfiles-root-relative path +BUNDLED_PYEXE_PATH="%BUNDLED_PYEXE_PATH%" +# Absolute path or single word +EXTERNAL_PYEXE_PATH="%EXTERNAL_PYEXE_PATH%" +# runfiles-root-relative path +STAGE2_BOOTSTRAP="%STAGE2_BOOTSTRAP%" +EXTRACT_DIR="%EXTRACT_DIR%" +ZIP_HASH="%ZIP_HASH%" +declare -a INTERPRETER_ARGS_FROM_TARGET=( +%INTERPRETER_ARGS% +) + +declare -a interpreter_env +declare -a interpreter_args +declare -a additional_interpreter_args + +if [[ -z "${PYTHONSAFEPATH+x}" ]]; then + # ${FOO-WORD} expands to WORD if $FOO is undefined, and $FOO otherwise + interpreter_env+=("PYTHONSAFEPATH=${PYTHONSAFEPATH-1}") +fi + + +if [[ -n "${RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS}" ]]; then + read -a additional_interpreter_args <<< "${RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS}" + interpreter_args+=("${additional_interpreter_args[@]}") + unset RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS +fi + + +if [[ -n "$RULES_PYTHON_EXTRACT_ROOT" ]]; then + zip_dir="$RULES_PYTHON_EXTRACT_ROOT/$EXTRACT_DIR/$ZIP_HASH" + if [[ ! -e "$zip_dir/__main__.py" ]]; then + mkdir -p "$zip_dir" + # Unzip emits a warning and exits 1 with the prelude + ( unzip -q -d "$zip_dir" "$0" 2>/dev/null || true ) + fi +else + # NOTE: Macs have an old version of mktemp, so we must use only the + # minimal functionality of it. + zip_dir=$(mktemp -d) + # Unzip emits a warning and exits 1 with the prelude + ( unzip -q -d "$zip_dir" "$0" 2>/dev/null || true ) + if [[ -n "$zip_dir" && -z "${RULES_PYTHON_BOOTSTRAP_VERBOSE:-}" ]]; then + trap 'rm -fr "$zip_dir"' EXIT + fi +fi + +export RUNFILES_DIR="$zip_dir/runfiles" +if [[ ! -d "$RUNFILES_DIR" ]]; then + echo "Runfiles dir not found: zip extraction likely failed" 1>&2 + echo "Run with RULES_PYTHON_BOOTSTRAP_VERBOSE=1 to aid debugging" 1>&2 + exit 1 +fi + +if [[ -n "$BUNDLED_PYEXE_PATH" ]]; then + python_exe="$RUNFILES_DIR/$BUNDLED_PYEXE_PATH" +else + python_exe="$EXTERNAL_PYEXE_PATH" +fi + +command=( + env + "${interpreter_env[@]}" + "$python_exe" + "-XRULES_PYTHON_ZIP_DIR=$zip_dir" + "${interpreter_args[@]}" + "${INTERPRETER_ARGS_FROM_TARGET[@]}" + "$RUNFILES_DIR/$STAGE2_BOOTSTRAP" + "$@" +) + +# NOTE: because exec isn't used, signals don't propagate to the child +# TODO: Use exec and let the program handle cleanup. Without exec, +# signals don't propagate to the child nicely. +# See https://github.com/bazel-contrib/rules_python/issues/2043#issuecomment-2215469971 +# for more information. +"${command[@]}" +# Explicit exit is needed because the implicit next line the zip file this +# template is prepended to. +exit 0 diff --git a/python/private/zipapp/zipapp_stage2_bootstrap_template.py b/python/private/zipapp/zipapp_stage2_bootstrap_template.py new file mode 100644 index 0000000000..9fd71fefb9 --- /dev/null +++ b/python/private/zipapp/zipapp_stage2_bootstrap_template.py @@ -0,0 +1,10 @@ +import runpy +import shutil +import sys + +try: + sys.argv.pop(0) # Remove zipapp_stage2_bootstrap from args + runpy.run_path(sys.argv[0], run_name="__main__") +finally: + if zip_dir := sys._xoptions.get("RULES_PYTHON_ZIP_DIR"): + shutil.rmtree(zip_dir, True) diff --git a/python/proto.bzl b/python/proto.bzl index 3f455aee58..2ea9bdb153 100644 --- a/python/proto.bzl +++ b/python/proto.bzl @@ -11,11 +11,11 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. - """ Python proto library. """ -load("//python/private/proto:py_proto_library.bzl", _py_proto_library = "py_proto_library") +load("@com_google_protobuf//bazel:py_proto_library.bzl", _py_proto_library = "py_proto_library") -py_proto_library = _py_proto_library +def py_proto_library(*, deprecation = "Use py_proto_library from protobuf repository", **kwargs): + _py_proto_library(deprecation = deprecation, **kwargs) diff --git a/python/proto/BUILD.bazel b/python/proto/BUILD.bazel index 9f60574f26..4d5a92a93f 100644 --- a/python/proto/BUILD.bazel +++ b/python/proto/BUILD.bazel @@ -14,5 +14,11 @@ package(default_visibility = ["//visibility:public"]) -# Toolchain type provided by proto_lang_toolchain rule and used by py_proto_library -toolchain_type(name = "toolchain_type") +# Deprecated; use @com_google_protobuf//bazel/private:python_toolchain_type instead. +# Alias is here to provide backward-compatibility; see #2604 +# It will be removed in a future release. +alias( + name = "toolchain_type", + actual = "@com_google_protobuf//bazel/private:python_toolchain_type", + deprecation = "Use @com_google_protobuf//bazel/private:python_toolchain_type instead", +) diff --git a/python/py_binary.bzl b/python/py_binary.bzl index c7d57dab49..1b59451893 100644 --- a/python/py_binary.bzl +++ b/python/py_binary.bzl @@ -14,13 +14,7 @@ """Public entry point for py_binary.""" -load("@rules_python_internal//:rules_python_config.bzl", "config") -load("//python/private:py_binary_macro.bzl", _starlark_py_binary = "py_binary") -load("//python/private:register_extension_info.bzl", "register_extension_info") -load("//python/private:util.bzl", "add_migration_tag") - -# buildifier: disable=native-python -_py_binary_impl = _starlark_py_binary if config.enable_pystar else native.py_binary +load("//python/private:py_binary_macro.bzl", _py_binary = "py_binary") def py_binary(**attrs): """Creates an executable Python program. @@ -34,17 +28,17 @@ def py_binary(**attrs): * `srcs_version`: cannot be `PY2` or `PY2ONLY` * `tags`: May have special marker values added, if not already present. + :::{versionchanged} 1.9.0 + The `PYTHONBREAKPOINT` environment variable is inherited. Use in combination + with {obj}`--debugger` to customize the debugger available and used. + ::: + Args: **attrs: Rule attributes forwarded onto the underlying {rule}`py_binary`. """ if attrs.get("python_version") == "PY2": - fail("Python 2 is no longer supported: https://github.com/bazelbuild/rules_python/issues/886") + fail("Python 2 is no longer supported: https://github.com/bazel-contrib/rules_python/issues/886") if attrs.get("srcs_version") in ("PY2", "PY2ONLY"): - fail("Python 2 is no longer supported: https://github.com/bazelbuild/rules_python/issues/886") - - _py_binary_impl(**add_migration_tag(attrs)) + fail("Python 2 is no longer supported: https://github.com/bazel-contrib/rules_python/issues/886") -register_extension_info( - extension = py_binary, - label_regex_for_dep = "{extension_name}", -) + _py_binary(**attrs) diff --git a/python/py_cc_link_params_info.bzl b/python/py_cc_link_params_info.bzl index 02eff71c4d..9f25f17f4b 100644 --- a/python/py_cc_link_params_info.bzl +++ b/python/py_cc_link_params_info.bzl @@ -1,10 +1,5 @@ """Public entry point for PyCcLinkParamsInfo.""" -load("@rules_python_internal//:rules_python_config.bzl", "config") -load("//python/private:py_cc_link_params_info.bzl", _starlark_PyCcLinkParamsInfo = "PyCcLinkParamsInfo") +load("//python/private:py_cc_link_params_info.bzl", _PyCcLinkParamsInfo = "PyCcLinkParamsInfo") -PyCcLinkParamsInfo = ( - _starlark_PyCcLinkParamsInfo if ( - config.enable_pystar or config.BuiltinPyCcLinkParamsProvider == None - ) else config.BuiltinPyCcLinkParamsProvider -) +PyCcLinkParamsInfo = _PyCcLinkParamsInfo diff --git a/python/py_info.bzl b/python/py_info.bzl index 5697f58419..8e46926c53 100644 --- a/python/py_info.bzl +++ b/python/py_info.bzl @@ -14,8 +14,17 @@ """Public entry point for PyInfo.""" -load("@rules_python_internal//:rules_python_config.bzl", "config") -load("//python/private:py_info.bzl", _starlark_PyInfo = "PyInfo") -load("//python/private:reexports.bzl", "BuiltinPyInfo") +load( + "//python/private:py_info.bzl", + _PyInfo = "PyInfo", + _VenvSymlinkEntry = "VenvSymlinkEntry", + _VenvSymlinkKind = "VenvSymlinkKind", +) -PyInfo = _starlark_PyInfo if config.enable_pystar or BuiltinPyInfo == None else BuiltinPyInfo +PyInfo = _PyInfo + +# buildifier: disable=name-conventions +VenvSymlinkEntry = _VenvSymlinkEntry + +# buildifier: disable=name-conventions +VenvSymlinkKind = _VenvSymlinkKind diff --git a/python/py_library.bzl b/python/py_library.bzl index 12354a7deb..c5cae241f6 100644 --- a/python/py_library.bzl +++ b/python/py_library.bzl @@ -14,13 +14,7 @@ """Public entry point for py_library.""" -load("@rules_python_internal//:rules_python_config.bzl", "config") -load("//python/private:py_library_macro.bzl", _starlark_py_library = "py_library") -load("//python/private:register_extension_info.bzl", "register_extension_info") -load("//python/private:util.bzl", "add_migration_tag") - -# buildifier: disable=native-python -_py_library_impl = _starlark_py_library if config.enable_pystar else native.py_library +load("//python/private:py_library_macro.bzl", _py_library = "py_library") def py_library(**attrs): """Creates an executable Python program. @@ -37,11 +31,6 @@ def py_library(**attrs): **attrs: Rule attributes forwarded onto {rule}`py_library`. """ if attrs.get("srcs_version") in ("PY2", "PY2ONLY"): - fail("Python 2 is no longer supported: https://github.com/bazelbuild/rules_python/issues/886") - - _py_library_impl(**add_migration_tag(attrs)) + fail("Python 2 is no longer supported: https://github.com/bazel-contrib/rules_python/issues/886") -register_extension_info( - extension = py_library, - label_regex_for_dep = "{extension_name}", -) + _py_library(**attrs) diff --git a/python/py_runtime.bzl b/python/py_runtime.bzl index 2c44523505..86f644a770 100644 --- a/python/py_runtime.bzl +++ b/python/py_runtime.bzl @@ -14,11 +14,7 @@ """Public entry point for py_runtime.""" -load("//python/private:py_runtime_macro.bzl", _starlark_py_runtime = "py_runtime") -load("//python/private:util.bzl", "IS_BAZEL_6_OR_HIGHER", "add_migration_tag") - -# buildifier: disable=native-python -_py_runtime_impl = _starlark_py_runtime if IS_BAZEL_6_OR_HIGHER else native.py_runtime +load("//python/private:py_runtime_macro.bzl", _py_runtime = "py_runtime") def py_runtime(**attrs): """Creates an executable Python program. @@ -37,6 +33,6 @@ def py_runtime(**attrs): **attrs: Rule attributes forwarded onto {rule}`py_runtime`. """ if attrs.get("python_version") == "PY2": - fail("Python 2 is no longer supported: see https://github.com/bazelbuild/rules_python/issues/886") + fail("Python 2 is no longer supported: see https://github.com/bazel-contrib/rules_python/issues/886") - _py_runtime_impl(**add_migration_tag(attrs)) + _py_runtime(**attrs) diff --git a/python/py_runtime_info.bzl b/python/py_runtime_info.bzl index 3a31c0f2f4..ac7b1b0fb6 100644 --- a/python/py_runtime_info.bzl +++ b/python/py_runtime_info.bzl @@ -14,8 +14,6 @@ """Public entry point for PyRuntimeInfo.""" -load("@rules_python_internal//:rules_python_config.bzl", "config") -load("//python/private:py_runtime_info.bzl", _starlark_PyRuntimeInfo = "PyRuntimeInfo") -load("//python/private:reexports.bzl", "BuiltinPyRuntimeInfo") +load("//python/private:py_runtime_info.bzl", _PyRuntimeInfo = "PyRuntimeInfo") -PyRuntimeInfo = _starlark_PyRuntimeInfo if config.enable_pystar else BuiltinPyRuntimeInfo +PyRuntimeInfo = _PyRuntimeInfo diff --git a/python/py_runtime_pair.bzl b/python/py_runtime_pair.bzl index b1e90414a2..f44c722d54 100644 --- a/python/py_runtime_pair.bzl +++ b/python/py_runtime_pair.bzl @@ -14,11 +14,7 @@ """Public entry point for py_runtime_pair.""" -load("@bazel_tools//tools/python:toolchain.bzl", _bazel_tools_impl = "py_runtime_pair") -load("//python/private:py_runtime_pair_macro.bzl", _starlark_impl = "py_runtime_pair") -load("//python/private:util.bzl", "IS_BAZEL_6_OR_HIGHER") - -_py_runtime_pair = _starlark_impl if IS_BAZEL_6_OR_HIGHER else _bazel_tools_impl +load("//python/private:py_runtime_pair_macro.bzl", _py_runtime_pair = "py_runtime_pair") # NOTE: This doc is copy/pasted from the builtin py_runtime_pair rule so our # doc generator gives useful API docs. @@ -85,7 +81,7 @@ def py_runtime_pair(name, py2_runtime = None, py3_runtime = None, **attrs): **attrs: Extra attrs passed onto the native rule """ if attrs.get("py2_runtime"): - fail("PYthon 2 is no longer supported: see https://github.com/bazelbuild/rules_python/issues/886") + fail("PYthon 2 is no longer supported: see https://github.com/bazel-contrib/rules_python/issues/886") _py_runtime_pair( name = name, py2_runtime = py2_runtime, diff --git a/python/py_test.bzl b/python/py_test.bzl index 7f6626e0e5..2f31e0865e 100644 --- a/python/py_test.bzl +++ b/python/py_test.bzl @@ -14,13 +14,7 @@ """Public entry point for py_test.""" -load("@rules_python_internal//:rules_python_config.bzl", "config") -load("//python/private:py_test_macro.bzl", _starlark_py_test = "py_test") -load("//python/private:register_extension_info.bzl", "register_extension_info") -load("//python/private:util.bzl", "add_migration_tag") - -# buildifier: disable=native-python -_py_test_impl = _starlark_py_test if config.enable_pystar else native.py_test +load("//python/private:py_test_macro.bzl", _py_test = "py_test") def py_test(**attrs): """Creates an executable Python program. @@ -38,14 +32,9 @@ def py_test(**attrs): **attrs: Rule attributes forwarded onto {rule}`py_test`. """ if attrs.get("python_version") == "PY2": - fail("Python 2 is no longer supported: https://github.com/bazelbuild/rules_python/issues/886") + fail("Python 2 is no longer supported: https://github.com/bazel-contrib/rules_python/issues/886") if attrs.get("srcs_version") in ("PY2", "PY2ONLY"): - fail("Python 2 is no longer supported: https://github.com/bazelbuild/rules_python/issues/886") + fail("Python 2 is no longer supported: https://github.com/bazel-contrib/rules_python/issues/886") # buildifier: disable=native-python - _py_test_impl(**add_migration_tag(attrs)) - -register_extension_info( - extension = py_test, - label_regex_for_dep = "{extension_name}", -) + _py_test(**attrs) diff --git a/python/python.bzl b/python/python.bzl index 3e739ca55d..cfbf25b5b5 100644 --- a/python/python.bzl +++ b/python/python.bzl @@ -14,11 +14,7 @@ """Re-exports for some of the core Bazel Python rules. -This file is deprecated; please use the exports in defs.bzl instead. This is to -follow the new naming convention of putting core rules for a language -underneath @rules_//:defs.bzl. The exports in this file will be -disallowed in a future Bazel release by -`--incompatible_load_python_rules_from_bzl`. +This file is deprecated; please use the exports in `.bzl` files instead. """ def py_library(*args, **kwargs): diff --git a/python/runfiles/BUILD.bazel b/python/runfiles/BUILD.bazel index c1fc027fa4..4e119eddbe 100644 --- a/python/runfiles/BUILD.bazel +++ b/python/runfiles/BUILD.bazel @@ -12,8 +12,8 @@ # See the License for the specific language governing permissions and # limitations under the License. -load("//python:defs.bzl", "py_library") load("//python:packaging.bzl", "py_wheel") +load("//python:py_library.bzl", "py_library") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") filegroup( @@ -22,24 +22,31 @@ filegroup( visibility = ["//python:__pkg__"], ) +filegroup( + name = "py_typed", + # See PEP 561: py.typed is a special file that indicates the code supports type checking + srcs = ["py.typed"], +) + py_library( name = "runfiles", srcs = [ "__init__.py", "runfiles.py", ], - data = ["py.typed"], + data = [":py_typed"], imports = [ # Add the repo root so `import python.runfiles.runfiles` works. This makes it agnostic # to the --experimental_python_import_all_repositories setting. "../..", ], + tags = ["pyrefly"], visibility = ["//visibility:public"], ) # This can be manually tested by running tests/runfiles/runfiles_wheel_integration_test.sh # We ought to have an automated integration test for it, too. -# see https://github.com/bazelbuild/rules_python/issues/1002 +# see https://github.com/bazel-contrib/rules_python/issues/1002 py_wheel( name = "wheel", # From https://pypi.org/classifiers/ @@ -50,12 +57,15 @@ py_wheel( description_file = "README.md", dist_folder = "dist", distribution = "bazel_runfiles", - homepage = "https://github.com/bazelbuild/rules_python", + homepage = "https://github.com/bazel-contrib/rules_python", python_requires = ">=3.7", strip_path_prefixes = ["python"], twine = None if BZLMOD_ENABLED else "@rules_python_publish_deps_twine//:pkg", # this can be replaced by building with --stamp --embed_label=1.2.3 version = "{BUILD_EMBED_LABEL}", visibility = ["//visibility:public"], - deps = [":runfiles"], + deps = [ + ":py_typed", + ":runfiles", + ], ) diff --git a/python/runfiles/README.md b/python/runfiles/README.md index 2a57c76846..4ed51a3b12 100644 --- a/python/runfiles/README.md +++ b/python/runfiles/README.md @@ -59,7 +59,19 @@ with open(r.Rlocation("my_workspace/path/to/my/data.txt"), "r") as f: # ... ``` -The code above creates a manifest- or directory-based implementation based on the environment variables in `os.environ`. See `Runfiles.Create()` for more info. +Here `my_workspace` is the name you specified via `module(name = "...")` in your `MODULE.bazel` file (with `--enable_bzlmod`, default as of Bazel 7) or `workspace(name = "...")` in `WORKSPACE` (with `--noenable_bzlmod`). + +The code above creates a manifest- or directory-based implementation based on +the environment variables in `os.environ`. See `Runfiles.Create()` for more +info. + +Alternatively, `Runfiles.CreateOrRaise()` can be used to raise an error +instead of returning `None` if runfiles cannot be found: + +```python +r = Runfiles.CreateOrRaise() +``` + If you want to explicitly create a manifest- or directory-based implementation, you can do so as follows: @@ -70,9 +82,7 @@ r1 = Runfiles.CreateManifestBased("path/to/foo.runfiles_manifest") r2 = Runfiles.CreateDirectoryBased("path/to/foo.runfiles/") ``` -If you want to start subprocesses, and the subprocess can't automatically -find the correct runfiles directory, you can explicitly set the right -environment variables for them: +If you want to start subprocesses that access runfiles, you have to set the right environment variables for them: ```python import subprocess diff --git a/python/runfiles/__init__.py b/python/runfiles/__init__.py index 3dc4141749..cdd7d9af00 100644 --- a/python/runfiles/__init__.py +++ b/python/runfiles/__init__.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -from .runfiles import * +from .runfiles import * # noqa: F403 diff --git a/python/runfiles/runfiles.py b/python/runfiles/runfiles.py index ea816c64fd..02fceb3020 100644 --- a/python/runfiles/runfiles.py +++ b/python/runfiles/runfiles.py @@ -15,12 +15,379 @@ """Runfiles lookup library for Bazel-built Python binaries and tests. See @rules_python//python/runfiles/README.md for usage instructions. + +:::{versionadded} 1.7.0 +Support for Bazel's `--incompatible_compact_repo_mapping_manifest` flag was added. +This enables prefix-based repository mappings to reduce memory usage for large +dependency graphs under bzlmod. +::: """ + import inspect import os +import pathlib import posixpath import sys -from typing import Dict, Optional, Tuple, Union +from collections import defaultdict +from typing import Dict, Generator, Optional, Tuple, Union + +if sys.version_info >= (3, 11): + from typing import Self +elif sys.version_info >= (3, 10): + from typing import TypeAlias + + Self: TypeAlias = "Path" # type: ignore +else: + from typing import Any as Self + + +class _RepositoryMapping: + """Repository mapping for resolving apparent repository names to canonical ones. + + Handles both exact mappings and prefix-based mappings introduced by the + --incompatible_compact_repo_mapping_manifest flag. + """ + + def __init__( + self, + exact_mappings: Dict[Tuple[str, str], str], + prefixed_mappings: Dict[Tuple[str, str], str], + ) -> None: + """Initialize repository mapping with exact and prefixed mappings. + + Args: + exact_mappings: Dict mapping (source_canonical, target_apparent) -> target_canonical + prefixed_mappings: Dict mapping (source_prefix, target_apparent) -> target_canonical + """ + self._exact_mappings = exact_mappings + + # Group prefixed mappings by target_apparent for faster lookups + self._grouped_prefixed_mappings = defaultdict(list) + for ( + prefix_source, + target_app, + ), target_canonical in prefixed_mappings.items(): + self._grouped_prefixed_mappings[target_app].append( + (prefix_source, target_canonical) + ) + + @staticmethod + def create_from_file(repo_mapping_path: Optional[str]) -> "_RepositoryMapping": + """Create RepositoryMapping from a repository mapping manifest file. + + Args: + repo_mapping_path: Path to the repository mapping file, or None if not available + + Returns: + RepositoryMapping instance with parsed mappings + """ + # If the repository mapping file can't be found, that is not an error: We + # might be running without Bzlmod enabled or there may not be any runfiles. + # In this case, just apply empty repo mappings. + if not repo_mapping_path: + return _RepositoryMapping({}, {}) + + try: + with open(repo_mapping_path, "r", encoding="utf-8", newline="\n") as f: + content = f.read() + except FileNotFoundError: + return _RepositoryMapping({}, {}) + + exact_mappings = {} + prefixed_mappings = {} + for line in content.splitlines(): + source_canonical, target_apparent, target_canonical = line.split(",") + if source_canonical.endswith("*"): + # This is a prefixed mapping - remove the '*' for prefix matching + prefix = source_canonical[:-1] + prefixed_mappings[(prefix, target_apparent)] = target_canonical + else: + # This is an exact mapping + exact_mappings[(source_canonical, target_apparent)] = target_canonical + + return _RepositoryMapping(exact_mappings, prefixed_mappings) + + def lookup(self, source_repo: Optional[str], target_apparent: str) -> Optional[str]: + """Look up repository mapping for the given source and target. + + This handles both exact mappings and prefix-based mappings introduced by the + --incompatible_compact_repo_mapping_manifest flag. Exact mappings are tried + first, followed by prefix-based mappings where order matters. + + Args: + source_repo: Source canonical repository name + target_apparent: Target apparent repository name + + Returns: + target_canonical repository name, or None if no mapping exists + """ + if source_repo is None: + return None + + key = (source_repo, target_apparent) + + # Try exact mapping first + if key in self._exact_mappings: + return self._exact_mappings[key] + + # Try prefixed mapping if no exact match found + if target_apparent in self._grouped_prefixed_mappings: + for prefix_source, target_canonical in self._grouped_prefixed_mappings[ + target_apparent + ]: + if source_repo.startswith(prefix_source): + return target_canonical + + # No mapping found + return None + + def is_empty(self) -> bool: + """Check if this repository mapping is empty (no exact or prefixed mappings). + + Returns: + True if there are no mappings, False otherwise + """ + return ( + len(self._exact_mappings) == 0 and len(self._grouped_prefixed_mappings) == 0 + ) + + +class Path(pathlib.Path): + """A pathlib-like path object for runfiles. + + This class extends `pathlib.Path` and resolves paths + using the associated `Runfiles` instance when converted to a string. + """ + + # Mypy isn't smart enough to realize `self` in the methods + # refers to our Path class instead of pathlib.Path + _runfiles: Optional["Runfiles"] + _source_repo: Optional[str] + + # For Python < 3.12 compatibility when subclassing Path directly + _flavour = getattr(type(pathlib.Path()), "_flavour", None) + + def __new__( + cls, + *args: Union[str, os.PathLike], + runfiles: Optional["Runfiles"] = None, + source_repo: Optional[str] = None, + ) -> Self: + """Private constructor. Use Runfiles.root() to create instances.""" + obj = super().__new__(cls, *args) + # Type checkers might complain about adding attributes to Path, + # but this is standard for pathlib subclasses. + obj._runfiles = runfiles # type: ignore + obj._source_repo = source_repo # type: ignore + return obj + + def __init__( + self, + *args: Union[str, os.PathLike], + runfiles: Optional["Runfiles"] = None, + source_repo: Optional[str] = None, + ) -> None: + # In Python 3.12+, pathlib was refactored and Path.__init__ now accepts + # *args. Prior to 3.12, Path did not define __init__, so + # super().__init__(*args) would fall through to object.__init__, which + # raises a TypeError because it takes no arguments. + if sys.version_info >= (3, 12): + super().__init__(*args) + else: + super().__init__() + + # We override resolve() and absolute() to ensure that in Python < 3.12, + # where pathlib internally uses object.__new__ instead of our custom + # __new__ or with_segments(), the runfiles state is preserved. We delegate + # to self._as_path() because super().resolve() creates intermediate objects + # that would otherwise crash during internal stat() calls. + # override + def resolve(self, strict: bool = False) -> Self: + return type(self)( + self._as_path().resolve(strict=strict), + runfiles=self._runfiles, + source_repo=self._source_repo, + ) + + # override + def absolute(self) -> Self: + return type(self)( + self._as_path().absolute(), + runfiles=self._runfiles, + source_repo=self._source_repo, + ) + + # override + def with_segments(self, *pathsegments: Union[str, os.PathLike]) -> Self: + """Used by Python 3.12+ pathlib to create new path objects.""" + return type(self)( + *pathsegments, + runfiles=self._runfiles, + source_repo=self._source_repo, + ) + + # For Python < 3.12 + # override + def _make_child(self, args: Tuple[str, ...]) -> Self: + obj = super()._make_child(args) # type: ignore + obj._runfiles = self._runfiles # type: ignore + obj._source_repo = self._source_repo # type: ignore + return obj + + # override + @property + def parents(self) -> Tuple[Self, ...]: + return tuple( + type(self)( + p, + runfiles=self._runfiles, + source_repo=self._source_repo, + ) + for p in super().parents + ) + + # override + @property + def parent(self) -> Self: + return type(self)( + super().parent, + runfiles=self._runfiles, + source_repo=self._source_repo, + ) + + @property + def runfile_path(self) -> str: + """Returns the runfiles-root relative path.""" + path_posix = super().__str__().replace("\\", "/") + if path_posix == ".": + return "" + return path_posix + + # override + def with_name(self, name: str) -> Self: + return type(self)( + super().with_name(name), + runfiles=self._runfiles, + source_repo=self._source_repo, + ) + + # override + def with_suffix(self, suffix: str) -> Self: + return type(self)( + super().with_suffix(suffix), + runfiles=self._runfiles, + source_repo=self._source_repo, + ) + + def _as_path(self) -> pathlib.Path: + return pathlib.Path(str(self)) + + # override + def stat(self, *, follow_symlinks: bool = True) -> os.stat_result: + return self._as_path().stat(follow_symlinks=follow_symlinks) + + # override + def lstat(self) -> os.stat_result: + return self._as_path().lstat() + + # override + def exists(self) -> bool: + return self._as_path().exists() + + # override + def is_dir(self) -> bool: + return self._as_path().is_dir() + + # override + def is_file(self) -> bool: + return self._as_path().is_file() + + # override + def is_symlink(self) -> bool: + return self._as_path().is_symlink() + + # override + def is_block_device(self) -> bool: + return self._as_path().is_block_device() + + # override + def is_char_device(self) -> bool: + return self._as_path().is_char_device() + + # override + def is_fifo(self) -> bool: + return self._as_path().is_fifo() + + # override + def is_socket(self) -> bool: + return self._as_path().is_socket() + + # override + def open( # pyrefly: ignore[bad-override] + self, + mode: str = "r", + buffering: int = -1, + encoding: Optional[str] = None, + errors: Optional[str] = None, + newline: Optional[str] = None, + ): + return self._as_path().open( + mode=mode, + buffering=buffering, + encoding=encoding, + errors=errors, + newline=newline, + ) + + # override + def read_bytes(self) -> bytes: + return self._as_path().read_bytes() + + # override + def read_text( + self, encoding: Optional[str] = None, errors: Optional[str] = None + ) -> str: + return self._as_path().read_text(encoding=encoding, errors=errors) + + # override + def iterdir(self) -> Generator[Self, None, None]: + resolved = self._as_path() + for p in resolved.iterdir(): + yield self / p.name + + # override + def glob(self, pattern: str) -> Generator[Self, None, None]: + resolved = self._as_path() + for p in resolved.glob(pattern): + yield self / p.relative_to(resolved) + + # override + def rglob(self, pattern: str) -> Generator[Self, None, None]: + resolved = self._as_path() + for p in resolved.rglob(pattern): + yield self / p.relative_to(resolved) + + def __repr__(self) -> str: + return "runfiles.Path({!r})".format(self.runfile_path) + + def __str__(self) -> str: + path_posix = super().__str__().replace("\\", "/") + if not path_posix or path_posix == ".": + # pylint: disable=protected-access + return self._runfiles._python_runfiles_root # type: ignore + resolved = self._runfiles.Rlocation(path_posix, source_repo=self._source_repo) # type: ignore + if resolved is not None: + return resolved + + # pylint: disable=protected-access + return posixpath.join(self._runfiles._python_runfiles_root, path_posix) # type: ignore + + def __fspath__(self) -> str: + return str(self) + + def runfiles_root(self) -> Self: + """Returns a Path object representing the runfiles root.""" + return self._runfiles.root(source_repo=self._source_repo) # type: ignore class _ManifestBased: @@ -56,7 +423,7 @@ def RlocationChecked(self, path: str) -> Optional[str]: def _LoadRunfiles(path: str) -> Dict[str, str]: """Loads the runfiles manifest.""" result = {} - with open(path, "r") as f: + with open(path, "r", encoding="utf-8", newline="\n") as f: for line in f: line = line.rstrip("\n") if line.startswith(" "): @@ -112,6 +479,9 @@ def RlocationChecked(self, path: str) -> str: # runfiles strategy on those platforms. return posixpath.join(self._runfiles_root, path) + def _GetRunfilesDir(self) -> str: + return self._runfiles_root + def EnvVars(self) -> Dict[str, str]: return { "RUNFILES_DIR": self._runfiles_root, @@ -129,11 +499,21 @@ class Runfiles: def __init__(self, strategy: Union[_ManifestBased, _DirectoryBased]) -> None: self._strategy = strategy - self._python_runfiles_root = _FindPythonRunfilesRoot() - self._repo_mapping = _ParseRepoMapping( + self._python_runfiles_root = strategy._GetRunfilesDir() + self._repo_mapping = _RepositoryMapping.create_from_file( strategy.RlocationChecked("_repo_mapping") ) + def root(self, source_repo: Optional[str] = None) -> Path: + """Returns a Path object representing the runfiles root. + + The repository mapping used by the returned Path object is that of the + caller of this method. + """ + if source_repo is None and not self._repo_mapping.is_empty(): + source_repo = self.CurrentRepository(frame=2) + return Path(runfiles=self, source_repo=source_repo) + def Rlocation(self, path: str, source_repo: Optional[str] = None) -> Optional[str]: """Returns the runtime path of a runfile. @@ -179,7 +559,7 @@ def Rlocation(self, path: str, source_repo: Optional[str] = None) -> Optional[st if os.path.isabs(path): return path - if source_repo is None and self._repo_mapping: + if source_repo is None and not self._repo_mapping.is_empty(): # Look up runfiles using the repository mapping of the caller of the # current method. If the repo mapping is empty, determining this # name is not necessary. @@ -188,7 +568,8 @@ def Rlocation(self, path: str, source_repo: Optional[str] = None) -> Optional[st # Split off the first path component, which contains the repository # name (apparent or canonical). target_repo, _, remainder = path.partition("/") - if not remainder or (source_repo, target_repo) not in self._repo_mapping: + target_canonical = self._repo_mapping.lookup(source_repo, target_repo) + if not remainder or target_canonical is None: # One of the following is the case: # - not using Bzlmod, so the repository mapping is empty and # apparent and canonical repository names are the same @@ -198,15 +579,17 @@ def Rlocation(self, path: str, source_repo: Optional[str] = None) -> Optional[st # which also should not be mapped. return self._strategy.RlocationChecked(path) - assert ( - source_repo is not None - ), "BUG: if the `source_repo` is None, we should never go past the `if` statement above" + assert source_repo is not None, ( + "BUG: if the `source_repo` is None, we should never go past the `if` statement above" + ) + + # Look up the target repository using the repository mapping + if target_canonical is not None: + return self._strategy.RlocationChecked(target_canonical + "/" + remainder) - # target_repo is an apparent repository name. Look up the corresponding - # canonical repository name with respect to the current repository, - # identified by its canonical name. - target_canonical = self._repo_mapping[(source_repo, target_repo)] - return self._strategy.RlocationChecked(target_canonical + "/" + remainder) + # No mapping found - assume target_repo is already canonical or + # we're not using Bzlmod + return self._strategy.RlocationChecked(path) def EnvVars(self) -> Dict[str, str]: """Returns environment variables for subprocesses. @@ -231,7 +614,7 @@ def CurrentRepository(self, frame: int = 1) -> str: More information about the difference between canonical repository names and the `@repo` part of labels is available at: - https://bazel.build/build/bzlmod#repository-names + NOTE: This function inspects the callstack to determine where in the runfiles the caller is located to determine which repository it came @@ -264,13 +647,16 @@ def CurrentRepository(self, frame: int = 1) -> str: # located in the main repository. # With Python 3.11 and higher, the Python launcher sets # PYTHONSAFEPATH, which prevents this behavior. + # On Windows, the current toolchain being used has a buggy zip file + # bootstrap, which leaves RUNFILES_DIR pointing at the first stage + # path and not the module path. In this case too, assume that the + # module is located in the main repository. # TODO: This doesn't cover the case of a script being run from an # external repository, which could be heuristically detected # by parsing the script's path. - if ( - sys.version_info.minor <= 10 - and sys.path[0] != self._python_runfiles_root - ): + if (sys.version_info.minor <= 10 or sys.platform == "win32") and sys.path[ + 0 + ] != self._python_runfiles_root: return "" raise ValueError( "{} does not lie under the runfiles root {}".format( @@ -342,45 +728,49 @@ def Create(env: Optional[Dict[str, str]] = None) -> Optional["Runfiles"]: return None + # TODO: Update return type to Self when 3.11 is the min version + # https://peps.python.org/pep-0673/ + @staticmethod + def CreateOrRaise(env: Optional[Dict[str, str]] = None) -> "Runfiles": + """Returns a new `Runfiles` instance, or raises an error. -# Support legacy imports by defining a private symbol. -_Runfiles = Runfiles + The returned object is either: + - manifest-based, meaning it looks up runfile paths from a manifest + file, or + - directory-based, meaning it looks up runfile paths under a given + directory path + + If `env` contains "RUNFILES_MANIFEST_FILE" with non-empty value, this + method returns a manifest-based implementation. The object eagerly + reads and caches the whole manifest file upon instantiation; this may + be relevant for performance consideration. + + Otherwise, if `env` contains "RUNFILES_DIR" with non-empty value + (checked in this priority order), this method returns a directory-based + implementation. + + If neither cases apply, this method raises a `RuntimeError`. + + Args: + env: {string: string}; optional; the map of environment variables. If + None, this function uses the environment variable map of this + process. + Raises: + RuntimeError: if runfiles cannot be found. + :::{versionadded} VERSION_NEXT_FEATURE + ::: + """ + runfiles = Runfiles.Create(env=env) + if runfiles is None: + raise RuntimeError( + "Cannot create Runfiles: $RUNFILES_MANIFEST_FILE and $RUNFILES_DIR are both unset or empty" + ) + return runfiles -def _FindPythonRunfilesRoot() -> str: - """Finds the root of the Python runfiles tree.""" - root = __file__ - # Walk up our own runfiles path to the root of the runfiles tree from which - # the current file is being run. This path coincides with what the Bazel - # Python stub sets up as sys.path[0]. Since that entry can be changed at - # runtime, we rederive it here. - for _ in range("rules_python/python/runfiles/runfiles.py".count("/") + 1): - root = os.path.dirname(root) - return root - - -def _ParseRepoMapping(repo_mapping_path: Optional[str]) -> Dict[Tuple[str, str], str]: - """Parses the repository mapping manifest.""" - # If the repository mapping file can't be found, that is not an error: We - # might be running without Bzlmod enabled or there may not be any runfiles. - # In this case, just apply an empty repo mapping. - if not repo_mapping_path: - return {} - try: - with open(repo_mapping_path, "r") as f: - content = f.read() - except FileNotFoundError: - return {} - - repo_mapping = {} - for line in content.split("\n"): - if not line: - # Empty line following the last line break - break - current_canonical, target_local, target_canonical = line.split(",") - repo_mapping[(current_canonical, target_local)] = target_canonical - - return repo_mapping + +# Support legacy imports by defining a private symbol. +_Runfiles = Runfiles def CreateManifestBased(manifest_path: str) -> Runfiles: @@ -393,3 +783,12 @@ def CreateDirectoryBased(runfiles_dir_path: str) -> Runfiles: def Create(env: Optional[Dict[str, str]] = None) -> Optional[Runfiles]: return Runfiles.Create(env) + + +def CreateOrRaise(env: Optional[Dict[str, str]] = None) -> Runfiles: + """Refer to `Runfiles.CreateOrRaise`. + + :::{versionadded} VERSION_NEXT_FEATURE + ::: + """ + return Runfiles.CreateOrRaise(env) diff --git a/python/runtime_env_toolchains/BUILD.bazel b/python/runtime_env_toolchains/BUILD.bazel index 21355ac939..5001d12556 100644 --- a/python/runtime_env_toolchains/BUILD.bazel +++ b/python/runtime_env_toolchains/BUILD.bazel @@ -17,3 +17,9 @@ load("//python/private:runtime_env_toolchain.bzl", "define_runtime_env_toolchain package(default_visibility = ["//:__subpackages__"]) define_runtime_env_toolchain(name = "runtime_env_toolchain") + +filegroup( + name = "distribution", + srcs = glob(["**"]), + visibility = ["//python:__pkg__"], +) diff --git a/python/uv/BUILD.bazel b/python/uv/BUILD.bazel index 383bdfcc3c..93aefbfbd7 100644 --- a/python/uv/BUILD.bazel +++ b/python/uv/BUILD.bazel @@ -27,9 +27,6 @@ filegroup( visibility = ["//:__subpackages__"], ) -# For stardoc to reference the files -exports_files(["defs.bzl"]) - toolchain_type( name = "uv_toolchain_type", visibility = ["//visibility:public"], @@ -48,34 +45,25 @@ current_toolchain( ) bzl_library( - name = "defs", - srcs = ["defs.bzl"], - # EXPERIMENTAL: Visibility is restricted to allow for changes. - visibility = ["//:__subpackages__"], + name = "lock", + srcs = ["lock.bzl"], + deps = ["//python/uv/private:lock"], ) bzl_library( - name = "extensions", - srcs = ["extensions.bzl"], - # EXPERIMENTAL: Visibility is restricted to allow for changes. - visibility = ["//:__subpackages__"], - deps = [":repositories"], + name = "uv", + srcs = ["uv.bzl"], + deps = ["//python/uv/private:uv"], ) bzl_library( - name = "repositories", - srcs = ["repositories.bzl"], - # EXPERIMENTAL: Visibility is restricted to allow for changes. - visibility = ["//:__subpackages__"], - deps = [ - "//python/uv/private:toolchains_repo", - "//python/uv/private:versions", - ], + name = "uv_toolchain", + srcs = ["uv_toolchain.bzl"], + deps = ["//python/uv/private:uv_toolchain"], ) bzl_library( - name = "toolchain", - srcs = ["toolchain.bzl"], - # EXPERIMENTAL: Visibility is restricted to allow for changes. - visibility = ["//:__subpackages__"], + name = "uv_toolchain_info", + srcs = ["uv_toolchain_info.bzl"], + deps = ["//python/uv/private:uv_toolchain_info"], ) diff --git a/python/uv/extensions.bzl b/python/uv/extensions.bzl deleted file mode 100644 index 82560eb17c..0000000000 --- a/python/uv/extensions.bzl +++ /dev/null @@ -1,50 +0,0 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -""" -EXPERIMENTAL: This is experimental and may be removed without notice - -A module extension for working with uv. -""" - -load("//python/uv:repositories.bzl", "uv_register_toolchains") - -_DOC = """\ -A module extension for working with uv. -""" - -uv_toolchain = tag_class(attrs = { - "uv_version": attr.string(doc = "Explicit version of uv.", mandatory = True), -}) - -def _uv_toolchain_extension(module_ctx): - for mod in module_ctx.modules: - for toolchain in mod.tags.toolchain: - if not mod.is_root: - fail( - "Only the root module may configure the uv toolchain.", - "This prevents conflicting registrations with any other modules.", - "NOTE: We may wish to enforce a policy where toolchain configuration is only allowed in the root module, or in rules_python. See https://github.com/bazelbuild/bazel/discussions/22024", - ) - - uv_register_toolchains( - uv_version = toolchain.uv_version, - register_toolchains = False, - ) - -uv = module_extension( - doc = _DOC, - implementation = _uv_toolchain_extension, - tag_classes = {"toolchain": uv_toolchain}, -) diff --git a/python/uv/lock.bzl b/python/uv/lock.bzl new file mode 100644 index 0000000000..7fd50082ea --- /dev/null +++ b/python/uv/lock.bzl @@ -0,0 +1,75 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""The `uv` locking rule. + +Differences with the legacy {obj}`compile_pip_requirements` rule: +- This is implemented as a rule that performs locking in a build action. +- Additionally one can use the runnable target. +- Uses `uv`. +- This does not error out if the output file does not exist yet. +- Supports transitions out of the box. + +Note, this does not provide a test target like {obj}`compile_pip_requirements` does. +The `uv pip compile` command is not hermetic and thus a test based on it would most likely be flaky: +- It may require auth injected into it, so most likely it requires a local tag added so that the bazel action runs without sandboxing. +- It requires network access. + +Given those points, a test target should be an explicit and properly documented target and not a hidden implicit target. +If, you would like to add a test target that always does the locking automatically to ensure that the +`requirements.txt` file is up-to-date, add something similar to: + +```starlark +load("@bazel_skylib//rules:native_binary.bzl", "native_test") +load("@rules_python//python/uv:lock.bzl", "lock") + +lock( + name = "requirements", + srcs = ["pyproject.toml"], +) + +native_test( + name = "requirements_test", + src = "requirements.update", +) +``` + +### `[tool.uv]` settings support + +When a `pyproject.toml` file is included in {attr}`lock.srcs`, the +`--project` flag is automatically passed to `uv pip compile` using the +directory of the shortest-path `pyproject.toml`. This causes `uv` to +read `[tool.uv]` settings such as `no-build-isolation`, +`exclude-dependencies`, and `[tool.uv.workspace]` from that file. + +If the auto-detection doesn't select the right project (e.g. in complex +workspace layouts), use the `project` parameter to override it: + +```starlark +lock( + name = "requirements", + srcs = [ + "pyproject.toml", + "requirements.in", + ], + project = "subproject", +) +``` + +EXPERIMENTAL: This is experimental and may be changed without notice. +""" + +load("//python/uv/private:lock.bzl", _lock = "lock") + +lock = _lock diff --git a/python/uv/private/BUILD.bazel b/python/uv/private/BUILD.bazel index 80fd23913f..ce0ce0ec81 100644 --- a/python/uv/private/BUILD.bazel +++ b/python/uv/private/BUILD.bazel @@ -13,6 +13,10 @@ # limitations under the License. load("@bazel_skylib//:bzl_library.bzl", "bzl_library") +load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility + +# public only because this is used from a macro to template +_NOT_REALLY_PUBLIC = ["//visibility:public"] filegroup( name = "distribution", @@ -20,29 +24,117 @@ filegroup( visibility = ["//python/uv:__pkg__"], ) +filegroup( + name = "lock_copier_template", + srcs = ["template/lock_copier.py"], + target_compatible_with = [] if BZLMOD_ENABLED else ["@platforms//:incompatible"], + visibility = _NOT_REALLY_PUBLIC, +) + +filegroup( + name = "uv_lock_template", + srcs = select({ + "@platforms//os:windows": ["template/uv_lock.bat"], + "//conditions:default": ["template/uv_lock.sh"], + }), + target_compatible_with = [] if BZLMOD_ENABLED else ["@platforms//:incompatible"], + visibility = _NOT_REALLY_PUBLIC, +) + +filegroup( + name = "uv_pip_compile_template", + srcs = select({ + "@platforms//os:windows": ["template/uv_pip_compile.bat"], + "//conditions:default": ["template/uv_pip_compile.sh"], + }), + target_compatible_with = [] if BZLMOD_ENABLED else ["@platforms//:incompatible"], + visibility = _NOT_REALLY_PUBLIC, +) + bzl_library( name = "current_toolchain", srcs = ["current_toolchain.bzl"], visibility = ["//python/uv:__subpackages__"], + deps = [":toolchain_types"], ) +# keep bzl_library( - name = "toolchain_types", - srcs = ["toolchain_types.bzl"], + name = "lock", + srcs = ["lock.bzl"], + visibility = [ + "//python/uv:__subpackages__", + "//tools/private:__subpackages__", + ], + deps = [ + ":toolchain_types", + "//python:py_binary", + "//python/private:bzlmod_enabled", + "//python/private:common_labels", + "//python/private:toolchain_types", + "@bazel_skylib//lib:shell", + ], +) + +bzl_library( + name = "toolchains_hub", + srcs = ["toolchains_hub.bzl"], visibility = ["//python/uv:__subpackages__"], + deps = [":toolchain_types"], ) bzl_library( - name = "toolchains_repo", - srcs = ["toolchains_repo.bzl"], + name = "uv", + srcs = ["uv.bzl"], visibility = ["//python/uv:__subpackages__"], deps = [ - "//python/private:text_util_bzl", + ":toolchain_types", + ":uv_repository", + ":uv_toolchains_repo", + "//python/private:auth", + "//python/private:common_labels", + ], +) + +bzl_library( + name = "uv_repository", + srcs = ["uv_repository.bzl"], + visibility = ["//python/uv:__subpackages__"], + deps = ["//python/private:auth"], +) + +bzl_library( + name = "uv_toolchain", + srcs = ["uv_toolchain.bzl"], + visibility = ["//python/uv:__subpackages__"], + deps = [":uv_toolchain_info"], +) + +bzl_library( + name = "uv_toolchains_repo", + srcs = ["uv_toolchains_repo.bzl"], + visibility = ["//python/uv:__subpackages__"], + deps = ["//python/private:text_util"], +) + +# keep +bzl_library( + name = "uv_lock_to_requirements", + srcs = ["uv_lock_to_requirements.bzl"], + visibility = [ + "//python/private:__subpackages__", + "//python/uv:__subpackages__", ], ) bzl_library( - name = "versions", - srcs = ["versions.bzl"], + name = "toolchain_types", + srcs = ["toolchain_types.bzl"], + visibility = ["//python/uv:__subpackages__"], +) + +bzl_library( + name = "uv_toolchain_info", + srcs = ["uv_toolchain_info.bzl"], visibility = ["//python/uv:__subpackages__"], ) diff --git a/python/uv/private/lock.bzl b/python/uv/private/lock.bzl index 217b6e4831..695591e924 100644 --- a/python/uv/private/lock.bzl +++ b/python/uv/private/lock.bzl @@ -12,119 +12,667 @@ # See the License for the specific language governing permissions and # limitations under the License. -"""A simple macro to lock the requirements. +"""An implementation for a simple macro to lock the requirements. """ -load("@bazel_skylib//rules:write_file.bzl", "write_file") +load("@bazel_skylib//lib:shell.bzl", "shell") load("//python:py_binary.bzl", "py_binary") -load("//python/config_settings:transition.bzl", transition_py_binary = "py_binary") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility +load("//python/private:common_labels.bzl", "labels") +load("//python/private:toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE") # buildifier: disable=bzl-visibility +load(":toolchain_types.bzl", "UV_TOOLCHAIN_TYPE") visibility(["//..."]) -_REQUIREMENTS_TARGET_COMPATIBLE_WITH = select({ - "@platforms//os:windows": ["@platforms//:incompatible"], - "//conditions:default": [], -}) if BZLMOD_ENABLED else ["@platforms//:incompatible"] +_RunLockInfo = provider( + doc = "", + fields = { + "args": "The args passed to the `uv` by default when running the runnable target.", + "env": "The env passed to the execution.", + "srcs": "Source files required to run the runnable target.", + "template": "The template file for writing a script.", + }, +) -def lock(*, name, srcs, out, upgrade = False, universal = True, python_version = None, args = [], **kwargs): - """Pin the requirements based on the src files. +def _args(ctx): + """A small helper to ensure that the right args are pushed to the _RunLockInfo provider""" + run_info = [] + args = ctx.actions.args() + + def _add_args(arg, maybe_value = None): + run_info.append(arg) + if maybe_value: + args.add(arg, maybe_value) + run_info.append(maybe_value) + else: + args.add(arg) + + def _add_all(name, all_args = None, **kwargs): + if not all_args and type(name) == "list": + all_args = name + name = None + + before_each = kwargs.get("before_each") + if name: + args.add_all(name, all_args, **kwargs) + run_info.append(name) + else: + args.add_all(all_args, **kwargs) + + for arg in all_args: + if before_each: + run_info.append(before_each) + run_info.append(arg) + + return struct( + run_info = run_info, + run_shell = args, + add = _add_args, + add_all = _add_all, + ) + +def _common_lock(ctx, locker): + fname = "{}.out".format(ctx.label.name) + + # TODO @aignas 2026-06-21: do not append python_version for uv.lock as it should work for all + # python versions + python_version = ctx.attr.python_version + if python_version: + fname = "{}.{}.out".format( + ctx.label.name, + python_version.replace(".", "_"), + ) + + output = ctx.actions.declare_file(fname) + toolchain_info = ctx.toolchains[UV_TOOLCHAIN_TYPE] + uv = toolchain_info.uv_toolchain_info.uv[DefaultInfo].files_to_run.executable + + args = _args(ctx) + args.add(uv) + + # The output params are: + # * srcs are the srcs for the locking command action inputs tracking + # * output_filename if set is to ensure that we can do special prep for uv.lock versus + # requirements.txt + # * mnemonic is for the action mnemonic + # * progress_message is the same + srcs, output_filename, mnemonic, progress_message = locker(args, output) + + args.add_all([ + "--no-python-downloads", + "--no-cache", + ]) + + project = None + if ctx.attr.project: + project = ctx.attr.project + else: + # Autodetect the project based on the `pyproject.toml` location - it will be the first src that + # we see that is named "pyproject.toml" + for src in srcs: + if src.basename == "pyproject.toml": + if project == None: + project = src.dirname + elif len(project) > len(src.dirname): + # select the shortest match + project = src.dirname + + if project == None: + project = ctx.label.package + + if project: + args.add_all([project], before_each = "--project") + + args.add_all(ctx.attr.args) + + exec_tools = ctx.toolchains[EXEC_TOOLS_TOOLCHAIN_TYPE].exec_tools + runtime = exec_tools.exec_interpreter[platform_common.ToolchainInfo].py3_runtime + python = runtime.interpreter or runtime.interpreter_path + python_files = runtime.files or depset() + args.add("--python", python) + + # These arguments does not change behaviour, but it reduces the output from + # the command, which is especially verbose in stderr. + args.add("--no-progress") + args.add("--quiet") + + if ctx.files.existing_output: + src_out = ctx.files.existing_output[0].path + elif output_filename: + # special case - the output filename has to be in the source tree and it has to have a + # special name, we use the project folder to determine this. + + if not project: + fail("Cannot lock this if the project dir is unset or cannot be infered") + + src_out = "{project}/{out_filename}".format( + project = project, + out_filename = output_filename, + ) + else: + src_out = "" + + is_windows = ctx.attr.is_windows + if is_windows: + path_sep = "\\" + ext = ".bat" + else: + path_sep = "/" + ext = "" + + output_path = output.path.replace("/", path_sep) if is_windows else output.path + src_out_path = src_out.replace("/", path_sep) if is_windows else src_out + + # On Windows, all args must be embedded in the .bat script because + # arguments are not passed on the command line. + if is_windows: + args_parts = [] + for i, arg in enumerate(args.run_info): + if hasattr(arg, "path"): + arg = arg.path + + # Only use backslashes for the executable itself (first arg) + # to ensure CMD can run it, but keep forward slashes for arguments + # so that uv writes consistent paths in comments. + if i == 0: + a = arg.replace("/", "\\") + else: + a = arg + a = a.replace('"', '""') + args_parts.append('"' + a + '"') + + # uv pip compile adds --output-file to run_shell (not run_info). + # For the lock case, output_filename is "uv.lock" and uv lock + # writes to the project directory without --output-file. + if not output_filename: + args_parts.append('"--output-file"') + args_parts.append('"' + output_path + '"') + windows_args = " ".join(args_parts) + else: + windows_args = " ".join([]) + + script = ctx.actions.declare_file(ctx.label.name + "_lock" + ext) + ctx.actions.expand_template( + template = ctx.files._template[0], + substitutions = { + '"{{args}}"': windows_args, + "{{out}}": output_path, + "{{src_out}}": src_out_path, + }, + output = script, + is_executable = True, + ) + + ctx.actions.run( + executable = script, + mnemonic = mnemonic, + inputs = srcs + ctx.files.existing_output, + outputs = [output], + # On Windows, the command line is embedded directly in the .bat + # script (with backslash paths). On POSIX, args are forwarded via + # exec "$@" in the .sh script. + arguments = [args.run_shell] if not ctx.attr.is_windows else [], + tools = [ + uv, + python_files, + script, + ], + + # User reported being unable to add `--action_env` and get it to work. + # Without this flag. + # + # Ref: https://app.slack.com/client/TA4K1KQ87/CA306CEV6 + use_default_shell_env = True, + progress_message = progress_message, + env = ctx.attr.env, + ) + + return [ + DefaultInfo(files = depset([output])), + _RunLockInfo( + args = args.run_info, + env = ctx.attr.env, + srcs = depset( + srcs + [uv], + transitive = [python_files], + ), + template = ctx.files._template[0], + ), + ] + +def _pip_compile_impl(ctx): + def _setup_args(args, output): + args.add_all(["pip", "compile"]) + pkg = ctx.label.package + update_target = ctx.attr.update_target + args.add("--custom-compile-command", "bazel run //{}:{}".format(pkg, update_target)) + + if ctx.attr.generate_hashes: + args.add("--generate-hashes") + if not ctx.attr.strip_extras: + args.add("--no-strip-extras") + + args.add_all(ctx.files.build_constraints, before_each = "--build-constraints") + args.add_all(ctx.files.constraints, before_each = "--constraints") + + args.run_shell.add("--output-file", output) + mnemonic = "PyRequirementsLockUv" + progress_message = "Creating a requirements.txt with uv: %{label}" + + args.add_all(ctx.files.srcs) + srcs = ctx.files.srcs + ctx.files.build_constraints + ctx.files.constraints + + return srcs, None, mnemonic, progress_message + + return _common_lock(ctx, _setup_args) + +def _transition_impl(input_settings, attr): + settings = { + labels.PYTHON_VERSION: input_settings[labels.PYTHON_VERSION], + } + if attr.python_version: + settings[labels.PYTHON_VERSION] = attr.python_version + return settings + +_python_version_transition = transition( + implementation = _transition_impl, + inputs = [labels.PYTHON_VERSION], + outputs = [labels.PYTHON_VERSION], +) + +_common_attrs = { + "args": attr.string_list( + doc = "Public, see the docs in the macro.", + ), + "env": attr.string_dict( + doc = "Public, see the docs in the macro.", + ), + "existing_output": attr.label( + mandatory = False, + allow_single_file = True, + doc = """\ +An already existing output file that is used as a basis for further +modifications and the locking is not done from scratch. +""", + ), + "is_windows": attr.bool(mandatory = True), + "output": attr.string( + doc = "Public, see the docs in the macro.", + mandatory = True, + ), + "project": attr.string( + doc = """\ +Overrides the `--project` directory passed to `uv pip compile`. +If not set, the project directory is auto-detected: when +`pyproject.toml` files are in {obj}`lock.srcs`, the one with the +shortest directory path is selected. This makes `uv` read +`[tool.uv]` settings (e.g. `no-build-isolation`, +`exclude-dependencies`) from that `pyproject.toml`. +""", + ), + "python_version": attr.string( + doc = "Public, see the docs in the macro.", + ), + "srcs": attr.label_list( + mandatory = True, + allow_files = True, + doc = "Public, see the docs in the macro.", + ), + "_allowlist_function_transition": attr.label( + default = "@bazel_tools//tools/allowlists/function_transition_allowlist", + ), +} + +_pip_compile = rule( + implementation = _pip_compile_impl, + doc = """\ +The lock rule that does the locking in a build action (that makes it possible +to use RBE) and also prepares information for a `bazel run` executable rule. + +:::{versionchanged} 2.1.0 +Added the {attr}`project` to configure the project setting if autodetection fails. +::: +""", + attrs = { + "build_constraints": attr.label_list( + allow_files = True, + doc = "Public, see the docs in the macro.", + ), + "constraints": attr.label_list( + allow_files = True, + doc = "Public, see the docs in the macro.", + ), + "generate_hashes": attr.bool( + doc = "Public, see the docs in the macro.", + default = True, + ), + "strip_extras": attr.bool( + doc = "Public, see the docs in the macro.", + default = False, + ), + "update_target": attr.string( + mandatory = True, + doc = """\ +The string to input for the 'uv pip compile'. +""", + ), + "_template": attr.label( + default = "//python/uv/private:uv_pip_compile_template", + doc = """\ +The template to be used for 'uv pip compile'. This is either .bat or bash +script depending on what the target platform is executed on. +""", + ), + } | _common_attrs, + toolchains = [ + EXEC_TOOLS_TOOLCHAIN_TYPE, + UV_TOOLCHAIN_TYPE, + ], + cfg = _python_version_transition, +) + +def _lock_impl(ctx): + def _setup_args(args, _output): + args.add("lock") + mnemonic = "PyUvLock" + progress_message = "Creating a uv.lock with uv: %{label}" + + return ctx.files.srcs, "uv.lock", mnemonic, progress_message + + return _common_lock(ctx, _setup_args) + +_lock = rule( + implementation = _lock_impl, + doc = """\ +The lock rule that does the locking in a build action and also prepares information for a `bazel +run` executable rule. + +:::{versionadded} 2.2.0 +::: +""", + attrs = _common_attrs | { + "_template": attr.label( + default = "//python/uv/private:uv_lock_template", + doc = """\ +The template to be used for 'uv lock'. Used when output ends with '.lock'. +""", + ), + }, + toolchains = [ + EXEC_TOOLS_TOOLCHAIN_TYPE, + UV_TOOLCHAIN_TYPE, + ], + cfg = _python_version_transition, +) + +def _run_impl(ctx): + if ctx.attr.is_windows: + path_sep = "\\" + ext = ".bat" + else: + path_sep = "/" + ext = "" + + def _maybe_path(arg): + if hasattr(arg, "short_path"): + arg = arg.short_path + + arg = arg.replace("/", path_sep) + if ctx.attr.is_windows: + # On Windows, CMD uses double quotes for quoting, and internal + # double quotes are escaped by doubling them. + return '"' + arg.replace('"', '""') + '"' + return shell.quote(arg) + + info = ctx.attr.lock[_RunLockInfo] + + executable = ctx.actions.declare_file(ctx.label.name + ext) + ctx.actions.expand_template( + template = info.template, + substitutions = { + '"{{args}}"': " ".join([_maybe_path(arg) for arg in info.args]), + "{{src_out}}": "{}/{}".format(ctx.label.package, ctx.attr.output).replace( + "/", + path_sep, + ), + }, + output = executable, + is_executable = True, + ) + + return [ + DefaultInfo( + executable = executable, + runfiles = ctx.runfiles(transitive_files = info.srcs), + ), + RunEnvironmentInfo( + environment = info.env, + ), + ] + +_run_locker = rule( + implementation = _run_impl, + doc = """\ +""", + attrs = { + "is_windows": attr.bool(mandatory = True), + "lock": attr.label( + doc = "The lock target that is doing locking in a build action.", + providers = [_RunLockInfo], + cfg = "exec", + ), + "output": attr.string( + doc = """\ +The output that we would be updated, relative to the package the macro is used in. +""", + ), + }, + executable = True, +) + +def _maybe_file(path): + """A small function to return a list of existing outputs. + + If the file referenced by the input argument exists, then it will return + it, otherwise it will return an empty list. This is useful to for programs + like pip-compile which behave differently if the output file exists and + update the output file in place. + + The API of the function ensures that path is not a glob itself. Args: - name: The name of the target to run for updating the requirements. - srcs: The srcs to use as inputs. - out: The output file. - upgrade: Tell `uv` to always upgrade the dependencies instead of - keeping them as they are. - universal: Tell `uv` to generate a universal lock file. - python_version: Tell `rules_python` to use a particular version. - Defaults to the default py toolchain. - args: Extra args to pass to the rule. - **kwargs: Extra kwargs passed to the binary rule. - - Differences with the current pip-compile rule: - - This is implemented in shell and uv. - - This does not error out if the output file does not exist yet. - - Supports transitions out of the box. + path: {type}`str` the file name. """ - pkg = native.package_name() - update_target = name + ".update" - - _args = [ - "--custom-compile-command='bazel run //{}:{}'".format(pkg, update_target), - "--generate-hashes", - "--emit-index-url", - "--no-strip-extras", - "--python=$(PYTHON3)", - ] + args + [ - "$(location {})".format(src) - for src in srcs - ] - if upgrade: - _args.append("--upgrade") - if universal: - _args.append("--universal") - _args.append("--output-file=$@") - cmd = "$(UV_BIN) pip compile " + " ".join(_args) + for p in native.glob([path], allow_empty = True): + if path != p: + continue + + return p + + return None + +def _expand_template_impl(ctx): + pkg = ctx.label.package + update_src = ctx.actions.declare_file(ctx.attr.update_target + ".py") - # Make a copy to ensure that we are not modifying the initial list - srcs = list(srcs) + # Fix the path construction to avoid absolute paths + # If package is empty (root), don't add a leading slash + dst = "{}/{}".format(pkg, ctx.attr.output) if pkg else ctx.attr.output + + ctx.actions.expand_template( + template = ctx.files._lock_copier_template[0], + substitutions = { + "{{dst}}": dst, + "{{src}}": "{}".format(ctx.files.src[0].short_path), + "{{update_target}}": "//{}:{}".format(pkg, ctx.attr.update_target), + }, + output = update_src, + ) + return DefaultInfo(files = depset([update_src])) + +_expand_template = rule( + implementation = _expand_template_impl, + attrs = { + "output": attr.string(mandatory = True), + "src": attr.label(mandatory = True), + "update_target": attr.string(mandatory = True), + "_lock_copier_template": attr.label( + default = "//python/uv/private:lock_copier_template", + allow_single_file = True, + ), + }, + doc = "Expand the template for the update script allowing us to use `select` statements in the {attr}`output` attribute.", +) + +def lock( + *, + name, + srcs, + out, + args = [], + build_constraints = [], + constraints = [], + env = None, + generate_hashes = True, + python_version = None, + project = None, + strip_extras = False, + **kwargs): + """Pin the requirements based on the src files. + + This macro creates the following targets: + - `name`: the target that creates the requirements.txt file in a build + action. This target will have `no-cache` and `requires-network` added + to its tags. + - `name.run`: a runnable target that can be used to pass extra parameters + to the same command that would be run in the `name` action. This will + update the source copy of the requirements file. You can customize the + args via the command line, but it requires being able to run `uv` (and + possibly `python`) directly on your host. + - `name.update`: a target that can be run to update the source-tree version + of the requirements lock file. The output can be fed to the + {obj}`pip.parse` bzlmod extension tag class. Note, you can use + `native_test` to wrap this target to make a test. You can't customize the + args via command line, but you can use RBE to generate requirements + (offload execution and run for different platforms). Note, that for RBE + to be usable, one needs to ensure that the nodes running the action have + internet connectivity or the indexes are provided in a different way for + a fully offline operation. + + :::{note} + All of the targets have `manual` tags as locking results cannot be cached. + ::: + + Args: + name: {type}`str` The prefix of all targets created by this macro. + srcs: {type}`list[Label]` The sources that will be used. Add all of the + files that would be passed as srcs to the `uv pip compile` command. + out: {type}`str` The output file relative to the package. + args: {type}`list[str]` The list of args to pass to uv. Note, these are + written into the runnable `name.run` target. + env: {type}`dict[str, str]` the environment variables to set. Note, this + is passed as is and the environment variables are not expanded. + build_constraints: {type}`list[Label]` The list of build constraints to use. + constraints: {type}`list[Label]` The list of constraints files to use. + generate_hashes: {type}`bool` Generate hashes for all of the + requirements. Only meaningful for `requirements.txt` style output. + Defaults to `True`. + strip_extras: {type}`bool` whether to strip extras from the output. + Currently `rules_python` requires `--no-strip-extras` to properly + function, but sometimes one may want to not have the extras if you + are compiling the requirements file for using it as a constraints + file. Defaults to `False`. + project: {type}`str | None` overrides the `--project` directory + passed to `uv pip compile`. By default the project directory + is auto-detected: when {obj}`lock.srcs` contains + `pyproject.toml` files, the one with the shortest directory + path is selected. This causes `uv` to read `[tool.uv]` + settings such as `no-build-isolation` and + `exclude-dependencies` from that `pyproject.toml`. If no + `pyproject.toml` is in `srcs` and no `project` is given, the + Bazel package directory is used as fallback. + {versionadded} 2.1.0 + python_version: {type}`str | None` the python_version to transition to + when locking the requirements. Defaults to the default python version + configured by the {obj}`python` module extension. + **kwargs: common kwargs passed to rules. + """ + update_target = "{}.update".format(name) + locker_target = "{}.run".format(name) # Check if the output file already exists, if yes, first copy it to the # output file location in order to make `uv` not change the requirements if # we are just running the command. - if native.glob([out]): - cmd = "cp -v $(location {}) $@; {}".format(out, cmd) - srcs.append(out) - - native.genrule( - name = name, - srcs = srcs, - outs = [out + ".new"], - cmd_bash = cmd, - tags = [ - "local", - "manual", + maybe_out = _maybe_file(out) + + tags = ["manual"] + kwargs.pop("tags", []) + if not BZLMOD_ENABLED: + kwargs["target_compatible_with"] = ["@platforms//:incompatible"] + + uv_kwargs = { + "is_windows": select({ + "@platforms//os:windows": True, + "//conditions:default": False, + }), + "output": out, + } | kwargs + + lock_target_kwargs = { + "args": args, + "env": env, + "existing_output": maybe_out, + "project": project, + "python_version": python_version, + "srcs": srcs, + "tags": [ "no-cache", - ], - target_compatible_with = _REQUIREMENTS_TARGET_COMPATIBLE_WITH, - toolchains = [ - Label("//python/uv:current_toolchain"), - Label("//python:current_py_toolchain"), - ], - ) - if python_version: - py_binary_rule = lambda *args, **kwargs: transition_py_binary(python_version = python_version, *args, **kwargs) + "requires-network", + ] + tags, + } | uv_kwargs + + # NOTE @aignas 2026-06-20: if the user passes these args the command will fail + # with an error message instead of silently ignoring the args + if build_constraints: + lock_target_kwargs["build_constraints"] = build_constraints + if constraints: + lock_target_kwargs["constraints"] = constraints + + if out.endswith(".lock"): + _lock(name = name, **lock_target_kwargs) else: - py_binary_rule = py_binary - - # Write a script that can be used for updating the in-tree version of the - # requirements file - write_file( - name = name + ".update_gen", - out = update_target + ".py", - content = [ - "from os import environ", - "from pathlib import Path", - "from sys import stderr", - "", - 'src = Path(environ["REQUIREMENTS_FILE"])', - 'assert src.exists(), f"the {src} file does not exist"', - 'dst = Path(environ["BUILD_WORKSPACE_DIRECTORY"]) / "{}" / "{}"'.format(pkg, out), - 'print(f"Writing requirements contents\\n from {src.absolute()}\\n to {dst.absolute()}", file=stderr)', - "dst.write_text(src.read_text())", - 'print("Success!", file=stderr)', - ], + _pip_compile( + name = name, + generate_hashes = generate_hashes, + strip_extras = strip_extras, + update_target = update_target, + **lock_target_kwargs + ) + + # A target for updating the in-tree version directly by skipping the in-action + # uv pip compile or uv lock, depending what is defined for the locker_target. + _run_locker( + name = locker_target, + lock = name, + tags = tags, + **uv_kwargs ) - py_binary_rule( + # FIXME @aignas 2025-03-20: is it possible to extend `py_binary` so that the + # srcs are generated before `py_binary` is run? I found that + # `ctx.files.srcs` usage in the base implementation is making it difficult. + template_target = "_{}_gen".format(name) + _expand_template( + name = template_target, + src = name, + output = out, + update_target = update_target, + tags = tags, + ) + + py_binary( name = update_target, - srcs = [update_target + ".py"], - main = update_target + ".py", - data = [name], - env = { - "REQUIREMENTS_FILE": "$(rootpath {})".format(name), - }, - tags = ["manual"], + srcs = [template_target], + data = [name] + ([maybe_out] if maybe_out else []), + tags = tags, **kwargs ) diff --git a/python/uv/private/template/lock_copier.py b/python/uv/private/template/lock_copier.py new file mode 100644 index 0000000000..8756fc4de6 --- /dev/null +++ b/python/uv/private/template/lock_copier.py @@ -0,0 +1,69 @@ +import sys +from difflib import unified_diff +from os import environ +from pathlib import Path + +_LINE = "=" * 80 + + +def main(): + src = "{{src}}" + dst = "{{dst}}" + + src = Path(src) + if not src.exists(): + raise AssertionError(f"The {src} file does not exist") + + if "TEST_SRCDIR" in environ: + # Running as a bazel test + dst = Path(dst) + a = dst.read_text() if dst.exists() else "\n" + b = src.read_text() + + diff = unified_diff( + a.splitlines(), + b.splitlines(), + str(dst), + str(src), + lineterm="", + ) + diff = "\n".join(list(diff)) + if not diff: + print( + f"""\ +{_LINE} +The in source file copy is up-to-date. +{_LINE} +""" + ) + return 0 + + print(diff) + print( + f"""\ +{_LINE} +The in source file copy is out of date, please run: + + bazel run {{update_target}} +{_LINE} +""" + ) + return 1 + + if "BUILD_WORKSPACE_DIRECTORY" not in environ: + raise RuntimeError( + "This must be either run as `bazel test` via a `native_test` or similar or via `bazel run`" + ) + + print(f"cp /{src.as_posix()} /{dst}") + build_workspace = Path(environ["BUILD_WORKSPACE_DIRECTORY"]) + + dst_real_path = build_workspace / dst + dst_real_path.parent.mkdir(parents=True, exist_ok=True) + dst_real_path.write_text(src.read_text()) + print(f"OK: updated {dst_real_path}") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/python/uv/private/template/uv_lock.bat b/python/uv/private/template/uv_lock.bat new file mode 100755 index 0000000000..d4f4558c53 --- /dev/null +++ b/python/uv/private/template/uv_lock.bat @@ -0,0 +1,21 @@ +@echo off +if not defined BUILD_WORKSPACE_DIRECTORY goto :not_in_workspace +"{{args}}" %* +exit /b %ERRORLEVEL% + +:not_in_workspace + +if not exist "{{src_out}}" goto :no_src_out +copy /y "{{src_out}}" "{{out}}" +del /f "{{src_out}}" +copy /y "{{out}}" "{{src_out}}" +"{{args}}" %* +set "exit_code=%ERRORLEVEL%" +copy /y "{{src_out}}" "{{out}}" +exit /b %exit_code% + +:no_src_out +"{{args}}" %* +set "exit_code=%ERRORLEVEL%" +copy /y "{{src_out}}" "{{out}}" +exit /b %exit_code% \ No newline at end of file diff --git a/python/uv/private/template/uv_lock.sh b/python/uv/private/template/uv_lock.sh new file mode 100755 index 0000000000..a6dd6d3c39 --- /dev/null +++ b/python/uv/private/template/uv_lock.sh @@ -0,0 +1,32 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [[ -n "${BUILD_WORKSPACE_DIRECTORY:-}" ]]; then + exec "{{args}}" "$@" +fi + +# Build action mode +# +# If the uv.lock exists, remove because the existing uv.lock file is read-only, then symlink so +# that we can reuse the existing contents and not do a full relock all the time. If +# nothing exists, just symlink. +# +# On Windows we do it with file copies: +# 1. If the file exists: +# 1. Copy the current file to out. +# 2. Rm the existing file +# 3. Copy the contents back +# 4. Run uv +# 5. Copy the contents to out. +# 1. If the current uv.lock does not exist yet +# 1. Run uv +# 2. Copy the contents to out. +readonly out="{{out}}" +if [[ -f "{{src_out}}" ]]; then + cp "{{src_out}}" "$out" + rm "{{src_out}}" + ln -s "$(pwd)"/"$out" "{{src_out}}" +else + ln -s "$(pwd)"/"$out" "{{src_out}}" +fi +exec "$@" diff --git a/python/uv/private/template/uv_pip_compile.bat b/python/uv/private/template/uv_pip_compile.bat new file mode 100755 index 0000000000..2ea1b5a44a --- /dev/null +++ b/python/uv/private/template/uv_pip_compile.bat @@ -0,0 +1,8 @@ +@echo off +if not defined BUILD_WORKSPACE_DIRECTORY goto :else +set "out=%BUILD_WORKSPACE_DIRECTORY%\{{src_out}}" +"{{args}}" --output-file "%out%" %* +exit /b %ERRORLEVEL% + +:else +"{{args}}" %* diff --git a/python/uv/private/template/uv_pip_compile.sh b/python/uv/private/template/uv_pip_compile.sh new file mode 100755 index 0000000000..ba379ed0f9 --- /dev/null +++ b/python/uv/private/template/uv_pip_compile.sh @@ -0,0 +1,15 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [[ -n "${BUILD_WORKSPACE_DIRECTORY:-}" ]]; then + readonly out="${BUILD_WORKSPACE_DIRECTORY}/{{src_out}}" + exec "{{args}}" --output-file "$out" "$@" +fi + +# Build action mode: seed the output with the source file, then run +# the full command (which includes --output-file from the action args). +readonly out="{{out}}" +if [[ -f "{{src_out}}" ]]; then + cp "{{src_out}}" "$out" +fi +exec "$@" diff --git a/python/uv/private/toolchains_hub.bzl b/python/uv/private/toolchains_hub.bzl new file mode 100644 index 0000000000..b39d84f0c2 --- /dev/null +++ b/python/uv/private/toolchains_hub.bzl @@ -0,0 +1,65 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""A macro used from the uv_toolchain hub repo.""" + +load(":toolchain_types.bzl", "UV_TOOLCHAIN_TYPE") + +def toolchains_hub( + *, + name, + toolchains, + implementations, + target_compatible_with, + target_settings): + """Define the toolchains so that the lexicographical order registration is deterministic. + + TODO @aignas 2025-03-09: see if this can be reused in the python toolchains. + + Args: + name: The prefix to all of the targets, which goes after a numeric prefix. + toolchains: The toolchain names for the targets defined by this macro. + The earlier occurring items take precedence over the later items if + they match the target platform and target settings. + implementations: The name to label mapping. + target_compatible_with: The name to target_compatible_with list mapping. + target_settings: The name to target_settings list mapping. + """ + if len(toolchains) != len(implementations): + fail("Each name must have an implementation") + + # We are defining the toolchains so that the order of toolchain matching is + # the same as the order of the toolchains, because: + # * the toolchains are matched by target settings and target_compatible_with + # * the first toolchain satisfying the above wins + # + # this means we need to register the toolchains prefixed with a number of + # format 00xy, where x and y are some digits and the leading zeros to + # ensure lexicographical sorting. + # + # Add 1 so that there is always a leading zero + prefix_len = len(str(len(toolchains))) + 1 + prefix = "0" * (prefix_len - 1) + + for i, toolchain in enumerate(toolchains): + # prefix with a prefix and then truncate the string. + number_prefix = "{}{}".format(prefix, i)[-prefix_len:] + + native.toolchain( + name = "{}_{}_{}".format(number_prefix, name, toolchain), + target_compatible_with = target_compatible_with.get(toolchain, []), + target_settings = target_settings.get(toolchain, []), + toolchain = implementations[toolchain], + toolchain_type = UV_TOOLCHAIN_TYPE, + ) diff --git a/python/uv/private/uv.bzl b/python/uv/private/uv.bzl new file mode 100644 index 0000000000..969b6f672d --- /dev/null +++ b/python/uv/private/uv.bzl @@ -0,0 +1,573 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +""" +EXPERIMENTAL: This is experimental and may be removed without notice + +A module extension for working with uv. +""" + +load("//python/private:auth.bzl", "AUTH_ATTRS", "get_auth") +load("//python/private:common_labels.bzl", "labels") +load(":toolchain_types.bzl", "UV_TOOLCHAIN_TYPE") +load(":uv_repository.bzl", "uv_repository") +load(":uv_toolchains_repo.bzl", "uv_toolchains_repo") + +_DOC = """\ +A module extension for working with uv. + +Basic usage: +```starlark +uv = use_extension( + "@rules_python//python/uv:uv.bzl", + "uv", + # Use `dev_dependency` so that the toolchains are not defined pulled when + # your module is used elsewhere. + dev_dependency = True, +) +uv.configure(version = "0.5.24") +``` + +Since this is only for locking the requirements files, it should be always +marked as a `dev_dependency`. +""" + +_DEFAULT_ATTRS = { + "base_url": attr.string( + doc = """\ +Base URL to download metadata about the binaries and the binaries themselves. +""", + ), + "compatible_with": attr.label_list( + doc = """\ +The compatible with constraint values for toolchain resolution. +""", + ), + "manifest_filename": attr.string( + doc = """\ +The distribution manifest filename to use for the metadata fetching from GH. The +defaults for this are set in `rules_python` MODULE.bazel file that one can override +for a specific version. +""", + default = "dist-manifest.json", + ), + "platform": attr.string( + doc = """\ +The platform string used in the UV repository to denote the platform triple. +""", + ), + "target_settings": attr.label_list( + doc = """\ +The `target_settings` to add to platform definitions that then get used in `toolchain` +definitions. +""", + ), + "version": attr.string( + doc = """\ +The version of uv to configure the sources for. If this is not specified it will be the +last version used in the module or the default version set by `rules_python`. +""", + ), +} | AUTH_ATTRS + +default = tag_class( + doc = """\ +Set the uv configuration defaults. +""", + attrs = _DEFAULT_ATTRS, +) + +configure = tag_class( + doc = """\ +Build the `uv` toolchain configuration by appending the provided configuration. +The information is appended to the version configuration that is specified by +{attr}`version` attribute, or if the version is unspecified, the version of the +last {obj}`uv.configure` call in the current module, or the version from the +defaults is used. + +Complex configuration example: +```starlark +# Configure the base_url for the default version. +uv.configure(base_url = "my_mirror") + +# Add an extra platform that can be used with your version. +uv.configure( + platform = "extra-platform", + target_settings = ["//my_config_setting_label"], + compatible_with = ["@platforms//os:exotic"], +) + +# Add an extra platform that can be used with your version. +uv.configure( + platform = "patched-binary", + target_settings = ["//my_super_config_setting"], + urls = ["https://example.zip"], + sha256 = "deadbeef", +) +``` +""", + attrs = _DEFAULT_ATTRS | { + "sha256": attr.string( + doc = "The sha256 of the downloaded artifact if the {attr}`urls` is specified.", + ), + "urls": attr.string_list( + doc = """\ +The urls to download the binary from. If this is used, {attr}`base_url` and +{attr}`manifest_filename` are ignored for the given version. + +::::note +If the `urls` are specified, they need to be specified for all of the platforms +for a particular version. +:::: +""", + ), + }, +) + +def _configure(config, *, platform, compatible_with, target_settings, auth_patterns, urls = [], sha256 = "", override = False, **values): + """Set the value in the config if the value is provided""" + for key, value in values.items(): + if not value: + continue + + if not override and config.get(key): + continue + + config[key] = value + + config.setdefault("auth_patterns", {}).update(auth_patterns) + config.setdefault("platforms", {}) + if not platform: + if compatible_with or target_settings or urls: + fail("`platform` name must be specified when specifying `compatible_with`, `target_settings` or `urls`") + elif compatible_with or target_settings: + if not override and config.get("platforms", {}).get(platform): + return + + config["platforms"][platform] = struct( + name = platform.replace("-", "_").lower(), + compatible_with = compatible_with, + target_settings = target_settings, + ) + elif urls: + if not override and config.get("urls", {}).get(platform): + return + + config.setdefault("urls", {})[platform] = struct( + sha256 = sha256, + urls = urls, + ) + else: + config["platforms"].pop(platform) + +def process_modules( + module_ctx, + hub_name = "uv", + uv_repository = uv_repository, + toolchain_type = str(UV_TOOLCHAIN_TYPE), + hub_repo = uv_toolchains_repo, + get_auth = get_auth): + """Parse the modules to get the config for 'uv' toolchains. + + Args: + module_ctx: the context. + hub_name: the name of the hub repository. + uv_repository: the rule to create a uv_repository override. + toolchain_type: the toolchain type to use here. + hub_repo: the hub repo factory function to use. + get_auth: the auth function to use. + + Returns: + the result of the hub_repo. Mainly used for tests. + """ + + # default values to apply for version specific config + defaults = { + "base_url": "", + "manifest_filename": "", + "platforms": { + # The structure is as follows: + # "platform_name": struct( + # compatible_with = [], + # target_settings = [], + # ), + # + # NOTE: urls and sha256 cannot be set in defaults + }, + "version": "", + } + for mod in module_ctx.modules: + if not (mod.is_root or mod.name == "rules_python"): + continue + + for tag in mod.tags.default: + _configure( + defaults, + version = tag.version, + base_url = tag.base_url, + manifest_filename = tag.manifest_filename, + platform = tag.platform, + compatible_with = tag.compatible_with, + target_settings = tag.target_settings, + override = mod.is_root, + netrc = tag.netrc, + auth_patterns = tag.auth_patterns, + ) + + for key in [ + "version", + "manifest_filename", + "platforms", + ]: + if not defaults.get(key, None): + fail("defaults need to be set for '{}'".format(key)) + + # resolved per-version configuration. The shape is something like: + # versions = { + # "1.0.0": { + # "base_url": "", + # "manifest_filename": "", + # "platforms": { + # "platform_name": struct( + # compatible_with = [], + # target_settings = [], + # urls = [], # can be unset + # sha256 = "", # can be unset + # ), + # }, + # }, + # } + versions = {} + for mod in module_ctx.modules: + if not (mod.is_root or mod.name == "rules_python"): + continue + + # last_version is the last version used in the MODULE.bazel or the default + last_version = None + for tag in mod.tags.configure: + last_version = tag.version or last_version or defaults["version"] + specific_config = versions.setdefault( + last_version, + { + "base_url": defaults["base_url"], + "manifest_filename": defaults["manifest_filename"], + # shallow copy is enough as the values are structs and will + # be replaced on modification + "platforms": dict(defaults["platforms"]), + }, + ) + + _configure( + specific_config, + base_url = tag.base_url, + manifest_filename = tag.manifest_filename, + platform = tag.platform, + compatible_with = tag.compatible_with, + target_settings = tag.target_settings, + sha256 = tag.sha256, + urls = tag.urls, + override = mod.is_root, + netrc = tag.netrc, + auth_patterns = tag.auth_patterns, + ) + + if not versions: + return hub_repo( + name = hub_name, + toolchain_type = toolchain_type, + toolchain_names = ["none"], + toolchain_implementations = { + # NOTE @aignas 2025-02-24: the label to the toolchain can be anything + "none": labels.NONE, + }, + toolchain_compatible_with = { + "none": ["@platforms//:incompatible"], + }, + toolchain_target_settings = {}, + ) + + toolchain_names = [] + toolchain_implementations = {} + toolchain_compatible_with_by_toolchain = {} + toolchain_target_settings = {} + for version, config in versions.items(): + platforms = config["platforms"] + + # Use the manually specified urls + urls = { + platform: src + for platform, src in config.get("urls", {}).items() + if src.urls + } + auth = { + "auth_patterns": config.get("auth_patterns"), + "netrc": config.get("netrc"), + } + auth = {k: v for k, v in auth.items() if v} + + # Or fallback to fetching them from GH manifest file + # Example file: https://github.com/astral-sh/uv/releases/download/0.6.3/dist-manifest.json + if not urls: + urls = _get_tool_urls_from_dist_manifest( + module_ctx, + base_url = "{base_url}/{version}".format( + version = version, + base_url = config["base_url"], + ), + manifest_filename = config["manifest_filename"], + platforms = sorted(platforms), + get_auth = get_auth, + **auth + ) + + for platform_name, platform in platforms.items(): + if platform_name not in urls: + continue + + toolchain_name = "{}_{}".format(version.replace(".", "_"), platform_name.lower().replace("-", "_")) + uv_repository_name = "{}_{}".format(hub_name, toolchain_name) + uv_repository( + name = uv_repository_name, + version = version, + platform = platform_name, + urls = urls[platform_name].urls, + sha256 = urls[platform_name].sha256, + **auth + ) + + toolchain_names.append(toolchain_name) + toolchain_implementations[toolchain_name] = "@{}//:uv_toolchain".format(uv_repository_name) + toolchain_compatible_with_by_toolchain[toolchain_name] = [ + str(label) + for label in platform.compatible_with + ] + if platform.target_settings: + toolchain_target_settings[toolchain_name] = [ + str(label) + for label in platform.target_settings + ] + + return hub_repo( + name = hub_name, + toolchain_type = toolchain_type, + toolchain_names = toolchain_names, + toolchain_implementations = toolchain_implementations, + toolchain_compatible_with = toolchain_compatible_with_by_toolchain, + toolchain_target_settings = toolchain_target_settings, + ) + +def _uv_toolchain_extension(module_ctx): + process_modules( + module_ctx, + hub_name = "uv", + ) + +def _overlap(first_collection, second_collection): + for x in first_collection: + if x in second_collection: + return True + + return False + +# See https://github.com/astral-sh/setup-uv/pull/809. +GITHUB_RELEASES_PREFIX = "https://github.com/astral-sh/uv/releases/download/" +ASTRAL_MIRROR_PREFIX = "https://releases.astral.sh/github/uv/releases/download/" + +def _get_tool_urls_from_dist_manifest(module_ctx, *, base_url, manifest_filename, platforms, get_auth = get_auth, **auth_attrs): + """Download the results about remote tool sources. + + This relies on the tools using the cargo packaging to infer the actual + sha256 values for each binary. + + Example manifest url: https://github.com/astral-sh/uv/releases/download/0.6.5/dist-manifest.json + + The example format is as below + + dist_version "0.28.0" + announcement_tag "0.6.5" + announcement_tag_is_implicit false + announcement_is_prerelease false + announcement_title "0.6.5" + announcement_changelog "text" + announcement_github_body "MD text" + releases [ + { + app_name "uv" + app_version "0.6.5" + env + install_dir_env_var "UV_INSTALL_DIR" + unmanaged_dir_env_var "UV_UNMANAGED_INSTALL" + disable_update_env_var "UV_DISABLE_UPDATE" + no_modify_path_env_var "UV_NO_MODIFY_PATH" + github_base_url_env_var "UV_INSTALLER_GITHUB_BASE_URL" + ghe_base_url_env_var "UV_INSTALLER_GHE_BASE_URL" + display_name "uv" + display true + artifacts [ + "source.tar.gz" + "source.tar.gz.sha256" + "uv-installer.sh" + "uv-installer.ps1" + "sha256.sum" + "uv-aarch64-apple-darwin.tar.gz" + "uv-aarch64-apple-darwin.tar.gz.sha256" + "... + ] + hosting + order + 0 "simple" + 1 "github" + github + artifact_base_url "https://github.com" + artifact_download_path "/astral-sh/uv/releases/download/0.11.2" + owner "astral-sh" + repo "uv" + simple + download_url "https://releases.astral.sh/github/uv/releases/download/0.11.2" + artifacts + uv-aarch64-apple-darwin.tar.gz + name "uv-aarch64-apple-darwin.tar.gz" + kind "executable-zip" + target_triples [ + "aarch64-apple-darwin" + assets [ + { + id "uv-aarch64-apple-darwin-exe-uv" + name "uv" + path "uv" + kind "executable" + }, + { + id "uv-aarch64-apple-darwin-exe-uvx" + name "uvx" + path "uvx" + kind "executable" + } + ] + checksum "uv-aarch64-apple-darwin.tar.gz.sha256" + checksums + sha256 "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef" + uv-aarch64-apple-darwin.tar.gz.sha256 + name "uv-aarch64-apple-darwin.tar.gz.sha256" + kind "checksum" + target_triples [ + "aarch64-apple-darwin" + ] + """ + auth_attr = struct(**auth_attrs) + dist_manifest = module_ctx.path(manifest_filename) + urls = [base_url + "/" + manifest_filename] + result = module_ctx.download( + url = urls, + output = dist_manifest, + auth = get_auth(module_ctx, urls, ctx_attr = auth_attr), + ) + if not result.success: + fail(result) + dist_manifest = json.decode(module_ctx.read(dist_manifest)) + + # Use the simple download_url from the manifest, when available. + dist_base_url = ( + dist_manifest + .get("releases", [{}])[0] + .get("hosting", {}) + .get("simple", {}) + .get("download_url", base_url) + ) + + # For official releases, add the astral mirror to improve availability. + # See https://github.com/astral-sh/setup-uv/pull/809. + if dist_base_url.startswith(GITHUB_RELEASES_PREFIX): + astral_base_url = ASTRAL_MIRROR_PREFIX + dist_base_url[len(GITHUB_RELEASES_PREFIX):] + base_urls = [ + astral_base_url, + dist_base_url, + ] + else: + base_urls = [dist_base_url] + + artifacts = dist_manifest["artifacts"] + tool_sources = {} + downloads = {} + for fname, artifact in artifacts.items(): + if artifact.get("kind") != "executable-zip": + continue + target_triples = artifact["target_triples"] + if not _overlap(target_triples, platforms): + # We are not interested in this platform, so skip. + continue + + # Releases of uv >= 0.11.0 have the sha256 directly inline. + sha256 = artifact.get("checksums", {}).get("sha256", "") + if len(sha256) > 0: + for platform in target_triples: + tool_sources[platform] = struct( + urls = [ + "{}/{}".format(base, fname) + for base in base_urls + ], + sha256 = sha256, + ) + continue + + # For uv < 0.11.0, we'll need to fetch the individual sha256 files. + checksum = artifacts[artifact["checksum"]] + checksum_fname = checksum["name"] + checksum_path = module_ctx.path(checksum_fname) + urls = ["{}/{}".format(dist_base_url, checksum_fname)] + downloads[checksum_path] = struct( + download = module_ctx.download( + url = urls, + output = checksum_path, + block = False, + auth = get_auth(module_ctx, urls, ctx_attr = auth_attr), + ), + archive_fname = fname, + platforms = target_triples, + ) + + for checksum_path, download in downloads.items(): + result = download.download.wait() + if not result.success: + fail(result) + + archive_fname = download.archive_fname + + sha256, _, checksummed_fname = module_ctx.read(checksum_path).partition(" ") + checksummed_fname = checksummed_fname.strip(" *\n") + if checksummed_fname and archive_fname != checksummed_fname: + fail("The checksum is for a different file, expected '{}' but got '{}'".format( + archive_fname, + checksummed_fname, + )) + + for platform in download.platforms: + tool_sources[platform] = struct( + urls = [ + "{}/{}".format(base, archive_fname) + for base in base_urls + ], + sha256 = sha256, + ) + + return tool_sources + +uv = module_extension( + doc = _DOC, + implementation = _uv_toolchain_extension, + tag_classes = { + "configure": configure, + "default": default, + }, +) diff --git a/python/uv/private/uv_lock_to_requirements.bzl b/python/uv/private/uv_lock_to_requirements.bzl new file mode 100644 index 0000000000..73d912bbfd --- /dev/null +++ b/python/uv/private/uv_lock_to_requirements.bzl @@ -0,0 +1,139 @@ +"""Convert a parsed uv.lock to requirements.txt format.""" + +def uv_lock_extras_map(uv_lock): + """Compute extras for each package from uv.lock data. + + Args: + uv_lock: a decoded JSON struct from a uv.lock file. + + Returns: + A dict of {package_name: [extra1, extra2, ...]} for packages with extras. + """ + extras_map = {} + for pkg in uv_lock.get("package", []): + pkg_name = pkg.get("name", "") + + for extra in pkg.get("provides-extras", pkg.get("extras", [])): + _add_extras(extras_map, pkg_name, [extra]) + + opt_deps = pkg.get("optional-dependencies", {}) + if opt_deps: + _add_extras(extras_map, pkg_name, _sorted(opt_deps.keys())) + + deps = pkg.get("dependencies", []) + for dep in deps: + dep_name = dep.get("name", "") + dep_extras_raw = dep.get("extra", []) + dep_extras = [dep_extras_raw] if type(dep_extras_raw) == "string" else dep_extras_raw + if dep_extras and dep_name != pkg_name: + _add_extras(extras_map, dep_name, dep_extras) + + metadata = pkg.get("metadata", {}) + for rd in metadata.get("requires-dist", []): + rd_name = rd.get("name", "") + rd_extras_raw = rd.get("extras", []) + rd_extras = [rd_extras_raw] if type(rd_extras_raw) == "string" else rd_extras_raw + if rd_extras and rd_name != pkg_name: + _add_extras(extras_map, rd_name, rd_extras) + + return extras_map + +def uv_lock_to_requirements(uv_lock): + """Convert a parsed uv.lock JSON struct to a requirements.txt formatted string. + + Args: + uv_lock: a decoded JSON struct from a uv.lock file. + + Returns: + A requirements.txt formatted string. + """ + packages = uv_lock.get("package", []) + extras_map = uv_lock_extras_map(uv_lock) + + dependents = {} + for pkg in packages: + pkg_name = pkg.get("name", "") + deps = pkg.get("dependencies", []) + for dep in deps: + dep_name = dep.get("name", "") + if dep_name != pkg_name: + _add_dependent(dependents, dep_name, pkg_name) + opt_deps = pkg.get("optional-dependencies", {}) + for _extra, deps in opt_deps.items(): + for dep in deps: + dep_name = dep.get("name", "") + if dep_name != pkg_name: + _add_dependent(dependents, dep_name, pkg_name) + + lines = [] + for pkg in packages: + source = pkg.get("source", {}) + if not source.get("registry"): + continue + + pkg_name = pkg.get("name", "") + version = pkg.get("version", "") + + markers = pkg.get("resolution-markers", []) + hashes = _collect_hashes(pkg) + + pkg_extras = extras_map.get(pkg_name, []) + if pkg_extras: + req = "{}[{}]=={}".format(pkg_name, ",".join(pkg_extras), version) + else: + req = "{}=={}".format(pkg_name, version) + if markers: + req += " ; " + " or ".join(markers) + + if hashes: + req += " \\" + lines.append(req) + + _emit_hashes(lines, hashes) + + dep_vias = _sorted(dependents.get(pkg_name, [])) + if dep_vias: + if len(dep_vias) == 1: + lines.append(" # via " + dep_vias[0]) + else: + lines.append(" # via") + for via in dep_vias: + lines.append(" # " + via) + + lines.append("") + + return "\n".join(lines) + +def _add_dependent(dependents, dep_name, dependent_name): + if dep_name not in dependents: + dependents[dep_name] = [] + if dependent_name not in dependents[dep_name]: + dependents[dep_name].append(dependent_name) + +def _add_extras(extras_map, pkg_name, extras): + existing = extras_map.get(pkg_name, []) + for extra in extras: + if extra not in existing: + existing.append(extra) + extras_map[pkg_name] = _sorted(existing) + +def _collect_hashes(pkg): + hashes = [] + for wheel in pkg.get("wheels", []): + whash = wheel.get("hash", "") + if whash.startswith("sha256:"): + hashes.append(whash[len("sha256:"):]) + sdist = pkg.get("sdist") + if sdist: + shash = sdist.get("hash", "") + if shash.startswith("sha256:"): + hashes.append(shash[len("sha256:"):]) + return _sorted(hashes) + +def _sorted(items): + return sorted(items) + +def _emit_hashes(lines, hashes): + for i, h in enumerate(hashes): + suffix = " \\" if i < len(hashes) - 1 else "" + lines.append(" --hash=sha256:{}{}".format(h, suffix)) diff --git a/python/uv/private/uv_repository.bzl b/python/uv/private/uv_repository.bzl new file mode 100644 index 0000000000..79a6495bdc --- /dev/null +++ b/python/uv/private/uv_repository.bzl @@ -0,0 +1,87 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +""" +EXPERIMENTAL: This is experimental and may be removed without notice + +Create repositories for uv toolchain dependencies +""" + +load("//python/private:auth.bzl", "AUTH_ATTRS", "get_auth") + +UV_BUILD_TMPL = """\ +# Generated by repositories.bzl +load("@rules_python//python/uv:uv_toolchain.bzl", "uv_toolchain") + +uv_toolchain( + name = "uv_toolchain", + uv = "{binary}", + version = "{version}", +) +""" + +def _uv_repo_impl(repository_ctx): + platform = repository_ctx.attr.platform + + is_windows = "windows" in platform + _, _, filename = repository_ctx.attr.urls[0].rpartition("/") + if filename.endswith(".tar.gz"): + strip_prefix = filename[:-len(".tar.gz")] + else: + strip_prefix = "" + + result = repository_ctx.download_and_extract( + url = repository_ctx.attr.urls, + sha256 = repository_ctx.attr.sha256, + stripPrefix = strip_prefix, + auth = get_auth(repository_ctx, repository_ctx.attr.urls), + ) + + binary = "uv.exe" if is_windows else "uv" + repository_ctx.file( + "BUILD.bazel", + UV_BUILD_TMPL.format( + binary = binary, + version = repository_ctx.attr.version, + ), + ) + + attrs = { + "name": repository_ctx.attr.name, + "platform": repository_ctx.attr.platform, + "sha256": result.sha256, + "urls": repository_ctx.attr.urls, + "version": repository_ctx.attr.version, + } + + # Bazel <8.3.0 lacks repository_ctx.repo_metadata + if not hasattr(repository_ctx, "repo_metadata"): + return attrs + + reproducible = repository_ctx.attr.sha256 != "" + return repository_ctx.repo_metadata( + reproducible = reproducible, + attrs_for_reproducibility = {} if reproducible else attrs, + ) + +uv_repository = repository_rule( + _uv_repo_impl, + doc = "Fetch external tools needed for uv toolchain", + attrs = { + "platform": attr.string(mandatory = True), + "sha256": attr.string(mandatory = False), + "urls": attr.string_list(mandatory = True), + "version": attr.string(mandatory = True), + } | AUTH_ATTRS, +) diff --git a/python/uv/toolchain.bzl b/python/uv/private/uv_toolchain.bzl similarity index 90% rename from python/uv/toolchain.bzl rename to python/uv/private/uv_toolchain.bzl index 3cd5850acd..bd82e7452f 100644 --- a/python/uv/toolchain.bzl +++ b/python/uv/private/uv_toolchain.bzl @@ -18,18 +18,20 @@ EXPERIMENTAL: This is experimental and may be removed without notice This module implements the uv toolchain rule """ -load("//python/uv/private:providers.bzl", "UvToolchainInfo") +load(":uv_toolchain_info.bzl", "UvToolchainInfo") def _uv_toolchain_impl(ctx): uv = ctx.attr.uv default_info = DefaultInfo( - files = uv.files, + files = uv[DefaultInfo].files, runfiles = uv[DefaultInfo].default_runfiles, ) uv_toolchain_info = UvToolchainInfo( uv = uv, version = ctx.attr.version, + # Exposed for testing/debugging + label = ctx.label, ) # Export all the providers inside our ToolchainInfo @@ -51,7 +53,7 @@ uv_toolchain = rule( mandatory = True, allow_single_file = True, executable = True, - cfg = "target", + cfg = "exec", ), "version": attr.string(mandatory = True, doc = "Version of the uv binary."), }, diff --git a/python/uv/private/providers.bzl b/python/uv/private/uv_toolchain_info.bzl similarity index 89% rename from python/uv/private/providers.bzl rename to python/uv/private/uv_toolchain_info.bzl index ac1ef310ea..5d70766e7f 100644 --- a/python/uv/private/providers.bzl +++ b/python/uv/private/uv_toolchain_info.bzl @@ -17,6 +17,11 @@ UvToolchainInfo = provider( doc = "Information about how to invoke the uv executable.", fields = { + "label": """ +:type: Label + +The uv toolchain implementation label returned by the toolchain. +""", "uv": """ :type: Target diff --git a/python/uv/private/toolchains_repo.bzl b/python/uv/private/uv_toolchains_repo.bzl similarity index 50% rename from python/uv/private/toolchains_repo.bzl rename to python/uv/private/uv_toolchains_repo.bzl index 9a8858f1b0..7e11e0adb6 100644 --- a/python/uv/private/toolchains_repo.bzl +++ b/python/uv/private/uv_toolchains_repo.bzl @@ -16,37 +16,44 @@ load("//python/private:text_util.bzl", "render") -_TOOLCHAIN_TEMPLATE = """ -toolchain( - name = "{name}", - target_compatible_with = {compatible_with}, - toolchain = "{toolchain_label}", - toolchain_type = "{toolchain_type}", -) -""" +_TEMPLATE = """\ +load("@rules_python//python/uv/private:toolchains_hub.bzl", "toolchains_hub") -def _toolchains_repo_impl(repository_ctx): - build_content = "" - for toolchain_name in repository_ctx.attr.toolchain_names: - toolchain_label = repository_ctx.attr.toolchain_labels[toolchain_name] - toolchain_compatible_with = repository_ctx.attr.toolchain_compatible_with[toolchain_name] +{} +""" - build_content += _TOOLCHAIN_TEMPLATE.format( - name = toolchain_name, - toolchain_type = repository_ctx.attr.toolchain_type, - toolchain_label = toolchain_label, - compatible_with = render.list(toolchain_compatible_with), - ) +def _non_empty(d): + return {k: v for k, v in d.items() if v} - repository_ctx.file("BUILD.bazel", build_content) +def _toolchains_repo_impl(repository_ctx): + contents = _TEMPLATE.format( + render.call( + "toolchains_hub", + name = repr("uv_toolchain"), + toolchains = render.list(repository_ctx.attr.toolchain_names), + implementations = render.dict( + repository_ctx.attr.toolchain_implementations, + ), + target_compatible_with = render.dict( + repository_ctx.attr.toolchain_compatible_with, + value_repr = render.list, + ), + target_settings = render.dict( + _non_empty(repository_ctx.attr.toolchain_target_settings), + value_repr = render.list, + ), + ), + ) + repository_ctx.file("BUILD.bazel", contents) uv_toolchains_repo = repository_rule( _toolchains_repo_impl, doc = "Generates a toolchain hub repository", attrs = { "toolchain_compatible_with": attr.string_list_dict(doc = "A list of platform constraints for this toolchain, keyed by toolchain name.", mandatory = True), - "toolchain_labels": attr.string_dict(doc = "The name of the toolchain implementation target, keyed by toolchain name.", mandatory = True), + "toolchain_implementations": attr.string_dict(doc = "The name of the toolchain implementation target, keyed by toolchain name.", mandatory = True), "toolchain_names": attr.string_list(doc = "List of toolchain names", mandatory = True), + "toolchain_target_settings": attr.string_list_dict(doc = "A list of target_settings constraints for this toolchain, keyed by toolchain name.", mandatory = True), "toolchain_type": attr.string(doc = "The toolchain type of the toolchains", mandatory = True), }, ) diff --git a/python/uv/private/versions.bzl b/python/uv/private/versions.bzl deleted file mode 100644 index 1d68302c74..0000000000 --- a/python/uv/private/versions.bzl +++ /dev/null @@ -1,94 +0,0 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""Version and integrity information for downloaded artifacts""" - -UV_PLATFORMS = { - "aarch64-apple-darwin": struct( - default_repo_name = "uv_darwin_aarch64", - compatible_with = [ - "@platforms//os:macos", - "@platforms//cpu:aarch64", - ], - ), - "aarch64-unknown-linux-gnu": struct( - default_repo_name = "uv_linux_aarch64", - compatible_with = [ - "@platforms//os:linux", - "@platforms//cpu:aarch64", - ], - ), - "powerpc64le-unknown-linux-gnu": struct( - default_repo_name = "uv_linux_ppc", - compatible_with = [ - "@platforms//os:linux", - "@platforms//cpu:ppc", - ], - ), - "s390x-unknown-linux-gnu": struct( - default_repo_name = "uv_linux_s390x", - compatible_with = [ - "@platforms//os:linux", - "@platforms//cpu:s390x", - ], - ), - "x86_64-apple-darwin": struct( - default_repo_name = "uv_darwin_x86_64", - compatible_with = [ - "@platforms//os:macos", - "@platforms//cpu:x86_64", - ], - ), - "x86_64-pc-windows-msvc": struct( - default_repo_name = "uv_windows_x86_64", - compatible_with = [ - "@platforms//os:windows", - "@platforms//cpu:x86_64", - ], - ), - "x86_64-unknown-linux-gnu": struct( - default_repo_name = "uv_linux_x86_64", - compatible_with = [ - "@platforms//os:linux", - "@platforms//cpu:x86_64", - ], - ), -} - -# From: https://github.com/astral-sh/uv/releases -UV_TOOL_VERSIONS = { - "0.4.25": { - "aarch64-apple-darwin": struct( - sha256 = "bb2ff4348114ef220ca52e44d5086640c4a1a18f797a5f1ab6f8559fc37b1230", - ), - "aarch64-unknown-linux-gnu": struct( - sha256 = "4485852eb8013530c4275cd222c0056ce123f92742321f012610f1b241463f39", - ), - "powerpc64le-unknown-linux-gnu": struct( - sha256 = "32421c61e8d497243171b28c7efd74f039251256ae9e57ce4a457fdd7d045e24", - ), - "s390x-unknown-linux-gnu": struct( - sha256 = "9afa342d87256f5178a592d3eeb44ece8a93e9359db37e31be1b092226338469", - ), - "x86_64-apple-darwin": struct( - sha256 = "f0ec1f79f4791294382bff242691c6502e95853acef080ae3f7c367a8e1beb6f", - ), - "x86_64-pc-windows-msvc": struct( - sha256 = "c5c7fa084ae4e8ac9e3b0b6c4c7b61e9355eb0c86801c4c7728c0cb142701f38", - ), - "x86_64-unknown-linux-gnu": struct( - sha256 = "6cb6eaf711cd7ce5fb1efaa539c5906374c762af547707a2041c9f6fd207769a", - ), - }, -} diff --git a/python/uv/repositories.bzl b/python/uv/repositories.bzl deleted file mode 100644 index 0125b2033b..0000000000 --- a/python/uv/repositories.bzl +++ /dev/null @@ -1,120 +0,0 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -""" -EXPERIMENTAL: This is experimental and may be removed without notice - -Create repositories for uv toolchain dependencies -""" - -load("//python/uv/private:toolchain_types.bzl", "UV_TOOLCHAIN_TYPE") -load("//python/uv/private:toolchains_repo.bzl", "uv_toolchains_repo") -load("//python/uv/private:versions.bzl", "UV_PLATFORMS", "UV_TOOL_VERSIONS") - -UV_BUILD_TMPL = """\ -# Generated by repositories.bzl -load("@rules_python//python/uv:toolchain.bzl", "uv_toolchain") - -uv_toolchain( - name = "uv_toolchain", - uv = "{binary}", - version = "{version}", -) -""" - -def _uv_repo_impl(repository_ctx): - platform = repository_ctx.attr.platform - uv_version = repository_ctx.attr.uv_version - - is_windows = "windows" in platform - - suffix = ".zip" if is_windows else ".tar.gz" - filename = "uv-{platform}{suffix}".format( - platform = platform, - suffix = suffix, - ) - url = "https://github.com/astral-sh/uv/releases/download/{version}/{filename}".format( - version = uv_version, - filename = filename, - ) - if filename.endswith(".tar.gz"): - strip_prefix = filename[:-len(".tar.gz")] - else: - strip_prefix = "" - - repository_ctx.download_and_extract( - url = url, - sha256 = UV_TOOL_VERSIONS[repository_ctx.attr.uv_version][repository_ctx.attr.platform].sha256, - stripPrefix = strip_prefix, - ) - - binary = "uv.exe" if is_windows else "uv" - repository_ctx.file( - "BUILD.bazel", - UV_BUILD_TMPL.format( - binary = binary, - version = uv_version, - ), - ) - -uv_repository = repository_rule( - _uv_repo_impl, - doc = "Fetch external tools needed for uv toolchain", - attrs = { - "platform": attr.string(mandatory = True, values = UV_PLATFORMS.keys()), - "uv_version": attr.string(mandatory = True, values = UV_TOOL_VERSIONS.keys()), - }, -) - -# buildifier: disable=unnamed-macro -def uv_register_toolchains(uv_version = None, register_toolchains = True): - """Convenience macro which does typical toolchain setup - - Skip this macro if you need more control over the toolchain setup. - - Args: - uv_version: The uv toolchain version to download. - register_toolchains: If true, repositories will be generated to produce and register `uv_toolchain` targets. - """ - if not uv_version: - fail("uv_version is required") - - toolchain_names = [] - toolchain_labels_by_toolchain = {} - toolchain_compatible_with_by_toolchain = {} - - for platform in UV_PLATFORMS.keys(): - uv_repository_name = UV_PLATFORMS[platform].default_repo_name - - uv_repository( - name = uv_repository_name, - uv_version = uv_version, - platform = platform, - ) - - toolchain_name = uv_repository_name + "_toolchain" - toolchain_names.append(toolchain_name) - toolchain_labels_by_toolchain[toolchain_name] = "@{}//:uv_toolchain".format(uv_repository_name) - toolchain_compatible_with_by_toolchain[toolchain_name] = UV_PLATFORMS[platform].compatible_with - - uv_toolchains_repo( - name = "uv_toolchains", - toolchain_type = str(UV_TOOLCHAIN_TYPE), - toolchain_names = toolchain_names, - toolchain_labels = toolchain_labels_by_toolchain, - toolchain_compatible_with = toolchain_compatible_with_by_toolchain, - ) - - if register_toolchains: - native.register_toolchains("@uv_toolchains//:all") diff --git a/python/uv/uv.bzl b/python/uv/uv.bzl new file mode 100644 index 0000000000..d72ab9dc3d --- /dev/null +++ b/python/uv/uv.bzl @@ -0,0 +1,22 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +""" EXPERIMENTAL: This is experimental and may be removed without notice. + +The uv toolchain extension. +""" + +load("//python/uv/private:uv.bzl", _uv = "uv") + +uv = _uv diff --git a/python/uv/uv_toolchain.bzl b/python/uv/uv_toolchain.bzl new file mode 100644 index 0000000000..a4b466cb1b --- /dev/null +++ b/python/uv/uv_toolchain.bzl @@ -0,0 +1,22 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""The `uv_toolchain` rule. + +EXPERIMENTAL: This is experimental and may be removed without notice +""" + +load("//python/uv/private:uv_toolchain.bzl", _uv_toolchain = "uv_toolchain") + +uv_toolchain = _uv_toolchain diff --git a/python/uv/defs.bzl b/python/uv/uv_toolchain_info.bzl similarity index 78% rename from python/uv/defs.bzl rename to python/uv/uv_toolchain_info.bzl index 20b426a355..1ae89636be 100644 --- a/python/uv/defs.bzl +++ b/python/uv/uv_toolchain_info.bzl @@ -1,4 +1,4 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. +# Copyright 2025 The Bazel Authors. All rights reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -12,12 +12,11 @@ # See the License for the specific language governing permissions and # limitations under the License. -""" -EXPERIMENTAL: This is experimental and may be removed without notice +"""The `UvToolchainInfo` provider. -A toolchain for uv +EXPERIMENTAL: This is experimental and may be removed without notice """ -load("//python/uv/private:providers.bzl", _UvToolchainInfo = "UvToolchainInfo") +load("//python/uv/private:uv_toolchain_info.bzl", _UvToolchainInfo = "UvToolchainInfo") UvToolchainInfo = _UvToolchainInfo diff --git a/python/versions.bzl b/python/versions.bzl index 1fd0649f12..6e3efb34d6 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -15,768 +15,233 @@ """The Python versions we use for the toolchains. """ +load("//python/private:pbs_manifest.bzl", "parse_runtime_manifest") +load("//python/private:platform_info.bzl", "platform_info") + +##load("@rules_python_internal//:manifest_tool_versions.bzl", "MANIFEST_ENTRIES") +load("//python/private:runtimes_manifest_workspace.bzl", "MANIFEST_TEXT") + # Values present in the @platforms//os package MACOS_NAME = "osx" LINUX_NAME = "linux" WINDOWS_NAME = "windows" -FREETHREADED = "freethreaded" + +FREETHREADED = "-freethreaded" +MUSL = "-musl" INSTALL_ONLY = "install_only" -DEFAULT_RELEASE_BASE_URL = "https://github.com/indygreg/python-build-standalone/releases/download" +_GITHUB_PREFIX = "https://github.com/astral-sh/python-build-standalone/releases/download" +_LEGACY_GITHUB_PREFIX = "https://github.com/indygreg/python-build-standalone/releases/download" +_ASTRAL_PREFIX = "https://releases.astral.sh/github/python-build-standalone/releases/download" -# When updating the versions and releases, run the following command to get -# the hashes: -# bazel run //python/private:print_toolchains_checksums --//python/config_settings:python_version={major}.{minor}.{patch} -# -# Note, to users looking at how to specify their tool versions, coverage_tool version for each -# interpreter can be specified by: -# "3.8.10": { -# "url": "20210506/cpython-{python_version}-{platform}-pgo+lto-20210506T0943.tar.zst", -# "sha256": { -# "x86_64-apple-darwin": "8d06bec08db8cdd0f64f4f05ee892cf2fcbc58cfb1dd69da2caab78fac420238", -# "x86_64-unknown-linux-gnu": "aec8c4c53373b90be7e2131093caa26063be6d9d826f599c935c0e1042af3355", -# }, -# "coverage_tool": { -# "x86_64-apple-darwin": """, -# "x86_64-unknown-linux-gnu": """, -# }, -# "strip_prefix": "python", -# }, -# -# It is possible to provide lists in "url". It is also possible to provide patches or patch_strip. -# -# buildifier: disable=unsorted-dict-items -TOOL_VERSIONS = { - "3.8.10": { - "url": "20210506/cpython-{python_version}-{platform}-pgo+lto-20210506T0943.tar.zst", - "sha256": { - "x86_64-apple-darwin": "8d06bec08db8cdd0f64f4f05ee892cf2fcbc58cfb1dd69da2caab78fac420238", - "x86_64-unknown-linux-gnu": "aec8c4c53373b90be7e2131093caa26063be6d9d826f599c935c0e1042af3355", - }, - "strip_prefix": "python/install", - }, - "3.8.12": { - "url": "20220227/cpython-{python_version}+20220227-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "f9a3cbb81e0463d6615125964762d133387d561b226a30199f5b039b20f1d944", - # no aarch64-unknown-linux-gnu build available for 3.8.12 - "x86_64-apple-darwin": "f323fbc558035c13a85ce2267d0fad9e89282268ecb810e364fff1d0a079d525", - "x86_64-pc-windows-msvc": "4658e08a00d60b1e01559b74d58ff4dd04da6df935d55f6268a15d6d0a679d74", - "x86_64-unknown-linux-gnu": "5be9c6d61e238b90dfd94755051c0d3a2d8023ebffdb4b0fa4e8fedd09a6cab6", - }, - "strip_prefix": "python", - }, - "3.8.13": { - "url": "20220802/cpython-{python_version}+20220802-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "ae4131253d890b013171cb5f7b03cadc585ae263719506f7b7e063a7cf6fde76", - # no aarch64-unknown-linux-gnu build available for 3.8.13 - "x86_64-apple-darwin": "cd6e7c0a27daf7df00f6882eaba01490dd963f698e99aeee9706877333e0df69", - "x86_64-pc-windows-msvc": "f20643f1b3e263a56287319aea5c3888530c09ad9de3a5629b1a5d207807e6b9", - "x86_64-unknown-linux-gnu": "fb566629ccb5f76ef56d275a3f8017d683f1c20c5beb5d5f38b155ed11e16187", - }, - "strip_prefix": "python", - }, - "3.8.15": { - "url": "20221106/cpython-{python_version}+20221106-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "1e0a92d1a4f5e6d4a99f86b1cbf9773d703fe7fd032590f3e9c285c7a5eeb00a", - "aarch64-unknown-linux-gnu": "886ab33ced13c84bf59ce8ff79eba6448365bfcafea1bf415bd1d75e21b690aa", - "x86_64-apple-darwin": "70b57f28c2b5e1e3dd89f0d30edd5bc414e8b20195766cf328e1b26bed7890e1", - "x86_64-pc-windows-msvc": "2fdc3fa1c95f982179bbbaedae2b328197658638799b6dcb63f9f494b0de59e2", - "x86_64-unknown-linux-gnu": "e47edfb2ceaf43fc699e20c179ec428b6f3e497cf8e2dcd8e9c936d4b96b1e56", - }, - "strip_prefix": "python", - }, - "3.8.16": { - "url": "20230116/cpython-{python_version}+20230116-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "d1f408569d8807c1053939d7822b082a17545e363697e1ce3cfb1ee75834c7be", - "aarch64-unknown-linux-gnu": "15d00bc8400ed6d94c665a797dc8ed7a491ae25c5022e738dcd665cd29beec42", - "x86_64-apple-darwin": "484ba901f64fc7888bec5994eb49343dc3f9d00ed43df17ee9c40935aad4aa18", - "x86_64-pc-windows-msvc": "b446bec833eaba1bac9063bb9b4aeadfdf67fa81783b4487a90c56d408fb7994", - "x86_64-unknown-linux-gnu": "c890de112f1ae31283a31fefd2061d5c97bdd4d1bdd795552c7abddef2697ea1", - }, - "strip_prefix": "python", - }, - "3.8.17": { - "url": "20230826/cpython-{python_version}+20230826-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "c6f7a130d0044a78e39648f4dae56dcff5a41eba91888a99f6e560507162e6a1", - "aarch64-unknown-linux-gnu": "9f6d585091fe26906ff1dbb80437a3fe37a1e3db34d6ecc0098f3d6a78356682", - "x86_64-apple-darwin": "155b06821607bae1a58ecc60a7d036b358c766f19e493b8876190765c883a5c2", - "x86_64-pc-windows-msvc": "6428e1b4e0b4482d390828de7d4c82815257443416cb786abe10cb2466ca68cd", - "x86_64-unknown-linux-gnu": "8d3e1826c0bb7821ec63288038644808a2d45553245af106c685ef5892fabcd8", - }, - "strip_prefix": "python", - }, - "3.8.18": { - "url": "20240224/cpython-{python_version}+20240224-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "4d493a1792bf211f37f98404cc1468f09bd781adc2602dea0df82ad264c11abc", - "aarch64-unknown-linux-gnu": "6588c9eed93833d9483d01fe40ac8935f691a1af8e583d404ec7666631b52487", - "x86_64-apple-darwin": "7d2cd8d289d5e3cdd0a8c06c028c7c621d3d00ce44b7e2f08c1724ae0471c626", - "x86_64-pc-windows-msvc": "dba923ee5df8f99db04f599e826be92880746c02247c8d8e4d955d4bc711af11", - "x86_64-unknown-linux-gnu": "5ae36825492372554c02708bdd26b8dcd57e3dbf34b3d6d599ad91d93540b2b7", - }, - "strip_prefix": "python", - }, - "3.8.19": { - "url": "20240726/cpython-{python_version}+20240726-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "fe4af1b6bc59478d027ede43f6249cf7b9143558e171bdf8711247337623af57", - "aarch64-unknown-linux-gnu": "8dc598aca7ad43ea20119324af98862d198d8990151c734a69f0fc9d16384b46", - "x86_64-apple-darwin": "4bc990b35384c83b5b0b3071e91455ec203517e569f29f691b159f1a6b2a19b2", - "x86_64-pc-windows-msvc": "4e8e9ddda82062d6e111108ab72f439acac4ba41b77d694548ef5dbf6b2b3319", - "x86_64-unknown-linux-gnu": "e81ea4dd16e6057c8121bdbcb7b64e2956068ca019f244c814bc3ad907cb2765", - }, - "strip_prefix": "python", - }, - "3.8.20": { - "url": "20241002/cpython-{python_version}+20241002-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "2ddfc04bdb3e240f30fb782fa1deec6323799d0e857e0b63fa299218658fd3d4", - "aarch64-unknown-linux-gnu": "9d8798f9e79e0fc0f36fcb95bfa28a1023407d51a8ea5944b4da711f1f75f1ed", - "x86_64-apple-darwin": "68d060cd373255d2ca5b8b3441363d5aa7cc45b0c11bbccf52b1717c2b5aa8bb", - "x86_64-pc-windows-msvc": "41b6709fec9c56419b7de1940d1f87fa62045aff81734480672dcb807eedc47e", - "x86_64-unknown-linux-gnu": "285e141c36f88b2e9357654c5f77d1f8fb29cc25132698fe35bb30d787f38e87", - }, - "strip_prefix": "python", - }, - "3.9.10": { - "url": "20220227/cpython-{python_version}+20220227-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "ad66c2a3e7263147e046a32694de7b897a46fb0124409d29d3a93ede631c8aee", - "aarch64-unknown-linux-gnu": "12dd1f125762f47975990ec744532a1cf3db74ad60f4dfb476ca42deb7f78ca4", - "x86_64-apple-darwin": "fdaf594142446029e314a9beb91f1ac75af866320b50b8b968181e592550cd68", - "x86_64-pc-windows-msvc": "c145d9d8143ce163670af124b623d7a2405143a3708b033b4d33eed355e61b24", - "x86_64-unknown-linux-gnu": "455089cc576bd9a58db45e919d1fc867ecdbb0208067dffc845cc9bbf0701b70", - }, - "strip_prefix": "python", - }, - "3.9.12": { - "url": "20220502/cpython-{python_version}+20220502-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "8dee06c07cc6429df34b6abe091a4684a86f7cec76f5d1ccc1c3ce2bd11168df", - "aarch64-unknown-linux-gnu": "2ee1426c181e65133e57dc55c6a685cb1fb5e63ef02d684b8a667d5c031c4203", - "x86_64-apple-darwin": "2453ba7f76b3df3310353b48c881d6cff622ba06e30d2b6ae91588b2bc9e481a", - "x86_64-pc-windows-msvc": "3024147fd987d9e1b064a3d94932178ff8e0fe98cfea955704213c0762fee8df", - "x86_64-unknown-linux-gnu": "ccca12f698b3b810d79c52f007078f520d588232a36bc12ede944ec3ea417816", - }, - "strip_prefix": "python", - }, - "3.9.13": { - "url": "20220802/cpython-{python_version}+20220802-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "d9603edc296a2dcbc59d7ada780fd12527f05c3e0b99f7545112daf11636d6e5", - "aarch64-unknown-linux-gnu": "80415aac1b96255b9211f6a4c300f31e9940c7e07a23d0dec12b53aa52c0d25e", - "x86_64-apple-darwin": "9540a7efb7c8a54a48aff1cb9480e49588d9c0a3f934ad53f5b167338174afa3", - "x86_64-pc-windows-msvc": "b538127025a467c64b3351babca2e4d2ea7bdfb7867d5febb3529c34456cdcd4", - "x86_64-unknown-linux-gnu": "ce1cfca2715e7e646dd618a8cb9baff93000e345ccc979b801fc6ccde7ce97df", - }, - "strip_prefix": "python", - }, - "3.9.15": { - "url": "20221106/cpython-{python_version}+20221106-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "64dc7e1013481c9864152c3dd806c41144c79d5e9cd3140e185c6a5060bdc9ab", - "aarch64-unknown-linux-gnu": "52a8c0a67fb919f80962d992da1bddb511cdf92faf382701ce7673e10a8ff98f", - "x86_64-apple-darwin": "f2bcade6fc976c472f18f2b3204d67202d43ae55cf6f9e670f95e488f780da08", - "x86_64-pc-windows-msvc": "022daacab215679b87f0d200d08b9068a721605fa4721ebeda38220fc641ccf6", - "x86_64-unknown-linux-gnu": "cdc3a4cfddcd63b6cebdd75b14970e02d8ef0ac5be4d350e57ab5df56c19e85e", - }, - "strip_prefix": "python", - }, - "3.9.16": { - "url": "20230507/cpython-{python_version}+20230507-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "c1de1d854717a6245f45262ef1bb17b09e2c587590e7e3f406593c143ff875bd", - "aarch64-unknown-linux-gnu": "f629b75ebfcafe9ceee2e796b7e4df5cf8dbd14f3c021afca078d159ab797acf", - "ppc64le-unknown-linux-gnu": "ff3ac35c58f67839aff9b5185a976abd3d1abbe61af02089f7105e876c1fe284", - "x86_64-apple-darwin": "3abc4d5fbbc80f5f848f280927ac5d13de8dc03aabb6ae65d8247cbb68e6f6bf", - "x86_64-pc-windows-msvc": "cdabb47204e96ce7ea31fbd0b5ed586114dd7d8f8eddf60a509a7f70b48a1c5e", - "x86_64-unknown-linux-gnu": "2b6e146234a4ef2a8946081fc3fbfffe0765b80b690425a49ebe40b47c33445b", - }, - "strip_prefix": "python", - }, - "3.9.17": { - "url": "20230726/cpython-{python_version}+20230726-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "73dbe2d702210b566221da9265acc274ba15275c5d0d1fa327f44ad86cde9aa1", - "aarch64-unknown-linux-gnu": "b77012ddaf7e0673e4aa4b1c5085275a06eee2d66f33442b5c54a12b62b96cbe", - "ppc64le-unknown-linux-gnu": "c591a28d943dce5cf9833e916125fdfbeb3120270c4866ee214493ccb5b83c3c", - "s390x-unknown-linux-gnu": "01454d7cc7c9c2fccde42ba868c4f372eaaafa48049d49dd94c9cf2875f497e6", - "x86_64-apple-darwin": "dfe1bea92c94b9cb779288b0b06e39157c5ff7e465cdd24032ac147c2af485c0", - "x86_64-pc-windows-msvc": "9b9a1e21eff29dcf043cea38180cf8ca3604b90117d00062a7b31605d4157714", - "x86_64-unknown-linux-gnu": "26c4a712b4b8e11ed5c027db5654eb12927c02da4857b777afb98f7a930ce637", - }, - "strip_prefix": "python", - }, - "3.9.18": { - "url": "20240224/cpython-{python_version}+20240224-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "2548f911a6e316575c303ba42bb51540dc9b47a9f76a06a2a37460d93b177aa2", - "aarch64-unknown-linux-gnu": "e5bc5196baa603d635ee6b0cd141e359752ad3e8ea76127eb9141a3155c51200", - "ppc64le-unknown-linux-gnu": "d6b18df7a25fe034fd5ce4e64216df2cc78b2d4d908d2a1c94058ae700d73d22", - "s390x-unknown-linux-gnu": "15d059507c7e900e9665f31e8d903e5a24a68ceed24f9a1c5ac06ab42a354f3f", - "x86_64-apple-darwin": "171d8b472fce0295be0e28bb702c43d5a2a39feccb3e72efe620ac3843c3e402", - "x86_64-pc-windows-msvc": "a9bdbd728ed4c353a4157ecf74386117fb2a2769a9353f491c528371cfe7f6cd", - "x86_64-unknown-linux-gnu": "0e5663025121186bd17d331538a44f48b41baff247891d014f3f962cbe2716b4", - }, - "strip_prefix": "python", - }, - "3.9.19": { - "url": "20240726/cpython-{python_version}+20240726-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "0e5a7aae57c53d7a849bc7f67764a947b626e3fe8d4d41a8eed11d9e4be0b1c6", - "aarch64-unknown-linux-gnu": "05ec896db9a9d4fe8004b4e4b6a6fdc588a015fedbddb475490885b0d9c7d9b3", - "ppc64le-unknown-linux-gnu": "bfff0e3d536b2f0c315e85926cc317b7b756701b6de781a8972cefbdbc991ca2", - "s390x-unknown-linux-gnu": "059ec97080b205ea5f1ddf71c18e22b691e8d68192bd37d13ad8f4359915299d", - "x86_64-apple-darwin": "f2ae9fcac044a329739b8c1676245e8cb6b3094416220e71823d2673bdea0bdb", - "x86_64-pc-windows-msvc": "a8df6a00140055c9accb0be632e7add951d587bbe3d63c40827bbd5145d8f557", - "x86_64-unknown-linux-gnu": "cbf94cb1c9d4b5501d9b3652f6e8400c2cab7c41dfea48d344d9e7f29692b91b", - }, - "strip_prefix": "python", - }, - "3.9.20": { - "url": "20241016/cpython-{python_version}+20241016-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "34ab2bc4c51502145e1a624b4e4ea06877e3d1934a88cc73ac2e0fd5fd439b75", - "aarch64-unknown-linux-gnu": "1e486c054a4e86666cf24e04f5e29456324ba9c2b95bf1cae1805be90d3da154", - "ppc64le-unknown-linux-gnu": "9a24ccdbfc7f67545d859128f02a3150a160ea6c2fc134b0773bf56f2d90b397", - "s390x-unknown-linux-gnu": "2cee381069bf344fb20eba609af92dfe7ba67eb75bea08eeccf11048a2c380c0", - "x86_64-apple-darwin": "193dc7f0284e4917d52b17a077924474882ee172872f2257cfe3375d6d468ed9", - "x86_64-pc-windows-msvc": "5069008a237b90f6f7a86956903f2a0221b90d471daa6e4a94831eaa399e3993", - "x86_64-unknown-linux-gnu": "c20ee831f7f46c58fa57919b75a40eb2b6a31e03fd29aaa4e8dab4b9c4b60d5d", - "x86_64-unknown-linux-musl": "5c1cc348e317fe7af1acd6a7f665b46eccb554b20d6533f0e76c53f44d4556cc", - }, - "strip_prefix": "python", - }, - "3.10.2": { - "url": "20220227/cpython-{python_version}+20220227-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "1409acd9a506e2d1d3b65c1488db4e40d8f19d09a7df099667c87a506f71c0ef", - "aarch64-unknown-linux-gnu": "8f351a8cc348bb45c0f95b8634c8345ec6e749e483384188ad865b7428342703", - "x86_64-apple-darwin": "8146ad4390710ec69b316a5649912df0247d35f4a42e2aa9615bffd87b3e235a", - "x86_64-pc-windows-msvc": "a1d9a594cd3103baa24937ad9150c1a389544b4350e859200b3e5c036ac352bd", - "x86_64-unknown-linux-gnu": "9b64eca2a94f7aff9409ad70bdaa7fbbf8148692662e764401883957943620dd", - }, - "strip_prefix": "python", - }, - "3.10.4": { - "url": "20220502/cpython-{python_version}+20220502-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "2c99983d1e83e4b6e7411ed9334019f193fba626344a50c36fba6c25d4de78a2", - "aarch64-unknown-linux-gnu": "d8098c0c54546637e7516f93b13403b11f9db285def8d7abd825c31407a13d7e", - "x86_64-apple-darwin": "f2711eaffff3477826a401d09a013c6802f11c04c63ab3686aa72664f1216a05", - "x86_64-pc-windows-msvc": "bee24a3a5c83325215521d261d73a5207ab7060ef3481f76f69b4366744eb81d", - "x86_64-unknown-linux-gnu": "f6f871e53a7b1469c13f9bd7920ad98c4589e549acad8e5a1e14760fff3dd5c9", - }, - "strip_prefix": "python", - }, - "3.10.6": { - "url": "20220802/cpython-{python_version}+20220802-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "efaf66acdb9a4eb33d57702607d2e667b1a319d58c167a43c96896b97419b8b7", - "aarch64-unknown-linux-gnu": "81625f5c97f61e2e3d7e9f62c484b1aa5311f21bd6545451714b949a29da5435", - "x86_64-apple-darwin": "7718411adf3ea1480f3f018a643eb0550282aefe39e5ecb3f363a4a566a9398c", - "x86_64-pc-windows-msvc": "91889a7dbdceea585ff4d3b7856a6bb8f8a4eca83a0ff52a73542c2e67220eaa", - "x86_64-unknown-linux-gnu": "55aa2190d28dcfdf414d96dc5dcea9fe048fadcd583dc3981fec020869826111", - }, - "strip_prefix": "python", - }, - "3.10.8": { - "url": "20221106/cpython-{python_version}+20221106-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "d52b03817bd245d28e0a8b2f715716cd0fcd112820ccff745636932c76afa20a", - "aarch64-unknown-linux-gnu": "33170bef18c811906b738be530f934640491b065bf16c4d276c6515321918132", - "x86_64-apple-darwin": "525b79c7ce5de90ab66bd07b0ac1008bafa147ddc8a41bef15ffb7c9c1e9e7c5", - "x86_64-pc-windows-msvc": "f2b6d2f77118f06dd2ca04dae1175e44aaa5077a5ed8ddc63333c15347182bfe", - "x86_64-unknown-linux-gnu": "6c8db44ae0e18e320320bbaaafd2d69cde8bfea171ae2d651b7993d1396260b7", - }, - "strip_prefix": "python", - }, - "3.10.9": { - "url": "20230116/cpython-{python_version}+20230116-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "018d05a779b2de7a476f3b3ff2d10f503d69d14efcedd0774e6dab8c22ef84ff", - "aarch64-unknown-linux-gnu": "2003750f40cd09d4bf7a850342613992f8d9454f03b3c067989911fb37e7a4d1", - "x86_64-apple-darwin": "0e685f98dce0e5bc8da93c7081f4e6c10219792e223e4b5886730fd73a7ba4c6", - "x86_64-pc-windows-msvc": "59c6970cecb357dc1d8554bd0540eb81ee7f6d16a07acf3d14ed294ece02c035", - "x86_64-unknown-linux-gnu": "d196347aeb701a53fe2bb2b095abec38d27d0fa0443f8a1c2023a1bed6e18cdf", - }, - "strip_prefix": "python", - }, - "3.10.11": { - "url": "20230507/cpython-{python_version}+20230507-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "8348bc3c2311f94ec63751fb71bd0108174be1c4def002773cf519ee1506f96f", - "aarch64-unknown-linux-gnu": "c7573fdb00239f86b22ea0e8e926ca881d24fde5e5890851339911d76110bc35", - "ppc64le-unknown-linux-gnu": "73a9d4c89ed51be39dd2de4e235078281087283e9fdedef65bec02f503e906ee", - "x86_64-apple-darwin": "bd3fc6e4da6f4033ebf19d66704e73b0804c22641ddae10bbe347c48f82374ad", - "x86_64-pc-windows-msvc": "9c2d3604a06fcd422289df73015cd00e7271d90de28d2c910f0e2309a7f73a68", - "x86_64-unknown-linux-gnu": "c5bcaac91bc80bfc29cf510669ecad12d506035ecb3ad85ef213416d54aecd79", - }, - "strip_prefix": "python", - }, - "3.10.12": { - "url": "20230726/cpython-{python_version}+20230726-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "bc66c706ea8c5fc891635fda8f9da971a1a901d41342f6798c20ad0b2a25d1d6", - "aarch64-unknown-linux-gnu": "fee80e221663eca5174bd794cb5047e40d3910dbeadcdf1f09d405a4c1c15fe4", - "ppc64le-unknown-linux-gnu": "bb5e8cb0d2e44241725fa9b342238245503e7849917660006b0246a9c97b1d6c", - "s390x-unknown-linux-gnu": "8d33d435ae6fb93ded7fc26798cc0a1a4f546a4e527012a1e2909cc314b332df", - "x86_64-apple-darwin": "8a6e3ed973a671de468d9c691ed9cb2c3a4858c5defffcf0b08969fba9c1dd04", - "x86_64-pc-windows-msvc": "c1a31c353ca44de7d1b1a3b6c55a823e9c1eed0423d4f9f66e617bdb1b608685", - "x86_64-unknown-linux-gnu": "a476dbca9184df9fc69fe6309cda5ebaf031d27ca9e529852437c94ec1bc43d3", - }, - "strip_prefix": "python", - }, - "3.10.13": { - "url": "20240224/cpython-{python_version}+20240224-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "5fdc0f6a5b5a90fd3c528e8b1da8e3aac931ea8690126c2fdb4254c84a3ff04a", - "aarch64-unknown-linux-gnu": "a898a88705611b372297bb8fe4d23cc16b8603ce5f24494c3a8cfa65d83787f9", - "ppc64le-unknown-linux-gnu": "c23706e138a0351fc1e9def2974af7b8206bac7ecbbb98a78f5aa9e7535fee42", - "s390x-unknown-linux-gnu": "09be8fb2cdfbb4a93d555f268f244dbe4d8ff1854b2658e8043aa4ec08aede3e", - "x86_64-apple-darwin": "6378dfd22f58bb553ddb02be28304d739cd730c1f95c15c74955c923a1bc3d6a", - "x86_64-pc-windows-msvc": "086f7fe9156b897bb401273db8359017104168ac36f60f3af4e31ac7acd6634e", - "x86_64-unknown-linux-gnu": "d995d032ca702afd2fc3a689c1f84a6c64972ecd82bba76a61d525f08eb0e195", - }, - "strip_prefix": "python", - }, - "3.10.14": { - "url": "20240726/cpython-{python_version}+20240726-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "164d89f0df2feb689981864ecc1dffb19e6aa3696c8880166de555494fe92607", - "aarch64-unknown-linux-gnu": "39bcd46b4d70e40da177c55259be16d5c2be7a3f7f93f1e3bde47e71b4833f29", - "ppc64le-unknown-linux-gnu": "549d38b9ef59cba9ab2990025255231bfa1cb32b4bc5eac321667640fdee19d1", - "s390x-unknown-linux-gnu": "de4bc878a8666c734f983db971610980870148f333bda8b0c34abfaeae88d7ec", - "x86_64-apple-darwin": "1a1455838cd1e8ed0da14a152a2d559a2fd3a6047ba7013e841db4a35a228c1d", - "x86_64-pc-windows-msvc": "7f68821a8b5445267eca480660364ebd06ec84632b336770c6e39de07ac0f6c3", - "x86_64-unknown-linux-gnu": "32b34cd13d9d745b3db3f3b8398ab2c07de74544829915dbebd8dce39bdc405e", - }, - "strip_prefix": "python", - }, - "3.10.15": { - "url": "20241016/cpython-{python_version}+20241016-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "f64776f455a44c24d50f947c813738cfb7b9ac43732c44891bc831fa7940a33c", - "aarch64-unknown-linux-gnu": "eb58581f85fde83d1f3e8e1f8c6f5a15c7ae4fdbe3b1d1083931f9167fdd8dbc", - "ppc64le-unknown-linux-gnu": "0c45af4e7525e2db59901606db32b2896ac1e9830c6f95551402207f537c2ce4", - "s390x-unknown-linux-gnu": "de205896b070e6f5259ac0f2b3379eead875ea84e6a6ef533b89886fcbb46a4c", - "x86_64-apple-darwin": "90b46dfb1abd98d45663c7a2a8c45d3047a59391d8586d71b459cec7b75f662b", - "x86_64-pc-windows-msvc": "e48952619796c66ec9719867b87be97edca791c2ef7fbf87d42c417c3331609e", - "x86_64-unknown-linux-gnu": "3db2171e03c1a7acdc599fba583c1b92306d3788b375c9323077367af1e9d9de", - "x86_64-unknown-linux-musl": "ed519c47d9620eb916a6f95ec2875396e7b1a9ab993ee40b2f31b837733f318c", - }, - "strip_prefix": "python", - }, - "3.11.1": { - "url": "20230116/cpython-{python_version}+20230116-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "4918cdf1cab742a90f85318f88b8122aeaa2d04705803c7b6e78e81a3dd40f80", - "aarch64-unknown-linux-gnu": "debf15783bdcb5530504f533d33fda75a7b905cec5361ae8f33da5ba6599f8b4", - "x86_64-apple-darwin": "20a4203d069dc9b710f70b09e7da2ce6f473d6b1110f9535fb6f4c469ed54733", - "x86_64-pc-windows-msvc": "edc08979cb0666a597466176511529c049a6f0bba8adf70df441708f766de5bf", - "x86_64-unknown-linux-gnu": "02a551fefab3750effd0e156c25446547c238688a32fabde2995c941c03a6423", - }, - "strip_prefix": "python", - }, - "3.11.3": { - "url": "20230507/cpython-{python_version}+20230507-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "09e412506a8d63edbb6901742b54da9aa7faf120b8dbdce56c57b303fc892c86", - "aarch64-unknown-linux-gnu": "8190accbbbbcf7620f1ff6d668e4dd090c639665d11188ce864b62554d40e5ab", - "ppc64le-unknown-linux-gnu": "767d24f3570b35fedb945f5ac66224c8983f2d556ab83c5cfaa5f3666e9c212c", - "x86_64-apple-darwin": "f710b8d60621308149c100d5175fec39274ed0b9c99645484fd93d1716ef4310", - "x86_64-pc-windows-msvc": "24741066da6f35a7ff67bee65ce82eae870d84e1181843e64a7076d1571e95af", - "x86_64-unknown-linux-gnu": "da50b87d1ec42b3cb577dfd22a3655e43a53150f4f98a4bfb40757c9d7839ab5", - }, - "strip_prefix": "python", - }, - "3.11.4": { - "url": "20230726/cpython-{python_version}+20230726-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "cb6d2948384a857321f2aa40fa67744cd9676a330f08b6dad7070bda0b6120a4", - "aarch64-unknown-linux-gnu": "2e84fc53f4e90e11963281c5c871f593abcb24fc796a50337fa516be99af02fb", - "ppc64le-unknown-linux-gnu": "df7b92ed9cec96b3bb658fb586be947722ecd8e420fb23cee13d2e90abcfcf25", - "s390x-unknown-linux-gnu": "e477f0749161f9aa7887964f089d9460a539f6b4a8fdab5166f898210e1a87a4", - "x86_64-apple-darwin": "47e1557d93a42585972772e82661047ca5f608293158acb2778dccf120eabb00", - "x86_64-pc-windows-msvc": "878614c03ea38538ae2f758e36c85d2c0eb1eaaca86cd400ff8c76693ee0b3e1", - "x86_64-unknown-linux-gnu": "e26247302bc8e9083a43ce9e8dd94905b40d464745b1603041f7bc9a93c65d05", - }, - "strip_prefix": "python", - }, - "3.11.5": { - "url": "20230826/cpython-{python_version}+20230826-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "dab64b3580118ad2073babd7c29fd2053b616479df5c107d31fe2af1f45e948b", - "aarch64-unknown-linux-gnu": "bb5c5d1ea0f199fe2d3f0996fff4b48ca6ddc415a3dbd98f50bff7fce48aac80", - "ppc64le-unknown-linux-gnu": "14121b53e9c8c6d0741f911ae00102a35adbcf5c3cdf732687ef7617b7d7304d", - "s390x-unknown-linux-gnu": "fe459da39874443579d6fe88c68777c6d3e331038e1fb92a0451879fb6beb16d", - "x86_64-apple-darwin": "4a4efa7378c72f1dd8ebcce1afb99b24c01b07023aa6b8fea50eaedb50bf2bfc", - "x86_64-pc-windows-msvc": "00f002263efc8aea896bcfaaf906b1f4dab3e5cd3db53e2b69ab9a10ba220b97", - "x86_64-unknown-linux-gnu": "fbed6f7694b2faae5d7c401a856219c945397f772eea5ca50c6eb825cbc9d1e1", - }, - "strip_prefix": "python", - }, - "3.11.6": { - "url": "20231002/cpython-{python_version}+20231002-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "916c35125b5d8323a21526d7a9154ca626453f63d0878e95b9f613a95006c990", - "aarch64-unknown-linux-gnu": "3e26a672df17708c4dc928475a5974c3fb3a34a9b45c65fb4bd1e50504cc84ec", - "ppc64le-unknown-linux-gnu": "7937035f690a624dba4d014ffd20c342e843dd46f89b0b0a1e5726b85deb8eaf", - "s390x-unknown-linux-gnu": "f9f19823dba3209cedc4647b00f46ed0177242917db20fb7fb539970e384531c", - "x86_64-apple-darwin": "178cb1716c2abc25cb56ae915096c1a083e60abeba57af001996e8bc6ce1a371", - "x86_64-pc-windows-msvc": "3933545e6d41462dd6a47e44133ea40995bc6efeed8c2e4cbdf1a699303e95ea", - "x86_64-unknown-linux-gnu": "ee37a7eae6e80148c7e3abc56e48a397c1664f044920463ad0df0fc706eacea8", - }, - "strip_prefix": "python", - }, - "3.11.7": { - "url": "20240107/cpython-{python_version}+20240107-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "b042c966920cf8465385ca3522986b12d745151a72c060991088977ca36d3883", - "aarch64-unknown-linux-gnu": "b102eaf865eb715aa98a8a2ef19037b6cc3ae7dfd4a632802650f29de635aa13", - "ppc64le-unknown-linux-gnu": "b44e1b74afe75c7b19143413632c4386708ae229117f8f950c2094e9681d34c7", - "s390x-unknown-linux-gnu": "49520e3ff494708020f306e30b0964f079170be83e956be4504f850557378a22", - "x86_64-apple-darwin": "a0e615eef1fafdc742da0008425a9030b7ea68a4ae4e73ac557ef27b112836d4", - "x86_64-pc-windows-msvc": "67077e6fa918e4f4fd60ba169820b00be7c390c497bf9bc9cab2c255ea8e6f3e", - "x86_64-unknown-linux-gnu": "4a51ce60007a6facf64e5495f4cf322e311ba9f39a8cd3f3e4c026eae488e140", - }, - "strip_prefix": "python", - }, - "3.11.8": { - "url": "20240224/cpython-{python_version}+20240224-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "389a51139f5abe071a0d70091ca5df3e7a3dfcfcbe3e0ba6ad85fb4c5638421e", - "aarch64-unknown-linux-gnu": "389b9005fb78dd5a6f68df5ea45ab7b30d9a4b3222af96999e94fd20d4ad0c6a", - "ppc64le-unknown-linux-gnu": "eb2b31f8e50309aae493c6a359c32b723a676f07c641f5e8fe4b6aa4dbb50946", - "s390x-unknown-linux-gnu": "844f64f4c16e24965778281da61d1e0e6cd1358a581df1662da814b1eed096b9", - "x86_64-apple-darwin": "097f467b0c36706bfec13f199a2eaf924e668f70c6e2bd1f1366806962f7e86e", - "x86_64-pc-windows-msvc": "b618f1f047349770ee1ef11d1b05899840abd53884b820fd25c7dfe2ec1664d4", - "x86_64-unknown-linux-gnu": "94e13d0e5ad417035b80580f3e893a72e094b0900d5d64e7e34ab08e95439987", - }, - "strip_prefix": "python", - }, - "3.11.9": { - "url": "20240726/cpython-{python_version}+20240726-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "cbdac9462bab9671c8e84650e425d3f43b775752a930a2ef954a0d457d5c00c3", - "aarch64-unknown-linux-gnu": "4d17cf988abe24449d649aad3ef974091ab76807904d41839907061925b4c9e3", - "ppc64le-unknown-linux-gnu": "fc4f3c9ef9bfac2ed0282126ff376e544697ad04a5408d6429d46899d7d3bf21", - "s390x-unknown-linux-gnu": "e69b66e53e926460df044f44846eef3fea642f630e829719e1a4112fc370dc56", - "x86_64-apple-darwin": "dc3174666a30f4c38d04e79a80c3159b4b3aa69597c4676701c8386696811611", - "x86_64-pc-windows-msvc": "f694be48bdfec1dace6d69a19906b6083f4dd7c7c61f1138ba520e433e5598f8", - "x86_64-unknown-linux-gnu": "f6e955dc9ddfcad74e77abe6f439dac48ebca14b101ed7c85a5bf3206ed2c53d", - }, - "strip_prefix": "python", - }, - "3.11.10": { - "url": "20241016/cpython-{python_version}+20241016-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "5a69382da99c4620690643517ca1f1f53772331b347e75f536088c42a4cf6620", - "aarch64-unknown-linux-gnu": "803e49259280af0f5466d32829cd9d65a302b0226e424b3f0b261f9daf6aee8f", - "ppc64le-unknown-linux-gnu": "92b666d103902001322f42badbd68da92adc5cebb826af9c1c906c33166e2f34", - "s390x-unknown-linux-gnu": "6d584317651c1ad4a857cb32d1999707e8bb3046fcb2f156d80381814fa19fde", - "x86_64-apple-darwin": "1e23ffe5bc473e1323ab8f51464da62d77399afb423babf67f8e13c82b69c674", - "x86_64-pc-windows-msvc": "647b66ff4552e70aec3bf634dd470891b4a2b291e8e8715b3bdb162f577d4c55", - "x86_64-unknown-linux-gnu": "8b50a442b04724a24c1eebb65a36a0c0e833d35374dbdf9c9470d8a97b164cd9", - "x86_64-unknown-linux-musl": "d36fc77a8dd76155a7530f6235999a693b9e7c48aa11afeb5610a091cae5aa6f", - }, - "strip_prefix": "python", - }, - "3.12.0": { - "url": "20231002/cpython-{python_version}+20231002-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "4734a2be2becb813830112c780c9879ac3aff111a0b0cd590e65ec7465774d02", - "aarch64-unknown-linux-gnu": "bccfe67cf5465a3dfb0336f053966e2613a9bc85a6588c2fcf1366ef930c4f88", - "ppc64le-unknown-linux-gnu": "b5dae075467ace32c594c7877fe6ebe0837681f814601d5d90ba4c0dfd87a1f2", - "s390x-unknown-linux-gnu": "5681621349dd85d9726d1b67c84a9686ce78f72e73a6f9e4cc4119911655759e", - "x86_64-apple-darwin": "5a9e88c8aa52b609d556777b52ebde464ae4b4f77e4aac4eb693af57395c9abf", - "x86_64-pc-windows-msvc": "facfaa1fbc8653f95057f3c4a0f8aa833dab0e0b316e24ee8686bc761d4b4f8d", - "x86_64-unknown-linux-gnu": "e51a5293f214053ddb4645b2c9f84542e2ef86870b8655704367bd4b29d39fe9", - }, - "strip_prefix": "python", - }, - "3.12.1": { - "url": "20240107/cpython-{python_version}+20240107-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "f93f8375ca6ac0a35d58ff007043cbd3a88d9609113f1cb59cf7c8d215f064af", - "aarch64-unknown-linux-gnu": "236533ef20e665007a111c2f36efb59c87ae195ad7dca223b6dc03fb07064f0b", - "ppc64le-unknown-linux-gnu": "78051f0d1411ee62bc2af5edfccf6e8400ac4ef82887a2affc19a7ace6a05267", - "s390x-unknown-linux-gnu": "60631211c701f8d2c56e5dd7b154e68868128a019b9db1d53a264f56c0d4aee2", - "x86_64-apple-darwin": "eca96158c1568dedd9a0b3425375637a83764d1fa74446438293089a8bfac1f8", - "x86_64-pc-windows-msvc": "fd5a9e0f41959d0341246d3643f2b8794f638adc0cec8dd5e1b6465198eae08a", - "x86_64-unknown-linux-gnu": "74e330b8212ca22fd4d9a2003b9eec14892155566738febc8e5e572f267b9472", - }, - "strip_prefix": "python", - }, - "3.12.2": { - "url": "20240224/cpython-{python_version}+20240224-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "01c064c00013b0175c7858b159989819ead53f4746d40580b5b0b35b6e80fba6", - "aarch64-unknown-linux-gnu": "e52550379e7c4ac27a87de832d172658bc04150e4e27d4e858e6d8cbb96fd709", - "ppc64le-unknown-linux-gnu": "74bc02c4bbbd26245c37b29b9e12d0a9c1b7ab93477fed8b651c988b6a9a6251", - "s390x-unknown-linux-gnu": "ecd6b0285e5eef94deb784b588b4b425a15a43ae671bf206556659dc141a9825", - "x86_64-apple-darwin": "a53a6670a202c96fec0b8c55ccc780ea3af5307eb89268d5b41a9775b109c094", - "x86_64-pc-windows-msvc": "1e5655a6ccb1a64a78460e4e3ee21036c70246800f176a6c91043a3fe3654a3b", - "x86_64-unknown-linux-gnu": "57a37b57f8243caa4cdac016176189573ad7620f0b6da5941c5e40660f9468ab", - }, - "strip_prefix": "python", - }, - "3.12.3": { - "url": "20240415/cpython-{python_version}+20240415-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "ccc40e5af329ef2af81350db2a88bbd6c17b56676e82d62048c15d548401519e", - "aarch64-unknown-linux-gnu": "ec8126de97945e629cca9aedc80a29c4ae2992c9d69f2655e27ae73906ba187d", - "ppc64le-unknown-linux-gnu": "c5dcf08b8077e617d949bda23027c49712f583120b3ed744f9b143da1d580572", - "s390x-unknown-linux-gnu": "872fc321363b8cdd826fd2cb1adfd1ceb813bc1281f9d410c1c2c4e177e8df86", - "x86_64-apple-darwin": "c37a22fca8f57d4471e3708de6d13097668c5f160067f264bb2b18f524c890c8", - "x86_64-pc-windows-msvc": "f7cfa4ad072feb4578c8afca5ba9a54ad591d665a441dd0d63aa366edbe19279", - "x86_64-unknown-linux-gnu": "a73ba777b5d55ca89edef709e6b8521e3f3d4289581f174c8699adfb608d09d6", - }, - "strip_prefix": "python", - }, - "3.12.4": { - "url": "20240726/cpython-{python_version}+20240726-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "1801025e825c04b3907e4ef6220a13607bc0397628c9485897073110ef7fde15", - "aarch64-unknown-linux-gnu": "a098b18b7e9fea0c66867b76c0124fce9465765017572b2e7b522154c87c78d7", - "ppc64le-unknown-linux-gnu": "04011c4c5b7fe34b0b895edf4ad8748e410686c1d69aaee11d6688d481023bcb", - "s390x-unknown-linux-gnu": "8f8f3e29cf0c2facdbcfee70660939fda7667ac24fee8656d3388fc72f3acc7c", - "x86_64-apple-darwin": "4c325838c1b0ed13698506fcd515be25c73dcbe195f8522cf98f9148a97601ed", - "x86_64-pc-windows-msvc": "74309b0f322716409883d38c621743ea7fa0376eb00927b8ee1e1671d3aff450", - "x86_64-unknown-linux-gnu": "e133dd6fc6a2d0033e2658637cc22e9c95f9d7073b80115037ee1f16417a54ac", - }, - "strip_prefix": "python", - }, - "3.12.7": { - "url": "20241016/cpython-{python_version}+20241016-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "4c18852bf9c1a11b56f21bcf0df1946f7e98ee43e9e4c0c5374b2b3765cf9508", - "aarch64-unknown-linux-gnu": "bba3c6be6153f715f2941da34f3a6a69c2d0035c9c5396bc5bb68c6d2bd1065a", - "ppc64le-unknown-linux-gnu": "0a1d1d92e33a969bd2f40a80af53c97b6c0cc1060d384ceff50ff801593bf9d6", - "s390x-unknown-linux-gnu": "935676a0c960b552f95e9ac2e1e385de5de4b34038ff65ffdc688838f1189c17", - "x86_64-apple-darwin": "60c5271e7edc3c2ab47440b7abf4ed50fbc693880b474f74f05768f5b657045a", - "x86_64-pc-windows-msvc": "f05531bff16fa77b53be0776587b97b466070e768e6d5920894de988bdcd547a", - "x86_64-unknown-linux-gnu": "43576f7db1033dd57b900307f09c2e86f371152ac8a2607133afa51cbfc36064", - "x86_64-unknown-linux-musl": "5ed4a4078db3cbac563af66403aaa156cd6e48831d90382a1820db2b120627b5", - }, - "strip_prefix": "python", - }, - "3.13.0": { - "url": "20241016/cpython-{python_version}+20241016-{platform}-{build}.{ext}", - "sha256": { - "aarch64-apple-darwin": "31397953849d275aa2506580f3fa1cb5a85b6a3d392e495f8030e8b6412f5556", - "aarch64-unknown-linux-gnu": "e8378c0162b2e0e4cc1f62b29443a3305d116d09583304dbb0149fecaff6347b", - "ppc64le-unknown-linux-gnu": "fc4b7f27c4e84c78f3c8e6c7f8e4023e4638d11f1b36b6b5ce457b1926cebb53", - "s390x-unknown-linux-gnu": "66b19e6a07717f6cfcd3a8ca953f0a2eaa232291142f3d26a8d17c979ec0f467", - "x86_64-apple-darwin": "cff1b7e7cd26f2d47acac1ad6590e27d29829776f77e8afa067e9419f2f6ce77", - "x86_64-pc-windows-msvc": "b25926e8ce4164cf103bacc4f4d154894ea53e07dd3fdd5ebb16fb1a82a7b1a0", - "x86_64-unknown-linux-gnu": "2c8cb15c6a2caadaa98af51df6fe78a8155b8471cb3dd7b9836038e0d3657fb4", - "x86_64-unknown-linux-musl": "2f61ee3b628a56aceea63b46c7afe2df3e22a61da706606b0c8efda57f953cf4", - "aarch64-apple-darwin-freethreaded": "efc2e71c0e05bc5bedb7a846e05f28dd26491b1744ded35ed82f8b49ccfa684b", - "aarch64-unknown-linux-gnu-freethreaded": "59b50df9826475d24bb7eff781fa3949112b5e9c92adb29e96a09cdf1216d5bd", - "ppc64le-unknown-linux-gnu-freethreaded": "1217efa5f4ce67fcc9f7eb64165b1bd0912b2a21bc25c1a7e2cb174a21a5df7e", - "s390x-unknown-linux-gnu-freethreaded": "6c3e1e4f19d2b018b65a7e3ef4cd4225c5b9adfbc490218628466e636d5c4b8c", - "x86_64-apple-darwin-freethreaded": "2e07dfea62fe2215738551a179c87dbed1cc79d1b3654f4d7559889a6d5ce4eb", - "x86_64-pc-windows-msvc-freethreaded": "bfd89f9acf866463bc4baf01733da5e767d13f5d0112175a4f57ba91f1541310", - "x86_64-unknown-linux-gnu-freethreaded": "a73adeda301ad843cce05f31a2d3e76222b656984535a7b87696a24a098b216c", - }, - "strip_prefix": { - "aarch64-apple-darwin": "python", - "aarch64-unknown-linux-gnu": "python", - "ppc64le-unknown-linux-gnu": "python", - "s390x-unknown-linux-gnu": "python", - "x86_64-apple-darwin": "python", - "x86_64-pc-windows-msvc": "python", - "x86_64-unknown-linux-gnu": "python", - "x86_64-unknown-linux-musl": "python", - "aarch64-apple-darwin-freethreaded": "python/install", - "aarch64-unknown-linux-gnu-freethreaded": "python/install", - "ppc64le-unknown-linux-gnu-freethreaded": "python/install", - "s390x-unknown-linux-gnu-freethreaded": "python/install", - "x86_64-apple-darwin-freethreaded": "python/install", - "x86_64-pc-windows-msvc-freethreaded": "python/install", - "x86_64-unknown-linux-gnu-freethreaded": "python/install", - }, - }, -} +DEFAULT_RELEASE_BASE_URL = _GITHUB_PREFIX +DEFAULT_RELEASE_BASE_URLS = [ + _GITHUB_PREFIX, + _ASTRAL_PREFIX, + _LEGACY_GITHUB_PREFIX, +] # buildifier: disable=unsorted-dict-items MINOR_MAPPING = { - "3.8": "3.8.20", - "3.9": "3.9.20", - "3.10": "3.10.15", - "3.11": "3.11.10", - "3.12": "3.12.7", - "3.13": "3.13.0", + "3.9": "3.9.25", + "3.10": "3.10.20", + "3.11": "3.11.15", + "3.12": "3.12.13", + "3.13": "3.13.13", + "3.14": "3.14.4", + "3.15": "3.15.0a8", } def _generate_platforms(): - libc = Label("//python/config_settings:py_linux_libc") + is_libc_glibc = str(Label("//python/config_settings:_is_py_linux_libc_glibc")) + is_libc_musl = str(Label("//python/config_settings:_is_py_linux_libc_musl")) platforms = { - "aarch64-apple-darwin": struct( + "aarch64-apple-darwin": platform_info( compatible_with = [ "@platforms//os:macos", "@platforms//cpu:aarch64", ], - flag_values = {}, os_name = MACOS_NAME, - # Matches the value in @platforms//cpu package arch = "aarch64", ), - "aarch64-unknown-linux-gnu": struct( + "aarch64-pc-windows-msvc": platform_info( + compatible_with = [ + "@platforms//os:windows", + "@platforms//cpu:aarch64", + ], + os_name = WINDOWS_NAME, + arch = "aarch64", + ), + "aarch64-unknown-linux-gnu": platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:aarch64", ], - flag_values = { - libc: "glibc", - }, + target_settings = [ + is_libc_glibc, + ], os_name = LINUX_NAME, - # Matches the value in @platforms//cpu package arch = "aarch64", ), - "armv7-unknown-linux-gnu": struct( + "arm64e-apple-darwin": platform_info( + compatible_with = [ + "@platforms//os:macos", + "@platforms//cpu:arm64e", + ], + os_name = MACOS_NAME, + arch = "aarch64", + ), + "armv7-unknown-linux-gnu": platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:armv7", ], - flag_values = { - libc: "glibc", - }, + target_settings = [ + is_libc_glibc, + ], os_name = LINUX_NAME, - # Matches the value in @platforms//cpu package arch = "arm", ), - "i386-unknown-linux-gnu": struct( + "i386-unknown-linux-gnu": platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:i386", ], - flag_values = { - libc: "glibc", - }, + target_settings = [ + is_libc_glibc, + ], os_name = LINUX_NAME, - # Matches the value in @platforms//cpu package arch = "x86_32", ), - "ppc64le-unknown-linux-gnu": struct( + "ppc64le-unknown-linux-gnu": platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:ppc", ], - flag_values = { - libc: "glibc", - }, + target_settings = [ + is_libc_glibc, + ], os_name = LINUX_NAME, - # Matches the value in @platforms//cpu package arch = "ppc", ), - "riscv64-unknown-linux-gnu": struct( + "riscv64-unknown-linux-gnu": platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:riscv64", ], - flag_values = { - Label("//python/config_settings:py_linux_libc"): "glibc", - }, + target_settings = [ + is_libc_glibc, + ], os_name = LINUX_NAME, - # Matches the value in @platforms//cpu package arch = "riscv64", ), - "s390x-unknown-linux-gnu": struct( + "s390x-unknown-linux-gnu": platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:s390x", ], - flag_values = { - Label("//python/config_settings:py_linux_libc"): "glibc", - }, + target_settings = [ + is_libc_glibc, + ], os_name = LINUX_NAME, - # Matches the value in @platforms//cpu package arch = "s390x", ), - "x86_64-apple-darwin": struct( + "x86_64-apple-darwin": platform_info( compatible_with = [ "@platforms//os:macos", "@platforms//cpu:x86_64", ], - flag_values = {}, os_name = MACOS_NAME, - # Matches the value in @platforms//cpu package arch = "x86_64", ), - "x86_64-pc-windows-msvc": struct( + "x86_64-pc-windows-msvc": platform_info( compatible_with = [ "@platforms//os:windows", "@platforms//cpu:x86_64", ], - flag_values = {}, os_name = WINDOWS_NAME, - # Matches the value in @platforms//cpu package arch = "x86_64", ), - "x86_64-unknown-linux-gnu": struct( + "x86_64-unknown-linux-gnu": platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:x86_64", ], - flag_values = { - libc: "glibc", - }, + target_settings = [ + is_libc_glibc, + ], os_name = LINUX_NAME, - # Matches the value in @platforms//cpu package arch = "x86_64", ), - "x86_64-unknown-linux-musl": struct( + "x86_64-unknown-linux-musl": platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:x86_64", ], - flag_values = { - libc: "musl", - }, + target_settings = [ + is_libc_musl, + ], os_name = LINUX_NAME, arch = "x86_64", ), } - freethreaded = Label("//python/config_settings:py_freethreaded") + is_freethreaded_yes = str(Label("//python/config_settings:_is_py_freethreaded_yes")) + is_freethreaded_no = str(Label("//python/config_settings:_is_py_freethreaded_no")) return { - p + suffix: struct( + p + suffix: platform_info( compatible_with = v.compatible_with, - flag_values = { - freethreaded: freethreaded_value, - } | v.flag_values, + target_settings = [ + freethreadedness, + ] + v.target_settings, os_name = v.os_name, arch = v.arch, ) for p, v in platforms.items() - for suffix, freethreaded_value in { - "": "no", - "-" + FREETHREADED: "yes", + for suffix, freethreadedness in { + "": is_freethreaded_no, + FREETHREADED: is_freethreaded_yes, }.items() } PLATFORMS = _generate_platforms() -def get_release_info(platform, python_version, base_url = DEFAULT_RELEASE_BASE_URL, tool_versions = TOOL_VERSIONS): +def get_release_info(platform, python_version, base_urls = DEFAULT_RELEASE_BASE_URLS, tool_versions = None): """Resolve the release URL for the requested interpreter version Args: platform: The platform string for the interpreter python_version: The version of the interpreter to get - base_url: The URL to prepend to the 'url' attr in the tool_versions dict + base_urls: The list of URLs to prepend to the 'url' attr in the tool_versions dict tool_versions: A dict listing the interpreter versions, their SHAs and URL Returns: A tuple of (filename, url, archive strip prefix, patches, patch_strip) """ + if tool_versions == None: + tool_versions = TOOL_VERSIONS + + if type(base_urls) == type(""): + base_urls = [base_urls] + elif base_urls == None: + base_urls = DEFAULT_RELEASE_BASE_URLS + + expanded_base_urls = [] + for b_url in base_urls: + if b_url not in expanded_base_urls: + expanded_base_urls.append(b_url) + if b_url.startswith(_GITHUB_PREFIX): + suffix = b_url[len(_GITHUB_PREFIX):] + a_url = _ASTRAL_PREFIX + suffix + if a_url not in expanded_base_urls: + expanded_base_urls.append(a_url) + elif b_url.startswith(_LEGACY_GITHUB_PREFIX): + suffix = b_url[len(_LEGACY_GITHUB_PREFIX):] + a_url = _ASTRAL_PREFIX + suffix + if a_url not in expanded_base_urls: + expanded_base_urls.append(a_url) + base_urls = expanded_base_urls url = tool_versions[python_version]["url"] @@ -793,25 +258,35 @@ def get_release_info(platform, python_version, base_url = DEFAULT_RELEASE_BASE_U release_filename = None rendered_urls = [] for u in url: - p, _, _ = platform.partition("-" + FREETHREADED) + p, _, _ = platform.partition(FREETHREADED) - if FREETHREADED in platform: + # Assume an unknown release_id is a newer url format + release_id = 99999999 + url_parts = u.split("/") + if len(url_parts) >= 2 and url_parts[-2].isdigit(): + maybe_release_id = url_parts[-2] + release_id = int(maybe_release_id) + + if FREETHREADED.lstrip("-") in platform and release_id < 20260325: build = "{}+{}-full".format( - FREETHREADED, + FREETHREADED.lstrip("-"), { "aarch64-apple-darwin": "pgo+lto", - "aarch64-unknown-linux-gnu": "lto", + "aarch64-pc-windows-msvc": "pgo", + "aarch64-unknown-linux-gnu": "lto" if release_id < 20250702 else "pgo+lto", "ppc64le-unknown-linux-gnu": "lto", + "riscv64-unknown-linux-gnu": "lto", "s390x-unknown-linux-gnu": "lto", "x86_64-apple-darwin": "pgo+lto", "x86_64-pc-windows-msvc": "pgo", "x86_64-unknown-linux-gnu": "pgo+lto", + "x86_64-unknown-linux-musl": "pgo+lto", }[p], ) else: build = INSTALL_ONLY - if WINDOWS_NAME in platform: + if WINDOWS_NAME in platform and release_id < 20250317: build = "shared-" + build release_filename = u.format( @@ -823,7 +298,8 @@ def get_release_info(platform, python_version, base_url = DEFAULT_RELEASE_BASE_U if "://" in release_filename: # is absolute url? rendered_urls.append(release_filename) else: - rendered_urls.append("/".join([base_url, release_filename])) + for b_url in base_urls: + rendered_urls.append("/".join([b_url, release_filename])) if release_filename == None: fail("release_filename should be set by now; were any download URLs given?") @@ -843,61 +319,74 @@ def get_release_info(platform, python_version, base_url = DEFAULT_RELEASE_BASE_U return (release_filename, rendered_urls, strip_prefix, patches, patch_strip) -def print_toolchains_checksums(name): - """A macro to print checksums for a particular Python interpreter version. +def gen_python_config_settings(name = ""): + for platform in PLATFORMS.keys(): + native.config_setting( + name = "{name}{platform}".format(name = name, platform = platform), + flag_values = PLATFORMS[platform].flag_values, + constraint_values = PLATFORMS[platform].compatible_with, + ) + +def _manifest_entry_sort_key(entry): + flavor_rank = {"full": 3, "install_only": 1, "install_only_stripped": 2}.get(entry.archive_flavor, 4) + microarch = entry.microarch + if not microarch: + microarch_rank = 0 + elif microarch.startswith("v") and microarch[1:].isdigit(): + microarch_rank = int(microarch[1:]) + else: + microarch_rank = 999 + return (flavor_rank, microarch_rank) + +def _tool_versions_from_manifest_entries(entries, base_url = DEFAULT_RELEASE_BASE_URL): + """Converts parsed manifest entries into the TOOL_VERSIONS dictionary format. Args: - name: {type}`str`: the name of the runnable target. + entries: {type}`list[struct]` the parsed manifest entries. + base_url: {type}`str` fallback base URL for standalone distributions. + + Returns: + {type}`dict[str, dict]` the tool versions map. """ - all_commands = [] - by_version = {} - for python_version in TOOL_VERSIONS.keys(): - by_version[python_version] = _commands_for_version(python_version) - all_commands.append(_commands_for_version(python_version)) + available_versions = {} + entries = sorted( + entries, + key = _manifest_entry_sort_key, + ) - template = """\ -cat > "$@" <<'EOF' -#!/bin/bash + for entry in entries: + location = entry.location + sha256 = entry.sha256 + py_version = entry.python_version -set -o errexit -o nounset -o pipefail + matched_platform = "{}-{}-{}".format(entry.arch, entry.vendor, entry.os) + if entry.libc: + matched_platform += "-" + entry.libc + if entry.freethreaded: + matched_platform += "-freethreaded" -echo "Fetching hashes..." + if matched_platform not in PLATFORMS: + continue -{commands} -EOF - """ + archive_flavor = entry.archive_flavor + if archive_flavor not in ["install_only", "install_only_stripped", "full"]: + continue - native.genrule( - name = name, - srcs = [], - outs = ["print_toolchains_checksums.sh"], - cmd = select({ - "//python/config_settings:is_python_{}".format(version): template.format( - commands = commands, - ) - for version, commands in by_version.items() - } | { - "//conditions:default": template.format(commands = "\n".join(all_commands)), - }), - executable = True, - ) + v_dict = available_versions.setdefault(py_version, {}) + if matched_platform in v_dict.get("sha256", {}): + continue -def _commands_for_version(python_version): - return "\n".join([ - "echo \"{python_version}: {platform}: $$(curl --location --fail {release_url_sha256} 2>/dev/null || curl --location --fail {release_url} 2>/dev/null | shasum -a 256 | awk '{{ print $$1 }}')\"".format( - python_version = python_version, - platform = platform, - release_url = release_url, - release_url_sha256 = release_url + ".sha256", - ) - for platform in TOOL_VERSIONS[python_version]["sha256"].keys() - for release_url in get_release_info(platform, python_version)[1] - ]) + if "://" in location: + urls = [location] + else: + urls = ["{}/{}".format(base_url, location)] -def gen_python_config_settings(name = ""): - for platform in PLATFORMS.keys(): - native.config_setting( - name = "{name}{platform}".format(name = name, platform = platform), - flag_values = PLATFORMS[platform].flag_values, - constraint_values = PLATFORMS[platform].compatible_with, - ) + strip_prefix = "python/install" if archive_flavor == "full" else "python" + + v_dict.setdefault("sha256", {})[matched_platform] = sha256 + v_dict.setdefault("url", {})[matched_platform] = urls + v_dict.setdefault("strip_prefix", {})[matched_platform] = strip_prefix + + return available_versions + +TOOL_VERSIONS = _tool_versions_from_manifest_entries(parse_runtime_manifest(MANIFEST_TEXT)) diff --git a/python/zipapp/BUILD.bazel b/python/zipapp/BUILD.bazel new file mode 100644 index 0000000000..77a8752701 --- /dev/null +++ b/python/zipapp/BUILD.bazel @@ -0,0 +1,30 @@ +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") + +package(default_visibility = ["//visibility:public"]) + +licenses(["notice"]) + +filegroup( + name = "distribution", + srcs = glob(["**"]), + visibility = ["//python:__pkg__"], +) + +bzl_library( + name = "py_zipapp_binary", + srcs = ["py_zipapp_binary.bzl"], + deps = [ + "//python/private:util", + "//python/private/zipapp:py_zipapp_rule", + ], +) + +# keep +bzl_library( + name = "py_zipapp_test", + srcs = ["py_zipapp_test.bzl"], + deps = [ + "//python/private:util", + "//python/private/zipapp:py_zipapp_rule", + ], +) diff --git a/python/zipapp/py_zipapp_binary.bzl b/python/zipapp/py_zipapp_binary.bzl new file mode 100644 index 0000000000..0b9c9bf95c --- /dev/null +++ b/python/zipapp/py_zipapp_binary.bzl @@ -0,0 +1,24 @@ +"""`py_zipapp_binary` macro. + +:::{seealso} + +{obj}`features.zipapp_rules` to detect if this rule is available. +::: +""" + +load("//python/private:util.bzl", "add_tag") +load("//python/private/zipapp:py_zipapp_rule.bzl", _py_zipapp_binary_rule = "py_zipapp_binary") + +def py_zipapp_binary(**kwargs): + """Builds a Python zipapp from a py_binary/py_test target. + + :::{versionadded} 1.9.0 + ::: + + Args: + **kwargs: Args passed onto {rule}`py_zipapp_binary`. + """ + add_tag(kwargs, "@rules_python//python:py_zipapp_binary") + _py_zipapp_binary_rule( + **kwargs + ) diff --git a/python/zipapp/py_zipapp_test.bzl b/python/zipapp/py_zipapp_test.bzl new file mode 100644 index 0000000000..261e584755 --- /dev/null +++ b/python/zipapp/py_zipapp_test.bzl @@ -0,0 +1,24 @@ +"""`py_zipapp_test` macro. + +:::{seealso} + +{obj}`features.zipapp_rules` to detect if this rule is available. +::: +""" + +load("//python/private:util.bzl", "add_tag") +load("//python/private/zipapp:py_zipapp_rule.bzl", _py_zipapp_test = "py_zipapp_test") + +def py_zipapp_test(**kwargs): + """Builds a Python zipapp from a py_binary/py_test target. + + :::{versionadded} 1.9.0 + ::: + + Args: + **kwargs: Args passed onto {rule}`py_zipapp_test`. + """ + add_tag(kwargs, "@rules_python//python:py_zipapp_test") + _py_zipapp_test( + **kwargs + ) diff --git a/replicate_ci b/replicate_ci new file mode 100755 index 0000000000..dd9b67e93e --- /dev/null +++ b/replicate_ci @@ -0,0 +1,167 @@ +#!/usr/bin/env python3 + +import argparse +import os +import shlex +import subprocess +import sys + +import yaml + + +def parse_args(): + parser = argparse.ArgumentParser( + description="Replicate and emulate BazelCI job configurations from presubmit.yml." + ) + parser.add_argument( + "job", + help="The key or name of the CI job to emulate (e.g., ubuntu_workspace).", + ) + return parser.parse_args() + + +def run_cmd(cmd, cwd=None, env=None, shell=False): + if shell: + cmd_str = cmd if isinstance(cmd, str) else " ".join(cmd) + else: + cmd_str = shlex.join(cmd) if isinstance(cmd, list) else str(cmd) + + print(f"\n🚀 Executing: {cmd_str}") + if cwd and cwd != os.getcwd(): + print(f"📁 Directory: {cwd}") + if env and "USE_BAZEL_VERSION" in env: + print(f"🔧 Bazel Version: {env['USE_BAZEL_VERSION']}") + + res = subprocess.run(cmd, cwd=cwd, env=env, shell=shell) + if res.returncode != 0: + print( + f"\n❌ Command failed with return code {res.returncode}: {cmd_str}", + file=sys.stderr, + ) + return False + return True + + +def resolve_bazel_version(task_bazel): + if not task_bazel or task_bazel.startswith("${{"): + return None + return task_bazel + + +def execute_ci_job(job_key, task, repo_root): + job_name = task.get("name", job_key) + print(f"\n{'=' * 80}\n🎯 Replicating CI Job: {job_key} ('{job_name}')\n{'=' * 80}") + + # Setup working directory + cwd = repo_root + if "working_directory" in task: + cwd = os.path.join(repo_root, task["working_directory"]) + if not os.path.exists(cwd): + print( + f"❌ Error: working_directory '{task['working_directory']}' does not exist at '{cwd}'", + file=sys.stderr, + ) + return False + + # Setup environment + env = os.environ.copy() + bzl_version = resolve_bazel_version(task.get("bazel")) + if bzl_version: + env["USE_BAZEL_VERSION"] = bzl_version + + # Execute pre-commands + is_windows = sys.platform.startswith("win") + pre_cmds = task.get("batch_commands" if is_windows else "shell_commands", []) + for pre_cmd in pre_cmds: + if not run_cmd(pre_cmd, cwd=cwd, env=env, shell=True): + return False + + # Execute Build Targets + build_targets = [t for t in task.get("build_targets", []) if t != "--"] + if build_targets: + build_flags = task.get("build_flags", []) + cmd = ["bazel", "build"] + build_flags + ["--"] + build_targets + if not run_cmd(cmd, cwd=cwd, env=env): + return False + + # Execute Test Targets + test_targets = [t for t in task.get("test_targets", []) if t != "--"] + if test_targets: + test_flags = task.get("test_flags", []) + if "--build_tests_only" not in test_flags: + test_flags = ["--build_tests_only"] + test_flags + cmd = ["bazel", "test"] + test_flags + ["--"] + test_targets + if not run_cmd(cmd, cwd=cwd, env=env): + return False + + # Execute Coverage Targets + coverage_targets = [t for t in task.get("coverage_targets", []) if t != "--"] + if coverage_targets: + coverage_flags = task.get("test_flags", []) + cmd = ["bazel", "coverage"] + coverage_flags + ["--"] + coverage_targets + if not run_cmd(cmd, cwd=cwd, env=env): + return False + + print(f"\n🎉 Successfully replicated CI Job: {job_key}") + return True + + +def main(): + args = parse_args() + + repo_root = os.path.abspath(os.path.dirname(__file__)) + presubmit_path = os.path.join(repo_root, ".bazelci/presubmit.yml") + if not os.path.exists(presubmit_path): + print( + f"❌ Error: Presubmit file not found at '{presubmit_path}'", + file=sys.stderr, + ) + sys.exit(1) + + with open(presubmit_path) as f: + presubmit = yaml.safe_load(f) + + tasks = presubmit.get("tasks", {}) + if not tasks: + print( + f"❌ Error: No tasks found in '{presubmit_path}'", + file=sys.stderr, + ) + sys.exit(1) + + # If no job specified, print available jobs and exit + if not args.job: + print("❌ Error: No CI job specified. Provide a job key.\n", file=sys.stderr) + print("📋 Available CI Job Keys:", file=sys.stderr) + for key in sorted(tasks.keys()): + name = tasks[key].get("name", key) + print(f" • {key} ({name})", file=sys.stderr) + sys.exit(1) + + # Match by key or by name + job_key = None + if args.job in tasks: + job_key = args.job + else: + for key, config in tasks.items(): + if config.get("name") == args.job: + job_key = key + break + + if not job_key: + print( + f"❌ Error: CI job '{args.job}' not found in '{presubmit_path}'\n", + file=sys.stderr, + ) + print("📋 Available CI Job Keys:", file=sys.stderr) + for key in sorted(tasks.keys()): + name = tasks[key].get("name", key) + print(f" • {key} ({name})", file=sys.stderr) + sys.exit(1) + + success = execute_ci_job(job_key, tasks[job_key], repo_root) + sys.exit(0 if success else 1) + + +if __name__ == "__main__": + main() diff --git a/ruff.toml b/ruff.toml new file mode 100644 index 0000000000..fc29594bad --- /dev/null +++ b/ruff.toml @@ -0,0 +1,29 @@ +# ruff.toml + +# Support Python 3.8 and above +target-version = "py38" + +# Match Black's default line length +line-length = 88 + +# Ignore testdata files +extend-exclude = ["**/testdata/**", "testdata"] + +[lint] +# Enable Pyflakes (F) and pycodestyle (E, W) for standard linting, +# and isort (I) for import sorting +select = ["F", "E", "W", "I"] + +# Like Black, allow certain violations that may conflict with its formatting +ignore = ["E501"] + +[lint.isort] +# Matches isort's profile = "black" behavior +combine-as-imports = true +force-single-line = false + +[format] +# The Ruff formatter defaults to Black's style +quote-style = "double" +indent-style = "space" +skip-magic-trailing-comma = false diff --git a/specialized_configs.bazelrc b/specialized_configs.bazelrc new file mode 100644 index 0000000000..46334e0a95 --- /dev/null +++ b/specialized_configs.bazelrc @@ -0,0 +1,12 @@ + +# Helper config to run most tests without waiting an inordinate amount +# of time or freezing the system +common:fast-tests --build_tests_only=true +common:fast-tests --build_tag_filters=-large,-enormous,-integration-test +common:fast-tests --test_tag_filters=-large,-enormous,-integration-test + +# Helper config for running a single test locally and investigating resulting state +common:testone --test_output=streamed +common:testone --test_strategy=standalone +common:testone --spawn_strategy=standalone +common:testone --strategy=standalone diff --git a/sphinxdocs/.bazelignore b/sphinxdocs/.bazelignore new file mode 100644 index 0000000000..48237cb876 --- /dev/null +++ b/sphinxdocs/.bazelignore @@ -0,0 +1,12 @@ +bazel-bin +bazel-out +bazel-testlogs +bazel-sphinxdocs +integration_tests/bcr/bazel-bin +integration_tests/bcr/bazel-bcr +integration_tests/bcr/bazel-out +integration_tests/bcr/bazel-testlogs +integration_tests/persistent_worker/workspace/bazel-bin +integration_tests/persistent_worker/workspace/bazel-out +integration_tests/persistent_worker/workspace/bazel-testlogs +integration_tests/persistent_worker/workspace/bazel-workspace diff --git a/sphinxdocs/.bazelrc b/sphinxdocs/.bazelrc new file mode 100644 index 0000000000..ce4d782113 --- /dev/null +++ b/sphinxdocs/.bazelrc @@ -0,0 +1,33 @@ +import %workspace%/.bazelrc.deleted_packages + +test --test_output=errors + +build --incompatible_default_to_explicit_init_py +build --@rules_python//python/config_settings:incompatible_default_to_explicit_init_py=True + +# Ensure ongoing compatibility with this flag. +common --incompatible_disallow_struct_provider_syntax +# Makes Bazel 7 act more like Bazel 8 +common --incompatible_use_plus_in_repo_names + +# Explicitly enable for Windows +build --enable_runfiles + +# Local disk cache greatly speeds up builds if the regular cache is lost +common --disk_cache=~/.cache/bazel/bazel-disk-cache +common --experimental_downloader_config=downloader_config.cfg +common --http_timeout_scaling=10.0 +common --experimental_repository_downloader_retries=10 + +common --incompatible_python_disallow_native_rules +common --incompatible_no_implicit_file_export +# Pass host PATH to build and test actions so bazel_from_env (used by rules_bazel_integration_test +# for local 'self' testing) can locate the user's bazel binary without disabling strict action env. +common --action_env=PATH +common --test_env=PATH + +build --lockfile_mode=update + +common:fast-tests --build_tests_only=true +common:fast-tests --build_tag_filters=-large,-enormous,-integration-test +common:fast-tests --test_tag_filters=-large,-enormous,-integration-test diff --git a/sphinxdocs/.bazelrc.deleted_packages b/sphinxdocs/.bazelrc.deleted_packages new file mode 100644 index 0000000000..9a90ef21be --- /dev/null +++ b/sphinxdocs/.bazelrc.deleted_packages @@ -0,0 +1,2 @@ +common --deleted_packages=integration_tests/bcr +common --deleted_packages=integration_tests/persistent_worker/workspace diff --git a/sphinxdocs/.bazelversion b/sphinxdocs/.bazelversion new file mode 100644 index 0000000000..512e4c889e --- /dev/null +++ b/sphinxdocs/.bazelversion @@ -0,0 +1 @@ +9.x diff --git a/sphinxdocs/BUILD.bazel b/sphinxdocs/BUILD.bazel index 9ad1e1eef9..d1eec30a09 100644 --- a/sphinxdocs/BUILD.bazel +++ b/sphinxdocs/BUILD.bazel @@ -1,66 +1,19 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -load("@bazel_skylib//:bzl_library.bzl", "bzl_library") -load("@bazel_skylib//rules:common_settings.bzl", "bool_flag") -load("//sphinxdocs/private:sphinx.bzl", "repeated_string_list_flag") - -package( - default_visibility = ["//:__subpackages__"], -) - -# Additional -D values to add to every Sphinx build. -# This is usually used to override the version when building -repeated_string_list_flag( - name = "extra_defines", - build_setting_default = [], -) - -repeated_string_list_flag( - name = "extra_env", - build_setting_default = [], -) - -# Whether to add the `-q` arg to Sphinx invocations, which determines if -# stdout has any output or not (logging INFO messages and progress messages). -# If true, add `-q`. If false, don't add `-q`. This is mostly useful for -# debugging invocations or developing extensions. -bool_flag( - name = "quiet", - build_setting_default = True, -) - -bzl_library( - name = "sphinx_bzl", - srcs = ["sphinx.bzl"], - deps = ["//sphinxdocs/private:sphinx_bzl"], -) - -bzl_library( - name = "sphinx_docs_library_bzl", - srcs = ["sphinx_docs_library.bzl"], - deps = ["//sphinxdocs/private:sphinx_docs_library_macro_bzl"], -) - -bzl_library( - name = "sphinx_stardoc_bzl", - srcs = ["sphinx_stardoc.bzl"], - deps = ["//sphinxdocs/private:sphinx_stardoc_bzl"], -) - -bzl_library( - name = "readthedocs_bzl", - srcs = ["readthedocs.bzl"], - deps = ["//sphinxdocs/private:readthedocs_bzl"], +package(default_visibility = ["//visibility:public"]) + +exports_files(["pyproject.toml"]) + +filegroup( + name = "distribution", + srcs = glob( + ["**/*"], + exclude = [ + "bazel-*/**", + "integration_tests/**", + "tests/**", + ], + ) + [ + "//dev:distribution", + "//sphinxdocs:distribution", + ], + visibility = ["//visibility:public"], ) diff --git a/sphinxdocs/MODULE.bazel b/sphinxdocs/MODULE.bazel new file mode 100644 index 0000000000..51de5e94f9 --- /dev/null +++ b/sphinxdocs/MODULE.bazel @@ -0,0 +1,42 @@ +module( + name = "sphinxdocs", + version = "0.0.0", + compatibility_level = 1, +) + +bazel_dep(name = "bazel_skylib", version = "1.8.2") +bazel_dep(name = "stardoc", version = "0.7.2", repo_name = "io_bazel_stardoc") +bazel_dep(name = "platforms", version = "0.0.11") +bazel_dep(name = "protobuf", version = "29.0-rc2", repo_name = "com_google_protobuf") +bazel_dep(name = "rules_python", version = "2.2.0") + +dev_pip = use_extension( + "@rules_python//python/extensions:pip.bzl", + "pip", + dev_dependency = True, +) +dev_pip.parse( + hub_name = "dev_pip", + python_version = "3.11", + requirements_lock = "//dev:requirements.txt", + uv_lock = "//dev:uv.lock", +) +use_repo(dev_pip, "dev_pip") + +bazel_dep(name = "rules_bazel_integration_test", version = "0.37.1", dev_dependency = True) + +bazel_binaries = use_extension( + "@rules_bazel_integration_test//:extensions.bzl", + "bazel_binaries", + dev_dependency = True, +) +bazel_binaries.local( + name = "self", + path = "integration_tests/bazel_from_env", +) +use_repo( + bazel_binaries, + "bazel_binaries", + "bazel_binaries_bazelisk", + "build_bazel_bazel_self", +) diff --git a/sphinxdocs/dev/BUILD.bazel b/sphinxdocs/dev/BUILD.bazel new file mode 100644 index 0000000000..41e88f2c59 --- /dev/null +++ b/sphinxdocs/dev/BUILD.bazel @@ -0,0 +1,30 @@ +load("@rules_python//python/uv:lock.bzl", "lock") + +package(default_visibility = ["//:__subpackages__"]) + +filegroup( + name = "distribution", + srcs = glob(["**/*"]), + visibility = ["//:__pkg__"], +) + +# Run bazel run //dev:uv_lock.update +lock( + name = "uv_lock", + srcs = ["//:pyproject.toml"], + out = "uv.lock", + python_version = "3.11", +) + +# Run bazel run //dev:requirements.update +lock( + name = "requirements", + srcs = ["//:pyproject.toml"], + out = "requirements.txt", + args = [ + "--emit-index-url", + "--universal", + "--upgrade", + ], + python_version = "3.11", +) diff --git a/sphinxdocs/dev/requirements.txt b/sphinxdocs/dev/requirements.txt new file mode 100644 index 0000000000..e24834fe1e --- /dev/null +++ b/sphinxdocs/dev/requirements.txt @@ -0,0 +1,556 @@ +# This file was autogenerated by uv via the following command: +# bazel run //dev:requirements.update +--index-url https://pypi.org/simple + +absl-py==2.3.1 ; python_full_version < '3.10' \ + --hash=sha256:a97820526f7fbfd2ec1bce83f3f25e3a14840dac0d8e02a0b71cd75db3f77fc9 \ + --hash=sha256:eeecf07f0c2a93ace0772c92e596ace6d3d3996c042b2128459aaae2a76de11d + # via rules-python-sphinxdocs-dev (dev/pyproject.toml) +absl-py==2.5.0 ; python_full_version >= '3.10' \ + --hash=sha256:0c996f25c0490700fadabe6351630f6111534fa0ae252cc6d2014ea3b141135f \ + --hash=sha256:0f17b89f2a4eaaedc4f28c622998aa690564b3012a396a4ffad0821007fe03ba + # via rules-python-sphinxdocs-dev (dev/pyproject.toml) +alabaster==0.7.16 ; python_full_version < '3.10' \ + --hash=sha256:75a8b99c28a5dad50dd7f8ccdd447a121ddb3892da9e53d1ca5cca3106d58d65 \ + --hash=sha256:b46733c07dce03ae4e150330b975c75737fa60f0a7c591b6c8bf4928a28e2c92 + # via sphinx +alabaster==1.0.0 ; python_full_version >= '3.10' \ + --hash=sha256:c00dca57bca26fa62a6d7d0a9fcce65f3e026e9bfe33e9c538fd3fbb2144fd9e \ + --hash=sha256:fc6786402dc3fcb2de3cabd5fe455a2db534b371124f1f21de8731783dec828b + # via sphinx +astroid==3.3.11 \ + --hash=sha256:1e5a5011af2920c7c67a53f65d536d65bfa7116feeaf2354d8b94f29573bb0ce \ + --hash=sha256:54c760ae8322ece1abd213057c4b5bba7c49818853fc901ef09719a60dbf9dec + # via sphinx-autodoc2 +babel==2.18.0 \ + --hash=sha256:b80b99a14bd085fcacfa15c9165f651fbb3406e66cc603abf11c5750937c992d \ + --hash=sha256:e2b422b277c2b9a9630c1d7903c2a00d0830c409c59ac8cae9081c92f1aeba35 + # via sphinx +certifi==2026.7.22 \ + --hash=sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775 \ + --hash=sha256:741e2c3b351ddf169a738da9f2c048608ff7f2c5cc02f1ebc6b118bb090d5d55 + # via requests +charset-normalizer==3.4.9 \ + --hash=sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380 \ + --hash=sha256:03d07803992c6c7bbc976327f34b18b6160327fc81cb82c9d504720ac0be3b62 \ + --hash=sha256:04ce310cb89c15df659582aee80a0603788732a5e017d5bd5c81158106ce249c \ + --hash=sha256:0d861473f743244d349b50f850d10eb87aeb22bbdcc8e64f79273c94af5a8226 \ + --hash=sha256:0e94703ec9684807f20cfb5eed95c70f67f2a8f21ad620146d7b5a13677b93e5 \ + --hash=sha256:0fa1aec2d32bcc03c8fa0f6f1712caad1adc38509f31142112e5c9daf5b9c833 \ + --hash=sha256:16b65ea0f2465b6fb52aa22de5eca612aa964ddfec00a912e26f4656cbef890b \ + --hash=sha256:16d10d789dd9bcca1173c95af82c58433122564b7bc39385124be735a35cbe99 \ + --hash=sha256:19ac87f93086ce37b86e098888555c4b4bc48102279bae3350098c0ed664b501 \ + --hash=sha256:1d22856ffbe153a602df38e4a5464f0b748a54002e0d69ac6d2ad0a197cc99ec \ + --hash=sha256:21e764fd1e70b6a3e205a0e46f3051701f98a8cb3fad66eeb80e48bb502f8698 \ + --hash=sha256:231ddcbb35e2ff8973e1365db41fe0572662893b99a05deb183b68ad4c0c8bd4 \ + --hash=sha256:253a4a220747e8b5faf57ec320c4f5efb0cef05f647420bf267143ec15dba10a \ + --hash=sha256:280081916dc341820640489a66e4696049401ef1cf6dd672f672e70ad915aca3 \ + --hash=sha256:2a441ea71902098ffe78c5abe6c494f44160b4af614ed16c3d9a3b1d17fd8ee2 \ + --hash=sha256:304b13570067b2547562e308af560b3963857b1fa90bd6afd978130130fe2d6a \ + --hash=sha256:32286a2c8d167e897177b673176c1e3e00d4057caf5d2b64eef9a3666b03018e \ + --hash=sha256:33bdcc2a32c0a0e861f60841a512c8acc658c87c2ac59d89e3a46dacf7d866e4 \ + --hash=sha256:375b83ed0aecfce76c16d198fbc21f3b11b337d68662bea0a995046682a11419 \ + --hash=sha256:3c09a49d6cde137258beb3d551994a2927fd35ad5cf96aed573f61bbd67c5f84 \ + --hash=sha256:3d92613ec25e43b05f042302531ec0f00b8445190e43325880cbd6ab7c2581da \ + --hash=sha256:40a126142a56b2dfc0aacbad1de8310cbf60da7656db0e6b16eebd48e3e93519 \ + --hash=sha256:416c229f77e5ea25b3dfd4b582f8d73d7e43c22320302b9ab128a2d3a0b38efe \ + --hash=sha256:432786d3561e69aeeae6c7e8648964ce0ad05736120135601f87ac26b9c83381 \ + --hash=sha256:43b9e366a31fdd1c87d0eb08f579b4a82b723ea54338f040d6b4e518a026ea29 \ + --hash=sha256:440eede837960000d74978f0eba527be106b5b9aee0daf779d395276ed0b0614 \ + --hash=sha256:45b0cc4e3556cd875e09102988d1ab8356c998b596c9fced84547c8138b487a0 \ + --hash=sha256:476743fe6dfe14a2da12e3ac79125dc84a3b2cf8094369a47a1529b0cd8549fe \ + --hash=sha256:4773092f8019072343a7447203308b176e10199920eb02d6195e81bbb3274c29 \ + --hash=sha256:4b3dac63058cc36820b0dd072f89898604e2d39686fe05321729d00d8ac185a0 \ + --hash=sha256:4d1c96a7a18b9690a4d46df09e3e3382406ae3213727cd1019ebade1c4a81917 \ + --hash=sha256:51307f5c71007673a2bf8232ad973483d281e74cb99c8c5a990af1eefa6277d9 \ + --hash=sha256:51447e9aa2684679af07ca5021c3db526e0284347ebf4ffcec1154c3350cfe32 \ + --hash=sha256:58150c9f9b9a552505912d182ccdf26f6396fb6094816ceebcbb20eecabaed94 \ + --hash=sha256:5b10cd92fc5c498b35a8635df6d5a100207f88b63a4dc1de7ef9a548e1e2cd63 \ + --hash=sha256:5e226f6218febc71f6c1fc2fafb91c226f75bdc1d8fb12d66823716e891608fd \ + --hash=sha256:609b3ba8fcc0fb5ab7af00719d0fb6ad0cb518e48e7712d12fd68f1327951198 \ + --hash=sha256:60f44ade2cf573dad7a277e6f8ca9a51a21dda572b13bd7d8539bb3cd5dbedde \ + --hash=sha256:611057cc5d5c0afc743ba8be6bd828c17e0aaa8643f9d0a9b9bb7dea80eb8012 \ + --hash=sha256:6366a16e1a25018694d6a5d784d09b046edc9eac40ea2b54065c3052672516a1 \ + --hash=sha256:65a7ff3f705e57d392f7261b6d0550fe137c3019477431f1c355e0db0a7d3e15 \ + --hash=sha256:673611bbd43f0810bec0b0f028ddeaaa501190339cac411f347ac76917c3ae7b \ + --hash=sha256:67830fc78e67501f47bb950471b2dcb9b35b140084429318e862895a8e89c993 \ + --hash=sha256:68ce9f4d6b26d5ccbf7fd4459bf75f74a0a146677ebba80597df60cbdb20e6f4 \ + --hash=sha256:68e5f26a1ad57ded6d1cfb85331d1c1a195314756471d97758c48498bb4dcdf5 \ + --hash=sha256:69b157c5d3292bcd443faca052f3096f637f1e074b98212a933c074ae23dc3b8 \ + --hash=sha256:75286256590a6320cf106a0d28970d3560aad9ee09aa7b34fb40524792436d35 \ + --hash=sha256:78841cccf1af7b40f6f716338d50c0902dbe88d9f800b3c973b7a9a0a693a642 \ + --hash=sha256:78fa18e436a1a0e58dbd7e02fc4473f3f32cceb12df9dfca542d075961c307d2 \ + --hash=sha256:79580094b00d1789d1f93ea55bc43cb2f611910c72235b7657f3482ddcc1b22d \ + --hash=sha256:7b86a2b16095d250c6f58b3d9b2eee6f4147754344f3dab0922f7c9bf7d226c9 \ + --hash=sha256:83aed2c10721ddd90f68140685391b50811a880af20654c59af6b6c66c40513c \ + --hash=sha256:84fd18bcc17526fc2b3c1af7d2b9217d32c9c04448c16ec693b9b4f1985c3d33 \ + --hash=sha256:871ff67ea1aad4dfd91736464934d56b32dac49f9fbe16cddba36198a7b3a0db \ + --hash=sha256:898f0e9068ca27d37f8e83a5b962821df851532e6c4a7d615c1c033f9da6eedf \ + --hash=sha256:8a79d9f4d8001473a30c163556b3c3bfebec837495a412dde78b51672f6134f9 \ + --hash=sha256:8c041122946b7ba21bb32c45b1aa57b1be35527690aeb3c5c234521085632eee \ + --hash=sha256:90c44bc373b7687f6948b693cceaea1348ae0975d7474746559494468e3c1d84 \ + --hash=sha256:9104ed0bd76a429d46f9ec0dbc9b08ad1d2dcdf2b00a5a0daa1c145329b35b44 \ + --hash=sha256:920079c3f7456fa213e0829ed2073aaa727fd39d889ead5b4f35d0de5460d04f \ + --hash=sha256:93d59d504b230e83c7a843251681959a0b6a9cd76f6e146ce1b8a80eb8739af9 \ + --hash=sha256:9b2aff1c7b3884512b9512c3eaadd9bab39fb45042ffaaa1dd08ff2b9f8109d9 \ + --hash=sha256:9b8e0f3107e2200b76f6054de99016eac3ee6762713587b36baaa7e4bd2ae177 \ + --hash=sha256:9bb41182d93ea91f60b4bc8fbf4c820c69ef8a12ab2d917f3f1834f1acad07e8 \ + --hash=sha256:9cdef90ae47919cae358d8ab15797a800ed41da7aba5d72419fb510729e2ed4b \ + --hash=sha256:a1786910334ed46ab1dd73222f2cd1e05c2c3bb39f6dddb4f8b36fc382058a39 \ + --hash=sha256:a4cfde78a9f2880208d16a93b795726a3017d5977e08d1e162a7a31322479c41 \ + --hash=sha256:a4fbdde9dd4a9ce5fd52c2b3a347bb50cc89483ef783f1cb00d408c13f7a96c0 \ + --hash=sha256:aa99adc8f081b475a12843953db36831eaf83ec33eb46a90629ca6a5de45a616 \ + --hash=sha256:ac351b3b8014eead140e77e9717e2992c6bbe30b63bc3422422eb84865412e3d \ + --hash=sha256:ad41ba96094304aa090f5a30cb6e4fb3b3f1c264c523394b4c39bbacc4dc92ba \ + --hash=sha256:b5314963fce9b0b12743891de876e724997864ee22aa496f903f426c7e2fa5b2 \ + --hash=sha256:bcf74c1df76758a395bf0af608c04c82257523f55c9868b334f06270d0f2112b \ + --hash=sha256:bd47ba7fc3ca94896759ea0109775132d3e7ab921fbf54038e1bab2e46c313c9 \ + --hash=sha256:c0323c9daef75ef2e5083624b4585018a0c9d5e3b40f607eed81a311270b934b \ + --hash=sha256:c1225416b463483160e4af85d5fc3a9690ccb53fd4b1865a6437825f5ede3209 \ + --hash=sha256:c1c948747b03be832dceed96ca815cef7360de9aa19d37c730f8e3f6101aca48 \ + --hash=sha256:c25fe15c70c59eb7c5ce8c06a1f3fa1da0ecc5ea1e7a5922c40fd2fa9b0d5046 \ + --hash=sha256:cc1b0fff8ead343dae06305f954eb8468ba0ec1a97881f42489d198e4ce3c632 \ + --hash=sha256:cd6280cf040f233bd7d3407b743b4b4c74f70e8e1c4199cb112a62c941c0772a \ + --hash=sha256:cd6c3d4b783c556fa00bf540854e42f135e2f256abd29669fcd0da0f2dec79c2 \ + --hash=sha256:d4d6fcde76f94f5cb9e43e9e9a61f16dacefd228cbbf6f1a09bd9b219a92f1a1 \ + --hash=sha256:ddf4af30b417d9fe16481e9b81c27ab2a7cde1ff7ba3e85653b02db7d145dc7b \ + --hash=sha256:df115d4d83168fdf2cae48ef1ff6d1cb4c466364e30861b37121de0f3bf1b990 \ + --hash=sha256:df7276909358e5635ae203673ab7e509ddd224225a8d6b0790bf13eb2bde1cc5 \ + --hash=sha256:e4fd89cc178bced6ad29cb3e6dd4aa63fa5017c3524dbd0b25998fb64a87cc8b \ + --hash=sha256:e9701d0049d92c16703a42771b98d560b95248949f23f8cf7b4eddd201814fb9 \ + --hash=sha256:ee2f2a527e3c1a6e6411eb4209642e138b544a2d72fe5d0d76daf77b24063534 \ + --hash=sha256:f7fb7d750cfa0a070d2c24e831fd3481019a60dd317ea2b39acbcebc08b6ed81 \ + --hash=sha256:f840ed6d8ecba8255df8c42b87fadeda98ddfc6eeec05e2dc66e26d46dd6f58a \ + --hash=sha256:f86c6358749bd4fda175388691e3ba8c46e24c5347d0afd20f9b7edfc9faf07d \ + --hash=sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf \ + --hash=sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115 + # via requests +colorama==0.4.6 ; sys_platform == 'win32' \ + --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \ + --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6 + # via sphinx +docutils==0.21.2 ; python_full_version < '3.11' \ + --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ + --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2 + # via + # myst-parser + # sphinx + # sphinx-rtd-theme +docutils==0.22.4 ; python_full_version >= '3.11' \ + --hash=sha256:4db53b1fde9abecbb74d91230d32ab626d94f6badfc575d6db9194a49df29968 \ + --hash=sha256:d0013f540772d1420576855455d050a2180186c91c15779301ac2ccb3eeb68de + # via + # myst-parser + # sphinx + # sphinx-rtd-theme +idna==3.18 \ + --hash=sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2 \ + --hash=sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848 + # via requests +imagesize==1.5.0 ; python_full_version < '3.10' \ + --hash=sha256:32677681b3f434c2cb496f00e89c5a291247b35b1f527589909e008057da5899 \ + --hash=sha256:8bfc5363a7f2133a89f0098451e0bcb1cd71aba4dc02bbcecb39d99d40e1b94f + # via sphinx +imagesize==2.0.0 ; python_full_version >= '3.10' \ + --hash=sha256:5667c5bbb57ab3f1fa4bc366f4fbc971db3d5ed011fd2715fd8001f782718d96 \ + --hash=sha256:8e8358c4a05c304f1fccf7ff96f036e7243a189e9e42e90851993c558cfe9ee3 + # via sphinx +importlib-metadata==8.7.1 ; python_full_version < '3.10' \ + --hash=sha256:49fef1ae6440c182052f407c8d34a68f72efc36db9ca90dc0113398f2fdde8bb \ + --hash=sha256:5a1f80bf1daa489495071efbb095d75a634cf28a8bc299581244063b53176151 + # via sphinx +jinja2==3.1.6 \ + --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ + --hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67 + # via + # myst-parser + # readthedocs-sphinx-ext + # sphinx +markdown-it-py==3.0.0 ; python_full_version < '3.11' \ + --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ + --hash=sha256:e3f60a94fa066dc52ec76661e37c851cb232d92f9886b15cb560aaada2df8feb + # via + # mdit-py-plugins + # myst-parser +markdown-it-py==4.2.0 ; python_full_version >= '3.11' \ + --hash=sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49 \ + --hash=sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a + # via + # mdit-py-plugins + # myst-parser +markupsafe==3.0.3 \ + --hash=sha256:0303439a41979d9e74d18ff5e2dd8c43ed6c6001fd40e5bf2e43f7bd9bbc523f \ + --hash=sha256:068f375c472b3e7acbe2d5318dea141359e6900156b5b2ba06a30b169086b91a \ + --hash=sha256:0bf2a864d67e76e5c9a34dc26ec616a66b9888e25e7b9460e1c76d3293bd9dbf \ + --hash=sha256:0db14f5dafddbb6d9208827849fad01f1a2609380add406671a26386cdf15a19 \ + --hash=sha256:0eb9ff8191e8498cca014656ae6b8d61f39da5f95b488805da4bb029cccbfbaf \ + --hash=sha256:0f4b68347f8c5eab4a13419215bdfd7f8c9b19f2b25520968adfad23eb0ce60c \ + --hash=sha256:1085e7fbddd3be5f89cc898938f42c0b3c711fdcb37d75221de2666af647c175 \ + --hash=sha256:116bb52f642a37c115f517494ea5feb03889e04df47eeff5b130b1808ce7c219 \ + --hash=sha256:12c63dfb4a98206f045aa9563db46507995f7ef6d83b2f68eda65c307c6829eb \ + --hash=sha256:133a43e73a802c5562be9bbcd03d090aa5a1fe899db609c29e8c8d815c5f6de6 \ + --hash=sha256:1353ef0c1b138e1907ae78e2f6c63ff67501122006b0f9abad68fda5f4ffc6ab \ + --hash=sha256:15d939a21d546304880945ca1ecb8a039db6b4dc49b2c5a400387cdae6a62e26 \ + --hash=sha256:177b5253b2834fe3678cb4a5f0059808258584c559193998be2601324fdeafb1 \ + --hash=sha256:1872df69a4de6aead3491198eaf13810b565bdbeec3ae2dc8780f14458ec73ce \ + --hash=sha256:1b4b79e8ebf6b55351f0d91fe80f893b4743f104bff22e90697db1590e47a218 \ + --hash=sha256:1b52b4fb9df4eb9ae465f8d0c228a00624de2334f216f178a995ccdcf82c4634 \ + --hash=sha256:1ba88449deb3de88bd40044603fafffb7bc2b055d626a330323a9ed736661695 \ + --hash=sha256:1cc7ea17a6824959616c525620e387f6dd30fec8cb44f649e31712db02123dad \ + --hash=sha256:218551f6df4868a8d527e3062d0fb968682fe92054e89978594c28e642c43a73 \ + --hash=sha256:26a5784ded40c9e318cfc2bdb30fe164bdb8665ded9cd64d500a34fb42067b1c \ + --hash=sha256:2713baf880df847f2bece4230d4d094280f4e67b1e813eec43b4c0e144a34ffe \ + --hash=sha256:2a15a08b17dd94c53a1da0438822d70ebcd13f8c3a95abe3a9ef9f11a94830aa \ + --hash=sha256:2f981d352f04553a7171b8e44369f2af4055f888dfb147d55e42d29e29e74559 \ + --hash=sha256:32001d6a8fc98c8cb5c947787c5d08b0a50663d139f1305bac5885d98d9b40fa \ + --hash=sha256:3524b778fe5cfb3452a09d31e7b5adefeea8c5be1d43c4f810ba09f2ceb29d37 \ + --hash=sha256:3537e01efc9d4dccdf77221fb1cb3b8e1a38d5428920e0657ce299b20324d758 \ + --hash=sha256:35add3b638a5d900e807944a078b51922212fb3dedb01633a8defc4b01a3c85f \ + --hash=sha256:38664109c14ffc9e7437e86b4dceb442b0096dfe3541d7864d9cbe1da4cf36c8 \ + --hash=sha256:3a7e8ae81ae39e62a41ec302f972ba6ae23a5c5396c8e60113e9066ef893da0d \ + --hash=sha256:3b562dd9e9ea93f13d53989d23a7e775fdfd1066c33494ff43f5418bc8c58a5c \ + --hash=sha256:457a69a9577064c05a97c41f4e65148652db078a3a509039e64d3467b9e7ef97 \ + --hash=sha256:4bd4cd07944443f5a265608cc6aab442e4f74dff8088b0dfc8238647b8f6ae9a \ + --hash=sha256:4e885a3d1efa2eadc93c894a21770e4bc67899e3543680313b09f139e149ab19 \ + --hash=sha256:4faffd047e07c38848ce017e8725090413cd80cbc23d86e55c587bf979e579c9 \ + --hash=sha256:509fa21c6deb7a7a273d629cf5ec029bc209d1a51178615ddf718f5918992ab9 \ + --hash=sha256:5678211cb9333a6468fb8d8be0305520aa073f50d17f089b5b4b477ea6e67fdc \ + --hash=sha256:591ae9f2a647529ca990bc681daebdd52c8791ff06c2bfa05b65163e28102ef2 \ + --hash=sha256:5a7d5dc5140555cf21a6fefbdbf8723f06fcd2f63ef108f2854de715e4422cb4 \ + --hash=sha256:69c0b73548bc525c8cb9a251cddf1931d1db4d2258e9599c28c07ef3580ef354 \ + --hash=sha256:6b5420a1d9450023228968e7e6a9ce57f65d148ab56d2313fcd589eee96a7a50 \ + --hash=sha256:722695808f4b6457b320fdc131280796bdceb04ab50fe1795cd540799ebe1698 \ + --hash=sha256:729586769a26dbceff69f7a7dbbf59ab6572b99d94576a5592625d5b411576b9 \ + --hash=sha256:77f0643abe7495da77fb436f50f8dab76dbc6e5fd25d39589a0f1fe6548bfa2b \ + --hash=sha256:795e7751525cae078558e679d646ae45574b47ed6e7771863fcc079a6171a0fc \ + --hash=sha256:7be7b61bb172e1ed687f1754f8e7484f1c8019780f6f6b0786e76bb01c2ae115 \ + --hash=sha256:7c3fb7d25180895632e5d3148dbdc29ea38ccb7fd210aa27acbd1201a1902c6e \ + --hash=sha256:7e68f88e5b8799aa49c85cd116c932a1ac15caaa3f5db09087854d218359e485 \ + --hash=sha256:83891d0e9fb81a825d9a6d61e3f07550ca70a076484292a70fde82c4b807286f \ + --hash=sha256:8485f406a96febb5140bfeca44a73e3ce5116b2501ac54fe953e488fb1d03b12 \ + --hash=sha256:8709b08f4a89aa7586de0aadc8da56180242ee0ada3999749b183aa23df95025 \ + --hash=sha256:8f71bc33915be5186016f675cd83a1e08523649b0e33efdb898db577ef5bb009 \ + --hash=sha256:915c04ba3851909ce68ccc2b8e2cd691618c4dc4c4232fb7982bca3f41fd8c3d \ + --hash=sha256:949b8d66bc381ee8b007cd945914c721d9aba8e27f71959d750a46f7c282b20b \ + --hash=sha256:94c6f0bb423f739146aec64595853541634bde58b2135f27f61c1ffd1cd4d16a \ + --hash=sha256:9a1abfdc021a164803f4d485104931fb8f8c1efd55bc6b748d2f5774e78b62c5 \ + --hash=sha256:9b79b7a16f7fedff2495d684f2b59b0457c3b493778c9eed31111be64d58279f \ + --hash=sha256:a320721ab5a1aba0a233739394eb907f8c8da5c98c9181d1161e77a0c8e36f2d \ + --hash=sha256:a4afe79fb3de0b7097d81da19090f4df4f8d3a2b3adaa8764138aac2e44f3af1 \ + --hash=sha256:ad2cf8aa28b8c020ab2fc8287b0f823d0a7d8630784c31e9ee5edea20f406287 \ + --hash=sha256:b8512a91625c9b3da6f127803b166b629725e68af71f8184ae7e7d54686a56d6 \ + --hash=sha256:bc51efed119bc9cfdf792cdeaa4d67e8f6fcccab66ed4bfdd6bde3e59bfcbb2f \ + --hash=sha256:bdc919ead48f234740ad807933cdf545180bfbe9342c2bb451556db2ed958581 \ + --hash=sha256:bdd37121970bfd8be76c5fb069c7751683bdf373db1ed6c010162b2a130248ed \ + --hash=sha256:be8813b57049a7dc738189df53d69395eba14fb99345e0a5994914a3864c8a4b \ + --hash=sha256:c0c0b3ade1c0b13b936d7970b1d37a57acde9199dc2aecc4c336773e1d86049c \ + --hash=sha256:c47a551199eb8eb2121d4f0f15ae0f923d31350ab9280078d1e5f12b249e0026 \ + --hash=sha256:c4ffb7ebf07cfe8931028e3e4c85f0357459a3f9f9490886198848f4fa002ec8 \ + --hash=sha256:ccfcd093f13f0f0b7fdd0f198b90053bf7b2f02a3927a30e63f3ccc9df56b676 \ + --hash=sha256:d2ee202e79d8ed691ceebae8e0486bd9a2cd4794cec4824e1c99b6f5009502f6 \ + --hash=sha256:d53197da72cc091b024dd97249dfc7794d6a56530370992a5e1a08983ad9230e \ + --hash=sha256:d6dd0be5b5b189d31db7cda48b91d7e0a9795f31430b7f271219ab30f1d3ac9d \ + --hash=sha256:d88b440e37a16e651bda4c7c2b930eb586fd15ca7406cb39e211fcff3bf3017d \ + --hash=sha256:de8a88e63464af587c950061a5e6a67d3632e36df62b986892331d4620a35c01 \ + --hash=sha256:df2449253ef108a379b8b5d6b43f4b1a8e81a061d6537becd5582fba5f9196d7 \ + --hash=sha256:e1c1493fb6e50ab01d20a22826e57520f1284df32f2d8601fdd90b6304601419 \ + --hash=sha256:e1cf1972137e83c5d4c136c43ced9ac51d0e124706ee1c8aa8532c1287fa8795 \ + --hash=sha256:e2103a929dfa2fcaf9bb4e7c091983a49c9ac3b19c9061b6d5427dd7d14d81a1 \ + --hash=sha256:e56b7d45a839a697b5eb268c82a71bd8c7f6c94d6fd50c3d577fa39a9f1409f5 \ + --hash=sha256:e8afc3f2ccfa24215f8cb28dcf43f0113ac3c37c2f0f0806d8c70e4228c5cf4d \ + --hash=sha256:e8fc20152abba6b83724d7ff268c249fa196d8259ff481f3b1476383f8f24e42 \ + --hash=sha256:eaa9599de571d72e2daf60164784109f19978b327a3910d3e9de8c97b5b70cfe \ + --hash=sha256:ec15a59cf5af7be74194f7ab02d0f59a62bdcf1a537677ce67a2537c9b87fcda \ + --hash=sha256:f190daf01f13c72eac4efd5c430a8de82489d9cff23c364c3ea822545032993e \ + --hash=sha256:f34c41761022dd093b4b6896d4810782ffbabe30f2d443ff5f083e0cbbb8c737 \ + --hash=sha256:f3e98bb3798ead92273dc0e5fd0f31ade220f59a266ffd8a4f6065e0a3ce0523 \ + --hash=sha256:f42d0984e947b8adf7dd6dde396e720934d12c506ce84eea8476409563607591 \ + --hash=sha256:f71a396b3bf33ecaa1626c255855702aca4d3d9fea5e051b41ac59a9c1c41edc \ + --hash=sha256:f9e130248f4462aaa8e2552d547f36ddadbeaa573879158d721bbd33dfe4743a \ + --hash=sha256:fed51ac40f757d41b7c48425901843666a6677e3e8eb0abcff09e4ba6e664f50 + # via + # rules-python-sphinxdocs-dev (dev/pyproject.toml) + # jinja2 +mdit-py-plugins==0.4.2 ; python_full_version < '3.10' \ + --hash=sha256:0c673c3f889399a33b95e88d2f0d111b4447bdfea7f237dab2d488f459835636 \ + --hash=sha256:5f2cd1fdb606ddf152d37ec30e46101a60512bc0e5fa1a7002c36647b09e26b5 + # via myst-parser +mdit-py-plugins==0.6.1 ; python_full_version >= '3.10' \ + --hash=sha256:214c82fb2ac524472ab6a5bcab1de80f73b50443e187f401bfd77efbc7c6481d \ + --hash=sha256:a2bca0f039f39dbd35fb74ae1b5f998608c437463371f0ff7f49a19a17a114d0 + # via myst-parser +mdurl==0.1.2 \ + --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ + --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba + # via markdown-it-py +myst-parser==3.0.1 ; python_full_version < '3.10' \ + --hash=sha256:6457aaa33a5d474aca678b8ead9b3dc298e89c68e67012e73146ea6fd54babf1 \ + --hash=sha256:88f0cb406cb363b077d176b51c476f62d60604d68a8dcdf4832e080441301a87 + # via rules-python-sphinxdocs-dev (dev/pyproject.toml) +myst-parser==4.0.1 ; python_full_version == '3.10.*' \ + --hash=sha256:5cfea715e4f3574138aecbf7d54132296bfd72bb614d31168f48c477a830a7c4 \ + --hash=sha256:9134e88959ec3b5780aedf8a99680ea242869d012e8821db3126d427edc9c95d + # via rules-python-sphinxdocs-dev (dev/pyproject.toml) +myst-parser==5.1.0 ; python_full_version >= '3.11' \ + --hash=sha256:9c91c52b3cdb4d94a6506e4fab4e2f296c7623a0da0dcbe6de1565c3dad67a8a \ + --hash=sha256:ab69322dc6719dcc7f296479dbb70181b66df6ed315064f92dbc85c0e1bf2f02 + # via rules-python-sphinxdocs-dev (dev/pyproject.toml) +packaging==26.2 \ + --hash=sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e \ + --hash=sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661 + # via + # readthedocs-sphinx-ext + # sphinx +pygments==2.20.0 \ + --hash=sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f \ + --hash=sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176 + # via sphinx +pyyaml==6.0.3 \ + --hash=sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c \ + --hash=sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a \ + --hash=sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3 \ + --hash=sha256:02ea2dfa234451bbb8772601d7b8e426c2bfa197136796224e50e35a78777956 \ + --hash=sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6 \ + --hash=sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c \ + --hash=sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65 \ + --hash=sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a \ + --hash=sha256:1ebe39cb5fc479422b83de611d14e2c0d3bb2a18bbcb01f229ab3cfbd8fee7a0 \ + --hash=sha256:214ed4befebe12df36bcc8bc2b64b396ca31be9304b8f59e25c11cf94a4c033b \ + --hash=sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1 \ + --hash=sha256:22ba7cfcad58ef3ecddc7ed1db3409af68d023b7f940da23c6c2a1890976eda6 \ + --hash=sha256:27c0abcb4a5dac13684a37f76e701e054692a9b2d3064b70f5e4eb54810553d7 \ + --hash=sha256:28c8d926f98f432f88adc23edf2e6d4921ac26fb084b028c733d01868d19007e \ + --hash=sha256:2e71d11abed7344e42a8849600193d15b6def118602c4c176f748e4583246007 \ + --hash=sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310 \ + --hash=sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4 \ + --hash=sha256:3c5677e12444c15717b902a5798264fa7909e41153cdf9ef7ad571b704a63dd9 \ + --hash=sha256:3ff07ec89bae51176c0549bc4c63aa6202991da2d9a6129d7aef7f1407d3f295 \ + --hash=sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea \ + --hash=sha256:418cf3f2111bc80e0933b2cd8cd04f286338bb88bdc7bc8e6dd775ebde60b5e0 \ + --hash=sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e \ + --hash=sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac \ + --hash=sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9 \ + --hash=sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7 \ + --hash=sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35 \ + --hash=sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb \ + --hash=sha256:5cf4e27da7e3fbed4d6c3d8e797387aaad68102272f8f9752883bc32d61cb87b \ + --hash=sha256:5e0b74767e5f8c593e8c9b5912019159ed0533c70051e9cce3e8b6aa699fcd69 \ + --hash=sha256:5ed875a24292240029e4483f9d4a4b8a1ae08843b9c54f43fcc11e404532a8a5 \ + --hash=sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b \ + --hash=sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c \ + --hash=sha256:6344df0d5755a2c9a276d4473ae6b90647e216ab4757f8426893b5dd2ac3f369 \ + --hash=sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd \ + --hash=sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824 \ + --hash=sha256:66291b10affd76d76f54fad28e22e51719ef9ba22b29e1d7d03d6777a9174198 \ + --hash=sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065 \ + --hash=sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c \ + --hash=sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c \ + --hash=sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764 \ + --hash=sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196 \ + --hash=sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b \ + --hash=sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00 \ + --hash=sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac \ + --hash=sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8 \ + --hash=sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e \ + --hash=sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28 \ + --hash=sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3 \ + --hash=sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5 \ + --hash=sha256:9c57bb8c96f6d1808c030b1687b9b5fb476abaa47f0db9c0101f5e9f394e97f4 \ + --hash=sha256:9c7708761fccb9397fe64bbc0395abcae8c4bf7b0eac081e12b809bf47700d0b \ + --hash=sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf \ + --hash=sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5 \ + --hash=sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702 \ + --hash=sha256:b30236e45cf30d2b8e7b3e85881719e98507abed1011bf463a8fa23e9c3e98a8 \ + --hash=sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788 \ + --hash=sha256:b865addae83924361678b652338317d1bd7e79b1f4596f96b96c77a5a34b34da \ + --hash=sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d \ + --hash=sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc \ + --hash=sha256:bdb2c67c6c1390b63c6ff89f210c8fd09d9a1217a465701eac7316313c915e4c \ + --hash=sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba \ + --hash=sha256:c2514fceb77bc5e7a2f7adfaa1feb2fb311607c9cb518dbc378688ec73d8292f \ + --hash=sha256:c3355370a2c156cffb25e876646f149d5d68f5e0a3ce86a5084dd0b64a994917 \ + --hash=sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5 \ + --hash=sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26 \ + --hash=sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f \ + --hash=sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b \ + --hash=sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be \ + --hash=sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c \ + --hash=sha256:efd7b85f94a6f21e4932043973a7ba2613b059c4a000551892ac9f1d11f5baf3 \ + --hash=sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6 \ + --hash=sha256:fa160448684b4e94d80416c0fa4aac48967a969efe22931448d853ada8baf926 \ + --hash=sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0 + # via myst-parser +readthedocs-sphinx-ext==2.2.5 \ + --hash=sha256:ee5fd5b99db9f0c180b2396cbce528aa36671951b9526bb0272dbfce5517bd27 \ + --hash=sha256:f8c56184ea011c972dd45a90122568587cc85b0127bc9cf064d17c68bc809daa + # via rules-python-sphinxdocs-dev (dev/pyproject.toml) +requests==2.32.5 ; python_full_version < '3.10' \ + --hash=sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6 \ + --hash=sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf + # via + # readthedocs-sphinx-ext + # sphinx +requests==2.34.2 ; python_full_version >= '3.10' \ + --hash=sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0 \ + --hash=sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed + # via + # readthedocs-sphinx-ext + # sphinx +roman-numerals==4.1.0 ; python_full_version >= '3.11' \ + --hash=sha256:1af8b147eb1405d5839e78aeb93131690495fe9da5c91856cb33ad55a7f1e5b2 \ + --hash=sha256:647ba99caddc2cc1e55a51e4360689115551bf4476d90e8162cf8c345fe233c7 + # via sphinx +snowballstemmer==3.1.1 \ + --hash=sha256:7e207fa178741da09cdee59d3ecec3827ad5f92b1fc5c9ff3755b639f71f5752 \ + --hash=sha256:e07bbc54a0d798fe6010a12398422e62a8bfbba95c394fd0956ef58cb4d3e260 + # via sphinx +sphinx==7.4.7 ; python_full_version < '3.10' \ + --hash=sha256:242f92a7ea7e6c5b406fdc2615413890ba9f699114a9c09192d7dfead2ee9cfe \ + --hash=sha256:c2419e2135d11f1951cd994d6eb18a1835bd8fdd8429f9ca375dc1f3281bd239 + # via + # rules-python-sphinxdocs-dev (dev/pyproject.toml) + # myst-parser + # sphinx-reredirects + # sphinx-rtd-theme + # sphinxcontrib-jquery +sphinx==8.1.3 ; python_full_version == '3.10.*' \ + --hash=sha256:09719015511837b76bf6e03e42eb7595ac8c2e41eeb9c29c5b755c6b677992a2 \ + --hash=sha256:43c1911eecb0d3e161ad78611bc905d1ad0e523e4ddc202a58a821773dc4c927 + # via + # rules-python-sphinxdocs-dev (dev/pyproject.toml) + # myst-parser + # sphinx-reredirects + # sphinx-rtd-theme + # sphinxcontrib-jquery +sphinx==9.0.4 ; python_full_version == '3.11.*' \ + --hash=sha256:594ef59d042972abbc581d8baa577404abe4e6c3b04ef61bd7fc2acbd51f3fa3 \ + --hash=sha256:5bebc595a5e943ea248b99c13814c1c5e10b3ece718976824ffa7959ff95fffb + # via + # rules-python-sphinxdocs-dev (dev/pyproject.toml) + # myst-parser + # sphinx-reredirects + # sphinx-rtd-theme + # sphinxcontrib-jquery +sphinx==9.1.0 ; python_full_version >= '3.12' \ + --hash=sha256:7741722357dd75f8190766926071fed3bdc211c74dd2d7d4df5404da95930ddb \ + --hash=sha256:c84fdd4e782504495fe4f2c0b3413d6c2bf388589bb352d439b2a3bb99991978 + # via + # rules-python-sphinxdocs-dev (dev/pyproject.toml) + # myst-parser + # sphinx-reredirects + # sphinx-rtd-theme + # sphinxcontrib-jquery +sphinx-autodoc2==0.5.0 \ + --hash=sha256:7d76044aa81d6af74447080182b6868c7eb066874edc835e8ddf810735b6565a \ + --hash=sha256:e867013b1512f9d6d7e6f6799f8b537d6884462acd118ef361f3f619a60b5c9e + # via rules-python-sphinxdocs-dev (dev/pyproject.toml) +sphinx-reredirects==0.1.6 ; python_full_version < '3.11' \ + --hash=sha256:c491cba545f67be9697508727818d8626626366245ae64456fe29f37e9bbea64 \ + --hash=sha256:efd50c766fbc5bf40cd5148e10c00f2c00d143027de5c5e48beece93cc40eeea + # via rules-python-sphinxdocs-dev (dev/pyproject.toml) +sphinx-reredirects==1.1.0 ; python_full_version >= '3.11' \ + --hash=sha256:4b5692273c72cd2d4d917f4c6f87d5919e4d6114a752d4be033f7f5f6310efd9 \ + --hash=sha256:fb9b195335ab14b43f8273287d0c7eeb637ba6c56c66581c11b47202f6718b29 + # via rules-python-sphinxdocs-dev (dev/pyproject.toml) +sphinx-rtd-theme==3.1.0 \ + --hash=sha256:1785824ae8e6632060490f67cf3a72d404a85d2d9fc26bce3619944de5682b89 \ + --hash=sha256:b44276f2c276e909239a4f6c955aa667aaafeb78597923b1c60babc76db78e4c + # via rules-python-sphinxdocs-dev (dev/pyproject.toml) +sphinxcontrib-applehelp==2.0.0 \ + --hash=sha256:2f29ef331735ce958efa4734873f084941970894c6090408b079c61b2e1c06d1 \ + --hash=sha256:4cd3f0ec4ac5dd9c17ec65e9ab272c9b867ea77425228e68ecf08d6b28ddbdb5 + # via sphinx +sphinxcontrib-devhelp==2.0.0 \ + --hash=sha256:411f5d96d445d1d73bb5d52133377b4248ec79db5c793ce7dbe59e074b4dd1ad \ + --hash=sha256:aefb8b83854e4b0998877524d1029fd3e6879210422ee3780459e28a1f03a8a2 + # via sphinx +sphinxcontrib-htmlhelp==2.1.0 \ + --hash=sha256:166759820b47002d22914d64a075ce08f4c46818e17cfc9470a9786b759b19f8 \ + --hash=sha256:c9e2916ace8aad64cc13a0d233ee22317f2b9025b9cf3295249fa985cc7082e9 + # via sphinx +sphinxcontrib-jquery==4.1 \ + --hash=sha256:1620739f04e36a2c779f1a131a2dfd49b2fd07351bf1968ced074365933abc7a \ + --hash=sha256:f936030d7d0147dd026a4f2b5a57343d233f1fc7b363f68b3d4f1cb0993878ae + # via sphinx-rtd-theme +sphinxcontrib-jsmath==1.0.1 \ + --hash=sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178 \ + --hash=sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8 + # via sphinx +sphinxcontrib-qthelp==2.0.0 \ + --hash=sha256:4fe7d0ac8fc171045be623aba3e2a8f613f8682731f9153bb2e40ece16b9bbab \ + --hash=sha256:b18a828cdba941ccd6ee8445dbe72ffa3ef8cbe7505d8cd1fa0d42d3f2d5f3eb + # via sphinx +sphinxcontrib-serializinghtml==2.0.0 \ + --hash=sha256:6e2cb0eef194e10c27ec0023bfeb25badbbb5868244cf5bc5bdc04e4464bf331 \ + --hash=sha256:e9d912827f872c029017a53f0ef2180b327c3f7fd23c87229f7a8e8b70031d4d + # via sphinx +tomli==2.4.1 ; python_full_version < '3.11' \ + --hash=sha256:01f520d4f53ef97964a240a035ec2a869fe1a37dde002b57ebc4417a27ccd853 \ + --hash=sha256:0d85819802132122da43cb86656f8d1f8c6587d54ae7dcaf30e90533028b49fe \ + --hash=sha256:136443dbd7e1dee43c68ac2694fde36b2849865fa258d39bf822c10e8068eac5 \ + --hash=sha256:1d8591993e228b0c930c4bb0db464bdad97b3289fb981255d6c9a41aedc84b2d \ + --hash=sha256:2190f2e9dd7508d2a90ded5ed369255980a1bcdd58e52f7fe24b8162bf9fedbd \ + --hash=sha256:2c1c351919aca02858f740c6d33adea0c5deea37f9ecca1cc1ef9e884a619d26 \ + --hash=sha256:36d2bd2ad5fb9eaddba5226aa02c8ec3fa4f192631e347b3ed28186d43be6b54 \ + --hash=sha256:3d48a93ee1c9b79c04bb38772ee1b64dcf18ff43085896ea460ca8dec96f35f6 \ + --hash=sha256:47149d5bd38761ac8be13a84864bf0b7b70bc051806bc3669ab1cbc56216b23c \ + --hash=sha256:4ab97e64ccda8756376892c53a72bd1f964e519c77236368527f758fbc36a53a \ + --hash=sha256:4b605484e43cdc43f0954ddae319fb75f04cc10dd80d830540060ee7cd0243cd \ + --hash=sha256:504aa796fe0569bb43171066009ead363de03675276d2d121ac1a4572397870f \ + --hash=sha256:51529d40e3ca50046d7606fa99ce3956a617f9b36380da3b7f0dd3dd28e68cb5 \ + --hash=sha256:52c8ef851d9a240f11a88c003eacb03c31fc1c9c4ec64a99a0f922b93874fda9 \ + --hash=sha256:559db847dc486944896521f68d8190be1c9e719fced785720d2216fe7022b662 \ + --hash=sha256:5a881ab208c0baf688221f8cecc5401bd291d67e38a1ac884d6736cbcd8247e9 \ + --hash=sha256:5cb41aa38891e073ee49d55fbc7839cfdb2bc0e600add13874d048c94aadddd1 \ + --hash=sha256:5e262d41726bc187e69af7825504c933b6794dc3fbd5945e41a79bb14c31f585 \ + --hash=sha256:5ee18d9ebdb417e384b58fe414e8d6af9f4e7a0ae761519fb50f721de398dd4e \ + --hash=sha256:7008df2e7655c495dd12d2a4ad038ff878d4ca4b81fccaf82b714e07eae4402c \ + --hash=sha256:734e20b57ba95624ecf1841e72b53f6e186355e216e5412de414e3c51e5e3c41 \ + --hash=sha256:7c7e1a961a0b2f2472c1ac5b69affa0ae1132c39adcb67aba98568702b9cc23f \ + --hash=sha256:7f86fd587c4ed9dd76f318225e7d9b29cfc5a9d43de44e5754db8d1128487085 \ + --hash=sha256:7f94b27a62cfad8496c8d2513e1a222dd446f095fca8987fceef261225538a15 \ + --hash=sha256:88dceee75c2c63af144e456745e10101eb67361050196b0b6af5d717254dddf7 \ + --hash=sha256:8a650c2dbafa08d42e51ba0b62740dae4ecb9338eefa093aa5c78ceb546fcd5c \ + --hash=sha256:8d65a2fbf9d2f8352685bc1364177ee3923d6baf5e7f43ea4959d7d8bc326a36 \ + --hash=sha256:96481a5786729fd470164b47cdb3e0e58062a496f455ee41b4403be77cb5a076 \ + --hash=sha256:a120733b01c45e9a0c34aeef92bf0cf1d56cfe81ed9d47d562f9ed591a9828ac \ + --hash=sha256:b1d22e6e9387bf4739fbe23bfa80e93f6b0373a7f1b96c6227c32bef95a4d7a8 \ + --hash=sha256:b8c198f8c1805dc42708689ed6864951fd2494f924149d3e4bce7710f8eb5232 \ + --hash=sha256:c2541745709bad0264b7d4705ad453b76ccd191e64aa6f0fc66b69a293a45ece \ + --hash=sha256:c742f741d58a28940ce01d58f0ab2ea3ced8b12402f162f4d534dfe18ba1cd6a \ + --hash=sha256:c7f2c7f2b9ca6bdeef8f0fa897f8e05085923eb091721675170254cbc5b02897 \ + --hash=sha256:d312ef37c91508b0ab2cee7da26ec0b3ed2f03ce12bd87a588d771ae15dcf82d \ + --hash=sha256:d4d8fe59808a54658fcc0160ecfb1b30f9089906c50b23bcb4c69eddc19ec2b4 \ + --hash=sha256:da25dc3563bff5965356133435b757a795a17b17d01dbc0f42fb32447ddfd917 \ + --hash=sha256:eab21f45c7f66c13f2a9e0e1535309cee140182a9cdae1e041d02e47291e8396 \ + --hash=sha256:eb0dc4e38e6a1fd579e5d50369aa2e10acfc9cace504579b2faabb478e76941a \ + --hash=sha256:ec9bfaf3ad2df51ace80688143a6a4ebc09a248f6ff781a9945e51937008fcbc \ + --hash=sha256:ede3e6487c5ef5d28634ba3f31f989030ad6af71edfb0055cbbd14189ff240ba \ + --hash=sha256:f3c6818a1a86dd6dca7ddcaaf76947d5ba31aecc28cb1b67009a5877c9a64f3f \ + --hash=sha256:f758f1b9299d059cc3f6546ae2af89670cb1c4d48ea29c3cacc4fe7de3058257 \ + --hash=sha256:f8f0fc26ec2cc2b965b7a3b87cd19c5c6b8c5e5f436b984e85f486d652285c30 \ + --hash=sha256:fd0409a3653af6c147209d267a0e4243f0ae46b011aa978b1080359fddc9b6cf \ + --hash=sha256:ff18e6a727ee0ab0388507b89d1bc6a22b138d1e2fa56d1ad494586d61d2eae9 \ + --hash=sha256:ff2983983d34813c1aeb0fa89091e76c3a22889ee83ab27c5eeb45100560c049 + # via + # sphinx + # sphinx-autodoc2 +typing-extensions==4.16.0 \ + --hash=sha256:481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8 \ + --hash=sha256:dc983d19a509c94dba722ee6abd33940f7c05a89e243c47e907eb4db6f1a43e5 + # via + # rules-python-sphinxdocs-dev (dev/pyproject.toml) + # astroid + # sphinx-autodoc2 +urllib3==2.6.3 ; python_full_version < '3.10' \ + --hash=sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed \ + --hash=sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4 + # via requests +urllib3==2.7.0 ; python_full_version >= '3.10' \ + --hash=sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c \ + --hash=sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897 + # via requests +zipp==3.23.1 ; python_full_version < '3.10' \ + --hash=sha256:0b3596c50a5c700c9cb40ba8d86d9f2cc4807e9bedb06bcdf7fac85633e444dc \ + --hash=sha256:32120e378d32cd9714ad503c1d024619063ec28aad2248dc6672ad13edfa5110 + # via importlib-metadata diff --git a/sphinxdocs/dev/uv.lock b/sphinxdocs/dev/uv.lock new file mode 100644 index 0000000000..39ce81556d --- /dev/null +++ b/sphinxdocs/dev/uv.lock @@ -0,0 +1,1073 @@ +version = 1 +revision = 3 +requires-python = ">=3.9" +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", + "python_full_version < '3.10'", +] + +[[package]] +name = "absl-py" +version = "2.3.1" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +sdist = { url = "https://files.pythonhosted.org/packages/10/2a/c93173ffa1b39c1d0395b7e842bbdc62e556ca9d8d3b5572926f3e4ca752/absl_py-2.3.1.tar.gz", hash = "sha256:a97820526f7fbfd2ec1bce83f3f25e3a14840dac0d8e02a0b71cd75db3f77fc9", size = 116588, upload-time = "2025-07-03T09:31:44.05Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/8f/aa/ba0014cc4659328dc818a28827be78e6d97312ab0cb98105a770924dc11e/absl_py-2.3.1-py3-none-any.whl", hash = "sha256:eeecf07f0c2a93ace0772c92e596ace6d3d3996c042b2128459aaae2a76de11d", size = 135811, upload-time = "2025-07-03T09:31:42.253Z" }, +] + +[[package]] +name = "absl-py" +version = "2.5.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +sdist = { url = "https://files.pythonhosted.org/packages/d0/4f/d79676ab82f2e42fc3611618139f13a9c4c31d0cff4b486982047679a802/absl_py-2.5.0.tar.gz", hash = "sha256:0c996f25c0490700fadabe6351630f6111534fa0ae252cc6d2014ea3b141135f", size = 118119, upload-time = "2026-07-03T10:57:48.157Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/58/0a/a10b45aab35b175aded078a462dc8d0c698f5b13946e7cb0869097b78bb6/absl_py-2.5.0-py3-none-any.whl", hash = "sha256:0f17b89f2a4eaaedc4f28c622998aa690564b3012a396a4ffad0821007fe03ba", size = 137410, upload-time = "2026-07-03T10:57:46.735Z" }, +] + +[[package]] +name = "alabaster" +version = "0.7.16" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +sdist = { url = "https://files.pythonhosted.org/packages/c9/3e/13dd8e5ed9094e734ac430b5d0eb4f2bb001708a8b7856cbf8e084e001ba/alabaster-0.7.16.tar.gz", hash = "sha256:75a8b99c28a5dad50dd7f8ccdd447a121ddb3892da9e53d1ca5cca3106d58d65", size = 23776, upload-time = "2024-01-10T00:56:10.189Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/32/34/d4e1c02d3bee589efb5dfa17f88ea08bdb3e3eac12bc475462aec52ed223/alabaster-0.7.16-py3-none-any.whl", hash = "sha256:b46733c07dce03ae4e150330b975c75737fa60f0a7c591b6c8bf4928a28e2c92", size = 13511, upload-time = "2024-01-10T00:56:08.388Z" }, +] + +[[package]] +name = "alabaster" +version = "1.0.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +sdist = { url = "https://files.pythonhosted.org/packages/a6/f8/d9c74d0daf3f742840fd818d69cfae176fa332022fd44e3469487d5a9420/alabaster-1.0.0.tar.gz", hash = "sha256:c00dca57bca26fa62a6d7d0a9fcce65f3e026e9bfe33e9c538fd3fbb2144fd9e", size = 24210, upload-time = "2024-07-26T18:15:03.762Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7e/b3/6b4067be973ae96ba0d615946e314c5ae35f9f993eca561b356540bb0c2b/alabaster-1.0.0-py3-none-any.whl", hash = "sha256:fc6786402dc3fcb2de3cabd5fe455a2db534b371124f1f21de8731783dec828b", size = 13929, upload-time = "2024-07-26T18:15:02.05Z" }, +] + +[[package]] +name = "astroid" +version = "3.3.11" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions", marker = "python_full_version < '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/18/74/dfb75f9ccd592bbedb175d4a32fc643cf569d7c218508bfbd6ea7ef9c091/astroid-3.3.11.tar.gz", hash = "sha256:1e5a5011af2920c7c67a53f65d536d65bfa7116feeaf2354d8b94f29573bb0ce", size = 400439, upload-time = "2025-07-13T18:04:23.177Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/af/0f/3b8fdc946b4d9cc8cc1e8af42c4e409468c84441b933d037e101b3d72d86/astroid-3.3.11-py3-none-any.whl", hash = "sha256:54c760ae8322ece1abd213057c4b5bba7c49818853fc901ef09719a60dbf9dec", size = 275612, upload-time = "2025-07-13T18:04:21.07Z" }, +] + +[[package]] +name = "babel" +version = "2.18.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/7d/b2/51899539b6ceeeb420d40ed3cd4b7a40519404f9baf3d4ac99dc413a834b/babel-2.18.0.tar.gz", hash = "sha256:b80b99a14bd085fcacfa15c9165f651fbb3406e66cc603abf11c5750937c992d", size = 9959554, upload-time = "2026-02-01T12:30:56.078Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/77/f5/21d2de20e8b8b0408f0681956ca2c69f1320a3848ac50e6e7f39c6159675/babel-2.18.0-py3-none-any.whl", hash = "sha256:e2b422b277c2b9a9630c1d7903c2a00d0830c409c59ac8cae9081c92f1aeba35", size = 10196845, upload-time = "2026-02-01T12:30:53.445Z" }, +] + +[[package]] +name = "certifi" +version = "2026.7.22" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/a3/c2/24167ea9858356b47a87a50d39908bfdb72ceeefe0041586e704e5376b3a/certifi-2026.7.22.tar.gz", hash = "sha256:741e2c3b351ddf169a738da9f2c048608ff7f2c5cc02f1ebc6b118bb090d5d55", size = 138112, upload-time = "2026-07-22T03:35:12.644Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0b/a7/71ac2cff56fec219ed242bb11b8efb69fcc4bec75db06fb7bfe35de520e6/certifi-2026.7.22-py3-none-any.whl", hash = "sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775", size = 136983, upload-time = "2026-07-22T03:35:11.276Z" }, +] + +[[package]] +name = "charset-normalizer" +version = "3.4.9" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/bd/2a/23f34ec9d04624958e137efdc394888716353190e75f25dd22c7a2c7a8aa/charset_normalizer-3.4.9.tar.gz", hash = "sha256:673611bbd43f0810bec0b0f028ddeaaa501190339cac411f347ac76917c3ae7b", size = 152439, upload-time = "2026-07-07T14:34:58.454Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ad/81/8e983840c6e5b93b33c2ba81aa3d52c2e42f0e9a690ce7607a2e61da4a5c/charset_normalizer-3.4.9-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:cd6280cf040f233bd7d3407b743b4b4c74f70e8e1c4199cb112a62c941c0772a", size = 322240, upload-time = "2026-07-07T14:32:36.236Z" }, + { url = "https://files.pythonhosted.org/packages/de/d1/b4319dc3229d8272fba305e206fc0a148e2de8d4087917ce62ae6382f359/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:aa99adc8f081b475a12843953db36831eaf83ec33eb46a90629ca6a5de45a616", size = 216475, upload-time = "2026-07-07T14:32:38.142Z" }, + { url = "https://files.pythonhosted.org/packages/80/33/6c99c1b3e6b8bf730e1bc809b9a2608f224145069114c479a2e9e1494346/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:c1225416b463483160e4af85d5fc3a9690ccb53fd4b1865a6437825f5ede3209", size = 238670, upload-time = "2026-07-07T14:32:39.658Z" }, + { url = "https://files.pythonhosted.org/packages/7f/f4/ffbb83546e1f198ecc70ecd372b65cf2b50f9068b380abd67640f17a8e18/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:16d10d789dd9bcca1173c95af82c58433122564b7bc39385124be735a35cbe99", size = 233476, upload-time = "2026-07-07T14:32:41.155Z" }, + { url = "https://files.pythonhosted.org/packages/e8/5f/b98b8da398637b551e427e7be922bdec19177dc54d6811dcdaa503f23aac/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9bb41182d93ea91f60b4bc8fbf4c820c69ef8a12ab2d917f3f1834f1acad07e8", size = 223817, upload-time = "2026-07-07T14:32:42.592Z" }, + { url = "https://files.pythonhosted.org/packages/36/31/a276bb2e66243072a3fd06fdcab9cbb61a305b02143d70d2bda21d888fa8/charset_normalizer-3.4.9-cp310-cp310-manylinux_2_31_armv7l.whl", hash = "sha256:bcf74c1df76758a395bf0af608c04c82257523f55c9868b334f06270d0f2112b", size = 207974, upload-time = "2026-07-07T14:32:44.258Z" }, + { url = "https://files.pythonhosted.org/packages/5e/be/7ee4453d7e88dfbc4104ccd34900b9f2c7c17dac22881865fe0e82424a25/charset_normalizer-3.4.9-cp310-cp310-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b5314963fce9b0b12743891de876e724997864ee22aa496f903f426c7e2fa5b2", size = 221655, upload-time = "2026-07-07T14:32:45.64Z" }, + { url = "https://files.pythonhosted.org/packages/1d/85/181c652953eb5276d198f375b1dd641047392050098100a3a02d6534f657/charset_normalizer-3.4.9-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:e9701d0049d92c16703a42771b98d560b95248949f23f8cf7b4eddd201814fb9", size = 219229, upload-time = "2026-07-07T14:32:47.376Z" }, + { url = "https://files.pythonhosted.org/packages/0c/e7/aaf6da33fc9f4691cda8f7efbc9f69179d3d39ec8a4799baf273ee1d8db0/charset_normalizer-3.4.9-cp310-cp310-musllinux_1_2_armv7l.whl", hash = "sha256:65a7ff3f705e57d392f7261b6d0550fe137c3019477431f1c355e0db0a7d3e15", size = 209704, upload-time = "2026-07-07T14:32:48.855Z" }, + { url = "https://files.pythonhosted.org/packages/63/01/f2fb3bd3a73be48b173ee0c6aa8d2497af97d5663a8c4c4b491de4c62f7a/charset_normalizer-3.4.9-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:79580094b00d1789d1f93ea55bc43cb2f611910c72235b7657f3482ddcc1b22d", size = 226243, upload-time = "2026-07-07T14:32:50.239Z" }, + { url = "https://files.pythonhosted.org/packages/c4/02/c57a22739fe05246b0b5783b3bfb6afaac4eebb46f3ececdfb2f048f780e/charset_normalizer-3.4.9-cp310-cp310-win32.whl", hash = "sha256:432786d3561e69aeeae6c7e8648964ce0ad05736120135601f87ac26b9c83381", size = 150935, upload-time = "2026-07-07T14:32:51.676Z" }, + { url = "https://files.pythonhosted.org/packages/37/8d/ca39a7559a4797505530d084fd3a49a2c959efbbbff146302fb7be4e3b35/charset_normalizer-3.4.9-cp310-cp310-win_amd64.whl", hash = "sha256:8c041122946b7ba21bb32c45b1aa57b1be35527690aeb3c5c234521085632eee", size = 162314, upload-time = "2026-07-07T14:32:53.193Z" }, + { url = "https://files.pythonhosted.org/packages/01/da/a44bd7a13d426e69e4894557106cd58669097bfad4a8681123b618fbfc5d/charset_normalizer-3.4.9-cp310-cp310-win_arm64.whl", hash = "sha256:375b83ed0aecfce76c16d198fbc21f3b11b337d68662bea0a995046682a11419", size = 153075, upload-time = "2026-07-07T14:32:54.554Z" }, + { url = "https://files.pythonhosted.org/packages/0b/e3/85ec501f206fb049259288c1f3506e53876937fb00edb47009348e66756b/charset_normalizer-3.4.9-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:0e94703ec9684807f20cfb5eed95c70f67f2a8f21ad620146d7b5a13677b93e5", size = 317075, upload-time = "2026-07-07T14:32:56.021Z" }, + { url = "https://files.pythonhosted.org/packages/c3/69/2a5385192e67175f7d8bd5ce4f57c24bc956439adeae5c13a99aa28a53d1/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:2a441ea71902098ffe78c5abe6c494f44160b4af614ed16c3d9a3b1d17fd8ee2", size = 213837, upload-time = "2026-07-07T14:32:57.78Z" }, + { url = "https://files.pythonhosted.org/packages/b3/46/03ddc7da576d814fe0a36dd1f0fd3258e95404b4b2e3c026b7923d7e133f/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:304b13570067b2547562e308af560b3963857b1fa90bd6afd978130130fe2d6a", size = 235503, upload-time = "2026-07-07T14:32:59.205Z" }, + { url = "https://files.pythonhosted.org/packages/4e/6e/de0229a7ef40f6f9d28a837eebf4ec47bdca5dab4e900c84f22919af636a/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:4773092f8019072343a7447203308b176e10199920eb02d6195e81bbb3274c29", size = 229944, upload-time = "2026-07-07T14:33:00.803Z" }, + { url = "https://files.pythonhosted.org/packages/a5/34/49b9060e8418b14fb5cba9cf6bfb383111e2538a03a1fb18e66a95aeb3d5/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:04ce310cb89c15df659582aee80a0603788732a5e017d5bd5c81158106ce249c", size = 221276, upload-time = "2026-07-07T14:33:02.199Z" }, + { url = "https://files.pythonhosted.org/packages/44/95/80282cce0fae9c3061203d723ee87da996aed79679e65d8935050ee7ca1f/charset_normalizer-3.4.9-cp311-cp311-manylinux_2_31_armv7l.whl", hash = "sha256:c0323c9daef75ef2e5083624b4585018a0c9d5e3b40f607eed81a311270b934b", size = 205260, upload-time = "2026-07-07T14:33:03.698Z" }, + { url = "https://files.pythonhosted.org/packages/0c/74/2f62c8821b969ea3bd67cc2e6976834f48ca5d12664d2559ebcd9bcfbed7/charset_normalizer-3.4.9-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:871ff67ea1aad4dfd91736464934d56b32dac49f9fbe16cddba36198a7b3a0db", size = 217786, upload-time = "2026-07-07T14:33:05.12Z" }, + { url = "https://files.pythonhosted.org/packages/d9/8d/feabb82cb49fcad14515b1d7d1ca4787b0da7fc723a212bf89bc9e0fac52/charset_normalizer-3.4.9-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:67830fc78e67501f47bb950471b2dcb9b35b140084429318e862895a8e89c993", size = 216798, upload-time = "2026-07-07T14:33:06.629Z" }, + { url = "https://files.pythonhosted.org/packages/a5/ff/c946d63bc3786d5b84d960b0f7ab7e25b828486a946b5aa997625bcaf6a6/charset_normalizer-3.4.9-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:3d92613ec25e43b05f042302531ec0f00b8445190e43325880cbd6ab7c2581da", size = 206429, upload-time = "2026-07-07T14:33:08.006Z" }, + { url = "https://files.pythonhosted.org/packages/af/ba/5e5007c370702f85d2ef75791fac7943ed41e080364a673b20142e430e3e/charset_normalizer-3.4.9-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:280081916dc341820640489a66e4696049401ef1cf6dd672f672e70ad915aca3", size = 223066, upload-time = "2026-07-07T14:33:09.783Z" }, + { url = "https://files.pythonhosted.org/packages/83/d5/9096aa3cf532dfad237861544eb47a0f20d5adbf1039760fed8eaae935d9/charset_normalizer-3.4.9-cp311-cp311-win32.whl", hash = "sha256:ac351b3b8014eead140e77e9717e2992c6bbe30b63bc3422422eb84865412e3d", size = 150456, upload-time = "2026-07-07T14:33:11.217Z" }, + { url = "https://files.pythonhosted.org/packages/ed/a1/e29995109e455dc8eff8d0fac6ae509be39561318a7cfeac5d33ad029213/charset_normalizer-3.4.9-cp311-cp311-win_amd64.whl", hash = "sha256:6366a16e1a25018694d6a5d784d09b046edc9eac40ea2b54065c3052672516a1", size = 161410, upload-time = "2026-07-07T14:33:12.743Z" }, + { url = "https://files.pythonhosted.org/packages/4f/8d/1569f4d0032d6ba2a4fe4591c35bf87868c600c41a71eb5c2e1ffa8464c2/charset_normalizer-3.4.9-cp311-cp311-win_arm64.whl", hash = "sha256:1d22856ffbe153a602df38e4a5464f0b748a54002e0d69ac6d2ad0a197cc99ec", size = 152649, upload-time = "2026-07-07T14:33:14.173Z" }, + { url = "https://files.pythonhosted.org/packages/70/4a/ecbd131485c07fcdfad54e28946d513e3da22ef3b4bd854dcafae54ec739/charset_normalizer-3.4.9-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:45b0cc4e3556cd875e09102988d1ab8356c998b596c9fced84547c8138b487a0", size = 319300, upload-time = "2026-07-07T14:33:15.666Z" }, + { url = "https://files.pythonhosted.org/packages/ec/96/5d9364e3342d69f3a045e1777bc47c85c383e6e9466d561b33fdb419d1f9/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9b2aff1c7b3884512b9512c3eaadd9bab39fb45042ffaaa1dd08ff2b9f8109d9", size = 215802, upload-time = "2026-07-07T14:33:17.031Z" }, + { url = "https://files.pythonhosted.org/packages/4b/4c/5361f9aa7f2cb58d94f2ab831b3d493f69efb1d239654b4744e3c09527cb/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:9104ed0bd76a429d46f9ec0dbc9b08ad1d2dcdf2b00a5a0daa1c145329b35b44", size = 237171, upload-time = "2026-07-07T14:33:18.576Z" }, + { url = "https://files.pythonhosted.org/packages/50/78/ce342ca4ff30b2eb49fe6d9578df85974f90c67d294113e94efdd9664cbd/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:7b86a2b16095d250c6f58b3d9b2eee6f4147754344f3dab0922f7c9bf7d226c9", size = 233075, upload-time = "2026-07-07T14:33:20.084Z" }, + { url = "https://files.pythonhosted.org/packages/01/c4/4fa4c8b3097a11f3c5f09a35b72ed6855fb1d332469504962ab7bafcc702/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5e226f6218febc71f6c1fc2fafb91c226f75bdc1d8fb12d66823716e891608fd", size = 224256, upload-time = "2026-07-07T14:33:21.747Z" }, + { url = "https://files.pythonhosted.org/packages/87/3a/ad914516df7e358a81aae018caa5e0470ba827fa6d763b1d2e87d920a5f6/charset_normalizer-3.4.9-cp312-cp312-manylinux_2_31_armv7l.whl", hash = "sha256:90c44bc373b7687f6948b693cceaea1348ae0975d7474746559494468e3c1d84", size = 208784, upload-time = "2026-07-07T14:33:23.313Z" }, + { url = "https://files.pythonhosted.org/packages/d7/74/3c12f9755717dfe5c5c87da63f35d765fa0c00382ec26bf23f7fae34f2ba/charset_normalizer-3.4.9-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:9cdef90ae47919cae358d8ab15797a800ed41da7aba5d72419fb510729e2ed4b", size = 219928, upload-time = "2026-07-07T14:33:24.814Z" }, + { url = "https://files.pythonhosted.org/packages/33/9a/895095b83e7907abd6d3d99aad3a38ad0d9686cc186cb0c94c24320fe63e/charset_normalizer-3.4.9-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:60f44ade2cf573dad7a277e6f8ca9a51a21dda572b13bd7d8539bb3cd5dbedde", size = 218489, upload-time = "2026-07-07T14:33:26.42Z" }, + { url = "https://files.pythonhosted.org/packages/a1/34/ef5c05f412f42520d7709b7d3784d19640839eb7366ded1755511585429f/charset_normalizer-3.4.9-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:a1786910334ed46ab1dd73222f2cd1e05c2c3bb39f6dddb4f8b36fc382058a39", size = 210267, upload-time = "2026-07-07T14:33:27.952Z" }, + { url = "https://files.pythonhosted.org/packages/83/dc/9b29fa4412b318bf3bfea985c35d67eb55e04b59a7c3f2237168b0e0be6f/charset_normalizer-3.4.9-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:03d07803992c6c7bbc976327f34b18b6160327fc81cb82c9d504720ac0be3b62", size = 226030, upload-time = "2026-07-07T14:33:29.397Z" }, + { url = "https://files.pythonhosted.org/packages/0e/42/6dbc00b8cd16011691203e33570fa42ed5746599a2e878112d16eab403a3/charset_normalizer-3.4.9-cp312-cp312-win32.whl", hash = "sha256:78841cccf1af7b40f6f716338d50c0902dbe88d9f800b3c973b7a9a0a693a642", size = 151185, upload-time = "2026-07-07T14:33:30.781Z" }, + { url = "https://files.pythonhosted.org/packages/80/cc/f920afd1a23c58ccd53c1d36085a71893a4737ff5e66e0371efab6809850/charset_normalizer-3.4.9-cp312-cp312-win_amd64.whl", hash = "sha256:4b3dac63058cc36820b0dd072f89898604e2d39686fe05321729d00d8ac185a0", size = 162557, upload-time = "2026-07-07T14:33:32.176Z" }, + { url = "https://files.pythonhosted.org/packages/f0/e6/0386d43a261ff4e4b30c5857af7df877254b46bec7b9d1b74b6bf969a90b/charset_normalizer-3.4.9-cp312-cp312-win_arm64.whl", hash = "sha256:78fa18e436a1a0e58dbd7e02fc4473f3f32cceb12df9dfca542d075961c307d2", size = 152665, upload-time = "2026-07-07T14:33:33.711Z" }, + { url = "https://files.pythonhosted.org/packages/b2/06/97ec2aeae780b31d742b6352218b43841a6871e2564578ca522dce4a45c3/charset_normalizer-3.4.9-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:440eede837960000d74978f0eba527be106b5b9aee0daf779d395276ed0b0614", size = 317688, upload-time = "2026-07-07T14:33:35.408Z" }, + { url = "https://files.pythonhosted.org/packages/d0/39/8ff066c672434225f8d25f8b739f992af250944392173dcc88362681c9bf/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:21e764fd1e70b6a3e205a0e46f3051701f98a8cb3fad66eeb80e48bb502f8698", size = 214982, upload-time = "2026-07-07T14:33:36.996Z" }, + { url = "https://files.pythonhosted.org/packages/92/8f/3a47a3667c83c2df9483d91644c6c107de3bf8874aa1793da9d3012eb986/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e4fd89cc178bced6ad29cb3e6dd4aa63fa5017c3524dbd0b25998fb64a87cc8b", size = 236460, upload-time = "2026-07-07T14:33:38.536Z" }, + { url = "https://files.pythonhosted.org/packages/f1/60/b22cdbee7e4013dab8b0d7647fc6181120fbbbc8f7025c226d15bd5a47fc/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:bd47ba7fc3ca94896759ea0109775132d3e7ab921fbf54038e1bab2e46c313c9", size = 232003, upload-time = "2026-07-07T14:33:40.059Z" }, + { url = "https://files.pythonhosted.org/packages/ea/f8/72eb13dcabe7257035cea8aefd922caad2f110d252bf9f67c4c2ca763aee/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:84fd18bcc17526fc2b3c1af7d2b9217d32c9c04448c16ec693b9b4f1985c3d33", size = 223149, upload-time = "2026-07-07T14:33:41.631Z" }, + { url = "https://files.pythonhosted.org/packages/b0/3e/faee8f9de92b14ee1198e9163252bb15efee7301b31256a3b6d9ebfdd0dd/charset_normalizer-3.4.9-cp313-cp313-manylinux_2_31_armv7l.whl", hash = "sha256:5b10cd92fc5c498b35a8635df6d5a100207f88b63a4dc1de7ef9a548e1e2cd63", size = 207901, upload-time = "2026-07-07T14:33:43.209Z" }, + { url = "https://files.pythonhosted.org/packages/3a/25/45f30093ae27dd7b92a793b61882a38685f993700113ca36e0c9c14965e1/charset_normalizer-3.4.9-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:a4fbdde9dd4a9ce5fd52c2b3a347bb50cc89483ef783f1cb00d408c13f7a96c0", size = 219176, upload-time = "2026-07-07T14:33:44.725Z" }, + { url = "https://files.pythonhosted.org/packages/48/18/c8f397329c35e32f6a837e488986f4ae03bd2abebc453b48714991630c2f/charset_normalizer-3.4.9-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:416c229f77e5ea25b3dfd4b582f8d73d7e43c22320302b9ab128a2d3a0b38efe", size = 217356, upload-time = "2026-07-07T14:33:46.192Z" }, + { url = "https://files.pythonhosted.org/packages/86/7e/5ce0bba863470fd1902d5e5843968951bddf38abe4742fc97116ef4598b3/charset_normalizer-3.4.9-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:75286256590a6320cf106a0d28970d3560aad9ee09aa7b34fb40524792436d35", size = 209614, upload-time = "2026-07-07T14:33:47.705Z" }, + { url = "https://files.pythonhosted.org/packages/6c/ef/2473d3c4d869155be4af1191111d59c4d5c4e0173026f7e85b176e23bf65/charset_normalizer-3.4.9-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:69b157c5d3292bcd443faca052f3096f637f1e074b98212a933c074ae23dc3b8", size = 224991, upload-time = "2026-07-07T14:33:49.238Z" }, + { url = "https://files.pythonhosted.org/packages/d0/a3/53ddae3db108a088156aa8ddfafd411ebbc1340f48c5573f697b27f69a39/charset_normalizer-3.4.9-cp313-cp313-win32.whl", hash = "sha256:51307f5c71007673a2bf8232ad973483d281e74cb99c8c5a990af1eefa6277d9", size = 150622, upload-time = "2026-07-07T14:33:50.711Z" }, + { url = "https://files.pythonhosted.org/packages/e8/ef/6953a77c7cf2c2ff9998e6f575ab3e380119f100223381565a4f94c1f836/charset_normalizer-3.4.9-cp313-cp313-win_amd64.whl", hash = "sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115", size = 161947, upload-time = "2026-07-07T14:33:52.197Z" }, + { url = "https://files.pythonhosted.org/packages/6e/fb/d560d1d1555debbfe7849d9cac6145c1b537709d79576bf22557ed803b82/charset_normalizer-3.4.9-cp313-cp313-win_arm64.whl", hash = "sha256:611057cc5d5c0afc743ba8be6bd828c17e0aaa8643f9d0a9b9bb7dea80eb8012", size = 152594, upload-time = "2026-07-07T14:33:53.486Z" }, + { url = "https://files.pythonhosted.org/packages/7e/8d/496817fa0944239ecae662dd57ea765cfeaec6a735f9f025d4b7b72e7143/charset_normalizer-3.4.9-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380", size = 317253, upload-time = "2026-07-07T14:33:54.994Z" }, + { url = "https://files.pythonhosted.org/packages/2b/f9/ef4a69ea338ad3c0deceea0f5f7d2380ae8b52132b06d652cb0d2cd86706/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8a79d9f4d8001473a30c163556b3c3bfebec837495a412dde78b51672f6134f9", size = 215898, upload-time = "2026-07-07T14:33:56.334Z" }, + { url = "https://files.pythonhosted.org/packages/8c/e7/5ddfd76fc061eb52de219658a4aa431cbacadf0a0219c8854f00da50d289/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:33bdcc2a32c0a0e861f60841a512c8acc658c87c2ac59d89e3a46dacf7d866e4", size = 236718, upload-time = "2026-07-07T14:33:57.9Z" }, + { url = "https://files.pythonhosted.org/packages/49/ba/768fa3f36048d81c477a0ce61f813bc1454d80917ccfe550abd9f44f5e24/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:f840ed6d8ecba8255df8c42b87fadeda98ddfc6eeec05e2dc66e26d46dd6f58a", size = 232519, upload-time = "2026-07-07T14:33:59.811Z" }, + { url = "https://files.pythonhosted.org/packages/f4/c4/b3e049d2aa3766180c78507110543d9d50894cc97f57de543f1be521dcdc/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c25fe15c70c59eb7c5ce8c06a1f3fa1da0ecc5ea1e7a5922c40fd2fa9b0d5046", size = 223143, upload-time = "2026-07-07T14:34:01.517Z" }, + { url = "https://files.pythonhosted.org/packages/19/79/55c32d06d76ae4feafe053f061f3e3ab70bcf19f4007797ce8c3efda7830/charset_normalizer-3.4.9-cp314-cp314-manylinux_2_31_armv7l.whl", hash = "sha256:f7fb7d750cfa0a070d2c24e831fd3481019a60dd317ea2b39acbcebc08b6ed81", size = 206742, upload-time = "2026-07-07T14:34:03.04Z" }, + { url = "https://files.pythonhosted.org/packages/10/e0/47c079dd82d217c807479cd59ffd30af56307ea31c108b75758970459ad3/charset_normalizer-3.4.9-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:4d1c96a7a18b9690a4d46df09e3e3382406ae3213727cd1019ebade1c4a81917", size = 219191, upload-time = "2026-07-07T14:34:04.657Z" }, + { url = "https://files.pythonhosted.org/packages/42/ab/b9bc2e77d6b44a7e46ef62ec5cac1c9a6ba7b9135a5d560f002696ec9995/charset_normalizer-3.4.9-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:a4cfde78a9f2880208d16a93b795726a3017d5977e08d1e162a7a31322479c41", size = 218328, upload-time = "2026-07-07T14:34:06.115Z" }, + { url = "https://files.pythonhosted.org/packages/f1/78/c9c71d599f5aa2d42bcdd35cbbd46d7f535351a57e40ff7d8e5a7e219401/charset_normalizer-3.4.9-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:d4d6fcde76f94f5cb9e43e9e9a61f16dacefd228cbbf6f1a09bd9b219a92f1a1", size = 207406, upload-time = "2026-07-07T14:34:07.554Z" }, + { url = "https://files.pythonhosted.org/packages/f6/39/c914445c321a845097ce4f6ac7de9a18228a77b766272125a1ce00d851eb/charset_normalizer-3.4.9-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:898f0e9068ca27d37f8e83a5b962821df851532e6c4a7d615c1c033f9da6eedf", size = 225157, upload-time = "2026-07-07T14:34:09.061Z" }, + { url = "https://files.pythonhosted.org/packages/9b/f2/c0d4b8508565a36bc5c624e88ed297f5b0b1095011034d7f5b83a69908b5/charset_normalizer-3.4.9-cp314-cp314-win32.whl", hash = "sha256:c1c948747b03be832dceed96ca815cef7360de9aa19d37c730f8e3f6101aca48", size = 151095, upload-time = "2026-07-07T14:34:10.901Z" }, + { url = "https://files.pythonhosted.org/packages/49/fd/a1d26144398c67486422a72bf5812cda22cb4ccfcd95a290fb41ceb4b8e2/charset_normalizer-3.4.9-cp314-cp314-win_amd64.whl", hash = "sha256:16b65ea0f2465b6fb52aa22de5eca612aa964ddfec00a912e26f4656cbef890b", size = 162796, upload-time = "2026-07-07T14:34:12.47Z" }, + { url = "https://files.pythonhosted.org/packages/20/95/d75e82f8ce9fd323ebf059c16c9aadefb22a1ecde13b7840b35835e4886c/charset_normalizer-3.4.9-cp314-cp314-win_arm64.whl", hash = "sha256:40a126142a56b2dfc0aacbad1de8310cbf60da7656db0e6b16eebd48e3e93519", size = 153334, upload-time = "2026-07-07T14:34:14.044Z" }, + { url = "https://files.pythonhosted.org/packages/00/5e/17398df3a139985ba9d11ed072531986f408c8fca952835ef1ab1820c02b/charset_normalizer-3.4.9-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:609b3ba8fcc0fb5ab7af00719d0fb6ad0cb518e48e7712d12fd68f1327951198", size = 338848, upload-time = "2026-07-07T14:34:15.688Z" }, + { url = "https://files.pythonhosted.org/packages/cd/91/7253a32e86b7e1d1239b1b36ba6dd0f021a21107ab33054b53119cc083b9/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:51447e9aa2684679af07ca5021c3db526e0284347ebf4ffcec1154c3350cfe32", size = 223022, upload-time = "2026-07-07T14:34:17.248Z" }, + { url = "https://files.pythonhosted.org/packages/cb/32/2e64bd2be10e89c61e57ebe6a93fd98ae88eb7ebe414b5121f22c96c69eb/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:cc1b0fff8ead343dae06305f954eb8468ba0ec1a97881f42489d198e4ce3c632", size = 241590, upload-time = "2026-07-07T14:34:18.813Z" }, + { url = "https://files.pythonhosted.org/packages/3d/ef/d96ec496cfea0c21db43b0ad03891308b02388d054cc902cf0e5a1ad6a88/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf", size = 239584, upload-time = "2026-07-07T14:34:20.52Z" }, + { url = "https://files.pythonhosted.org/packages/d4/ce/9af95f7876194bd7a14e3dfe4a4de2e0bff02666a3910d72beafd06cc297/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:df115d4d83168fdf2cae48ef1ff6d1cb4c466364e30861b37121de0f3bf1b990", size = 230224, upload-time = "2026-07-07T14:34:22.189Z" }, + { url = "https://files.pythonhosted.org/packages/52/94/af74dde74a3996bd959c350709bfe50e297823d70a8c1cbd54b838880863/charset_normalizer-3.4.9-cp314-cp314t-manylinux_2_31_armv7l.whl", hash = "sha256:f86c6358749bd4fda175388691e3ba8c46e24c5347d0afd20f9b7edfc9faf07d", size = 212667, upload-time = "2026-07-07T14:34:23.857Z" }, + { url = "https://files.pythonhosted.org/packages/ee/f0/f1c4fe746c395922961b5916ed1d7d6e7d4c84851d19ed43cc89980ec953/charset_normalizer-3.4.9-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:32286a2c8d167e897177b673176c1e3e00d4057caf5d2b64eef9a3666b03018e", size = 227179, upload-time = "2026-07-07T14:34:25.586Z" }, + { url = "https://files.pythonhosted.org/packages/e4/56/6c745619ac397e8871e2bcd3cea1eec86b877488f33888b3aef5c3ed506e/charset_normalizer-3.4.9-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:83aed2c10721ddd90f68140685391b50811a880af20654c59af6b6c66c40513c", size = 225372, upload-time = "2026-07-07T14:34:27.212Z" }, + { url = "https://files.pythonhosted.org/packages/78/ad/98aae8630ac71f16711968e38a5acfecce41b778bf2f0312851020f565a8/charset_normalizer-3.4.9-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:cd6c3d4b783c556fa00bf540854e42f135e2f256abd29669fcd0da0f2dec79c2", size = 215222, upload-time = "2026-07-07T14:34:28.774Z" }, + { url = "https://files.pythonhosted.org/packages/f7/40/9593d54209765207a7f11073c06494c1721e4ca4a0a426c597679bf7f91e/charset_normalizer-3.4.9-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:ee2f2a527e3c1a6e6411eb4209642e138b544a2d72fe5d0d76daf77b24063534", size = 231958, upload-time = "2026-07-07T14:34:30.345Z" }, + { url = "https://files.pythonhosted.org/packages/b1/27/693ee5e8a18191eb38647360c51cd505013e2bd3b366aa43fd5344c21e3c/charset_normalizer-3.4.9-cp314-cp314t-win32.whl", hash = "sha256:0d861473f743244d349b50f850d10eb87aeb22bbdcc8e64f79273c94af5a8226", size = 155580, upload-time = "2026-07-07T14:34:31.884Z" }, + { url = "https://files.pythonhosted.org/packages/80/3f/bd97d3d9c613013d07cb7733d299385b41df37f0471310f5a73dc359f0b8/charset_normalizer-3.4.9-cp314-cp314t-win_amd64.whl", hash = "sha256:9b8e0f3107e2200b76f6054de99016eac3ee6762713587b36baaa7e4bd2ae177", size = 167620, upload-time = "2026-07-07T14:34:33.438Z" }, + { url = "https://files.pythonhosted.org/packages/3d/c6/eee9dca4439b1061f76373f06ea855678cc4a64c1c3c90b50e479edbb8eb/charset_normalizer-3.4.9-cp314-cp314t-win_arm64.whl", hash = "sha256:19ac87f93086ce37b86e098888555c4b4bc48102279bae3350098c0ed664b501", size = 158037, upload-time = "2026-07-07T14:34:35.018Z" }, + { url = "https://files.pythonhosted.org/packages/a6/ec/81e22253f4b7091eca6515bb3da5e45d05a663f7f567bb745695dc60f892/charset_normalizer-3.4.9-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:253a4a220747e8b5faf57ec320c4f5efb0cef05f647420bf267143ec15dba10a", size = 306122, upload-time = "2026-07-07T14:34:36.607Z" }, + { url = "https://files.pythonhosted.org/packages/c8/53/a8c042eb9eee4716f4d42a0f5a571eb32a09ec429be9fb0b8b9d765393ba/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:68ce9f4d6b26d5ccbf7fd4459bf75f74a0a146677ebba80597df60cbdb20e6f4", size = 206284, upload-time = "2026-07-07T14:34:38.166Z" }, + { url = "https://files.pythonhosted.org/packages/14/cb/1db8b96547ee3186cd2dd7f2e59dd560a9b80748f3604171f3c153d62811/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:58150c9f9b9a552505912d182ccdf26f6396fb6094816ceebcbb20eecabaed94", size = 226837, upload-time = "2026-07-07T14:34:39.77Z" }, + { url = "https://files.pythonhosted.org/packages/6a/05/c94d5cd23396289c54c93b02e0273b4dd8921641d9968c4828caf9bbaad9/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:df7276909358e5635ae203673ab7e509ddd224225a8d6b0790bf13eb2bde1cc5", size = 222199, upload-time = "2026-07-07T14:34:41.391Z" }, + { url = "https://files.pythonhosted.org/packages/6d/46/79847edd07244a4a2d443c6655a7b6ee94203c21539414b059f32713c357/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:3c09a49d6cde137258beb3d551994a2927fd35ad5cf96aed573f61bbd67c5f84", size = 214344, upload-time = "2026-07-07T14:34:42.986Z" }, + { url = "https://files.pythonhosted.org/packages/ec/b4/ef5a49b2e77c00deb43bb3256592b115ba9e4346016e82c516b8d215bf68/charset_normalizer-3.4.9-cp39-cp39-manylinux_2_31_armv7l.whl", hash = "sha256:231ddcbb35e2ff8973e1365db41fe0572662893b99a05deb183b68ad4c0c8bd4", size = 199988, upload-time = "2026-07-07T14:34:44.685Z" }, + { url = "https://files.pythonhosted.org/packages/3d/ca/ad1d7c7d3077dab873f539d3e1d083c0845a762cb0bafdfbe3ef93add598/charset_normalizer-3.4.9-cp39-cp39-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:920079c3f7456fa213e0829ed2073aaa727fd39d889ead5b4f35d0de5460d04f", size = 211908, upload-time = "2026-07-07T14:34:46.227Z" }, + { url = "https://files.pythonhosted.org/packages/ed/61/710738687f90d01c06a04ed52d6ca1e62dd9b1d8cc2567098167c4691034/charset_normalizer-3.4.9-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:0fa1aec2d32bcc03c8fa0f6f1712caad1adc38509f31142112e5c9daf5b9c833", size = 209320, upload-time = "2026-07-07T14:34:47.753Z" }, + { url = "https://files.pythonhosted.org/packages/5f/c0/6eec7bdabe6cbbcc274ec04596f6d93865751a0541d33d60d1ce179bd372/charset_normalizer-3.4.9-cp39-cp39-musllinux_1_2_armv7l.whl", hash = "sha256:ad41ba96094304aa090f5a30cb6e4fb3b3f1c264c523394b4c39bbacc4dc92ba", size = 200980, upload-time = "2026-07-07T14:34:49.362Z" }, + { url = "https://files.pythonhosted.org/packages/eb/78/59344ff9a4a7b5f6530bf7bec2c980047cc42c3a616596cdbd8cb5c1a1af/charset_normalizer-3.4.9-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:43b9e366a31fdd1c87d0eb08f579b4a82b723ea54338f040d6b4e518a026ea29", size = 216545, upload-time = "2026-07-07T14:34:50.98Z" }, + { url = "https://files.pythonhosted.org/packages/17/6d/bff78a4bacc4891bc63ec5bdc6776d8c85e47fab93d0d5f6223068fad0a4/charset_normalizer-3.4.9-cp39-cp39-win32.whl", hash = "sha256:93d59d504b230e83c7a843251681959a0b6a9cd76f6e146ce1b8a80eb8739af9", size = 146256, upload-time = "2026-07-07T14:34:52.509Z" }, + { url = "https://files.pythonhosted.org/packages/a2/55/86048bde1c9d0352940bd7b87d825091a52aef67d01cde6c6f7342c5b552/charset_normalizer-3.4.9-cp39-cp39-win_amd64.whl", hash = "sha256:ddf4af30b417d9fe16481e9b81c27ab2a7cde1ff7ba3e85653b02db7d145dc7b", size = 156413, upload-time = "2026-07-07T14:34:54.117Z" }, + { url = "https://files.pythonhosted.org/packages/28/e9/9fb6099b868c82a40698a748ae0fbd4f31ccc13844c176a07158ba2abbfd/charset_normalizer-3.4.9-cp39-cp39-win_arm64.whl", hash = "sha256:476743fe6dfe14a2da12e3ac79125dc84a3b2cf8094369a47a1529b0cd8549fe", size = 147887, upload-time = "2026-07-07T14:34:55.51Z" }, + { url = "https://files.pythonhosted.org/packages/98/2b/f97f1c193fb855c345d678f5077d6926034db0722df74c8f057020e05a25/charset_normalizer-3.4.9-py3-none-any.whl", hash = "sha256:68e5f26a1ad57ded6d1cfb85331d1c1a195314756471d97758c48498bb4dcdf5", size = 64538, upload-time = "2026-07-07T14:34:56.993Z" }, +] + +[[package]] +name = "colorama" +version = "0.4.6" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" }, +] + +[[package]] +name = "docutils" +version = "0.21.2" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version == '3.10.*'", + "python_full_version < '3.10'", +] +sdist = { url = "https://files.pythonhosted.org/packages/ae/ed/aefcc8cd0ba62a0560c3c18c33925362d46c6075480bfa4df87b28e169a9/docutils-0.21.2.tar.gz", hash = "sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f", size = 2204444, upload-time = "2024-04-23T18:57:18.24Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/8f/d7/9322c609343d929e75e7e5e6255e614fcc67572cfd083959cdef3b7aad79/docutils-0.21.2-py3-none-any.whl", hash = "sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2", size = 587408, upload-time = "2024-04-23T18:57:14.835Z" }, +] + +[[package]] +name = "docutils" +version = "0.22.4" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", +] +sdist = { url = "https://files.pythonhosted.org/packages/ae/b6/03bb70946330e88ffec97aefd3ea75ba575cb2e762061e0e62a213befee8/docutils-0.22.4.tar.gz", hash = "sha256:4db53b1fde9abecbb74d91230d32ab626d94f6badfc575d6db9194a49df29968", size = 2291750, upload-time = "2025-12-18T19:00:26.443Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/02/10/5da547df7a391dcde17f59520a231527b8571e6f46fc8efb02ccb370ab12/docutils-0.22.4-py3-none-any.whl", hash = "sha256:d0013f540772d1420576855455d050a2180186c91c15779301ac2ccb3eeb68de", size = 633196, upload-time = "2025-12-18T19:00:18.077Z" }, +] + +[[package]] +name = "idna" +version = "3.18" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/cd/63/9496c57188a2ee585e0f1db071d75089a11e98aa86eb99d9d7618fc1edce/idna-3.18.tar.gz", hash = "sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848", size = 196711, upload-time = "2026-06-02T14:34:07.794Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/1e/5e/d4e9f1a599fb8e573b7b87160658329fbf28d19eac2718f51fc3def3aa5a/idna-3.18-py3-none-any.whl", hash = "sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2", size = 65455, upload-time = "2026-06-02T14:34:06.319Z" }, +] + +[[package]] +name = "imagesize" +version = "1.5.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +sdist = { url = "https://files.pythonhosted.org/packages/cf/59/4b0dd64676aa6fb4986a755790cb6fc558559cf0084effad516820208ec3/imagesize-1.5.0.tar.gz", hash = "sha256:8bfc5363a7f2133a89f0098451e0bcb1cd71aba4dc02bbcecb39d99d40e1b94f", size = 1281127, upload-time = "2026-03-03T01:59:54.651Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/1e/b1/a0662b03103c66cf77101a187f396ea91167cd9b7d5d3a2e465ad2c7ee9b/imagesize-1.5.0-py2.py3-none-any.whl", hash = "sha256:32677681b3f434c2cb496f00e89c5a291247b35b1f527589909e008057da5899", size = 5763, upload-time = "2026-03-03T01:59:52.343Z" }, +] + +[[package]] +name = "imagesize" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +sdist = { url = "https://files.pythonhosted.org/packages/6c/e6/7bf14eeb8f8b7251141944835abd42eb20a658d89084b7e1f3e5fe394090/imagesize-2.0.0.tar.gz", hash = "sha256:8e8358c4a05c304f1fccf7ff96f036e7243a189e9e42e90851993c558cfe9ee3", size = 1773045, upload-time = "2026-03-03T14:18:29.941Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5f/53/fb7122b71361a0d121b669dcf3d31244ef75badbbb724af388948de543e2/imagesize-2.0.0-py2.py3-none-any.whl", hash = "sha256:5667c5bbb57ab3f1fa4bc366f4fbc971db3d5ed011fd2715fd8001f782718d96", size = 9441, upload-time = "2026-03-03T14:18:27.892Z" }, +] + +[[package]] +name = "importlib-metadata" +version = "8.7.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "zipp", marker = "python_full_version < '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/f3/49/3b30cad09e7771a4982d9975a8cbf64f00d4a1ececb53297f1d9a7be1b10/importlib_metadata-8.7.1.tar.gz", hash = "sha256:49fef1ae6440c182052f407c8d34a68f72efc36db9ca90dc0113398f2fdde8bb", size = 57107, upload-time = "2025-12-21T10:00:19.278Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/fa/5e/f8e9a1d23b9c20a551a8a02ea3637b4642e22c2626e3a13a9a29cdea99eb/importlib_metadata-8.7.1-py3-none-any.whl", hash = "sha256:5a1f80bf1daa489495071efbb095d75a634cf28a8bc299581244063b53176151", size = 27865, upload-time = "2025-12-21T10:00:18.329Z" }, +] + +[[package]] +name = "jinja2" +version = "3.1.6" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "markupsafe" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/df/bf/f7da0350254c0ed7c72f3e33cef02e048281fec7ecec5f032d4aac52226b/jinja2-3.1.6.tar.gz", hash = "sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d", size = 245115, upload-time = "2025-03-05T20:05:02.478Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/62/a1/3d680cbfd5f4b8f15abc1d571870c5fc3e594bb582bc3b64ea099db13e56/jinja2-3.1.6-py3-none-any.whl", hash = "sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67", size = 134899, upload-time = "2025-03-05T20:05:00.369Z" }, +] + +[[package]] +name = "markdown-it-py" +version = "3.0.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version == '3.10.*'", + "python_full_version < '3.10'", +] +dependencies = [ + { name = "mdurl", marker = "python_full_version < '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/38/71/3b932df36c1a044d397a1f92d1cf91ee0a503d91e470cbd670aa66b07ed0/markdown-it-py-3.0.0.tar.gz", hash = "sha256:e3f60a94fa066dc52ec76661e37c851cb232d92f9886b15cb560aaada2df8feb", size = 74596, upload-time = "2023-06-03T06:41:14.443Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/42/d7/1ec15b46af6af88f19b8e5ffea08fa375d433c998b8a7639e76935c14f1f/markdown_it_py-3.0.0-py3-none-any.whl", hash = "sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1", size = 87528, upload-time = "2023-06-03T06:41:11.019Z" }, +] + +[[package]] +name = "markdown-it-py" +version = "4.2.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", +] +dependencies = [ + { name = "mdurl", marker = "python_full_version >= '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/06/ff/7841249c247aa650a76b9ee4bbaeae59370dc8bfd2f6c01f3630c35eb134/markdown_it_py-4.2.0.tar.gz", hash = "sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49", size = 82454, upload-time = "2026-05-07T12:08:28.36Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b3/81/4da04ced5a082363ecfa159c010d200ecbd959ae410c10c0264a38cac0f5/markdown_it_py-4.2.0-py3-none-any.whl", hash = "sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a", size = 91687, upload-time = "2026-05-07T12:08:27.182Z" }, +] + +[[package]] +name = "markupsafe" +version = "3.0.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/7e/99/7690b6d4034fffd95959cbe0c02de8deb3098cc577c67bb6a24fe5d7caa7/markupsafe-3.0.3.tar.gz", hash = "sha256:722695808f4b6457b320fdc131280796bdceb04ab50fe1795cd540799ebe1698", size = 80313, upload-time = "2025-09-27T18:37:40.426Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e8/4b/3541d44f3937ba468b75da9eebcae497dcf67adb65caa16760b0a6807ebb/markupsafe-3.0.3-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:2f981d352f04553a7171b8e44369f2af4055f888dfb147d55e42d29e29e74559", size = 11631, upload-time = "2025-09-27T18:36:05.558Z" }, + { url = "https://files.pythonhosted.org/packages/98/1b/fbd8eed11021cabd9226c37342fa6ca4e8a98d8188a8d9b66740494960e4/markupsafe-3.0.3-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:e1c1493fb6e50ab01d20a22826e57520f1284df32f2d8601fdd90b6304601419", size = 12057, upload-time = "2025-09-27T18:36:07.165Z" }, + { url = "https://files.pythonhosted.org/packages/40/01/e560d658dc0bb8ab762670ece35281dec7b6c1b33f5fbc09ebb57a185519/markupsafe-3.0.3-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1ba88449deb3de88bd40044603fafffb7bc2b055d626a330323a9ed736661695", size = 22050, upload-time = "2025-09-27T18:36:08.005Z" }, + { url = "https://files.pythonhosted.org/packages/af/cd/ce6e848bbf2c32314c9b237839119c5a564a59725b53157c856e90937b7a/markupsafe-3.0.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f42d0984e947b8adf7dd6dde396e720934d12c506ce84eea8476409563607591", size = 20681, upload-time = "2025-09-27T18:36:08.881Z" }, + { url = "https://files.pythonhosted.org/packages/c9/2a/b5c12c809f1c3045c4d580b035a743d12fcde53cf685dbc44660826308da/markupsafe-3.0.3-cp310-cp310-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:c0c0b3ade1c0b13b936d7970b1d37a57acde9199dc2aecc4c336773e1d86049c", size = 20705, upload-time = "2025-09-27T18:36:10.131Z" }, + { url = "https://files.pythonhosted.org/packages/cf/e3/9427a68c82728d0a88c50f890d0fc072a1484de2f3ac1ad0bfc1a7214fd5/markupsafe-3.0.3-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:0303439a41979d9e74d18ff5e2dd8c43ed6c6001fd40e5bf2e43f7bd9bbc523f", size = 21524, upload-time = "2025-09-27T18:36:11.324Z" }, + { url = "https://files.pythonhosted.org/packages/bc/36/23578f29e9e582a4d0278e009b38081dbe363c5e7165113fad546918a232/markupsafe-3.0.3-cp310-cp310-musllinux_1_2_riscv64.whl", hash = "sha256:d2ee202e79d8ed691ceebae8e0486bd9a2cd4794cec4824e1c99b6f5009502f6", size = 20282, upload-time = "2025-09-27T18:36:12.573Z" }, + { url = "https://files.pythonhosted.org/packages/56/21/dca11354e756ebd03e036bd8ad58d6d7168c80ce1fe5e75218e4945cbab7/markupsafe-3.0.3-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:177b5253b2834fe3678cb4a5f0059808258584c559193998be2601324fdeafb1", size = 20745, upload-time = "2025-09-27T18:36:13.504Z" }, + { url = "https://files.pythonhosted.org/packages/87/99/faba9369a7ad6e4d10b6a5fbf71fa2a188fe4a593b15f0963b73859a1bbd/markupsafe-3.0.3-cp310-cp310-win32.whl", hash = "sha256:2a15a08b17dd94c53a1da0438822d70ebcd13f8c3a95abe3a9ef9f11a94830aa", size = 14571, upload-time = "2025-09-27T18:36:14.779Z" }, + { url = "https://files.pythonhosted.org/packages/d6/25/55dc3ab959917602c96985cb1253efaa4ff42f71194bddeb61eb7278b8be/markupsafe-3.0.3-cp310-cp310-win_amd64.whl", hash = "sha256:c4ffb7ebf07cfe8931028e3e4c85f0357459a3f9f9490886198848f4fa002ec8", size = 15056, upload-time = "2025-09-27T18:36:16.125Z" }, + { url = "https://files.pythonhosted.org/packages/d0/9e/0a02226640c255d1da0b8d12e24ac2aa6734da68bff14c05dd53b94a0fc3/markupsafe-3.0.3-cp310-cp310-win_arm64.whl", hash = "sha256:e2103a929dfa2fcaf9bb4e7c091983a49c9ac3b19c9061b6d5427dd7d14d81a1", size = 13932, upload-time = "2025-09-27T18:36:17.311Z" }, + { url = "https://files.pythonhosted.org/packages/08/db/fefacb2136439fc8dd20e797950e749aa1f4997ed584c62cfb8ef7c2be0e/markupsafe-3.0.3-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:1cc7ea17a6824959616c525620e387f6dd30fec8cb44f649e31712db02123dad", size = 11631, upload-time = "2025-09-27T18:36:18.185Z" }, + { url = "https://files.pythonhosted.org/packages/e1/2e/5898933336b61975ce9dc04decbc0a7f2fee78c30353c5efba7f2d6ff27a/markupsafe-3.0.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:4bd4cd07944443f5a265608cc6aab442e4f74dff8088b0dfc8238647b8f6ae9a", size = 12058, upload-time = "2025-09-27T18:36:19.444Z" }, + { url = "https://files.pythonhosted.org/packages/1d/09/adf2df3699d87d1d8184038df46a9c80d78c0148492323f4693df54e17bb/markupsafe-3.0.3-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6b5420a1d9450023228968e7e6a9ce57f65d148ab56d2313fcd589eee96a7a50", size = 24287, upload-time = "2025-09-27T18:36:20.768Z" }, + { url = "https://files.pythonhosted.org/packages/30/ac/0273f6fcb5f42e314c6d8cd99effae6a5354604d461b8d392b5ec9530a54/markupsafe-3.0.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0bf2a864d67e76e5c9a34dc26ec616a66b9888e25e7b9460e1c76d3293bd9dbf", size = 22940, upload-time = "2025-09-27T18:36:22.249Z" }, + { url = "https://files.pythonhosted.org/packages/19/ae/31c1be199ef767124c042c6c3e904da327a2f7f0cd63a0337e1eca2967a8/markupsafe-3.0.3-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:bc51efed119bc9cfdf792cdeaa4d67e8f6fcccab66ed4bfdd6bde3e59bfcbb2f", size = 21887, upload-time = "2025-09-27T18:36:23.535Z" }, + { url = "https://files.pythonhosted.org/packages/b2/76/7edcab99d5349a4532a459e1fe64f0b0467a3365056ae550d3bcf3f79e1e/markupsafe-3.0.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:068f375c472b3e7acbe2d5318dea141359e6900156b5b2ba06a30b169086b91a", size = 23692, upload-time = "2025-09-27T18:36:24.823Z" }, + { url = "https://files.pythonhosted.org/packages/a4/28/6e74cdd26d7514849143d69f0bf2399f929c37dc2b31e6829fd2045b2765/markupsafe-3.0.3-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:7be7b61bb172e1ed687f1754f8e7484f1c8019780f6f6b0786e76bb01c2ae115", size = 21471, upload-time = "2025-09-27T18:36:25.95Z" }, + { url = "https://files.pythonhosted.org/packages/62/7e/a145f36a5c2945673e590850a6f8014318d5577ed7e5920a4b3448e0865d/markupsafe-3.0.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:f9e130248f4462aaa8e2552d547f36ddadbeaa573879158d721bbd33dfe4743a", size = 22923, upload-time = "2025-09-27T18:36:27.109Z" }, + { url = "https://files.pythonhosted.org/packages/0f/62/d9c46a7f5c9adbeeeda52f5b8d802e1094e9717705a645efc71b0913a0a8/markupsafe-3.0.3-cp311-cp311-win32.whl", hash = "sha256:0db14f5dafddbb6d9208827849fad01f1a2609380add406671a26386cdf15a19", size = 14572, upload-time = "2025-09-27T18:36:28.045Z" }, + { url = "https://files.pythonhosted.org/packages/83/8a/4414c03d3f891739326e1783338e48fb49781cc915b2e0ee052aa490d586/markupsafe-3.0.3-cp311-cp311-win_amd64.whl", hash = "sha256:de8a88e63464af587c950061a5e6a67d3632e36df62b986892331d4620a35c01", size = 15077, upload-time = "2025-09-27T18:36:29.025Z" }, + { url = "https://files.pythonhosted.org/packages/35/73/893072b42e6862f319b5207adc9ae06070f095b358655f077f69a35601f0/markupsafe-3.0.3-cp311-cp311-win_arm64.whl", hash = "sha256:3b562dd9e9ea93f13d53989d23a7e775fdfd1066c33494ff43f5418bc8c58a5c", size = 13876, upload-time = "2025-09-27T18:36:29.954Z" }, + { url = "https://files.pythonhosted.org/packages/5a/72/147da192e38635ada20e0a2e1a51cf8823d2119ce8883f7053879c2199b5/markupsafe-3.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:d53197da72cc091b024dd97249dfc7794d6a56530370992a5e1a08983ad9230e", size = 11615, upload-time = "2025-09-27T18:36:30.854Z" }, + { url = "https://files.pythonhosted.org/packages/9a/81/7e4e08678a1f98521201c3079f77db69fb552acd56067661f8c2f534a718/markupsafe-3.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:1872df69a4de6aead3491198eaf13810b565bdbeec3ae2dc8780f14458ec73ce", size = 12020, upload-time = "2025-09-27T18:36:31.971Z" }, + { url = "https://files.pythonhosted.org/packages/1e/2c/799f4742efc39633a1b54a92eec4082e4f815314869865d876824c257c1e/markupsafe-3.0.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3a7e8ae81ae39e62a41ec302f972ba6ae23a5c5396c8e60113e9066ef893da0d", size = 24332, upload-time = "2025-09-27T18:36:32.813Z" }, + { url = "https://files.pythonhosted.org/packages/3c/2e/8d0c2ab90a8c1d9a24f0399058ab8519a3279d1bd4289511d74e909f060e/markupsafe-3.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d6dd0be5b5b189d31db7cda48b91d7e0a9795f31430b7f271219ab30f1d3ac9d", size = 22947, upload-time = "2025-09-27T18:36:33.86Z" }, + { url = "https://files.pythonhosted.org/packages/2c/54/887f3092a85238093a0b2154bd629c89444f395618842e8b0c41783898ea/markupsafe-3.0.3-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:94c6f0bb423f739146aec64595853541634bde58b2135f27f61c1ffd1cd4d16a", size = 21962, upload-time = "2025-09-27T18:36:35.099Z" }, + { url = "https://files.pythonhosted.org/packages/c9/2f/336b8c7b6f4a4d95e91119dc8521402461b74a485558d8f238a68312f11c/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:be8813b57049a7dc738189df53d69395eba14fb99345e0a5994914a3864c8a4b", size = 23760, upload-time = "2025-09-27T18:36:36.001Z" }, + { url = "https://files.pythonhosted.org/packages/32/43/67935f2b7e4982ffb50a4d169b724d74b62a3964bc1a9a527f5ac4f1ee2b/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:83891d0e9fb81a825d9a6d61e3f07550ca70a076484292a70fde82c4b807286f", size = 21529, upload-time = "2025-09-27T18:36:36.906Z" }, + { url = "https://files.pythonhosted.org/packages/89/e0/4486f11e51bbba8b0c041098859e869e304d1c261e59244baa3d295d47b7/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:77f0643abe7495da77fb436f50f8dab76dbc6e5fd25d39589a0f1fe6548bfa2b", size = 23015, upload-time = "2025-09-27T18:36:37.868Z" }, + { url = "https://files.pythonhosted.org/packages/2f/e1/78ee7a023dac597a5825441ebd17170785a9dab23de95d2c7508ade94e0e/markupsafe-3.0.3-cp312-cp312-win32.whl", hash = "sha256:d88b440e37a16e651bda4c7c2b930eb586fd15ca7406cb39e211fcff3bf3017d", size = 14540, upload-time = "2025-09-27T18:36:38.761Z" }, + { url = "https://files.pythonhosted.org/packages/aa/5b/bec5aa9bbbb2c946ca2733ef9c4ca91c91b6a24580193e891b5f7dbe8e1e/markupsafe-3.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:26a5784ded40c9e318cfc2bdb30fe164bdb8665ded9cd64d500a34fb42067b1c", size = 15105, upload-time = "2025-09-27T18:36:39.701Z" }, + { url = "https://files.pythonhosted.org/packages/e5/f1/216fc1bbfd74011693a4fd837e7026152e89c4bcf3e77b6692fba9923123/markupsafe-3.0.3-cp312-cp312-win_arm64.whl", hash = "sha256:35add3b638a5d900e807944a078b51922212fb3dedb01633a8defc4b01a3c85f", size = 13906, upload-time = "2025-09-27T18:36:40.689Z" }, + { url = "https://files.pythonhosted.org/packages/38/2f/907b9c7bbba283e68f20259574b13d005c121a0fa4c175f9bed27c4597ff/markupsafe-3.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:e1cf1972137e83c5d4c136c43ced9ac51d0e124706ee1c8aa8532c1287fa8795", size = 11622, upload-time = "2025-09-27T18:36:41.777Z" }, + { url = "https://files.pythonhosted.org/packages/9c/d9/5f7756922cdd676869eca1c4e3c0cd0df60ed30199ffd775e319089cb3ed/markupsafe-3.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:116bb52f642a37c115f517494ea5feb03889e04df47eeff5b130b1808ce7c219", size = 12029, upload-time = "2025-09-27T18:36:43.257Z" }, + { url = "https://files.pythonhosted.org/packages/00/07/575a68c754943058c78f30db02ee03a64b3c638586fba6a6dd56830b30a3/markupsafe-3.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:133a43e73a802c5562be9bbcd03d090aa5a1fe899db609c29e8c8d815c5f6de6", size = 24374, upload-time = "2025-09-27T18:36:44.508Z" }, + { url = "https://files.pythonhosted.org/packages/a9/21/9b05698b46f218fc0e118e1f8168395c65c8a2c750ae2bab54fc4bd4e0e8/markupsafe-3.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ccfcd093f13f0f0b7fdd0f198b90053bf7b2f02a3927a30e63f3ccc9df56b676", size = 22980, upload-time = "2025-09-27T18:36:45.385Z" }, + { url = "https://files.pythonhosted.org/packages/7f/71/544260864f893f18b6827315b988c146b559391e6e7e8f7252839b1b846a/markupsafe-3.0.3-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:509fa21c6deb7a7a273d629cf5ec029bc209d1a51178615ddf718f5918992ab9", size = 21990, upload-time = "2025-09-27T18:36:46.916Z" }, + { url = "https://files.pythonhosted.org/packages/c2/28/b50fc2f74d1ad761af2f5dcce7492648b983d00a65b8c0e0cb457c82ebbe/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a4afe79fb3de0b7097d81da19090f4df4f8d3a2b3adaa8764138aac2e44f3af1", size = 23784, upload-time = "2025-09-27T18:36:47.884Z" }, + { url = "https://files.pythonhosted.org/packages/ed/76/104b2aa106a208da8b17a2fb72e033a5a9d7073c68f7e508b94916ed47a9/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:795e7751525cae078558e679d646ae45574b47ed6e7771863fcc079a6171a0fc", size = 21588, upload-time = "2025-09-27T18:36:48.82Z" }, + { url = "https://files.pythonhosted.org/packages/b5/99/16a5eb2d140087ebd97180d95249b00a03aa87e29cc224056274f2e45fd6/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:8485f406a96febb5140bfeca44a73e3ce5116b2501ac54fe953e488fb1d03b12", size = 23041, upload-time = "2025-09-27T18:36:49.797Z" }, + { url = "https://files.pythonhosted.org/packages/19/bc/e7140ed90c5d61d77cea142eed9f9c303f4c4806f60a1044c13e3f1471d0/markupsafe-3.0.3-cp313-cp313-win32.whl", hash = "sha256:bdd37121970bfd8be76c5fb069c7751683bdf373db1ed6c010162b2a130248ed", size = 14543, upload-time = "2025-09-27T18:36:51.584Z" }, + { url = "https://files.pythonhosted.org/packages/05/73/c4abe620b841b6b791f2edc248f556900667a5a1cf023a6646967ae98335/markupsafe-3.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:9a1abfdc021a164803f4d485104931fb8f8c1efd55bc6b748d2f5774e78b62c5", size = 15113, upload-time = "2025-09-27T18:36:52.537Z" }, + { url = "https://files.pythonhosted.org/packages/f0/3a/fa34a0f7cfef23cf9500d68cb7c32dd64ffd58a12b09225fb03dd37d5b80/markupsafe-3.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:7e68f88e5b8799aa49c85cd116c932a1ac15caaa3f5db09087854d218359e485", size = 13911, upload-time = "2025-09-27T18:36:53.513Z" }, + { url = "https://files.pythonhosted.org/packages/e4/d7/e05cd7efe43a88a17a37b3ae96e79a19e846f3f456fe79c57ca61356ef01/markupsafe-3.0.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:218551f6df4868a8d527e3062d0fb968682fe92054e89978594c28e642c43a73", size = 11658, upload-time = "2025-09-27T18:36:54.819Z" }, + { url = "https://files.pythonhosted.org/packages/99/9e/e412117548182ce2148bdeacdda3bb494260c0b0184360fe0d56389b523b/markupsafe-3.0.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:3524b778fe5cfb3452a09d31e7b5adefeea8c5be1d43c4f810ba09f2ceb29d37", size = 12066, upload-time = "2025-09-27T18:36:55.714Z" }, + { url = "https://files.pythonhosted.org/packages/bc/e6/fa0ffcda717ef64a5108eaa7b4f5ed28d56122c9a6d70ab8b72f9f715c80/markupsafe-3.0.3-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4e885a3d1efa2eadc93c894a21770e4bc67899e3543680313b09f139e149ab19", size = 25639, upload-time = "2025-09-27T18:36:56.908Z" }, + { url = "https://files.pythonhosted.org/packages/96/ec/2102e881fe9d25fc16cb4b25d5f5cde50970967ffa5dddafdb771237062d/markupsafe-3.0.3-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8709b08f4a89aa7586de0aadc8da56180242ee0ada3999749b183aa23df95025", size = 23569, upload-time = "2025-09-27T18:36:57.913Z" }, + { url = "https://files.pythonhosted.org/packages/4b/30/6f2fce1f1f205fc9323255b216ca8a235b15860c34b6798f810f05828e32/markupsafe-3.0.3-cp313-cp313t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b8512a91625c9b3da6f127803b166b629725e68af71f8184ae7e7d54686a56d6", size = 23284, upload-time = "2025-09-27T18:36:58.833Z" }, + { url = "https://files.pythonhosted.org/packages/58/47/4a0ccea4ab9f5dcb6f79c0236d954acb382202721e704223a8aafa38b5c8/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:9b79b7a16f7fedff2495d684f2b59b0457c3b493778c9eed31111be64d58279f", size = 24801, upload-time = "2025-09-27T18:36:59.739Z" }, + { url = "https://files.pythonhosted.org/packages/6a/70/3780e9b72180b6fecb83a4814d84c3bf4b4ae4bf0b19c27196104149734c/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_riscv64.whl", hash = "sha256:12c63dfb4a98206f045aa9563db46507995f7ef6d83b2f68eda65c307c6829eb", size = 22769, upload-time = "2025-09-27T18:37:00.719Z" }, + { url = "https://files.pythonhosted.org/packages/98/c5/c03c7f4125180fc215220c035beac6b9cb684bc7a067c84fc69414d315f5/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:8f71bc33915be5186016f675cd83a1e08523649b0e33efdb898db577ef5bb009", size = 23642, upload-time = "2025-09-27T18:37:01.673Z" }, + { url = "https://files.pythonhosted.org/packages/80/d6/2d1b89f6ca4bff1036499b1e29a1d02d282259f3681540e16563f27ebc23/markupsafe-3.0.3-cp313-cp313t-win32.whl", hash = "sha256:69c0b73548bc525c8cb9a251cddf1931d1db4d2258e9599c28c07ef3580ef354", size = 14612, upload-time = "2025-09-27T18:37:02.639Z" }, + { url = "https://files.pythonhosted.org/packages/2b/98/e48a4bfba0a0ffcf9925fe2d69240bfaa19c6f7507b8cd09c70684a53c1e/markupsafe-3.0.3-cp313-cp313t-win_amd64.whl", hash = "sha256:1b4b79e8ebf6b55351f0d91fe80f893b4743f104bff22e90697db1590e47a218", size = 15200, upload-time = "2025-09-27T18:37:03.582Z" }, + { url = "https://files.pythonhosted.org/packages/0e/72/e3cc540f351f316e9ed0f092757459afbc595824ca724cbc5a5d4263713f/markupsafe-3.0.3-cp313-cp313t-win_arm64.whl", hash = "sha256:ad2cf8aa28b8c020ab2fc8287b0f823d0a7d8630784c31e9ee5edea20f406287", size = 13973, upload-time = "2025-09-27T18:37:04.929Z" }, + { url = "https://files.pythonhosted.org/packages/33/8a/8e42d4838cd89b7dde187011e97fe6c3af66d8c044997d2183fbd6d31352/markupsafe-3.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:eaa9599de571d72e2daf60164784109f19978b327a3910d3e9de8c97b5b70cfe", size = 11619, upload-time = "2025-09-27T18:37:06.342Z" }, + { url = "https://files.pythonhosted.org/packages/b5/64/7660f8a4a8e53c924d0fa05dc3a55c9cee10bbd82b11c5afb27d44b096ce/markupsafe-3.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:c47a551199eb8eb2121d4f0f15ae0f923d31350ab9280078d1e5f12b249e0026", size = 12029, upload-time = "2025-09-27T18:37:07.213Z" }, + { url = "https://files.pythonhosted.org/packages/da/ef/e648bfd021127bef5fa12e1720ffed0c6cbb8310c8d9bea7266337ff06de/markupsafe-3.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f34c41761022dd093b4b6896d4810782ffbabe30f2d443ff5f083e0cbbb8c737", size = 24408, upload-time = "2025-09-27T18:37:09.572Z" }, + { url = "https://files.pythonhosted.org/packages/41/3c/a36c2450754618e62008bf7435ccb0f88053e07592e6028a34776213d877/markupsafe-3.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:457a69a9577064c05a97c41f4e65148652db078a3a509039e64d3467b9e7ef97", size = 23005, upload-time = "2025-09-27T18:37:10.58Z" }, + { url = "https://files.pythonhosted.org/packages/bc/20/b7fdf89a8456b099837cd1dc21974632a02a999ec9bf7ca3e490aacd98e7/markupsafe-3.0.3-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e8afc3f2ccfa24215f8cb28dcf43f0113ac3c37c2f0f0806d8c70e4228c5cf4d", size = 22048, upload-time = "2025-09-27T18:37:11.547Z" }, + { url = "https://files.pythonhosted.org/packages/9a/a7/591f592afdc734f47db08a75793a55d7fbcc6902a723ae4cfbab61010cc5/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:ec15a59cf5af7be74194f7ab02d0f59a62bdcf1a537677ce67a2537c9b87fcda", size = 23821, upload-time = "2025-09-27T18:37:12.48Z" }, + { url = "https://files.pythonhosted.org/packages/7d/33/45b24e4f44195b26521bc6f1a82197118f74df348556594bd2262bda1038/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:0eb9ff8191e8498cca014656ae6b8d61f39da5f95b488805da4bb029cccbfbaf", size = 21606, upload-time = "2025-09-27T18:37:13.485Z" }, + { url = "https://files.pythonhosted.org/packages/ff/0e/53dfaca23a69fbfbbf17a4b64072090e70717344c52eaaaa9c5ddff1e5f0/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:2713baf880df847f2bece4230d4d094280f4e67b1e813eec43b4c0e144a34ffe", size = 23043, upload-time = "2025-09-27T18:37:14.408Z" }, + { url = "https://files.pythonhosted.org/packages/46/11/f333a06fc16236d5238bfe74daccbca41459dcd8d1fa952e8fbd5dccfb70/markupsafe-3.0.3-cp314-cp314-win32.whl", hash = "sha256:729586769a26dbceff69f7a7dbbf59ab6572b99d94576a5592625d5b411576b9", size = 14747, upload-time = "2025-09-27T18:37:15.36Z" }, + { url = "https://files.pythonhosted.org/packages/28/52/182836104b33b444e400b14f797212f720cbc9ed6ba34c800639d154e821/markupsafe-3.0.3-cp314-cp314-win_amd64.whl", hash = "sha256:bdc919ead48f234740ad807933cdf545180bfbe9342c2bb451556db2ed958581", size = 15341, upload-time = "2025-09-27T18:37:16.496Z" }, + { url = "https://files.pythonhosted.org/packages/6f/18/acf23e91bd94fd7b3031558b1f013adfa21a8e407a3fdb32745538730382/markupsafe-3.0.3-cp314-cp314-win_arm64.whl", hash = "sha256:5a7d5dc5140555cf21a6fefbdbf8723f06fcd2f63ef108f2854de715e4422cb4", size = 14073, upload-time = "2025-09-27T18:37:17.476Z" }, + { url = "https://files.pythonhosted.org/packages/3c/f0/57689aa4076e1b43b15fdfa646b04653969d50cf30c32a102762be2485da/markupsafe-3.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:1353ef0c1b138e1907ae78e2f6c63ff67501122006b0f9abad68fda5f4ffc6ab", size = 11661, upload-time = "2025-09-27T18:37:18.453Z" }, + { url = "https://files.pythonhosted.org/packages/89/c3/2e67a7ca217c6912985ec766c6393b636fb0c2344443ff9d91404dc4c79f/markupsafe-3.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1085e7fbddd3be5f89cc898938f42c0b3c711fdcb37d75221de2666af647c175", size = 12069, upload-time = "2025-09-27T18:37:19.332Z" }, + { url = "https://files.pythonhosted.org/packages/f0/00/be561dce4e6ca66b15276e184ce4b8aec61fe83662cce2f7d72bd3249d28/markupsafe-3.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1b52b4fb9df4eb9ae465f8d0c228a00624de2334f216f178a995ccdcf82c4634", size = 25670, upload-time = "2025-09-27T18:37:20.245Z" }, + { url = "https://files.pythonhosted.org/packages/50/09/c419f6f5a92e5fadde27efd190eca90f05e1261b10dbd8cbcb39cd8ea1dc/markupsafe-3.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fed51ac40f757d41b7c48425901843666a6677e3e8eb0abcff09e4ba6e664f50", size = 23598, upload-time = "2025-09-27T18:37:21.177Z" }, + { url = "https://files.pythonhosted.org/packages/22/44/a0681611106e0b2921b3033fc19bc53323e0b50bc70cffdd19f7d679bb66/markupsafe-3.0.3-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:f190daf01f13c72eac4efd5c430a8de82489d9cff23c364c3ea822545032993e", size = 23261, upload-time = "2025-09-27T18:37:22.167Z" }, + { url = "https://files.pythonhosted.org/packages/5f/57/1b0b3f100259dc9fffe780cfb60d4be71375510e435efec3d116b6436d43/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e56b7d45a839a697b5eb268c82a71bd8c7f6c94d6fd50c3d577fa39a9f1409f5", size = 24835, upload-time = "2025-09-27T18:37:23.296Z" }, + { url = "https://files.pythonhosted.org/packages/26/6a/4bf6d0c97c4920f1597cc14dd720705eca0bf7c787aebc6bb4d1bead5388/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:f3e98bb3798ead92273dc0e5fd0f31ade220f59a266ffd8a4f6065e0a3ce0523", size = 22733, upload-time = "2025-09-27T18:37:24.237Z" }, + { url = "https://files.pythonhosted.org/packages/14/c7/ca723101509b518797fedc2fdf79ba57f886b4aca8a7d31857ba3ee8281f/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:5678211cb9333a6468fb8d8be0305520aa073f50d17f089b5b4b477ea6e67fdc", size = 23672, upload-time = "2025-09-27T18:37:25.271Z" }, + { url = "https://files.pythonhosted.org/packages/fb/df/5bd7a48c256faecd1d36edc13133e51397e41b73bb77e1a69deab746ebac/markupsafe-3.0.3-cp314-cp314t-win32.whl", hash = "sha256:915c04ba3851909ce68ccc2b8e2cd691618c4dc4c4232fb7982bca3f41fd8c3d", size = 14819, upload-time = "2025-09-27T18:37:26.285Z" }, + { url = "https://files.pythonhosted.org/packages/1a/8a/0402ba61a2f16038b48b39bccca271134be00c5c9f0f623208399333c448/markupsafe-3.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4faffd047e07c38848ce017e8725090413cd80cbc23d86e55c587bf979e579c9", size = 15426, upload-time = "2025-09-27T18:37:27.316Z" }, + { url = "https://files.pythonhosted.org/packages/70/bc/6f1c2f612465f5fa89b95bead1f44dcb607670fd42891d8fdcd5d039f4f4/markupsafe-3.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:32001d6a8fc98c8cb5c947787c5d08b0a50663d139f1305bac5885d98d9b40fa", size = 14146, upload-time = "2025-09-27T18:37:28.327Z" }, + { url = "https://files.pythonhosted.org/packages/56/23/0d8c13a44bde9154821586520840643467aee574d8ce79a17da539ee7fed/markupsafe-3.0.3-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:15d939a21d546304880945ca1ecb8a039db6b4dc49b2c5a400387cdae6a62e26", size = 11623, upload-time = "2025-09-27T18:37:29.296Z" }, + { url = "https://files.pythonhosted.org/packages/fd/23/07a2cb9a8045d5f3f0890a8c3bc0859d7a47bfd9a560b563899bec7b72ed/markupsafe-3.0.3-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:f71a396b3bf33ecaa1626c255855702aca4d3d9fea5e051b41ac59a9c1c41edc", size = 12049, upload-time = "2025-09-27T18:37:30.234Z" }, + { url = "https://files.pythonhosted.org/packages/bc/e4/6be85eb81503f8e11b61c0b6369b6e077dcf0a74adbd9ebf6b349937b4e9/markupsafe-3.0.3-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0f4b68347f8c5eab4a13419215bdfd7f8c9b19f2b25520968adfad23eb0ce60c", size = 21923, upload-time = "2025-09-27T18:37:31.177Z" }, + { url = "https://files.pythonhosted.org/packages/6f/bc/4dc914ead3fe6ddaef035341fee0fc956949bbd27335b611829292b89ee2/markupsafe-3.0.3-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e8fc20152abba6b83724d7ff268c249fa196d8259ff481f3b1476383f8f24e42", size = 20543, upload-time = "2025-09-27T18:37:32.168Z" }, + { url = "https://files.pythonhosted.org/packages/89/6e/5fe81fbcfba4aef4093d5f856e5c774ec2057946052d18d168219b7bd9f9/markupsafe-3.0.3-cp39-cp39-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:949b8d66bc381ee8b007cd945914c721d9aba8e27f71959d750a46f7c282b20b", size = 20585, upload-time = "2025-09-27T18:37:33.166Z" }, + { url = "https://files.pythonhosted.org/packages/f6/f6/e0e5a3d3ae9c4020f696cd055f940ef86b64fe88de26f3a0308b9d3d048c/markupsafe-3.0.3-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:3537e01efc9d4dccdf77221fb1cb3b8e1a38d5428920e0657ce299b20324d758", size = 21387, upload-time = "2025-09-27T18:37:34.185Z" }, + { url = "https://files.pythonhosted.org/packages/c8/25/651753ef4dea08ea790f4fbb65146a9a44a014986996ca40102e237aa49a/markupsafe-3.0.3-cp39-cp39-musllinux_1_2_riscv64.whl", hash = "sha256:591ae9f2a647529ca990bc681daebdd52c8791ff06c2bfa05b65163e28102ef2", size = 20133, upload-time = "2025-09-27T18:37:35.138Z" }, + { url = "https://files.pythonhosted.org/packages/dc/0a/c3cf2b4fef5f0426e8a6d7fce3cb966a17817c568ce59d76b92a233fdbec/markupsafe-3.0.3-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:a320721ab5a1aba0a233739394eb907f8c8da5c98c9181d1161e77a0c8e36f2d", size = 20588, upload-time = "2025-09-27T18:37:36.096Z" }, + { url = "https://files.pythonhosted.org/packages/cd/1b/a7782984844bd519ad4ffdbebbba2671ec5d0ebbeac34736c15fb86399e8/markupsafe-3.0.3-cp39-cp39-win32.whl", hash = "sha256:df2449253ef108a379b8b5d6b43f4b1a8e81a061d6537becd5582fba5f9196d7", size = 14566, upload-time = "2025-09-27T18:37:37.09Z" }, + { url = "https://files.pythonhosted.org/packages/18/1f/8d9c20e1c9440e215a44be5ab64359e207fcb4f675543f1cf9a2a7f648d0/markupsafe-3.0.3-cp39-cp39-win_amd64.whl", hash = "sha256:7c3fb7d25180895632e5d3148dbdc29ea38ccb7fd210aa27acbd1201a1902c6e", size = 15053, upload-time = "2025-09-27T18:37:38.054Z" }, + { url = "https://files.pythonhosted.org/packages/4e/d3/fe08482b5cd995033556d45041a4f4e76e7f0521112a9c9991d40d39825f/markupsafe-3.0.3-cp39-cp39-win_arm64.whl", hash = "sha256:38664109c14ffc9e7437e86b4dceb442b0096dfe3541d7864d9cbe1da4cf36c8", size = 13928, upload-time = "2025-09-27T18:37:39.037Z" }, +] + +[[package]] +name = "mdit-py-plugins" +version = "0.4.2" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +dependencies = [ + { name = "markdown-it-py", version = "3.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/19/03/a2ecab526543b152300717cf232bb4bb8605b6edb946c845016fa9c9c9fd/mdit_py_plugins-0.4.2.tar.gz", hash = "sha256:5f2cd1fdb606ddf152d37ec30e46101a60512bc0e5fa1a7002c36647b09e26b5", size = 43542, upload-time = "2024-09-09T20:27:49.564Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a7/f7/7782a043553ee469c1ff49cfa1cdace2d6bf99a1f333cf38676b3ddf30da/mdit_py_plugins-0.4.2-py3-none-any.whl", hash = "sha256:0c673c3f889399a33b95e88d2f0d111b4447bdfea7f237dab2d488f459835636", size = 55316, upload-time = "2024-09-09T20:27:48.397Z" }, +] + +[[package]] +name = "mdit-py-plugins" +version = "0.6.1" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +dependencies = [ + { name = "markdown-it-py", version = "3.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "markdown-it-py", version = "4.2.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/59/fc/f8d0863f8862f25602c0404d75568e89fb6b4109804645e5cdfb1be5cf56/mdit_py_plugins-0.6.1.tar.gz", hash = "sha256:a2bca0f039f39dbd35fb74ae1b5f998608c437463371f0ff7f49a19a17a114d0", size = 56114, upload-time = "2026-05-13T09:03:38.91Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a5/69/6da5581c6a7fede7dc261bf4e67d6adca4196f176b43288b55b3db395b6e/mdit_py_plugins-0.6.1-py3-none-any.whl", hash = "sha256:214c82fb2ac524472ab6a5bcab1de80f73b50443e187f401bfd77efbc7c6481d", size = 66663, upload-time = "2026-05-13T09:03:37.76Z" }, +] + +[[package]] +name = "mdurl" +version = "0.1.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d6/54/cfe61301667036ec958cb99bd3efefba235e65cdeb9c84d24a8293ba1d90/mdurl-0.1.2.tar.gz", hash = "sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba", size = 8729, upload-time = "2022-08-14T12:40:10.846Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl", hash = "sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", size = 9979, upload-time = "2022-08-14T12:40:09.779Z" }, +] + +[[package]] +name = "myst-parser" +version = "3.0.1" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +dependencies = [ + { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "jinja2", marker = "python_full_version < '3.10'" }, + { name = "markdown-it-py", version = "3.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "mdit-py-plugins", version = "0.4.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "pyyaml", marker = "python_full_version < '3.10'" }, + { name = "sphinx", version = "7.4.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/49/64/e2f13dac02f599980798c01156393b781aec983b52a6e4057ee58f07c43a/myst_parser-3.0.1.tar.gz", hash = "sha256:88f0cb406cb363b077d176b51c476f62d60604d68a8dcdf4832e080441301a87", size = 92392, upload-time = "2024-04-28T20:22:42.116Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e2/de/21aa8394f16add8f7427f0a1326ccd2b3a2a8a3245c9252bc5ac034c6155/myst_parser-3.0.1-py3-none-any.whl", hash = "sha256:6457aaa33a5d474aca678b8ead9b3dc298e89c68e67012e73146ea6fd54babf1", size = 83163, upload-time = "2024-04-28T20:22:39.985Z" }, +] + +[[package]] +name = "myst-parser" +version = "4.0.1" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version == '3.10.*'", +] +dependencies = [ + { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "jinja2", marker = "python_full_version == '3.10.*'" }, + { name = "markdown-it-py", version = "3.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "mdit-py-plugins", version = "0.6.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "pyyaml", marker = "python_full_version == '3.10.*'" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/66/a5/9626ba4f73555b3735ad86247a8077d4603aa8628537687c839ab08bfe44/myst_parser-4.0.1.tar.gz", hash = "sha256:5cfea715e4f3574138aecbf7d54132296bfd72bb614d31168f48c477a830a7c4", size = 93985, upload-time = "2025-02-12T10:53:03.833Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5f/df/76d0321c3797b54b60fef9ec3bd6f4cfd124b9e422182156a1dd418722cf/myst_parser-4.0.1-py3-none-any.whl", hash = "sha256:9134e88959ec3b5780aedf8a99680ea242869d012e8821db3126d427edc9c95d", size = 84579, upload-time = "2025-02-12T10:53:02.078Z" }, +] + +[[package]] +name = "myst-parser" +version = "5.1.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", +] +dependencies = [ + { name = "docutils", version = "0.22.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "jinja2", marker = "python_full_version >= '3.11'" }, + { name = "markdown-it-py", version = "4.2.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "mdit-py-plugins", version = "0.6.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "pyyaml", marker = "python_full_version >= '3.11'" }, + { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/21/dc/603751677fff302f34396e206b610f556a59d7fe58b9a2145f54e96b48e8/myst_parser-5.1.0.tar.gz", hash = "sha256:ab69322dc6719dcc7f296479dbb70181b66df6ed315064f92dbc85c0e1bf2f02", size = 101182, upload-time = "2026-05-13T09:38:19.361Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/09/dc/f3dfb7488b770f3f67e6545085bf2abea5172e88f57b8ad25ef860ca704c/myst_parser-5.1.0-py3-none-any.whl", hash = "sha256:9c91c52b3cdb4d94a6506e4fab4e2f296c7623a0da0dcbe6de1565c3dad67a8a", size = 85817, upload-time = "2026-05-13T09:38:17.904Z" }, +] + +[[package]] +name = "packaging" +version = "26.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d7/f1/e7a6dd94a8d4a5626c03e4e99c87f241ba9e350cd9e6d75123f992427270/packaging-26.2.tar.gz", hash = "sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661", size = 228134, upload-time = "2026-04-24T20:15:23.917Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/df/b2/87e62e8c3e2f4b32e5fe99e0b86d576da1312593b39f47d8ceef365e95ed/packaging-26.2-py3-none-any.whl", hash = "sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e", size = 100195, upload-time = "2026-04-24T20:15:22.081Z" }, +] + +[[package]] +name = "pygments" +version = "2.20.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz", hash = "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", size = 4955991, upload-time = "2026-03-29T13:29:33.898Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" }, +] + +[[package]] +name = "pyyaml" +version = "6.0.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/05/8e/961c0007c59b8dd7729d542c61a4d537767a59645b82a0b521206e1e25c2/pyyaml-6.0.3.tar.gz", hash = "sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f", size = 130960, upload-time = "2025-09-25T21:33:16.546Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f4/a0/39350dd17dd6d6c6507025c0e53aef67a9293a6d37d3511f23ea510d5800/pyyaml-6.0.3-cp310-cp310-macosx_10_13_x86_64.whl", hash = "sha256:214ed4befebe12df36bcc8bc2b64b396ca31be9304b8f59e25c11cf94a4c033b", size = 184227, upload-time = "2025-09-25T21:31:46.04Z" }, + { url = "https://files.pythonhosted.org/packages/05/14/52d505b5c59ce73244f59c7a50ecf47093ce4765f116cdb98286a71eeca2/pyyaml-6.0.3-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:02ea2dfa234451bbb8772601d7b8e426c2bfa197136796224e50e35a78777956", size = 174019, upload-time = "2025-09-25T21:31:47.706Z" }, + { url = "https://files.pythonhosted.org/packages/43/f7/0e6a5ae5599c838c696adb4e6330a59f463265bfa1e116cfd1fbb0abaaae/pyyaml-6.0.3-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b30236e45cf30d2b8e7b3e85881719e98507abed1011bf463a8fa23e9c3e98a8", size = 740646, upload-time = "2025-09-25T21:31:49.21Z" }, + { url = "https://files.pythonhosted.org/packages/2f/3a/61b9db1d28f00f8fd0ae760459a5c4bf1b941baf714e207b6eb0657d2578/pyyaml-6.0.3-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:66291b10affd76d76f54fad28e22e51719ef9ba22b29e1d7d03d6777a9174198", size = 840793, upload-time = "2025-09-25T21:31:50.735Z" }, + { url = "https://files.pythonhosted.org/packages/7a/1e/7acc4f0e74c4b3d9531e24739e0ab832a5edf40e64fbae1a9c01941cabd7/pyyaml-6.0.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9c7708761fccb9397fe64bbc0395abcae8c4bf7b0eac081e12b809bf47700d0b", size = 770293, upload-time = "2025-09-25T21:31:51.828Z" }, + { url = "https://files.pythonhosted.org/packages/8b/ef/abd085f06853af0cd59fa5f913d61a8eab65d7639ff2a658d18a25d6a89d/pyyaml-6.0.3-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:418cf3f2111bc80e0933b2cd8cd04f286338bb88bdc7bc8e6dd775ebde60b5e0", size = 732872, upload-time = "2025-09-25T21:31:53.282Z" }, + { url = "https://files.pythonhosted.org/packages/1f/15/2bc9c8faf6450a8b3c9fc5448ed869c599c0a74ba2669772b1f3a0040180/pyyaml-6.0.3-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:5e0b74767e5f8c593e8c9b5912019159ed0533c70051e9cce3e8b6aa699fcd69", size = 758828, upload-time = "2025-09-25T21:31:54.807Z" }, + { url = "https://files.pythonhosted.org/packages/a3/00/531e92e88c00f4333ce359e50c19b8d1de9fe8d581b1534e35ccfbc5f393/pyyaml-6.0.3-cp310-cp310-win32.whl", hash = "sha256:28c8d926f98f432f88adc23edf2e6d4921ac26fb084b028c733d01868d19007e", size = 142415, upload-time = "2025-09-25T21:31:55.885Z" }, + { url = "https://files.pythonhosted.org/packages/2a/fa/926c003379b19fca39dd4634818b00dec6c62d87faf628d1394e137354d4/pyyaml-6.0.3-cp310-cp310-win_amd64.whl", hash = "sha256:bdb2c67c6c1390b63c6ff89f210c8fd09d9a1217a465701eac7316313c915e4c", size = 158561, upload-time = "2025-09-25T21:31:57.406Z" }, + { url = "https://files.pythonhosted.org/packages/6d/16/a95b6757765b7b031c9374925bb718d55e0a9ba8a1b6a12d25962ea44347/pyyaml-6.0.3-cp311-cp311-macosx_10_13_x86_64.whl", hash = "sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e", size = 185826, upload-time = "2025-09-25T21:31:58.655Z" }, + { url = "https://files.pythonhosted.org/packages/16/19/13de8e4377ed53079ee996e1ab0a9c33ec2faf808a4647b7b4c0d46dd239/pyyaml-6.0.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824", size = 175577, upload-time = "2025-09-25T21:32:00.088Z" }, + { url = "https://files.pythonhosted.org/packages/0c/62/d2eb46264d4b157dae1275b573017abec435397aa59cbcdab6fc978a8af4/pyyaml-6.0.3-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c", size = 775556, upload-time = "2025-09-25T21:32:01.31Z" }, + { url = "https://files.pythonhosted.org/packages/10/cb/16c3f2cf3266edd25aaa00d6c4350381c8b012ed6f5276675b9eba8d9ff4/pyyaml-6.0.3-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00", size = 882114, upload-time = "2025-09-25T21:32:03.376Z" }, + { url = "https://files.pythonhosted.org/packages/71/60/917329f640924b18ff085ab889a11c763e0b573da888e8404ff486657602/pyyaml-6.0.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d", size = 806638, upload-time = "2025-09-25T21:32:04.553Z" }, + { url = "https://files.pythonhosted.org/packages/dd/6f/529b0f316a9fd167281a6c3826b5583e6192dba792dd55e3203d3f8e655a/pyyaml-6.0.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a", size = 767463, upload-time = "2025-09-25T21:32:06.152Z" }, + { url = "https://files.pythonhosted.org/packages/f2/6a/b627b4e0c1dd03718543519ffb2f1deea4a1e6d42fbab8021936a4d22589/pyyaml-6.0.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4", size = 794986, upload-time = "2025-09-25T21:32:07.367Z" }, + { url = "https://files.pythonhosted.org/packages/45/91/47a6e1c42d9ee337c4839208f30d9f09caa9f720ec7582917b264defc875/pyyaml-6.0.3-cp311-cp311-win32.whl", hash = "sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b", size = 142543, upload-time = "2025-09-25T21:32:08.95Z" }, + { url = "https://files.pythonhosted.org/packages/da/e3/ea007450a105ae919a72393cb06f122f288ef60bba2dc64b26e2646fa315/pyyaml-6.0.3-cp311-cp311-win_amd64.whl", hash = "sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf", size = 158763, upload-time = "2025-09-25T21:32:09.96Z" }, + { url = "https://files.pythonhosted.org/packages/d1/33/422b98d2195232ca1826284a76852ad5a86fe23e31b009c9886b2d0fb8b2/pyyaml-6.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196", size = 182063, upload-time = "2025-09-25T21:32:11.445Z" }, + { url = "https://files.pythonhosted.org/packages/89/a0/6cf41a19a1f2f3feab0e9c0b74134aa2ce6849093d5517a0c550fe37a648/pyyaml-6.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0", size = 173973, upload-time = "2025-09-25T21:32:12.492Z" }, + { url = "https://files.pythonhosted.org/packages/ed/23/7a778b6bd0b9a8039df8b1b1d80e2e2ad78aa04171592c8a5c43a56a6af4/pyyaml-6.0.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28", size = 775116, upload-time = "2025-09-25T21:32:13.652Z" }, + { url = "https://files.pythonhosted.org/packages/65/30/d7353c338e12baef4ecc1b09e877c1970bd3382789c159b4f89d6a70dc09/pyyaml-6.0.3-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c", size = 844011, upload-time = "2025-09-25T21:32:15.21Z" }, + { url = "https://files.pythonhosted.org/packages/8b/9d/b3589d3877982d4f2329302ef98a8026e7f4443c765c46cfecc8858c6b4b/pyyaml-6.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc", size = 807870, upload-time = "2025-09-25T21:32:16.431Z" }, + { url = "https://files.pythonhosted.org/packages/05/c0/b3be26a015601b822b97d9149ff8cb5ead58c66f981e04fedf4e762f4bd4/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e", size = 761089, upload-time = "2025-09-25T21:32:17.56Z" }, + { url = "https://files.pythonhosted.org/packages/be/8e/98435a21d1d4b46590d5459a22d88128103f8da4c2d4cb8f14f2a96504e1/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea", size = 790181, upload-time = "2025-09-25T21:32:18.834Z" }, + { url = "https://files.pythonhosted.org/packages/74/93/7baea19427dcfbe1e5a372d81473250b379f04b1bd3c4c5ff825e2327202/pyyaml-6.0.3-cp312-cp312-win32.whl", hash = "sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5", size = 137658, upload-time = "2025-09-25T21:32:20.209Z" }, + { url = "https://files.pythonhosted.org/packages/86/bf/899e81e4cce32febab4fb42bb97dcdf66bc135272882d1987881a4b519e9/pyyaml-6.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b", size = 154003, upload-time = "2025-09-25T21:32:21.167Z" }, + { url = "https://files.pythonhosted.org/packages/1a/08/67bd04656199bbb51dbed1439b7f27601dfb576fb864099c7ef0c3e55531/pyyaml-6.0.3-cp312-cp312-win_arm64.whl", hash = "sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd", size = 140344, upload-time = "2025-09-25T21:32:22.617Z" }, + { url = "https://files.pythonhosted.org/packages/d1/11/0fd08f8192109f7169db964b5707a2f1e8b745d4e239b784a5a1dd80d1db/pyyaml-6.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8", size = 181669, upload-time = "2025-09-25T21:32:23.673Z" }, + { url = "https://files.pythonhosted.org/packages/b1/16/95309993f1d3748cd644e02e38b75d50cbc0d9561d21f390a76242ce073f/pyyaml-6.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1", size = 173252, upload-time = "2025-09-25T21:32:25.149Z" }, + { url = "https://files.pythonhosted.org/packages/50/31/b20f376d3f810b9b2371e72ef5adb33879b25edb7a6d072cb7ca0c486398/pyyaml-6.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c", size = 767081, upload-time = "2025-09-25T21:32:26.575Z" }, + { url = "https://files.pythonhosted.org/packages/49/1e/a55ca81e949270d5d4432fbbd19dfea5321eda7c41a849d443dc92fd1ff7/pyyaml-6.0.3-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5", size = 841159, upload-time = "2025-09-25T21:32:27.727Z" }, + { url = "https://files.pythonhosted.org/packages/74/27/e5b8f34d02d9995b80abcef563ea1f8b56d20134d8f4e5e81733b1feceb2/pyyaml-6.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6", size = 801626, upload-time = "2025-09-25T21:32:28.878Z" }, + { url = "https://files.pythonhosted.org/packages/f9/11/ba845c23988798f40e52ba45f34849aa8a1f2d4af4b798588010792ebad6/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6", size = 753613, upload-time = "2025-09-25T21:32:30.178Z" }, + { url = "https://files.pythonhosted.org/packages/3d/e0/7966e1a7bfc0a45bf0a7fb6b98ea03fc9b8d84fa7f2229e9659680b69ee3/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be", size = 794115, upload-time = "2025-09-25T21:32:31.353Z" }, + { url = "https://files.pythonhosted.org/packages/de/94/980b50a6531b3019e45ddeada0626d45fa85cbe22300844a7983285bed3b/pyyaml-6.0.3-cp313-cp313-win32.whl", hash = "sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26", size = 137427, upload-time = "2025-09-25T21:32:32.58Z" }, + { url = "https://files.pythonhosted.org/packages/97/c9/39d5b874e8b28845e4ec2202b5da735d0199dbe5b8fb85f91398814a9a46/pyyaml-6.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c", size = 154090, upload-time = "2025-09-25T21:32:33.659Z" }, + { url = "https://files.pythonhosted.org/packages/73/e8/2bdf3ca2090f68bb3d75b44da7bbc71843b19c9f2b9cb9b0f4ab7a5a4329/pyyaml-6.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb", size = 140246, upload-time = "2025-09-25T21:32:34.663Z" }, + { url = "https://files.pythonhosted.org/packages/9d/8c/f4bd7f6465179953d3ac9bc44ac1a8a3e6122cf8ada906b4f96c60172d43/pyyaml-6.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac", size = 181814, upload-time = "2025-09-25T21:32:35.712Z" }, + { url = "https://files.pythonhosted.org/packages/bd/9c/4d95bb87eb2063d20db7b60faa3840c1b18025517ae857371c4dd55a6b3a/pyyaml-6.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310", size = 173809, upload-time = "2025-09-25T21:32:36.789Z" }, + { url = "https://files.pythonhosted.org/packages/92/b5/47e807c2623074914e29dabd16cbbdd4bf5e9b2db9f8090fa64411fc5382/pyyaml-6.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7", size = 766454, upload-time = "2025-09-25T21:32:37.966Z" }, + { url = "https://files.pythonhosted.org/packages/02/9e/e5e9b168be58564121efb3de6859c452fccde0ab093d8438905899a3a483/pyyaml-6.0.3-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788", size = 836355, upload-time = "2025-09-25T21:32:39.178Z" }, + { url = "https://files.pythonhosted.org/packages/88/f9/16491d7ed2a919954993e48aa941b200f38040928474c9e85ea9e64222c3/pyyaml-6.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5", size = 794175, upload-time = "2025-09-25T21:32:40.865Z" }, + { url = "https://files.pythonhosted.org/packages/dd/3f/5989debef34dc6397317802b527dbbafb2b4760878a53d4166579111411e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764", size = 755228, upload-time = "2025-09-25T21:32:42.084Z" }, + { url = "https://files.pythonhosted.org/packages/d7/ce/af88a49043cd2e265be63d083fc75b27b6ed062f5f9fd6cdc223ad62f03e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35", size = 789194, upload-time = "2025-09-25T21:32:43.362Z" }, + { url = "https://files.pythonhosted.org/packages/23/20/bb6982b26a40bb43951265ba29d4c246ef0ff59c9fdcdf0ed04e0687de4d/pyyaml-6.0.3-cp314-cp314-win_amd64.whl", hash = "sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac", size = 156429, upload-time = "2025-09-25T21:32:57.844Z" }, + { url = "https://files.pythonhosted.org/packages/f4/f4/a4541072bb9422c8a883ab55255f918fa378ecf083f5b85e87fc2b4eda1b/pyyaml-6.0.3-cp314-cp314-win_arm64.whl", hash = "sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3", size = 143912, upload-time = "2025-09-25T21:32:59.247Z" }, + { url = "https://files.pythonhosted.org/packages/7c/f9/07dd09ae774e4616edf6cda684ee78f97777bdd15847253637a6f052a62f/pyyaml-6.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3", size = 189108, upload-time = "2025-09-25T21:32:44.377Z" }, + { url = "https://files.pythonhosted.org/packages/4e/78/8d08c9fb7ce09ad8c38ad533c1191cf27f7ae1effe5bb9400a46d9437fcf/pyyaml-6.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba", size = 183641, upload-time = "2025-09-25T21:32:45.407Z" }, + { url = "https://files.pythonhosted.org/packages/7b/5b/3babb19104a46945cf816d047db2788bcaf8c94527a805610b0289a01c6b/pyyaml-6.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c", size = 831901, upload-time = "2025-09-25T21:32:48.83Z" }, + { url = "https://files.pythonhosted.org/packages/8b/cc/dff0684d8dc44da4d22a13f35f073d558c268780ce3c6ba1b87055bb0b87/pyyaml-6.0.3-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702", size = 861132, upload-time = "2025-09-25T21:32:50.149Z" }, + { url = "https://files.pythonhosted.org/packages/b1/5e/f77dc6b9036943e285ba76b49e118d9ea929885becb0a29ba8a7c75e29fe/pyyaml-6.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c", size = 839261, upload-time = "2025-09-25T21:32:51.808Z" }, + { url = "https://files.pythonhosted.org/packages/ce/88/a9db1376aa2a228197c58b37302f284b5617f56a5d959fd1763fb1675ce6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065", size = 805272, upload-time = "2025-09-25T21:32:52.941Z" }, + { url = "https://files.pythonhosted.org/packages/da/92/1446574745d74df0c92e6aa4a7b0b3130706a4142b2d1a5869f2eaa423c6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65", size = 829923, upload-time = "2025-09-25T21:32:54.537Z" }, + { url = "https://files.pythonhosted.org/packages/f0/7a/1c7270340330e575b92f397352af856a8c06f230aa3e76f86b39d01b416a/pyyaml-6.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9", size = 174062, upload-time = "2025-09-25T21:32:55.767Z" }, + { url = "https://files.pythonhosted.org/packages/f1/12/de94a39c2ef588c7e6455cfbe7343d3b2dc9d6b6b2f40c4c6565744c873d/pyyaml-6.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b", size = 149341, upload-time = "2025-09-25T21:32:56.828Z" }, + { url = "https://files.pythonhosted.org/packages/9f/62/67fc8e68a75f738c9200422bf65693fb79a4cd0dc5b23310e5202e978090/pyyaml-6.0.3-cp39-cp39-macosx_10_13_x86_64.whl", hash = "sha256:b865addae83924361678b652338317d1bd7e79b1f4596f96b96c77a5a34b34da", size = 184450, upload-time = "2025-09-25T21:33:00.618Z" }, + { url = "https://files.pythonhosted.org/packages/ae/92/861f152ce87c452b11b9d0977952259aa7df792d71c1053365cc7b09cc08/pyyaml-6.0.3-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:c3355370a2c156cffb25e876646f149d5d68f5e0a3ce86a5084dd0b64a994917", size = 174319, upload-time = "2025-09-25T21:33:02.086Z" }, + { url = "https://files.pythonhosted.org/packages/d0/cd/f0cfc8c74f8a030017a2b9c771b7f47e5dd702c3e28e5b2071374bda2948/pyyaml-6.0.3-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3c5677e12444c15717b902a5798264fa7909e41153cdf9ef7ad571b704a63dd9", size = 737631, upload-time = "2025-09-25T21:33:03.25Z" }, + { url = "https://files.pythonhosted.org/packages/ef/b2/18f2bd28cd2055a79a46c9b0895c0b3d987ce40ee471cecf58a1a0199805/pyyaml-6.0.3-cp39-cp39-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5ed875a24292240029e4483f9d4a4b8a1ae08843b9c54f43fcc11e404532a8a5", size = 836795, upload-time = "2025-09-25T21:33:05.014Z" }, + { url = "https://files.pythonhosted.org/packages/73/b9/793686b2d54b531203c160ef12bec60228a0109c79bae6c1277961026770/pyyaml-6.0.3-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a", size = 750767, upload-time = "2025-09-25T21:33:06.398Z" }, + { url = "https://files.pythonhosted.org/packages/a9/86/a137b39a611def2ed78b0e66ce2fe13ee701a07c07aebe55c340ed2a050e/pyyaml-6.0.3-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:fa160448684b4e94d80416c0fa4aac48967a969efe22931448d853ada8baf926", size = 727982, upload-time = "2025-09-25T21:33:08.708Z" }, + { url = "https://files.pythonhosted.org/packages/dd/62/71c27c94f457cf4418ef8ccc71735324c549f7e3ea9d34aba50874563561/pyyaml-6.0.3-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:27c0abcb4a5dac13684a37f76e701e054692a9b2d3064b70f5e4eb54810553d7", size = 755677, upload-time = "2025-09-25T21:33:09.876Z" }, + { url = "https://files.pythonhosted.org/packages/29/3d/6f5e0d58bd924fb0d06c3a6bad00effbdae2de5adb5cda5648006ffbd8d3/pyyaml-6.0.3-cp39-cp39-win32.whl", hash = "sha256:1ebe39cb5fc479422b83de611d14e2c0d3bb2a18bbcb01f229ab3cfbd8fee7a0", size = 142592, upload-time = "2025-09-25T21:33:10.983Z" }, + { url = "https://files.pythonhosted.org/packages/f0/0c/25113e0b5e103d7f1490c0e947e303fe4a696c10b501dea7a9f49d4e876c/pyyaml-6.0.3-cp39-cp39-win_amd64.whl", hash = "sha256:2e71d11abed7344e42a8849600193d15b6def118602c4c176f748e4583246007", size = 158777, upload-time = "2025-09-25T21:33:15.55Z" }, +] + +[[package]] +name = "readthedocs-sphinx-ext" +version = "2.2.5" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "jinja2" }, + { name = "packaging" }, + { name = "requests", version = "2.32.5", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "requests", version = "2.34.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/e8/ce/38130d8dec600bf5413eb89a3413dd38f204c7c728c4947e12ff8cb793b7/readthedocs-sphinx-ext-2.2.5.tar.gz", hash = "sha256:ee5fd5b99db9f0c180b2396cbce528aa36671951b9526bb0272dbfce5517bd27", size = 12303, upload-time = "2023-12-19T10:00:49.573Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/64/71/c89e7709a0d4f93af1848e9855112299a820b470d84f917b4dd5998bdd07/readthedocs_sphinx_ext-2.2.5-py2.py3-none-any.whl", hash = "sha256:f8c56184ea011c972dd45a90122568587cc85b0127bc9cf064d17c68bc809daa", size = 11332, upload-time = "2023-12-19T10:00:43.972Z" }, +] + +[[package]] +name = "requests" +version = "2.32.5" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +dependencies = [ + { name = "certifi", marker = "python_full_version < '3.10'" }, + { name = "charset-normalizer", marker = "python_full_version < '3.10'" }, + { name = "idna", marker = "python_full_version < '3.10'" }, + { name = "urllib3", version = "2.6.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/c9/74/b3ff8e6c8446842c3f5c837e9c3dfcfe2018ea6ecef224c710c85ef728f4/requests-2.32.5.tar.gz", hash = "sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf", size = 134517, upload-time = "2025-08-18T20:46:02.573Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/1e/db/4254e3eabe8020b458f1a747140d32277ec7a271daf1d235b70dc0b4e6e3/requests-2.32.5-py3-none-any.whl", hash = "sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6", size = 64738, upload-time = "2025-08-18T20:46:00.542Z" }, +] + +[[package]] +name = "requests" +version = "2.34.2" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +dependencies = [ + { name = "certifi", marker = "python_full_version >= '3.10'" }, + { name = "charset-normalizer", marker = "python_full_version >= '3.10'" }, + { name = "idna", marker = "python_full_version >= '3.10'" }, + { name = "urllib3", version = "2.7.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/ac/c3/e2a2b89f2d3e2179abd6d00ebd70bff6273f37fb3e0cc209f48b39d00cbf/requests-2.34.2.tar.gz", hash = "sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed", size = 142856, upload-time = "2026-05-14T19:25:27.735Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a0/f4/c67b0b3f1b9245e8d266f0f112c500d50e5b4e83cb6f3b71b6528104182a/requests-2.34.2-py3-none-any.whl", hash = "sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0", size = 73075, upload-time = "2026-05-14T19:25:26.443Z" }, +] + +[[package]] +name = "roman-numerals" +version = "4.1.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/ae/f9/41dc953bbeb056c17d5f7a519f50fdf010bd0553be2d630bc69d1e022703/roman_numerals-4.1.0.tar.gz", hash = "sha256:1af8b147eb1405d5839e78aeb93131690495fe9da5c91856cb33ad55a7f1e5b2", size = 9077, upload-time = "2025-12-17T18:25:34.381Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/04/54/6f679c435d28e0a568d8e8a7c0a93a09010818634c3c3907fc98d8983770/roman_numerals-4.1.0-py3-none-any.whl", hash = "sha256:647ba99caddc2cc1e55a51e4360689115551bf4476d90e8162cf8c345fe233c7", size = 7676, upload-time = "2025-12-17T18:25:33.098Z" }, +] + +[[package]] +name = "rules-python-sphinxdocs-dev" +version = "0.0.0" +source = { virtual = "." } +dependencies = [ + { name = "absl-py", version = "2.3.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "absl-py", version = "2.5.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, + { name = "markupsafe" }, + { name = "myst-parser", version = "3.0.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "myst-parser", version = "4.0.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "myst-parser", version = "5.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "readthedocs-sphinx-ext" }, + { name = "sphinx", version = "7.4.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "sphinx-autodoc2" }, + { name = "sphinx-reredirects", version = "0.1.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "sphinx-reredirects", version = "1.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "sphinx-rtd-theme" }, + { name = "typing-extensions" }, +] + +[package.metadata] +requires-dist = [ + { name = "absl-py" }, + { name = "markupsafe" }, + { name = "myst-parser" }, + { name = "readthedocs-sphinx-ext" }, + { name = "sphinx" }, + { name = "sphinx-autodoc2" }, + { name = "sphinx-reredirects" }, + { name = "sphinx-rtd-theme", specifier = ">=2.0" }, + { name = "typing-extensions" }, +] + +[[package]] +name = "snowballstemmer" +version = "3.1.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/43/f8/0a71edf031f03c40db17503cb8ca78a69a171254e568e7db241b0ab57ea1/snowballstemmer-3.1.1.tar.gz", hash = "sha256:e07bbc54a0d798fe6010a12398422e62a8bfbba95c394fd0956ef58cb4d3e260", size = 123314, upload-time = "2026-06-03T00:56:40.194Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/4c/07/2ebca9b11fb9be7340a818d8d6f63feaebb146be2c4afbd6061701d6df6e/snowballstemmer-3.1.1-py3-none-any.whl", hash = "sha256:7e207fa178741da09cdee59d3ecec3827ad5f92b1fc5c9ff3755b639f71f5752", size = 104164, upload-time = "2026-06-03T00:56:38.614Z" }, +] + +[[package]] +name = "sphinx" +version = "7.4.7" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +dependencies = [ + { name = "alabaster", version = "0.7.16", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "babel", marker = "python_full_version < '3.10'" }, + { name = "colorama", marker = "python_full_version < '3.10' and sys_platform == 'win32'" }, + { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "imagesize", version = "1.5.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "importlib-metadata", marker = "python_full_version < '3.10'" }, + { name = "jinja2", marker = "python_full_version < '3.10'" }, + { name = "packaging", marker = "python_full_version < '3.10'" }, + { name = "pygments", marker = "python_full_version < '3.10'" }, + { name = "requests", version = "2.32.5", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "snowballstemmer", marker = "python_full_version < '3.10'" }, + { name = "sphinxcontrib-applehelp", marker = "python_full_version < '3.10'" }, + { name = "sphinxcontrib-devhelp", marker = "python_full_version < '3.10'" }, + { name = "sphinxcontrib-htmlhelp", marker = "python_full_version < '3.10'" }, + { name = "sphinxcontrib-jsmath", marker = "python_full_version < '3.10'" }, + { name = "sphinxcontrib-qthelp", marker = "python_full_version < '3.10'" }, + { name = "sphinxcontrib-serializinghtml", marker = "python_full_version < '3.10'" }, + { name = "tomli", marker = "python_full_version < '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/5b/be/50e50cb4f2eff47df05673d361095cafd95521d2a22521b920c67a372dcb/sphinx-7.4.7.tar.gz", hash = "sha256:242f92a7ea7e6c5b406fdc2615413890ba9f699114a9c09192d7dfead2ee9cfe", size = 8067911, upload-time = "2024-07-20T14:46:56.059Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0d/ef/153f6803c5d5f8917dbb7f7fcf6d34a871ede3296fa89c2c703f5f8a6c8e/sphinx-7.4.7-py3-none-any.whl", hash = "sha256:c2419e2135d11f1951cd994d6eb18a1835bd8fdd8429f9ca375dc1f3281bd239", size = 3401624, upload-time = "2024-07-20T14:46:52.142Z" }, +] + +[[package]] +name = "sphinx" +version = "8.1.3" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version == '3.10.*'", +] +dependencies = [ + { name = "alabaster", version = "1.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "babel", marker = "python_full_version == '3.10.*'" }, + { name = "colorama", marker = "python_full_version == '3.10.*' and sys_platform == 'win32'" }, + { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "imagesize", version = "2.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "jinja2", marker = "python_full_version == '3.10.*'" }, + { name = "packaging", marker = "python_full_version == '3.10.*'" }, + { name = "pygments", marker = "python_full_version == '3.10.*'" }, + { name = "requests", version = "2.34.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "snowballstemmer", marker = "python_full_version == '3.10.*'" }, + { name = "sphinxcontrib-applehelp", marker = "python_full_version == '3.10.*'" }, + { name = "sphinxcontrib-devhelp", marker = "python_full_version == '3.10.*'" }, + { name = "sphinxcontrib-htmlhelp", marker = "python_full_version == '3.10.*'" }, + { name = "sphinxcontrib-jsmath", marker = "python_full_version == '3.10.*'" }, + { name = "sphinxcontrib-qthelp", marker = "python_full_version == '3.10.*'" }, + { name = "sphinxcontrib-serializinghtml", marker = "python_full_version == '3.10.*'" }, + { name = "tomli", marker = "python_full_version == '3.10.*'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/6f/6d/be0b61178fe2cdcb67e2a92fc9ebb488e3c51c4f74a36a7824c0adf23425/sphinx-8.1.3.tar.gz", hash = "sha256:43c1911eecb0d3e161ad78611bc905d1ad0e523e4ddc202a58a821773dc4c927", size = 8184611, upload-time = "2024-10-13T20:27:13.93Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/26/60/1ddff83a56d33aaf6f10ec8ce84b4c007d9368b21008876fceda7e7381ef/sphinx-8.1.3-py3-none-any.whl", hash = "sha256:09719015511837b76bf6e03e42eb7595ac8c2e41eeb9c29c5b755c6b677992a2", size = 3487125, upload-time = "2024-10-13T20:27:10.448Z" }, +] + +[[package]] +name = "sphinx" +version = "9.0.4" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version == '3.11.*'", +] +dependencies = [ + { name = "alabaster", version = "1.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "babel", marker = "python_full_version == '3.11.*'" }, + { name = "colorama", marker = "python_full_version == '3.11.*' and sys_platform == 'win32'" }, + { name = "docutils", version = "0.22.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "imagesize", version = "2.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "jinja2", marker = "python_full_version == '3.11.*'" }, + { name = "packaging", marker = "python_full_version == '3.11.*'" }, + { name = "pygments", marker = "python_full_version == '3.11.*'" }, + { name = "requests", version = "2.34.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "roman-numerals", marker = "python_full_version == '3.11.*'" }, + { name = "snowballstemmer", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-applehelp", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-devhelp", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-htmlhelp", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-jsmath", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-qthelp", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-serializinghtml", marker = "python_full_version == '3.11.*'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/42/50/a8c6ccc36d5eacdfd7913ddccd15a9cee03ecafc5ee2bc40e1f168d85022/sphinx-9.0.4.tar.gz", hash = "sha256:594ef59d042972abbc581d8baa577404abe4e6c3b04ef61bd7fc2acbd51f3fa3", size = 8710502, upload-time = "2025-12-04T07:45:27.343Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c6/3f/4bbd76424c393caead2e1eb89777f575dee5c8653e2d4b6afd7a564f5974/sphinx-9.0.4-py3-none-any.whl", hash = "sha256:5bebc595a5e943ea248b99c13814c1c5e10b3ece718976824ffa7959ff95fffb", size = 3917713, upload-time = "2025-12-04T07:45:24.944Z" }, +] + +[[package]] +name = "sphinx" +version = "9.1.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", +] +dependencies = [ + { name = "alabaster", version = "1.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "babel", marker = "python_full_version >= '3.12'" }, + { name = "colorama", marker = "python_full_version >= '3.12' and sys_platform == 'win32'" }, + { name = "docutils", version = "0.22.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "imagesize", version = "2.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "jinja2", marker = "python_full_version >= '3.12'" }, + { name = "packaging", marker = "python_full_version >= '3.12'" }, + { name = "pygments", marker = "python_full_version >= '3.12'" }, + { name = "requests", version = "2.34.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "roman-numerals", marker = "python_full_version >= '3.12'" }, + { name = "snowballstemmer", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-applehelp", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-devhelp", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-htmlhelp", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-jsmath", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-qthelp", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-serializinghtml", marker = "python_full_version >= '3.12'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/cd/bd/f08eb0f4eed5c83f1ba2a3bd18f7745a2b1525fad70660a1c00224ec468a/sphinx-9.1.0.tar.gz", hash = "sha256:7741722357dd75f8190766926071fed3bdc211c74dd2d7d4df5404da95930ddb", size = 8718324, upload-time = "2025-12-31T15:09:27.646Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/73/f7/b1884cb3188ab181fc81fa00c266699dab600f927a964df02ec3d5d1916a/sphinx-9.1.0-py3-none-any.whl", hash = "sha256:c84fdd4e782504495fe4f2c0b3413d6c2bf388589bb352d439b2a3bb99991978", size = 3921742, upload-time = "2025-12-31T15:09:25.561Z" }, +] + +[[package]] +name = "sphinx-autodoc2" +version = "0.5.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "astroid" }, + { name = "tomli", marker = "python_full_version < '3.11'" }, + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/17/5f/5350046d1aa1a56b063ae08b9ad871025335c9d55fe2372896ea48711da9/sphinx_autodoc2-0.5.0.tar.gz", hash = "sha256:7d76044aa81d6af74447080182b6868c7eb066874edc835e8ddf810735b6565a", size = 115077, upload-time = "2023-11-27T07:27:51.407Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/19/e6/48d47961bbdae755ba9c17dfc65d89356312c67668dcb36c87cfadfa1964/sphinx_autodoc2-0.5.0-py3-none-any.whl", hash = "sha256:e867013b1512f9d6d7e6f6799f8b537d6884462acd118ef361f3f619a60b5c9e", size = 43385, upload-time = "2023-11-27T07:27:49.929Z" }, +] + +[[package]] +name = "sphinx-reredirects" +version = "0.1.6" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version == '3.10.*'", + "python_full_version < '3.10'", +] +dependencies = [ + { name = "sphinx", version = "7.4.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/16/6b/bcca2785de4071f604a722444d4d7ba8a9d40de3c14ad52fce93e6d92694/sphinx_reredirects-0.1.6.tar.gz", hash = "sha256:c491cba545f67be9697508727818d8626626366245ae64456fe29f37e9bbea64", size = 7080, upload-time = "2025-03-22T10:52:30.271Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ac/6f/0b3625be30a1a50f9e4c2cb2ec147b08f15ed0e9f8444efcf274b751300b/sphinx_reredirects-0.1.6-py3-none-any.whl", hash = "sha256:efd50c766fbc5bf40cd5148e10c00f2c00d143027de5c5e48beece93cc40eeea", size = 5675, upload-time = "2025-03-22T10:52:29.113Z" }, +] + +[[package]] +name = "sphinx-reredirects" +version = "1.1.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", +] +dependencies = [ + { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/1b/8d/0e39fe2740d7d71417edf9a6424aa80ca2c27c17fc21282cdc39f90d5a40/sphinx_reredirects-1.1.0.tar.gz", hash = "sha256:fb9b195335ab14b43f8273287d0c7eeb637ba6c56c66581c11b47202f6718b29", size = 614624, upload-time = "2025-12-22T08:28:02.792Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/51/81/b5dd07067f3daac6d23687ec737b2d593740671ebcd145830c8f92d381c5/sphinx_reredirects-1.1.0-py3-none-any.whl", hash = "sha256:4b5692273c72cd2d4d917f4c6f87d5919e4d6114a752d4be033f7f5f6310efd9", size = 6351, upload-time = "2025-12-22T08:27:59.724Z" }, +] + +[[package]] +name = "sphinx-rtd-theme" +version = "3.1.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "docutils", version = "0.22.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "sphinx", version = "7.4.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-jquery" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/84/68/a1bfbf38c0f7bccc9b10bbf76b94606f64acb1552ae394f0b8285bfaea25/sphinx_rtd_theme-3.1.0.tar.gz", hash = "sha256:b44276f2c276e909239a4f6c955aa667aaafeb78597923b1c60babc76db78e4c", size = 7620915, upload-time = "2026-01-12T16:03:31.17Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/87/c7/b5c8015d823bfda1a346adb2c634a2101d50bb75d421eb6dcb31acd25ebc/sphinx_rtd_theme-3.1.0-py2.py3-none-any.whl", hash = "sha256:1785824ae8e6632060490f67cf3a72d404a85d2d9fc26bce3619944de5682b89", size = 7655617, upload-time = "2026-01-12T16:03:28.101Z" }, +] + +[[package]] +name = "sphinxcontrib-applehelp" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/ba/6e/b837e84a1a704953c62ef8776d45c3e8d759876b4a84fe14eba2859106fe/sphinxcontrib_applehelp-2.0.0.tar.gz", hash = "sha256:2f29ef331735ce958efa4734873f084941970894c6090408b079c61b2e1c06d1", size = 20053, upload-time = "2024-07-29T01:09:00.465Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5d/85/9ebeae2f76e9e77b952f4b274c27238156eae7979c5421fba91a28f4970d/sphinxcontrib_applehelp-2.0.0-py3-none-any.whl", hash = "sha256:4cd3f0ec4ac5dd9c17ec65e9ab272c9b867ea77425228e68ecf08d6b28ddbdb5", size = 119300, upload-time = "2024-07-29T01:08:58.99Z" }, +] + +[[package]] +name = "sphinxcontrib-devhelp" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/f6/d2/5beee64d3e4e747f316bae86b55943f51e82bb86ecd325883ef65741e7da/sphinxcontrib_devhelp-2.0.0.tar.gz", hash = "sha256:411f5d96d445d1d73bb5d52133377b4248ec79db5c793ce7dbe59e074b4dd1ad", size = 12967, upload-time = "2024-07-29T01:09:23.417Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/35/7a/987e583882f985fe4d7323774889ec58049171828b58c2217e7f79cdf44e/sphinxcontrib_devhelp-2.0.0-py3-none-any.whl", hash = "sha256:aefb8b83854e4b0998877524d1029fd3e6879210422ee3780459e28a1f03a8a2", size = 82530, upload-time = "2024-07-29T01:09:21.945Z" }, +] + +[[package]] +name = "sphinxcontrib-htmlhelp" +version = "2.1.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/43/93/983afd9aa001e5201eab16b5a444ed5b9b0a7a010541e0ddfbbfd0b2470c/sphinxcontrib_htmlhelp-2.1.0.tar.gz", hash = "sha256:c9e2916ace8aad64cc13a0d233ee22317f2b9025b9cf3295249fa985cc7082e9", size = 22617, upload-time = "2024-07-29T01:09:37.889Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0a/7b/18a8c0bcec9182c05a0b3ec2a776bba4ead82750a55ff798e8d406dae604/sphinxcontrib_htmlhelp-2.1.0-py3-none-any.whl", hash = "sha256:166759820b47002d22914d64a075ce08f4c46818e17cfc9470a9786b759b19f8", size = 98705, upload-time = "2024-07-29T01:09:36.407Z" }, +] + +[[package]] +name = "sphinxcontrib-jquery" +version = "4.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "sphinx", version = "7.4.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/de/f3/aa67467e051df70a6330fe7770894b3e4f09436dea6881ae0b4f3d87cad8/sphinxcontrib-jquery-4.1.tar.gz", hash = "sha256:1620739f04e36a2c779f1a131a2dfd49b2fd07351bf1968ced074365933abc7a", size = 122331, upload-time = "2023-03-14T15:01:01.944Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/76/85/749bd22d1a68db7291c89e2ebca53f4306c3f205853cf31e9de279034c3c/sphinxcontrib_jquery-4.1-py2.py3-none-any.whl", hash = "sha256:f936030d7d0147dd026a4f2b5a57343d233f1fc7b363f68b3d4f1cb0993878ae", size = 121104, upload-time = "2023-03-14T15:01:00.356Z" }, +] + +[[package]] +name = "sphinxcontrib-jsmath" +version = "1.0.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/b2/e8/9ed3830aeed71f17c026a07a5097edcf44b692850ef215b161b8ad875729/sphinxcontrib-jsmath-1.0.1.tar.gz", hash = "sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8", size = 5787, upload-time = "2019-01-21T16:10:16.347Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c2/42/4c8646762ee83602e3fb3fbe774c2fac12f317deb0b5dbeeedd2d3ba4b77/sphinxcontrib_jsmath-1.0.1-py2.py3-none-any.whl", hash = "sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178", size = 5071, upload-time = "2019-01-21T16:10:14.333Z" }, +] + +[[package]] +name = "sphinxcontrib-qthelp" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/68/bc/9104308fc285eb3e0b31b67688235db556cd5b0ef31d96f30e45f2e51cae/sphinxcontrib_qthelp-2.0.0.tar.gz", hash = "sha256:4fe7d0ac8fc171045be623aba3e2a8f613f8682731f9153bb2e40ece16b9bbab", size = 17165, upload-time = "2024-07-29T01:09:56.435Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/27/83/859ecdd180cacc13b1f7e857abf8582a64552ea7a061057a6c716e790fce/sphinxcontrib_qthelp-2.0.0-py3-none-any.whl", hash = "sha256:b18a828cdba941ccd6ee8445dbe72ffa3ef8cbe7505d8cd1fa0d42d3f2d5f3eb", size = 88743, upload-time = "2024-07-29T01:09:54.885Z" }, +] + +[[package]] +name = "sphinxcontrib-serializinghtml" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/3b/44/6716b257b0aa6bfd51a1b31665d1c205fb12cb5ad56de752dfa15657de2f/sphinxcontrib_serializinghtml-2.0.0.tar.gz", hash = "sha256:e9d912827f872c029017a53f0ef2180b327c3f7fd23c87229f7a8e8b70031d4d", size = 16080, upload-time = "2024-07-29T01:10:09.332Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/52/a7/d2782e4e3f77c8450f727ba74a8f12756d5ba823d81b941f1b04da9d033a/sphinxcontrib_serializinghtml-2.0.0-py3-none-any.whl", hash = "sha256:6e2cb0eef194e10c27ec0023bfeb25badbbb5868244cf5bc5bdc04e4464bf331", size = 92072, upload-time = "2024-07-29T01:10:08.203Z" }, +] + +[[package]] +name = "tomli" +version = "2.4.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/22/de/48c59722572767841493b26183a0d1cc411d54fd759c5607c4590b6563a6/tomli-2.4.1.tar.gz", hash = "sha256:7c7e1a961a0b2f2472c1ac5b69affa0ae1132c39adcb67aba98568702b9cc23f", size = 17543, upload-time = "2026-03-25T20:22:03.828Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f4/11/db3d5885d8528263d8adc260bb2d28ebf1270b96e98f0e0268d32b8d9900/tomli-2.4.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:f8f0fc26ec2cc2b965b7a3b87cd19c5c6b8c5e5f436b984e85f486d652285c30", size = 154704, upload-time = "2026-03-25T20:21:10.473Z" }, + { url = "https://files.pythonhosted.org/packages/6d/f7/675db52c7e46064a9aa928885a9b20f4124ecb9bc2e1ce74c9106648d202/tomli-2.4.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:4ab97e64ccda8756376892c53a72bd1f964e519c77236368527f758fbc36a53a", size = 149454, upload-time = "2026-03-25T20:21:12.036Z" }, + { url = "https://files.pythonhosted.org/packages/61/71/81c50943cf953efa35bce7646caab3cf457a7d8c030b27cfb40d7235f9ee/tomli-2.4.1-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:96481a5786729fd470164b47cdb3e0e58062a496f455ee41b4403be77cb5a076", size = 237561, upload-time = "2026-03-25T20:21:13.098Z" }, + { url = "https://files.pythonhosted.org/packages/48/c1/f41d9cb618acccca7df82aaf682f9b49013c9397212cb9f53219e3abac37/tomli-2.4.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5a881ab208c0baf688221f8cecc5401bd291d67e38a1ac884d6736cbcd8247e9", size = 243824, upload-time = "2026-03-25T20:21:14.569Z" }, + { url = "https://files.pythonhosted.org/packages/22/e4/5a816ecdd1f8ca51fb756ef684b90f2780afc52fc67f987e3c61d800a46d/tomli-2.4.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:47149d5bd38761ac8be13a84864bf0b7b70bc051806bc3669ab1cbc56216b23c", size = 242227, upload-time = "2026-03-25T20:21:15.712Z" }, + { url = "https://files.pythonhosted.org/packages/6b/49/2b2a0ef529aa6eec245d25f0c703e020a73955ad7edf73e7f54ddc608aa5/tomli-2.4.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:ec9bfaf3ad2df51ace80688143a6a4ebc09a248f6ff781a9945e51937008fcbc", size = 247859, upload-time = "2026-03-25T20:21:17.001Z" }, + { url = "https://files.pythonhosted.org/packages/83/bd/6c1a630eaca337e1e78c5903104f831bda934c426f9231429396ce3c3467/tomli-2.4.1-cp311-cp311-win32.whl", hash = "sha256:ff2983983d34813c1aeb0fa89091e76c3a22889ee83ab27c5eeb45100560c049", size = 97204, upload-time = "2026-03-25T20:21:18.079Z" }, + { url = "https://files.pythonhosted.org/packages/42/59/71461df1a885647e10b6bb7802d0b8e66480c61f3f43079e0dcd315b3954/tomli-2.4.1-cp311-cp311-win_amd64.whl", hash = "sha256:5ee18d9ebdb417e384b58fe414e8d6af9f4e7a0ae761519fb50f721de398dd4e", size = 108084, upload-time = "2026-03-25T20:21:18.978Z" }, + { url = "https://files.pythonhosted.org/packages/b8/83/dceca96142499c069475b790e7913b1044c1a4337e700751f48ed723f883/tomli-2.4.1-cp311-cp311-win_arm64.whl", hash = "sha256:c2541745709bad0264b7d4705ad453b76ccd191e64aa6f0fc66b69a293a45ece", size = 95285, upload-time = "2026-03-25T20:21:20.309Z" }, + { url = "https://files.pythonhosted.org/packages/c1/ba/42f134a3fe2b370f555f44b1d72feebb94debcab01676bf918d0cb70e9aa/tomli-2.4.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:c742f741d58a28940ce01d58f0ab2ea3ced8b12402f162f4d534dfe18ba1cd6a", size = 155924, upload-time = "2026-03-25T20:21:21.626Z" }, + { url = "https://files.pythonhosted.org/packages/dc/c7/62d7a17c26487ade21c5422b646110f2162f1fcc95980ef7f63e73c68f14/tomli-2.4.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:7f86fd587c4ed9dd76f318225e7d9b29cfc5a9d43de44e5754db8d1128487085", size = 150018, upload-time = "2026-03-25T20:21:23.002Z" }, + { url = "https://files.pythonhosted.org/packages/5c/05/79d13d7c15f13bdef410bdd49a6485b1c37d28968314eabee452c22a7fda/tomli-2.4.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ff18e6a727ee0ab0388507b89d1bc6a22b138d1e2fa56d1ad494586d61d2eae9", size = 244948, upload-time = "2026-03-25T20:21:24.04Z" }, + { url = "https://files.pythonhosted.org/packages/10/90/d62ce007a1c80d0b2c93e02cab211224756240884751b94ca72df8a875ca/tomli-2.4.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:136443dbd7e1dee43c68ac2694fde36b2849865fa258d39bf822c10e8068eac5", size = 253341, upload-time = "2026-03-25T20:21:25.177Z" }, + { url = "https://files.pythonhosted.org/packages/1a/7e/caf6496d60152ad4ed09282c1885cca4eea150bfd007da84aea07bcc0a3e/tomli-2.4.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:5e262d41726bc187e69af7825504c933b6794dc3fbd5945e41a79bb14c31f585", size = 248159, upload-time = "2026-03-25T20:21:26.364Z" }, + { url = "https://files.pythonhosted.org/packages/99/e7/c6f69c3120de34bbd882c6fba7975f3d7a746e9218e56ab46a1bc4b42552/tomli-2.4.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:5cb41aa38891e073ee49d55fbc7839cfdb2bc0e600add13874d048c94aadddd1", size = 253290, upload-time = "2026-03-25T20:21:27.46Z" }, + { url = "https://files.pythonhosted.org/packages/d6/2f/4a3c322f22c5c66c4b836ec58211641a4067364f5dcdd7b974b4c5da300c/tomli-2.4.1-cp312-cp312-win32.whl", hash = "sha256:da25dc3563bff5965356133435b757a795a17b17d01dbc0f42fb32447ddfd917", size = 98141, upload-time = "2026-03-25T20:21:28.492Z" }, + { url = "https://files.pythonhosted.org/packages/24/22/4daacd05391b92c55759d55eaee21e1dfaea86ce5c571f10083360adf534/tomli-2.4.1-cp312-cp312-win_amd64.whl", hash = "sha256:52c8ef851d9a240f11a88c003eacb03c31fc1c9c4ec64a99a0f922b93874fda9", size = 108847, upload-time = "2026-03-25T20:21:29.386Z" }, + { url = "https://files.pythonhosted.org/packages/68/fd/70e768887666ddd9e9f5d85129e84910f2db2796f9096aa02b721a53098d/tomli-2.4.1-cp312-cp312-win_arm64.whl", hash = "sha256:f758f1b9299d059cc3f6546ae2af89670cb1c4d48ea29c3cacc4fe7de3058257", size = 95088, upload-time = "2026-03-25T20:21:30.677Z" }, + { url = "https://files.pythonhosted.org/packages/07/06/b823a7e818c756d9a7123ba2cda7d07bc2dd32835648d1a7b7b7a05d848d/tomli-2.4.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:36d2bd2ad5fb9eaddba5226aa02c8ec3fa4f192631e347b3ed28186d43be6b54", size = 155866, upload-time = "2026-03-25T20:21:31.65Z" }, + { url = "https://files.pythonhosted.org/packages/14/6f/12645cf7f08e1a20c7eb8c297c6f11d31c1b50f316a7e7e1e1de6e2e7b7e/tomli-2.4.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:eb0dc4e38e6a1fd579e5d50369aa2e10acfc9cace504579b2faabb478e76941a", size = 149887, upload-time = "2026-03-25T20:21:33.028Z" }, + { url = "https://files.pythonhosted.org/packages/5c/e0/90637574e5e7212c09099c67ad349b04ec4d6020324539297b634a0192b0/tomli-2.4.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c7f2c7f2b9ca6bdeef8f0fa897f8e05085923eb091721675170254cbc5b02897", size = 243704, upload-time = "2026-03-25T20:21:34.51Z" }, + { url = "https://files.pythonhosted.org/packages/10/8f/d3ddb16c5a4befdf31a23307f72828686ab2096f068eaf56631e136c1fdd/tomli-2.4.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f3c6818a1a86dd6dca7ddcaaf76947d5ba31aecc28cb1b67009a5877c9a64f3f", size = 251628, upload-time = "2026-03-25T20:21:36.012Z" }, + { url = "https://files.pythonhosted.org/packages/e3/f1/dbeeb9116715abee2485bf0a12d07a8f31af94d71608c171c45f64c0469d/tomli-2.4.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:d312ef37c91508b0ab2cee7da26ec0b3ed2f03ce12bd87a588d771ae15dcf82d", size = 247180, upload-time = "2026-03-25T20:21:37.136Z" }, + { url = "https://files.pythonhosted.org/packages/d3/74/16336ffd19ed4da28a70959f92f506233bd7cfc2332b20bdb01591e8b1d1/tomli-2.4.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:51529d40e3ca50046d7606fa99ce3956a617f9b36380da3b7f0dd3dd28e68cb5", size = 251674, upload-time = "2026-03-25T20:21:38.298Z" }, + { url = "https://files.pythonhosted.org/packages/16/f9/229fa3434c590ddf6c0aa9af64d3af4b752540686cace29e6281e3458469/tomli-2.4.1-cp313-cp313-win32.whl", hash = "sha256:2190f2e9dd7508d2a90ded5ed369255980a1bcdd58e52f7fe24b8162bf9fedbd", size = 97976, upload-time = "2026-03-25T20:21:39.316Z" }, + { url = "https://files.pythonhosted.org/packages/6a/1e/71dfd96bcc1c775420cb8befe7a9d35f2e5b1309798f009dca17b7708c1e/tomli-2.4.1-cp313-cp313-win_amd64.whl", hash = "sha256:8d65a2fbf9d2f8352685bc1364177ee3923d6baf5e7f43ea4959d7d8bc326a36", size = 108755, upload-time = "2026-03-25T20:21:40.248Z" }, + { url = "https://files.pythonhosted.org/packages/83/7a/d34f422a021d62420b78f5c538e5b102f62bea616d1d75a13f0a88acb04a/tomli-2.4.1-cp313-cp313-win_arm64.whl", hash = "sha256:4b605484e43cdc43f0954ddae319fb75f04cc10dd80d830540060ee7cd0243cd", size = 95265, upload-time = "2026-03-25T20:21:41.219Z" }, + { url = "https://files.pythonhosted.org/packages/3c/fb/9a5c8d27dbab540869f7c1f8eb0abb3244189ce780ba9cd73f3770662072/tomli-2.4.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:fd0409a3653af6c147209d267a0e4243f0ae46b011aa978b1080359fddc9b6cf", size = 155726, upload-time = "2026-03-25T20:21:42.23Z" }, + { url = "https://files.pythonhosted.org/packages/62/05/d2f816630cc771ad836af54f5001f47a6f611d2d39535364f148b6a92d6b/tomli-2.4.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:a120733b01c45e9a0c34aeef92bf0cf1d56cfe81ed9d47d562f9ed591a9828ac", size = 149859, upload-time = "2026-03-25T20:21:43.386Z" }, + { url = "https://files.pythonhosted.org/packages/ce/48/66341bdb858ad9bd0ceab5a86f90eddab127cf8b046418009f2125630ecb/tomli-2.4.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:559db847dc486944896521f68d8190be1c9e719fced785720d2216fe7022b662", size = 244713, upload-time = "2026-03-25T20:21:44.474Z" }, + { url = "https://files.pythonhosted.org/packages/df/6d/c5fad00d82b3c7a3ab6189bd4b10e60466f22cfe8a08a9394185c8a8111c/tomli-2.4.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:01f520d4f53ef97964a240a035ec2a869fe1a37dde002b57ebc4417a27ccd853", size = 252084, upload-time = "2026-03-25T20:21:45.62Z" }, + { url = "https://files.pythonhosted.org/packages/00/71/3a69e86f3eafe8c7a59d008d245888051005bd657760e96d5fbfb0b740c2/tomli-2.4.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7f94b27a62cfad8496c8d2513e1a222dd446f095fca8987fceef261225538a15", size = 247973, upload-time = "2026-03-25T20:21:46.937Z" }, + { url = "https://files.pythonhosted.org/packages/67/50/361e986652847fec4bd5e4a0208752fbe64689c603c7ae5ea7cb16b1c0ca/tomli-2.4.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:ede3e6487c5ef5d28634ba3f31f989030ad6af71edfb0055cbbd14189ff240ba", size = 256223, upload-time = "2026-03-25T20:21:48.467Z" }, + { url = "https://files.pythonhosted.org/packages/8c/9a/b4173689a9203472e5467217e0154b00e260621caa227b6fa01feab16998/tomli-2.4.1-cp314-cp314-win32.whl", hash = "sha256:3d48a93ee1c9b79c04bb38772ee1b64dcf18ff43085896ea460ca8dec96f35f6", size = 98973, upload-time = "2026-03-25T20:21:49.526Z" }, + { url = "https://files.pythonhosted.org/packages/14/58/640ac93bf230cd27d002462c9af0d837779f8773bc03dee06b5835208214/tomli-2.4.1-cp314-cp314-win_amd64.whl", hash = "sha256:88dceee75c2c63af144e456745e10101eb67361050196b0b6af5d717254dddf7", size = 109082, upload-time = "2026-03-25T20:21:50.506Z" }, + { url = "https://files.pythonhosted.org/packages/d5/2f/702d5e05b227401c1068f0d386d79a589bb12bf64c3d2c72ce0631e3bc49/tomli-2.4.1-cp314-cp314-win_arm64.whl", hash = "sha256:b8c198f8c1805dc42708689ed6864951fd2494f924149d3e4bce7710f8eb5232", size = 96490, upload-time = "2026-03-25T20:21:51.474Z" }, + { url = "https://files.pythonhosted.org/packages/45/4b/b877b05c8ba62927d9865dd980e34a755de541eb65fffba52b4cc495d4d2/tomli-2.4.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:d4d8fe59808a54658fcc0160ecfb1b30f9089906c50b23bcb4c69eddc19ec2b4", size = 164263, upload-time = "2026-03-25T20:21:52.543Z" }, + { url = "https://files.pythonhosted.org/packages/24/79/6ab420d37a270b89f7195dec5448f79400d9e9c1826df982f3f8e97b24fd/tomli-2.4.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:7008df2e7655c495dd12d2a4ad038ff878d4ca4b81fccaf82b714e07eae4402c", size = 160736, upload-time = "2026-03-25T20:21:53.674Z" }, + { url = "https://files.pythonhosted.org/packages/02/e0/3630057d8eb170310785723ed5adcdfb7d50cb7e6455f85ba8a3deed642b/tomli-2.4.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1d8591993e228b0c930c4bb0db464bdad97b3289fb981255d6c9a41aedc84b2d", size = 270717, upload-time = "2026-03-25T20:21:55.129Z" }, + { url = "https://files.pythonhosted.org/packages/7a/b4/1613716072e544d1a7891f548d8f9ec6ce2faf42ca65acae01d76ea06bb0/tomli-2.4.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:734e20b57ba95624ecf1841e72b53f6e186355e216e5412de414e3c51e5e3c41", size = 278461, upload-time = "2026-03-25T20:21:56.228Z" }, + { url = "https://files.pythonhosted.org/packages/05/38/30f541baf6a3f6df77b3df16b01ba319221389e2da59427e221ef417ac0c/tomli-2.4.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:8a650c2dbafa08d42e51ba0b62740dae4ecb9338eefa093aa5c78ceb546fcd5c", size = 274855, upload-time = "2026-03-25T20:21:57.653Z" }, + { url = "https://files.pythonhosted.org/packages/77/a3/ec9dd4fd2c38e98de34223b995a3b34813e6bdadf86c75314c928350ed14/tomli-2.4.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:504aa796fe0569bb43171066009ead363de03675276d2d121ac1a4572397870f", size = 283144, upload-time = "2026-03-25T20:21:59.089Z" }, + { url = "https://files.pythonhosted.org/packages/ef/be/605a6261cac79fba2ec0c9827e986e00323a1945700969b8ee0b30d85453/tomli-2.4.1-cp314-cp314t-win32.whl", hash = "sha256:b1d22e6e9387bf4739fbe23bfa80e93f6b0373a7f1b96c6227c32bef95a4d7a8", size = 108683, upload-time = "2026-03-25T20:22:00.214Z" }, + { url = "https://files.pythonhosted.org/packages/12/64/da524626d3b9cc40c168a13da8335fe1c51be12c0a63685cc6db7308daae/tomli-2.4.1-cp314-cp314t-win_amd64.whl", hash = "sha256:2c1c351919aca02858f740c6d33adea0c5deea37f9ecca1cc1ef9e884a619d26", size = 121196, upload-time = "2026-03-25T20:22:01.169Z" }, + { url = "https://files.pythonhosted.org/packages/5a/cd/e80b62269fc78fc36c9af5a6b89c835baa8af28ff5ad28c7028d60860320/tomli-2.4.1-cp314-cp314t-win_arm64.whl", hash = "sha256:eab21f45c7f66c13f2a9e0e1535309cee140182a9cdae1e041d02e47291e8396", size = 100393, upload-time = "2026-03-25T20:22:02.137Z" }, + { url = "https://files.pythonhosted.org/packages/7b/61/cceae43728b7de99d9b847560c262873a1f6c98202171fd5ed62640b494b/tomli-2.4.1-py3-none-any.whl", hash = "sha256:0d85819802132122da43cb86656f8d1f8c6587d54ae7dcaf30e90533028b49fe", size = 14583, upload-time = "2026-03-25T20:22:03.012Z" }, +] + +[[package]] +name = "typing-extensions" +version = "4.16.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/f6/cc/6253133b5bb138fc3306cebfbda2c520f545d36b5be2c7255cc528bb45d6/typing_extensions-4.16.0.tar.gz", hash = "sha256:dc983d19a509c94dba722ee6abd33940f7c05a89e243c47e907eb4db6f1a43e5", size = 113555, upload-time = "2026-07-02T08:40:05.92Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/49/d3/b8441a820a491ddfc024b0b0cf0393375b75ea13866d9c66727e54c2fc80/typing_extensions-4.16.0-py3-none-any.whl", hash = "sha256:481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8", size = 45571, upload-time = "2026-07-02T08:40:04.659Z" }, +] + +[[package]] +name = "urllib3" +version = "2.6.3" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +sdist = { url = "https://files.pythonhosted.org/packages/c7/24/5f1b3bdffd70275f6661c76461e25f024d5a38a46f04aaca912426a2b1d3/urllib3-2.6.3.tar.gz", hash = "sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed", size = 435556, upload-time = "2026-01-07T16:24:43.925Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/39/08/aaaad47bc4e9dc8c725e68f9d04865dbcb2052843ff09c97b08904852d84/urllib3-2.6.3-py3-none-any.whl", hash = "sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4", size = 131584, upload-time = "2026-01-07T16:24:42.685Z" }, +] + +[[package]] +name = "urllib3" +version = "2.7.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +sdist = { url = "https://files.pythonhosted.org/packages/53/0c/06f8b233b8fd13b9e5ee11424ef85419ba0d8ba0b3138bf360be2ff56953/urllib3-2.7.0.tar.gz", hash = "sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c", size = 433602, upload-time = "2026-05-07T16:13:18.596Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7f/3e/5db95bcf282c52709639744ca2a8b149baccf648e39c8cc87553df9eae0c/urllib3-2.7.0-py3-none-any.whl", hash = "sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897", size = 131087, upload-time = "2026-05-07T16:13:17.151Z" }, +] + +[[package]] +name = "zipp" +version = "3.23.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/30/21/093488dfc7cc8964ded15ab726fad40f25fd3d788fd741cc1c5a17d78ee8/zipp-3.23.1.tar.gz", hash = "sha256:32120e378d32cd9714ad503c1d024619063ec28aad2248dc6672ad13edfa5110", size = 25965, upload-time = "2026-04-13T23:21:46.6Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/08/8a/0861bec20485572fbddf3dfba2910e38fe249796cb73ecdeb74e07eeb8d3/zipp-3.23.1-py3-none-any.whl", hash = "sha256:0b3596c50a5c700c9cb40ba8d86d9f2cc4807e9bedb06bcdf7fac85633e444dc", size = 10378, upload-time = "2026-04-13T23:21:45.386Z" }, +] diff --git a/sphinxdocs/docs/BUILD.bazel b/sphinxdocs/docs/BUILD.bazel index 070e0485d7..58c9e7fc18 100644 --- a/sphinxdocs/docs/BUILD.bazel +++ b/sphinxdocs/docs/BUILD.bazel @@ -1,8 +1,8 @@ -load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility load("//sphinxdocs:sphinx_docs_library.bzl", "sphinx_docs_library") load("//sphinxdocs:sphinx_stardoc.bzl", "sphinx_stardocs") +load("//sphinxdocs/private:util.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility -package(default_visibility = ["//:__subpackages__"]) +package(default_visibility = ["//sphinxdocs:__subpackages__"]) # We only build for Linux and Mac because: # 1. The actual doc process only runs on Linux @@ -18,6 +18,7 @@ _TARGET_COMPATIBLE_WITH = select({ sphinx_docs_library( name = "docs_lib", + visibility = ["//visibility:public"], deps = [ ":artisian_api_docs", ":bzl_docs", @@ -47,11 +48,11 @@ sphinx_docs_library( sphinx_stardocs( name = "bzl_docs", srcs = [ - "//sphinxdocs:readthedocs_bzl", - "//sphinxdocs:sphinx_bzl", - "//sphinxdocs:sphinx_docs_library_bzl", - "//sphinxdocs:sphinx_stardoc_bzl", - "//sphinxdocs/private:sphinx_docs_library_bzl", + "//sphinxdocs:readthedocs", + "//sphinxdocs:sphinx", + "//sphinxdocs:sphinx_docs_library", + "//sphinxdocs:sphinx_stardoc", + "//sphinxdocs/private:sphinx_docs_library", ], prefix = "api/sphinxdocs/", target_compatible_with = _TARGET_COMPATIBLE_WITH, diff --git a/sphinxdocs/docs/api/sphinxdocs/index.md b/sphinxdocs/docs/api/sphinxdocs/index.md index bd4e9b6eec..a59edb2b65 100644 --- a/sphinxdocs/docs/api/sphinxdocs/index.md +++ b/sphinxdocs/docs/api/sphinxdocs/index.md @@ -1,7 +1,7 @@ -:::{bzl:currentfile} //sphinxdocs:BUILD.bazel +:::{bzl:currentfile} //:BUILD.bazel ::: -# //sphinxdocs +# // :::{bzl:flag} extra_defines Additional `-D` values to add to every Sphinx build. diff --git a/sphinxdocs/docs/api/sphinxdocs/inventories/index.md b/sphinxdocs/docs/api/sphinxdocs/inventories/index.md index a03645ed44..d0b983fe7f 100644 --- a/sphinxdocs/docs/api/sphinxdocs/inventories/index.md +++ b/sphinxdocs/docs/api/sphinxdocs/inventories/index.md @@ -1,7 +1,7 @@ -:::{bzl:currentfile} //sphinxdocs/inventories:BUILD.bazel +:::{bzl:currentfile} //inventories:BUILD.bazel ::: -# //sphinxdocs/inventories +# //inventories :::{bzl:target} bazel_inventory A Sphinx inventory of Bazel objects. diff --git a/sphinxdocs/docs/index.md b/sphinxdocs/docs/index.md index bd6448ced9..43bbc14f60 100644 --- a/sphinxdocs/docs/index.md +++ b/sphinxdocs/docs/index.md @@ -11,6 +11,29 @@ documentation. It comes with: While it is primarily oriented towards docgen for Starlark code, the core of it is agnostic as to what is being documented. +## Optimization + +Normally, Sphinx keeps various cache files to improve incremental building. +Unfortunately, programs performing their own caching don't interact well +with Bazel's model of precisely declaring and strictly enforcing what are +inputs, what are outputs, and what files are available when running a program. +The net effect is programs don't have a prior invocation's cache files +available. + +There are two mechanisms available to make some cache available to Sphinx under +Bazel: + +* Disable sandboxing, which allows some files from prior invocations to be + visible to subsequent invocations. This can be done multiple ways: + * Set `tags = ["no-sandbox"]` on the `sphinx_docs` target + * `--modify_execution_info=SphinxBuildDocs=+no-sandbox` (Bazel flag) + * `--strategy=SphinxBuildDocs=local` (Bazel flag) +* Use persistent workers (enabled by default) by setting + `allow_persistent_workers=True` on the `sphinx_docs` target. Note that other + Bazel flags can disable using workers even if an action supports it. Setting + `--strategy=SphinxBuildDocs=dynamic,worker,local,sandbox` should tell Bazel + to use workers if possible, otherwise fallback to non-worker invocations. + ```{toctree} :hidden: diff --git a/sphinxdocs/docs/readthedocs.md b/sphinxdocs/docs/readthedocs.md index 66e4be82ea..bcdae833d9 100644 --- a/sphinxdocs/docs/readthedocs.md +++ b/sphinxdocs/docs/readthedocs.md @@ -26,8 +26,8 @@ In the example below, `npm` is used to install Bazelisk and a helper shell script, `readthedocs_build.sh` is used to construct the Bazel invocation. The key purpose of the shell script it to set the -`--@rules_python//sphinxdocs:extra_env` and -`--@rules_python//sphinxdocs:extra_defines` flags. These are used to communicate +`@sphinxdocs//sphinxdocs::extra_env` and +`@sphinxdocs//sphinxdocs::extra_defines` flags. These are used to communicate `READTHEDOCS*` environment variables and settings to the Bazel invocation. ## BUILD config @@ -73,7 +73,7 @@ build: ``` # File: docs/BUILD -load("@rules_python//sphinxdocs:readthedocs.bzl.bzl", "readthedocs_install") +load("//:readthedocs.bzl", "readthedocs_install") readthedocs_install( name = "readthedocs_install", docs = [":docs"], @@ -90,17 +90,17 @@ set -eou pipefail declare -a extra_env while IFS='=' read -r -d '' name value; do if [[ "$name" == READTHEDOCS* ]]; then - extra_env+=("--@rules_python//sphinxdocs:extra_env=$name=$value") + extra_env+=("--@sphinxdocs//sphinxdocs:extra_env=$name=$value") fi done < <(env -0) # In order to get the build number, we extract it from the host name -extra_env+=("--@rules_python//sphinxdocs:extra_env=HOSTNAME=$HOSTNAME") +extra_env+=("--@sphinxdocs//sphinxdocs:extra_env=HOSTNAME=$HOSTNAME") set -x bazel run \ --stamp \ - "--@rules_python//sphinxdocs:extra_defines=version=$READTHEDOCS_VERSION" \ + "--@sphinxdocs//sphinxdocs:extra_defines=version=$READTHEDOCS_VERSION" \ "${extra_env[@]}" \ //docs:readthedocs_install ``` @@ -119,7 +119,7 @@ if os.environ.get("READTHEDOCS") == "True": # Insert after the main extension extensions.insert(1, "readthedocs_ext.external_version_warning") readthedocs_vcs_url = ( - "http://github.com/bazelbuild/rules_python/pull/{}".format( + "http://github.com/bazel-contrib/rules_python/pull/{}".format( os.environ.get("READTHEDOCS_VERSION", "") ) ) diff --git a/sphinxdocs/docs/sphinx-bzl.md b/sphinxdocs/docs/sphinx-bzl.md index 8376f60679..c96061b984 100644 --- a/sphinxdocs/docs/sphinx-bzl.md +++ b/sphinxdocs/docs/sphinx-bzl.md @@ -11,7 +11,7 @@ a well known target. ## Configuring Sphinx To enable the plugin in Sphinx, depend on -`@rules_python//sphinxdocs/src/sphinx_bzl` and enable it in `conf.py`: +`//sphinxdocs/src/sphinx_bzl` and enable it in `conf.py`: ``` extensions = [ @@ -265,7 +265,7 @@ Documents a flag. It has the same format as `{bzl:target}` ::::::{rst:directive} .. bzl:typedef:: typename Documents a user-defined structural "type". These are typically generated by -the {obj}`sphinx_stardoc` rule after following [User-defined types] to create a +the {bzl:obj}`sphinx_stardoc` rule after following [User-defined types] to create a struct with a `TYPEDEF` field, but can also be manually defined if there's no natural place for it in code, e.g. some ad-hoc structural type. diff --git a/sphinxdocs/docs/starlark-docgen.md b/sphinxdocs/docs/starlark-docgen.md index ba4ab516f5..0b89393ced 100644 --- a/sphinxdocs/docs/starlark-docgen.md +++ b/sphinxdocs/docs/starlark-docgen.md @@ -13,7 +13,7 @@ While the `sphinx_stardoc` rule doesn't require Sphinx itself, the source it generates requires some additional Sphinx plugins and config settings. When defining the `sphinx_build_binary` target, also depend on: -* `@rules_python//sphinxdocs/src/sphinx_bzl:sphinx_bzl` +* `//sphinxdocs/src/sphinx_bzl:sphinx_bzl` * `myst_parser` (e.g. `@pypi//myst_parser`) * `typing_extensions` (e.g. `@pypi//myst_parser`) @@ -21,7 +21,7 @@ When defining the `sphinx_build_binary` target, also depend on: sphinx_build_binary( name = "sphinx-build", deps = [ - "@rules_python//sphinxdocs/src/sphinx_bzl", + "//sphinxdocs/src/sphinx_bzl", "@pypi//myst_parser", "@pypi//typing_extensions", ... @@ -81,7 +81,7 @@ still possible to create such objects using `struct` and lambdas. For the purposes of documentation, they can be documented by creating a module-level `struct` with matching fields *and* also a field named `TYPEDEF`. When the `sphinx_stardoc` rule sees a struct with a `TYPEDEF` field, it generates doc -using the {rst:directive}`bzl:typedef` directive and puts all the struct's fields +using the {rst:dir}`bzl:typedef` directive and puts all the struct's fields within the typedef. The net result is the rendered docs look similar to how a class would be documented in other programming languages. diff --git a/sphinxdocs/downloader_config.cfg b/sphinxdocs/downloader_config.cfg new file mode 100644 index 0000000000..3fa6264eda --- /dev/null +++ b/sphinxdocs/downloader_config.cfg @@ -0,0 +1,21 @@ +# Try GitHub first (primary) +rewrite ^github\.com/bazel-contrib/bazel_features/(.*) github.com/bazel-contrib/bazel_features/$1 +rewrite ^github\.com/bazel-contrib/rules_go/(.*) github.com/bazel-contrib/rules_go/$1 +rewrite ^github\.com/bazelbuild/bazel-skylib/(.*) github.com/bazelbuild/bazel-skylib/$1 +rewrite ^github\.com/bazelbuild/platforms/(.*) github.com/bazelbuild/platforms/$1 +rewrite ^github\.com/bazelbuild/rules_kotlin/(.*) github.com/bazelbuild/rules_kotlin/$1 +rewrite ^github\.com/bazelbuild/rules_shell/(.*) github.com/bazelbuild/rules_shell/$1 +rewrite ^github\.com/bazelbuild/rules_java/(.*) github.com/bazelbuild/rules_java/$1 +rewrite ^github\.com/bazelbuild/stardoc/(.*) github.com/bazelbuild/stardoc/$1 + + +# Fall back to mirror (secondary) +# Tracking upstream BCR mirror addition: https://github.com/bazelbuild/platforms/issues/139 +rewrite ^github\.com/bazel-contrib/bazel_features/(.*) mirror.bazel.build/github.com/bazel-contrib/bazel_features/$1 +rewrite ^github\.com/bazel-contrib/rules_go/(.*) mirror.bazel.build/github.com/bazel-contrib/rules_go/$1 +rewrite ^github\.com/bazelbuild/bazel-skylib/(.*) mirror.bazel.build/github.com/bazelbuild/bazel-skylib/$1 +rewrite ^github\.com/bazelbuild/platforms/(.*) mirror.bazel.build/github.com/bazelbuild/platforms/$1 +rewrite ^github\.com/bazelbuild/rules_kotlin/(.*) mirror.bazel.build/github.com/bazelbuild/rules_kotlin/$1 +rewrite ^github\.com/bazelbuild/rules_shell/(.*) mirror.bazel.build/github.com/bazelbuild/rules_shell/$1 +rewrite ^github\.com/bazelbuild/rules_java/(.*) mirror.bazel.build/github.com/bazelbuild/rules_java/$1 +rewrite ^github\.com/bazelbuild/stardoc/(.*) mirror.bazel.build/github.com/bazelbuild/stardoc/$1 diff --git a/sphinxdocs/integration_tests/BUILD.bazel b/sphinxdocs/integration_tests/BUILD.bazel new file mode 100644 index 0000000000..45e894fc2e --- /dev/null +++ b/sphinxdocs/integration_tests/BUILD.bazel @@ -0,0 +1,8 @@ +load("@rules_python//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "runner_lib", + srcs = ["runner.py"], +) diff --git a/sphinxdocs/integration_tests/bazel_from_env b/sphinxdocs/integration_tests/bazel_from_env new file mode 100755 index 0000000000..a372736f32 --- /dev/null +++ b/sphinxdocs/integration_tests/bazel_from_env @@ -0,0 +1,6 @@ +#!/usr/bin/env bash +# +# A simple wrapper so rules_bazel_integration_test can use the +# bazel version inherited from the environment. + +bazel "$@" diff --git a/sphinxdocs/integration_tests/bcr/BUILD.bazel b/sphinxdocs/integration_tests/bcr/BUILD.bazel new file mode 100644 index 0000000000..412ba56ec3 --- /dev/null +++ b/sphinxdocs/integration_tests/bcr/BUILD.bazel @@ -0,0 +1,23 @@ +load("@bazel_skylib//rules:build_test.bzl", "build_test") +load("@sphinxdocs//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs") + +sphinx_docs( + name = "docs", + srcs = ["index.md"], + config = "conf.py", + formats = ["html"], + sphinx = ":sphinx-build", +) + +sphinx_build_binary( + name = "sphinx-build", + deps = [ + "@dev_pip//myst_parser", + "@dev_pip//sphinx", + ], +) + +build_test( + name = "docs_build_test", + targets = [":docs"], +) diff --git a/sphinxdocs/integration_tests/bcr/MODULE.bazel b/sphinxdocs/integration_tests/bcr/MODULE.bazel new file mode 100644 index 0000000000..451f2e920d --- /dev/null +++ b/sphinxdocs/integration_tests/bcr/MODULE.bazel @@ -0,0 +1,26 @@ +module( + name = "sphinxdocs_example", + version = "0.0.0", +) + +bazel_dep(name = "sphinxdocs", version = "0.0.0") +local_path_override( + module_name = "sphinxdocs", + path = "../..", +) + +bazel_dep(name = "rules_python", version = "1.8.5") + +dev_pip = use_extension( + "@rules_python//python/extensions:pip.bzl", + "pip", + dev_dependency = True, +) +dev_pip.parse( + hub_name = "dev_pip", + python_version = "3.11", + requirements_lock = "@sphinxdocs//dev:requirements.txt", +) +use_repo(dev_pip, "dev_pip") + +bazel_dep(name = "bazel_skylib", version = "1.8.2") diff --git a/sphinxdocs/integration_tests/bcr/conf.py b/sphinxdocs/integration_tests/bcr/conf.py new file mode 100644 index 0000000000..3b751bf402 --- /dev/null +++ b/sphinxdocs/integration_tests/bcr/conf.py @@ -0,0 +1,2 @@ +extensions = ["myst_parser"] +master_doc = "index" diff --git a/sphinxdocs/integration_tests/bcr/index.md b/sphinxdocs/integration_tests/bcr/index.md new file mode 100644 index 0000000000..bfe100937d --- /dev/null +++ b/sphinxdocs/integration_tests/bcr/index.md @@ -0,0 +1 @@ +# Sphinxdocs example diff --git a/sphinxdocs/integration_tests/integration_test.bzl b/sphinxdocs/integration_tests/integration_test.bzl new file mode 100644 index 0000000000..b6786b9c6b --- /dev/null +++ b/sphinxdocs/integration_tests/integration_test.bzl @@ -0,0 +1,68 @@ +"""Helpers for running bazel-in-bazel integration tests for sphinxdocs.""" + +load( + "@rules_bazel_integration_test//bazel_integration_test:defs.bzl", + "bazel_integration_test", + "integration_test_utils", +) +load("@rules_python//python:py_test.bzl", "py_test") + +def _test_runner(*, name, bazel_version, py_main, py_deps): + test_runner = "{}_bazel_{}_py_runner".format(name, bazel_version) + py_test( + name = test_runner, + srcs = [py_main], + main = py_main, + deps = ["//integration_tests:runner_lib"] + py_deps, + tags = ["manual"], + ) + return test_runner + +def sphinxdocs_integration_test( + name, + workspace_path = "workspace", + tags = None, + py_main = None, + py_deps = None, + bazel_versions = None, + **kwargs): + """Runs a bazel-in-bazel integration test for sphinxdocs. + + Args: + name: Name of the test. + workspace_path: The directory name of the sub-workspace. + tags: Test tags. + py_main: Main Python test runner script. + py_deps: Dependencies for py_main. + bazel_versions: List of bazel versions to test. + **kwargs: Passed to bazel_integration_test. + """ + workspace_files = integration_test_utils.glob_workspace_files(workspace_path) + native.filegroup( + name = name + "_workspace_files", + srcs = workspace_files + [ + "//:distribution", + ], + ) + kwargs.setdefault("size", "large") + for bazel_version in bazel_versions or ["self"]: + test_runner = _test_runner( + name = name, + bazel_version = bazel_version, + py_main = py_main, + py_deps = py_deps or [], + ) + bazel_integration_test( + name = "{}_bazel_{}".format(name, bazel_version), + workspace_path = workspace_path, + test_runner = test_runner, + bazel_version = bazel_version, + workspace_files = [name + "_workspace_files"], + tags = (tags or []) + [ + "exclusive", + "no-sandbox", + "no-remote-exec", + "integration-test", + ], + **kwargs + ) diff --git a/sphinxdocs/integration_tests/persistent_worker/BUILD.bazel b/sphinxdocs/integration_tests/persistent_worker/BUILD.bazel new file mode 100644 index 0000000000..60fb4cbc6c --- /dev/null +++ b/sphinxdocs/integration_tests/persistent_worker/BUILD.bazel @@ -0,0 +1,8 @@ +load("//integration_tests:integration_test.bzl", "sphinxdocs_integration_test") + +package(default_visibility = ["//visibility:public"]) + +sphinxdocs_integration_test( + name = "persistent_worker_test", + py_main = "persistent_worker_test.py", +) diff --git a/sphinxdocs/integration_tests/persistent_worker/persistent_worker_test.py b/sphinxdocs/integration_tests/persistent_worker/persistent_worker_test.py new file mode 100644 index 0000000000..30a8b46c00 --- /dev/null +++ b/sphinxdocs/integration_tests/persistent_worker/persistent_worker_test.py @@ -0,0 +1,54 @@ +import unittest + +from integration_tests import runner + + +class PersistentWorkerTest(runner.TestCase): + def _check_index_html(self, text: str, should_exist: bool): + index_html = ( + self.repo_root / "bazel-bin" / "docs" / "_build" / "html" / "index.html" + ) + if not index_html.exists(): + self.fail(f"Could not find index.html at {index_html}") + content = index_html.read_text() + if should_exist: + self.assertIn( + text, + content, + f"Expected '{text}' in index.html after build, but not found.", + ) + else: + self.assertNotIn( + text, + content, + f"Expected '{text}' NOT to be in index.html after build, but found it.", + ) + + def test_incremental_add_and_remove_files(self): + # 1. Initial build + result = self.run_bazel("build", "//:docs") + self.assert_result_matches(result, "bazel-bin") + index_html = ( + self.repo_root / "bazel-bin" / "docs" / "_build" / "html" / "index.html" + ) + self.assertTrue( + index_html.exists(), "index.html should exist after initial build" + ) + + # 2. Add a new markdown file and verify it is included across incremental build + page2_md = self.repo_root / "page2.md" + page2_md.write_text("# Page 2\n\nThis is a newly added page.\n") + result = self.run_bazel("build", "//:docs") + self.assert_result_matches(result, "bazel-bin") + self._check_index_html("page2.html", should_exist=True) + + # 3. Remove the added markdown file and verify the persistent worker cleans up + # stale source files and invalidates toctrees without errors or warnings. + page2_md.unlink() + result = self.run_bazel("build", "//:docs") + self.assert_result_matches(result, "bazel-bin") + self._check_index_html("page2.html", should_exist=False) + + +if __name__ == "__main__": + unittest.main() diff --git a/sphinxdocs/integration_tests/persistent_worker/workspace/.bazelrc b/sphinxdocs/integration_tests/persistent_worker/workspace/.bazelrc new file mode 100644 index 0000000000..f3162b9547 --- /dev/null +++ b/sphinxdocs/integration_tests/persistent_worker/workspace/.bazelrc @@ -0,0 +1,8 @@ +# Disable disk caching and remote action caching so Bazel is forced to dispatch builds +# directly to the running persistent worker instead of restoring pre-built HTML cache results across test steps. +common --disk_cache= +build --noremote_accept_cached +common --http_timeout_scaling=10.0 +common --experimental_repository_downloader_retries=10 +common --experimental_downloader_config=downloader_config.cfg +common --lockfile_mode=off diff --git a/sphinxdocs/integration_tests/persistent_worker/workspace/BUILD.bazel b/sphinxdocs/integration_tests/persistent_worker/workspace/BUILD.bazel new file mode 100644 index 0000000000..60e66004fd --- /dev/null +++ b/sphinxdocs/integration_tests/persistent_worker/workspace/BUILD.bazel @@ -0,0 +1,17 @@ +load("@sphinxdocs//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs") + +sphinx_docs( + name = "docs", + srcs = glob(["*.md"]), + config = "conf.py", + formats = ["html"], + sphinx = ":sphinx-build", +) + +sphinx_build_binary( + name = "sphinx-build", + deps = [ + "@dev_pip//myst_parser", + "@dev_pip//sphinx", + ], +) diff --git a/sphinxdocs/integration_tests/persistent_worker/workspace/MODULE.bazel b/sphinxdocs/integration_tests/persistent_worker/workspace/MODULE.bazel new file mode 100644 index 0000000000..976718f312 --- /dev/null +++ b/sphinxdocs/integration_tests/persistent_worker/workspace/MODULE.bazel @@ -0,0 +1,23 @@ +module(version = "0.0.0") + +bazel_dep(name = "sphinxdocs", version = "0.0.0") +local_path_override( + module_name = "sphinxdocs", + path = "../../..", +) + +bazel_dep(name = "rules_python", version = "1.8.5") + +dev_pip = use_extension( + "@rules_python//python/extensions:pip.bzl", + "pip", + dev_dependency = True, +) +dev_pip.parse( + hub_name = "dev_pip", + python_version = "3.11", + requirements_lock = "@sphinxdocs//dev:requirements.txt", +) +use_repo(dev_pip, "dev_pip") + +bazel_dep(name = "bazel_skylib", version = "1.8.2") diff --git a/sphinxdocs/integration_tests/persistent_worker/workspace/WORKSPACE b/sphinxdocs/integration_tests/persistent_worker/workspace/WORKSPACE new file mode 100644 index 0000000000..4be37fab94 --- /dev/null +++ b/sphinxdocs/integration_tests/persistent_worker/workspace/WORKSPACE @@ -0,0 +1 @@ +# Bzlmod-enabled workspace; this empty WORKSPACE file satisfies rules_bazel_integration_test checks. diff --git a/sphinxdocs/integration_tests/persistent_worker/workspace/conf.py b/sphinxdocs/integration_tests/persistent_worker/workspace/conf.py new file mode 100644 index 0000000000..de33bde101 --- /dev/null +++ b/sphinxdocs/integration_tests/persistent_worker/workspace/conf.py @@ -0,0 +1 @@ +extensions = ["myst_parser"] diff --git a/sphinxdocs/integration_tests/persistent_worker/workspace/downloader_config.cfg b/sphinxdocs/integration_tests/persistent_worker/workspace/downloader_config.cfg new file mode 100644 index 0000000000..3fa6264eda --- /dev/null +++ b/sphinxdocs/integration_tests/persistent_worker/workspace/downloader_config.cfg @@ -0,0 +1,21 @@ +# Try GitHub first (primary) +rewrite ^github\.com/bazel-contrib/bazel_features/(.*) github.com/bazel-contrib/bazel_features/$1 +rewrite ^github\.com/bazel-contrib/rules_go/(.*) github.com/bazel-contrib/rules_go/$1 +rewrite ^github\.com/bazelbuild/bazel-skylib/(.*) github.com/bazelbuild/bazel-skylib/$1 +rewrite ^github\.com/bazelbuild/platforms/(.*) github.com/bazelbuild/platforms/$1 +rewrite ^github\.com/bazelbuild/rules_kotlin/(.*) github.com/bazelbuild/rules_kotlin/$1 +rewrite ^github\.com/bazelbuild/rules_shell/(.*) github.com/bazelbuild/rules_shell/$1 +rewrite ^github\.com/bazelbuild/rules_java/(.*) github.com/bazelbuild/rules_java/$1 +rewrite ^github\.com/bazelbuild/stardoc/(.*) github.com/bazelbuild/stardoc/$1 + + +# Fall back to mirror (secondary) +# Tracking upstream BCR mirror addition: https://github.com/bazelbuild/platforms/issues/139 +rewrite ^github\.com/bazel-contrib/bazel_features/(.*) mirror.bazel.build/github.com/bazel-contrib/bazel_features/$1 +rewrite ^github\.com/bazel-contrib/rules_go/(.*) mirror.bazel.build/github.com/bazel-contrib/rules_go/$1 +rewrite ^github\.com/bazelbuild/bazel-skylib/(.*) mirror.bazel.build/github.com/bazelbuild/bazel-skylib/$1 +rewrite ^github\.com/bazelbuild/platforms/(.*) mirror.bazel.build/github.com/bazelbuild/platforms/$1 +rewrite ^github\.com/bazelbuild/rules_kotlin/(.*) mirror.bazel.build/github.com/bazelbuild/rules_kotlin/$1 +rewrite ^github\.com/bazelbuild/rules_shell/(.*) mirror.bazel.build/github.com/bazelbuild/rules_shell/$1 +rewrite ^github\.com/bazelbuild/rules_java/(.*) mirror.bazel.build/github.com/bazelbuild/rules_java/$1 +rewrite ^github\.com/bazelbuild/stardoc/(.*) mirror.bazel.build/github.com/bazelbuild/stardoc/$1 diff --git a/sphinxdocs/integration_tests/persistent_worker/workspace/index.md b/sphinxdocs/integration_tests/persistent_worker/workspace/index.md new file mode 100644 index 0000000000..d1dca92350 --- /dev/null +++ b/sphinxdocs/integration_tests/persistent_worker/workspace/index.md @@ -0,0 +1,7 @@ +# Test Documentation + +```{toctree} +:glob: true + +* +``` diff --git a/sphinxdocs/integration_tests/runner.py b/sphinxdocs/integration_tests/runner.py new file mode 100644 index 0000000000..cab9730bb8 --- /dev/null +++ b/sphinxdocs/integration_tests/runner.py @@ -0,0 +1,99 @@ +import logging +import os +import os.path +import pathlib +import re +import shlex +import subprocess +import unittest + +_logger = logging.getLogger(__name__) + + +class ExecuteError(Exception): + def __init__(self, result): + self.result = result + + def __str__(self): + return self.result.describe() + + +class ExecuteResult: + def __init__( + self, + args: list[str], + env: dict[str, str], + cwd: pathlib.Path, + proc_result: subprocess.CompletedProcess, + ): + self.args = args + self.env = env + self.cwd = cwd + self.exit_code = proc_result.returncode + self.stdout = proc_result.stdout + self.stderr = proc_result.stderr + + def describe(self) -> str: + env_lines = [ + " " + shlex.quote(f"{key}={value}") + for key, value in sorted(self.env.items()) + ] + env = " \\\n".join(env_lines) + args = shlex.join(self.args) + maybe_stdout_nl = "" if self.stdout.endswith("\n") else "\n" + maybe_stderr_nl = "" if self.stderr.endswith("\n") else "\n" + return f"""\ +COMMAND: +cd {self.cwd} && \\ +env \\ +{env} \\ + {args} +RESULT: exit_code: {self.exit_code} +===== STDOUT START ===== +{self.stdout}{maybe_stdout_nl}===== STDOUT END ===== +===== STDERR START ===== +{self.stderr}{maybe_stderr_nl}===== STDERR END ===== +""" + + +class TestCase(unittest.TestCase): + def setUp(self): + super().setUp() + self.repo_root = pathlib.Path(os.environ["BIT_WORKSPACE_DIR"]) + self.bazel = pathlib.Path(os.environ["BIT_BAZEL_BINARY"]) + outer_test_tmpdir = pathlib.Path(os.environ["TEST_TMPDIR"]) + self.test_tmp_dir = outer_test_tmpdir / "bit_test_tmp" + self.tmp_dir = outer_test_tmpdir / "bit_tmp" + self.bazel_env = { + "PATH": os.environ["PATH"], + "TEST_TMPDIR": str(self.test_tmp_dir), + "TMP": str(self.tmp_dir), + "RUNFILES_DIR": os.environ["TEST_SRCDIR"], + } + + def run_bazel(self, *args: str, check: bool = True) -> ExecuteResult: + args = [str(self.bazel), *args] + env = self.bazel_env + _logger.info("executing: %s", shlex.join(args)) + cwd = self.repo_root + proc_result = subprocess.run( + args=args, + text=True, + capture_output=True, + cwd=cwd, + env=env, + check=False, + ) + exec_result = ExecuteResult(args, env, cwd, proc_result) + if check and exec_result.exit_code: + raise ExecuteError(exec_result) + else: + return exec_result + + def assert_result_matches(self, result: ExecuteResult, regex: str) -> None: + if not re.search(regex, result.stdout + result.stderr): + self.fail( + "Bazel output did not match expected pattern\n" + + f"expected pattern: {regex}\n" + + f"invocation details:\n{result.describe()}" + ) diff --git a/sphinxdocs/private/sphinx_build.py b/sphinxdocs/private/sphinx_build.py deleted file mode 100644 index 3b7b32eaf6..0000000000 --- a/sphinxdocs/private/sphinx_build.py +++ /dev/null @@ -1,8 +0,0 @@ -import os -import pathlib -import sys - -from sphinx.cmd.build import main - -if __name__ == "__main__": - sys.exit(main()) diff --git a/sphinxdocs/pyproject.toml b/sphinxdocs/pyproject.toml new file mode 100644 index 0000000000..89330debc6 --- /dev/null +++ b/sphinxdocs/pyproject.toml @@ -0,0 +1,15 @@ +[project] +name = "sphinxdocs" +version = "0.0.0" + +dependencies = [ + "absl-py", + "markupsafe", + "myst-parser", + "readthedocs-sphinx-ext", + "sphinx", + "sphinx-autodoc2", + "sphinx-reredirects", + "sphinx_rtd_theme >=2.0", # uv insists on downgrading for some reason + "typing-extensions", +] diff --git a/sphinxdocs/sphinxdocs/BUILD.bazel b/sphinxdocs/sphinxdocs/BUILD.bazel new file mode 100644 index 0000000000..9a871f5268 --- /dev/null +++ b/sphinxdocs/sphinxdocs/BUILD.bazel @@ -0,0 +1,104 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") +load("@bazel_skylib//rules:common_settings.bzl", "bool_flag") +load("//sphinxdocs/private:sphinx.bzl", "repeated_string_list_flag") + +package( + default_visibility = ["//:__subpackages__"], +) + +filegroup( + name = "distribution", + srcs = glob(["**/*"]) + [ + "//sphinxdocs/private:distribution", + ], + visibility = ["//:__pkg__"], +) + +# Additional -D values to add to every Sphinx build. +# This is usually used to override the version when building +repeated_string_list_flag( + name = "extra_defines", + build_setting_default = [], +) + +repeated_string_list_flag( + name = "extra_env", + build_setting_default = [], +) + +# Whether to add the `-q` arg to Sphinx invocations, which determines if +# stdout has any output or not (logging INFO messages and progress messages). +# If true, add `-q`. If false, don't add `-q`. This is mostly useful for +# debugging invocations or developing extensions. +bool_flag( + name = "quiet", + build_setting_default = True, +) + +bzl_library( + name = "sphinx", + srcs = ["sphinx.bzl"], + visibility = ["//visibility:public"], + deps = ["//sphinxdocs/private:sphinx"], +) + +bzl_library( + name = "sphinx_docs_library", + srcs = ["sphinx_docs_library.bzl"], + visibility = ["//visibility:public"], + deps = ["//sphinxdocs/private:sphinx_docs_library_macro"], +) + +bzl_library( + name = "sphinx_stardoc", + srcs = ["sphinx_stardoc.bzl"], + visibility = ["//visibility:public"], + deps = ["//sphinxdocs/private:sphinx_stardoc"], +) + +bzl_library( + name = "readthedocs", + srcs = ["readthedocs.bzl"], + visibility = ["//visibility:public"], + deps = ["//sphinxdocs/private:readthedocs"], +) + +# ========= Deprecated aliases for backwards compatibility ========= + +alias( + name = "sphinx_bzl", + actual = ":sphinx", + deprecation = "Use //sphinxdocs:sphinx instead", +) + +alias( + name = "sphinx_docs_library_bzl", + actual = ":sphinx_docs_library", + deprecation = "Use //sphinxdocs:sphinx_docs_library instead", +) + +alias( + name = "sphinx_stardoc_bzl", + actual = ":sphinx_stardoc", + deprecation = "Use //sphinxdocs:sphinx_stardoc instead", +) + +alias( + name = "readthedocs_bzl", + actual = ":readthedocs", + deprecation = "Use //sphinxdocs:readthedocs instead", +) diff --git a/sphinxdocs/inventories/BUILD.bazel b/sphinxdocs/sphinxdocs/inventories/BUILD.bazel similarity index 100% rename from sphinxdocs/inventories/BUILD.bazel rename to sphinxdocs/sphinxdocs/inventories/BUILD.bazel diff --git a/sphinxdocs/inventories/bazel_inventory.txt b/sphinxdocs/sphinxdocs/inventories/bazel_inventory.txt similarity index 83% rename from sphinxdocs/inventories/bazel_inventory.txt rename to sphinxdocs/sphinxdocs/inventories/bazel_inventory.txt index 969c772386..866be33886 100644 --- a/sphinxdocs/inventories/bazel_inventory.txt +++ b/sphinxdocs/sphinxdocs/inventories/bazel_inventory.txt @@ -3,8 +3,10 @@ # Version: 7.3.0 # The remainder of this file is compressed using zlib Action bzl:type 1 rules/lib/Action - +Attribute bzl:type 1 rules/lib/builtins/Attribute - CcInfo bzl:provider 1 rules/lib/providers/CcInfo - CcInfo.linking_context bzl:provider-field 1 rules/lib/providers/CcInfo#linking_context - +DefaultInfo bzl:type 1 rules/lib/providers/DefaultInfo - ExecutionInfo bzl:type 1 rules/lib/providers/ExecutionInfo - File bzl:type 1 rules/lib/File - Label bzl:type 1 rules/lib/Label - @@ -13,16 +15,35 @@ RBE bzl:obj 1 remote/rbe - RunEnvironmentInfo bzl:type 1 rules/lib/providers/RunEnvironmentInfo - Target bzl:type 1 rules/lib/builtins/Target - ToolchainInfo bzl:type 1 rules/lib/providers/ToolchainInfo.html - +alias bzl:rule 1 reference/be/general#alias - attr.bool bzl:type 1 rules/lib/toplevel/attr#bool - attr.int bzl:type 1 rules/lib/toplevel/attr#int - +attr.int_list bzl:type 1 rules/lib/toplevel/attr#int_list - attr.label bzl:type 1 rules/lib/toplevel/attr#label - +attr.label_keyed_string_dict bzl:type 1 rules/lib/toplevel/attr#label_keyed_string_dict - attr.label_list bzl:type 1 rules/lib/toplevel/attr#label_list - +attr.output bzl:type 1 rules/lib/toplevel/attr#output - +attr.output_list bzl:type 1 rules/lib/toplevel/attr#output_list - attr.string bzl:type 1 rules/lib/toplevel/attr#string - +attr.string_dict bzl:type 1 rules/lib/toplevel/attr#string_dict - +attr.string_keyed_label_dict bzl:type 1 rules/lib/toplevel/attr#string_keyed_label_dict - attr.string_list bzl:type 1 rules/lib/toplevel/attr#string_list - +attr.string_list_dict bzl:type 1 rules/lib/toplevel/attr#string_list_dict - bool bzl:type 1 rules/lib/bool - callable bzl:type 1 rules/lib/core/function - +config bzl:obj 1 rules/lib/toplevel/config - +config.bool bzl:function 1 rules/lib/toplevel/config#bool - +config.exec bzl:function 1 rules/lib/toplevel/config#exec - +config.int bzl:function 1 rules/lib/toplevel/config#int - +config.none bzl:function 1 rules/lib/toplevel/config#none - +config.string bzl:function 1 rules/lib/toplevel/config#string - +config.string_list bzl:function 1 rules/lib/toplevel/config#string_list - +config.target bzl:function 1 rules/lib/toplevel/config#target - config_common.FeatureFlagInfo bzl:type 1 rules/lib/toplevel/config_common#FeatureFlagInfo - config_common.toolchain_type bzl:function 1 rules/lib/toplevel/config_common#toolchain_type - +config_setting bzl:rule 1 reference/be/general#config_setting - +configuration_field bzl:type 1 rules/lib/builtins/configuration_field - +ctx bzl:type 1 rules/lib/builtins/repository_ctx - ctx.actions bzl:obj 1 rules/lib/builtins/ctx#actions - ctx.aspect_ids bzl:obj 1 rules/lib/builtins/ctx#aspect_ids - ctx.attr bzl:obj 1 rules/lib/builtins/ctx#attr - @@ -60,6 +81,9 @@ ctx.workspace_name bzl:obj 1 rules/lib/builtins/ctx#workspace_name - depset bzl:type 1 rules/lib/depset - dict bzl:type 1 rules/lib/dict - exec_compatible_with bzl:attr 1 reference/be/common-definitions#common.exec_compatible_with - +exec_group bzl:function 1 rules/lib/globals/bzl#exec_group - +filegroup bzl:rule 1 reference/be/general#filegroup - +filegroup.data bzl:attr 1 reference/be/general#filegroup.data - int bzl:type 1 rules/lib/int - label bzl:type 1 concepts/labels - list bzl:type 1 rules/lib/list - @@ -80,6 +104,7 @@ module_ctx.report_progress bzl:function 1 rules/lib/builtins/module_ctx#report_p module_ctx.root_module_has_non_dev_dependency bzl:function 1 rules/lib/builtins/module_ctx#root_module_has_non_dev_dependency - module_ctx.watch bzl:function 1 rules/lib/builtins/module_ctx#watch - module_ctx.which bzl:function 1 rules/lib/builtins/module_ctx#which - +native bzl:obj 1 rules/lib/toplevel/native - native.existing_rule bzl:function 1 rules/lib/toplevel/native#existing_rule - native.existing_rules bzl:function 1 rules/lib/toplevel/native#existing_rules - native.exports_files bzl:function 1 rules/lib/toplevel/native#exports_files - @@ -124,6 +149,8 @@ repository_os bzl:type 1 rules/lib/builtins/repository_os - repository_os.arch bzl:obj 1 rules/lib/builtins/repository_os#arch repository_os.environ bzl:obj 1 rules/lib/builtins/repository_os#environ repository_os.name bzl:obj 1 rules/lib/builtins/repository_os#name +rule bzl:type 1 rules/lib/builtins/rule - +rule bzl:function rules/lib/globals/bzl.html#rule - runfiles bzl:type 1 rules/lib/builtins/runfiles - runfiles.empty_filenames bzl:type 1 rules/lib/builtins/runfiles#empty_filenames - runfiles.files bzl:type 1 rules/lib/builtins/runfiles#files - @@ -131,6 +158,7 @@ runfiles.merge bzl:type 1 rules/lib/builtins/runfiles#merge - runfiles.merge_all bzl:type 1 rules/lib/builtins/runfiles#merge_all - runfiles.root_symlinks bzl:type 1 rules/lib/builtins/runfiles#root_symlinks - runfiles.symlinks bzl:type 1 rules/lib/builtins/runfiles#symlinks - +stamp bzl:flag 1 reference/command-line-reference#flag--stamp - str bzl:type 1 rules/lib/string - struct bzl:type 1 rules/lib/builtins/struct - target_compatible_with bzl:attr 1 reference/be/common-definitions#common.target_compatible_with - @@ -140,6 +168,9 @@ testing.TestEnvironment bzl:function 1 rules/lib/toplevel/testing#TestEnvironmen testing.analysis_test bzl:rule 1 rules/lib/toplevel/testing#analysis_test - toolchain bzl:rule 1 reference/be/platforms-and-toolchains#toolchain - toolchain.exec_compatible_with bzl:rule 1 reference/be/platforms-and-toolchains#toolchain.exec_compatible_with - -toolchain.target_settings bzl:attr 1 reference/be/platforms-and-toolchains#toolchain.target_settings - toolchain.target_compatible_with bzl:attr 1 reference/be/platforms-and-toolchains#toolchain.target_compatible_with - +toolchain.target_settings bzl:attr 1 reference/be/platforms-and-toolchains#toolchain.target_settings - toolchain_type bzl:type 1 rules/lib/builtins/toolchain_type.html - +transition bzl:type 1 rules/lib/builtins/transition - +tuple bzl:type 1 rules/lib/core/tuple - +workspace_status_command bzl:flag 1 reference/command-line-reference#build-flag--workspace_status_command - diff --git a/sphinxdocs/private/BUILD.bazel b/sphinxdocs/sphinxdocs/private/BUILD.bazel similarity index 63% rename from sphinxdocs/private/BUILD.bazel rename to sphinxdocs/sphinxdocs/private/BUILD.bazel index c4246ed0de..823f07fe73 100644 --- a/sphinxdocs/private/BUILD.bazel +++ b/sphinxdocs/sphinxdocs/private/BUILD.bazel @@ -13,17 +13,18 @@ # limitations under the License. load("@bazel_skylib//:bzl_library.bzl", "bzl_library") -load("//python:proto.bzl", "py_proto_library") -load("//python:py_binary.bzl", "py_binary") -load("//python:py_library.bzl", "py_library") +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_python//python:py_binary.bzl", "py_binary") +load("@rules_python//python:py_library.bzl", "py_library") +load(":visibility.bzl", "NOT_ACTUALLY_PUBLIC") package( - default_visibility = ["//sphinxdocs:__subpackages__"], + default_visibility = ["//:__subpackages__"], ) -# These are only exported because they're passed as files to the //sphinxdocs +# These are only exported because they're passed as files to the @sphinxdocs # macros, and thus must be visible to other packages. They should only be -# referenced by the //sphinxdocs macros. +# referenced by the @sphinxdocs macros. exports_files( [ "readthedocs_install.py", @@ -34,48 +35,63 @@ exports_files( visibility = ["//visibility:public"], ) +filegroup( + name = "distribution", + srcs = glob(["**/*"]), + visibility = ["//sphinxdocs:__pkg__"], +) + +bzl_library( + name = "util", + srcs = ["util.bzl"], + deps = [ + "@bazel_skylib//lib:types", + ], +) + bzl_library( - name = "sphinx_docs_library_macro_bzl", + name = "sphinx_docs_library_macro", srcs = ["sphinx_docs_library_macro.bzl"], deps = [ - ":sphinx_docs_library_bzl", - "//python/private:util_bzl", + ":sphinx_docs_library", + ":util", ], ) bzl_library( - name = "sphinx_docs_library_bzl", + name = "sphinx_docs_library", srcs = ["sphinx_docs_library.bzl"], - deps = [":sphinx_docs_library_info_bzl"], + deps = [":sphinx_docs_library_info"], ) bzl_library( - name = "sphinx_docs_library_info_bzl", + name = "sphinx_docs_library_info", srcs = ["sphinx_docs_library_info.bzl"], ) bzl_library( - name = "sphinx_bzl", + name = "sphinx", srcs = ["sphinx.bzl"], deps = [ - ":sphinx_docs_library_info_bzl", - "//python:py_binary_bzl", + ":sphinx_docs_library_info", + ":util", "@bazel_skylib//:bzl_library", "@bazel_skylib//lib:paths", "@bazel_skylib//lib:types", "@bazel_skylib//rules:build_test", "@bazel_skylib//rules:common_settings", "@io_bazel_stardoc//stardoc:stardoc_lib", + "@rules_python//python:py_binary_bzl", ], ) bzl_library( - name = "sphinx_stardoc_bzl", + name = "sphinx_stardoc", srcs = ["sphinx_stardoc.bzl"], deps = [ - ":sphinx_docs_library_macro_bzl", - "//python/private:util_bzl", - "//sphinxdocs:sphinx_bzl", + ":sphinx_docs_library_macro", + ":util", + "//sphinxdocs:sphinx", "@bazel_skylib//:bzl_library", "@bazel_skylib//lib:paths", "@bazel_skylib//lib:types", @@ -85,31 +101,45 @@ bzl_library( ) bzl_library( - name = "readthedocs_bzl", + name = "readthedocs", srcs = ["readthedocs.bzl"], - deps = ["//python:py_binary_bzl"], + deps = [ + ":util", + "@rules_python//python:py_binary_bzl", + ], +) + +bzl_library( + name = "visibility", + srcs = ["visibility.bzl"], ) py_binary( name = "inventory_builder", srcs = ["inventory_builder.py"], # Only public because it's an implicit attribute - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) py_binary( name = "proto_to_markdown", srcs = ["proto_to_markdown.py"], # Only public because it's an implicit attribute - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, deps = [":proto_to_markdown_lib"], ) +py_library( + name = "sphinx_build_lib", + srcs = ["sphinx_build.py"], + visibility = ["//:__subpackages__"], +) + py_library( name = "proto_to_markdown_lib", srcs = ["proto_to_markdown.py"], # Only public because it's an implicit attribute - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, deps = [ ":stardoc_output_proto_py_pb2", ], diff --git a/sphinxdocs/private/inventory_builder.py b/sphinxdocs/sphinxdocs/private/inventory_builder.py similarity index 100% rename from sphinxdocs/private/inventory_builder.py rename to sphinxdocs/sphinxdocs/private/inventory_builder.py diff --git a/sphinxdocs/private/proto_to_markdown.py b/sphinxdocs/sphinxdocs/private/proto_to_markdown.py similarity index 99% rename from sphinxdocs/private/proto_to_markdown.py rename to sphinxdocs/sphinxdocs/private/proto_to_markdown.py index 18fbd12ede..05278a5c02 100644 --- a/sphinxdocs/private/proto_to_markdown.py +++ b/sphinxdocs/sphinxdocs/private/proto_to_markdown.py @@ -13,11 +13,9 @@ # limitations under the License. import argparse -import io import itertools import pathlib import sys -import textwrap from typing import Callable, TextIO, TypeVar from stardoc.proto import stardoc_output_pb2 @@ -197,7 +195,7 @@ def _render_module_extension(self, mod_ext: stardoc_output_pb2.ModuleExtensionIn # Ensure a newline between the directive and the doc fields, # otherwise they get parsed as directive options instead. if not doc_string and tag.attribute: - self.write("\n") + self._write("\n") self._render_attributes(tag.attribute) self._write(":::::\n") self._write("::::::\n") @@ -216,7 +214,9 @@ def _render_repository_rule(self, repo_rule: stardoc_output_pb2.RepositoryRuleIn self._render_attributes(repo_rule.attribute) if repo_rule.environ: self._write(":envvars: ", ", ".join(sorted(repo_rule.environ))) - self._write("\n") + self._write("\n\n") + + self._write("::::::\n") def _render_rule(self, rule: stardoc_output_pb2.RuleInfo): rule_name = rule.rule_name diff --git a/sphinxdocs/private/readthedocs.bzl b/sphinxdocs/sphinxdocs/private/readthedocs.bzl similarity index 87% rename from sphinxdocs/private/readthedocs.bzl rename to sphinxdocs/sphinxdocs/private/readthedocs.bzl index a62c51b86a..14bb1c9fcd 100644 --- a/sphinxdocs/private/readthedocs.bzl +++ b/sphinxdocs/sphinxdocs/private/readthedocs.bzl @@ -13,8 +13,8 @@ # limitations under the License. """Starlark rules for integrating Sphinx and Readthedocs.""" -load("//python:py_binary.bzl", "py_binary") -load("//python/private:util.bzl", "add_tag") # buildifier: disable=bzl-visibility +load("@rules_python//python:py_binary.bzl", "py_binary") +load("//sphinxdocs/private:util.bzl", "add_tag") # buildifier: disable=bzl-visibility _INSTALL_MAIN_SRC = Label("//sphinxdocs/private:readthedocs_install.py") @@ -33,7 +33,7 @@ def readthedocs_install(name, docs, **kwargs): is typically a single {obj}`sphinx_stardocs` target. **kwargs: {type}`dict` additional kwargs to pass onto the installer """ - add_tag(kwargs, "@rules_python//sphinxdocs:readthedocs_install") + add_tag(kwargs, "//sphinxdocs:readthedocs_install") py_binary( name = name, srcs = [_INSTALL_MAIN_SRC], @@ -43,6 +43,6 @@ def readthedocs_install(name, docs, **kwargs): "$(rlocationpaths {})".format(d) for d in docs ], - deps = [Label("//python/runfiles")], + deps = [Label("@rules_python//python/runfiles")], **kwargs ) diff --git a/sphinxdocs/private/readthedocs_install.py b/sphinxdocs/sphinxdocs/private/readthedocs_install.py similarity index 100% rename from sphinxdocs/private/readthedocs_install.py rename to sphinxdocs/sphinxdocs/private/readthedocs_install.py diff --git a/sphinxdocs/private/sphinx.bzl b/sphinxdocs/sphinxdocs/private/sphinx.bzl similarity index 81% rename from sphinxdocs/private/sphinx.bzl rename to sphinxdocs/sphinxdocs/private/sphinx.bzl index 7ec35f9ab4..24c5e9f9d1 100644 --- a/sphinxdocs/private/sphinx.bzl +++ b/sphinxdocs/sphinxdocs/private/sphinx.bzl @@ -15,10 +15,9 @@ """Implementation of sphinx rules.""" load("@bazel_skylib//lib:paths.bzl", "paths") -load("@bazel_skylib//rules:build_test.bzl", "build_test") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") -load("//python:py_binary.bzl", "py_binary") -load("//python/private:util.bzl", "add_tag", "copy_propagating_kwargs") # buildifier: disable=bzl-visibility +load("@rules_python//python:py_binary.bzl", "py_binary") +load("//sphinxdocs/private:util.bzl", "add_tag", "copy_propagating_kwargs") # buildifier: disable=bzl-visibility load(":sphinx_docs_library_info.bzl", "SphinxDocsLibraryInfo") _SPHINX_BUILD_MAIN_SRC = Label("//sphinxdocs/private:sphinx_build.py") @@ -84,7 +83,7 @@ def sphinx_build_binary(name, py_binary_rule = py_binary, **kwargs): **kwargs: {type}`dict` Additional kwargs to pass onto `py_binary`. The `srcs` and `main` attributes must not be specified. """ - add_tag(kwargs, "@rules_python//sphinxdocs:sphinx_build_binary") + add_tag(kwargs, "//sphinxdocs:sphinx_build_binary") py_binary_rule( name = name, srcs = [_SPHINX_BUILD_MAIN_SRC], @@ -101,9 +100,10 @@ def sphinx_docs( sphinx, config, formats, - strip_prefix = "", + strip_prefix = None, extra_opts = [], tools = [], + allow_persistent_workers = True, **kwargs): """Generate docs using Sphinx. @@ -133,19 +133,23 @@ def sphinx_docs( config: {type}`label` the Sphinx config file (`conf.py`) to use. formats: (list of str) the formats (`-b` flag) to generate documentation in. Each format will become an output group. - strip_prefix: {type}`str` A prefix to remove from the file paths of the - source files. e.g., given `//docs:foo.md`, stripping `docs/` makes - Sphinx see `foo.md` in its generated source directory. If not - specified, then {any}`native.package_name` is used. + strip_prefix: {type}`str | None` A prefix to remove from the file paths of the + source files. An empty string (`""`) means no prefix stripping; `None` + means {any}`native.package_name` is used. e.g., given `//docs:foo.md`, + stripping `docs/` makes Sphinx see `foo.md` in its generated source directory. extra_opts: {type}`list[str]` Additional options to pass onto Sphinx building. On each provided option, a location expansion is performed. See {any}`ctx.expand_location`. tools: {type}`list[label]` Additional tools that are used by Sphinx and its plugins. This just makes the tools available during Sphinx execution. To locate them, use {obj}`extra_opts` and `$(location)`. + allow_persistent_workers: {type}`bool` (experimental) If true, allow + using persistent workers for running Sphinx, if Bazel decides to do so. + This can improve incremental building of docs. **kwargs: {type}`dict` Common attributes to pass onto rules. """ - add_tag(kwargs, "@rules_python//sphinxdocs:sphinx_docs") + strip_prefix = strip_prefix if strip_prefix != None else native.package_name() + add_tag(kwargs, "//sphinxdocs:sphinx_docs") common_kwargs = copy_propagating_kwargs(kwargs) internal_name = "_{}".format(name.lstrip("_")) @@ -166,6 +170,7 @@ def sphinx_docs( source_tree = internal_name + "/_sources", extra_opts = extra_opts, tools = tools, + allow_persistent_workers = allow_persistent_workers, **kwargs ) @@ -177,26 +182,24 @@ def sphinx_docs( **common_kwargs ) + common_kwargs_with_manual_tag = dict(common_kwargs) + common_kwargs_with_manual_tag["tags"] = list(common_kwargs.get("tags") or []) + ["manual"] + py_binary( name = name + ".serve", srcs = [_SPHINX_SERVE_MAIN_SRC], main = _SPHINX_SERVE_MAIN_SRC, data = [html_name], + deps = [Label("@rules_python//python/runfiles")], args = [ - "$(execpath {})".format(html_name), + "$(rlocationpath {})".format(html_name), ], - **common_kwargs + **common_kwargs_with_manual_tag ) sphinx_run( name = name + ".run", docs = name, - **common_kwargs - ) - - build_test( - name = name + "_build_test", - targets = [name], - **kwargs # kwargs used to pick up target_compatible_with + **common_kwargs_with_manual_tag ) def _sphinx_docs_impl(ctx): @@ -213,6 +216,7 @@ def _sphinx_docs_impl(ctx): source_path = source_dir_path, output_prefix = paths.join(ctx.label.name, "_build"), inputs = inputs, + allow_persistent_workers = ctx.attr.allow_persistent_workers, ) outputs[format] = output_dir per_format_args[format] = args_env @@ -233,6 +237,10 @@ def _sphinx_docs_impl(ctx): _sphinx_docs = rule( implementation = _sphinx_docs_impl, attrs = { + "allow_persistent_workers": attr.bool( + doc = "(experimental) Whether to invoke Sphinx as a persistent worker.", + default = False, + ), "extra_opts": attr.string_list( doc = "Additional options to pass onto Sphinx. These are added after " + "other options, but before the source/output args.", @@ -258,16 +266,27 @@ _sphinx_docs = rule( }, ) -def _run_sphinx(ctx, format, source_path, inputs, output_prefix): +def _run_sphinx(ctx, format, source_path, inputs, output_prefix, allow_persistent_workers): output_dir = ctx.actions.declare_directory(paths.join(output_prefix, format)) run_args = [] # Copy of the args to forward along to debug runner args = ctx.actions.args() # Args passed to the action + # An args file is required for persistent workers, but we don't know if + # the action will use worker mode or not (settings we can't see may + # force non-worker mode). For consistency, always use a params file. + args.use_param_file("@%s", use_always = True) + args.set_param_file_format("multiline") + + # NOTE: sphinx_build.py relies on the first two args being the srcdir and + # outputdir, in that order. + args.add(source_path) + args.add(output_dir.path) + args.add("--show-traceback") # Full tracebacks on error run_args.append("--show-traceback") - args.add("--builder", format) - run_args.extend(("--builder", format)) + args.add(format, format = "--builder=%s") + run_args.append("--builder={}".format(format)) if ctx.attr._quiet_flag[BuildSettingInfo].value: # Not added to run_args because run_args is for debugging @@ -275,11 +294,17 @@ def _run_sphinx(ctx, format, source_path, inputs, output_prefix): # Build in parallel, if possible # Don't add to run_args: parallel building breaks interactive debugging - args.add("--jobs", "auto") - args.add("--fresh-env") # Don't try to use cache files. Bazel can't make use of them. - run_args.append("--fresh-env") - args.add("--write-all") # Write all files; don't try to detect "changed" files - run_args.append("--write-all") + args.add("--jobs=auto") + + # Put the doctree dir outside of the output directory. + # This allows it to be reused between invocations when possible; Bazel + # clears the output directory every action invocation. + # * For workers, they can fully re-use it. + # * For non-workers, it can be reused when sandboxing is disabled via + # the `no-sandbox` tag or execution requirement. + # + # We also use a non-dot prefixed name so it shows up more visibly. + args.add(paths.join(output_dir.path + "_doctrees"), format = "--doctree-dir=%s") for opt in ctx.attr.extra_opts: expanded = ctx.expand_location(opt) @@ -291,9 +316,6 @@ def _run_sphinx(ctx, format, source_path, inputs, output_prefix): for define in extra_defines: run_args.extend(("--define", define)) - args.add(source_path) - args.add(output_dir.path) - env = dict([ v.split("=", 1) for v in ctx.attr._extra_env_flag[_FlagInfo].value @@ -303,6 +325,14 @@ def _run_sphinx(ctx, format, source_path, inputs, output_prefix): for tool in ctx.attr.tools: tools.append(tool[DefaultInfo].files_to_run) + # NOTE: Command line flags or RBE capabilities may override the execution + # requirements and disable workers. Thus, we can't assume that these + # exec requirements will actually be respected. + execution_requirements = {} + if allow_persistent_workers: + execution_requirements["supports-workers"] = "1" + execution_requirements["requires-worker-protocol"] = "json" + ctx.actions.run( executable = ctx.executable.sphinx, arguments = [args], @@ -312,6 +342,7 @@ def _run_sphinx(ctx, format, source_path, inputs, output_prefix): mnemonic = "SphinxBuildDocs", progress_message = "Sphinx building {} for %{{label}}".format(format), env = env, + execution_requirements = execution_requirements, ) return output_dir, struct(args = run_args, env = env) @@ -325,17 +356,24 @@ def _sphinx_source_tree_impl(ctx): # Materialize a file under the `_sources` dir def _relocate(source_file, dest_path = None): if not dest_path: - dest_path = source_file.short_path.removeprefix(ctx.attr.strip_prefix) + # Strip leading slash if strip_prefix lacks a trailing slash, + # preventing paths.join from treating it as an absolute root. + dest_path = source_file.short_path.removeprefix(ctx.attr.strip_prefix).lstrip("/") dest_path = paths.join(source_prefix, dest_path) if source_file.is_directory: dest_file = ctx.actions.declare_directory(dest_path) + progress_message = "Symlinking Sphinx source directory %{input} to %{output}" else: dest_file = ctx.actions.declare_file(dest_path) + progress_message = "Symlinking Sphinx source %{input} to %{output}" + + # NOTE: Sphinx/MyST will read through symlinks, which can break relative + # xref lookup. Files are copied during the action phase to prevent this. ctx.actions.symlink( output = dest_file, target_file = source_file, - progress_message = "Symlinking Sphinx source %{input} to %{output}", + progress_message = progress_message, ) sphinx_source_files.append(dest_file) return dest_file @@ -343,7 +381,9 @@ def _sphinx_source_tree_impl(ctx): # Though Sphinx has a -c flag, we move the config file into the sources # directory to make the config more intuitive because some configuration # options are relative to the config location, not the sources directory. - source_conf_file = _relocate(ctx.file.config) + # Sphinx requires the configuration file to be named conf.py: + # https://www.sphinx-doc.org/en/master/usage/configuration.html#module-conf + source_conf_file = _relocate(ctx.file.config, "conf.py") sphinx_source_dir_path = paths.dirname(source_conf_file.path) for src in ctx.attr.srcs: @@ -357,7 +397,7 @@ def _sphinx_source_tree_impl(ctx): _relocate(orig_file) for src_target, dest in ctx.attr.renamed_srcs.items(): - src_files = src_target.files.to_list() + src_files = src_target[DefaultInfo].files.to_list() if len(src_files) != 1: fail("A single file must be specified to be renamed. Target {} " + "generate {} files: {}".format( @@ -452,7 +492,7 @@ def sphinx_inventory(*, name, src, **kwargs): the value `-` to indicate it is the same as `name` :::{seealso} - {bzl:obj}`//sphinxdocs/inventories` for inventories of Bazel objects. + {bzl:obj}`//inventories` for inventories of Bazel objects. ::: Args: @@ -469,6 +509,7 @@ def _sphinx_inventory_impl(ctx): args.add(output) ctx.actions.run( executable = ctx.executable._builder, + mnemonic = "SphinxInventoryBuilder", arguments = [args], inputs = depset([ctx.file.src]), outputs = [output], diff --git a/sphinxdocs/sphinxdocs/private/sphinx_build.py b/sphinxdocs/sphinxdocs/private/sphinx_build.py new file mode 100644 index 0000000000..52a334d9b9 --- /dev/null +++ b/sphinxdocs/sphinxdocs/private/sphinx_build.py @@ -0,0 +1,518 @@ +import concurrent.futures +import contextlib +import io +import json +import logging +import os +import pathlib +import shutil +import stat +import sys +import threading +import traceback +import typing + +import sphinx.application +from sphinx.cmd.build import main + +WorkRequest = object +WorkResponse = object + + +class SphinxMainError(Exception): + def __init__(self, message, exit_code): + super().__init__(message) + self.exit_code = exit_code + + +logger = logging.getLogger("sphinxdocs_build") + +_WORKER_SPHINX_EXT_MODULE_NAME = "bazel_worker_sphinx_ext" + +# Config value name for getting the path to the request info file +_REQUEST_INFO_CONFIG_NAME = "bazel_worker_request_info_path" + + +class DirectorySyncerError(Exception): + """Raised when one or more errors occur during directory synchronization.""" + + def __init__(self, errors: typing.List[BaseException]): + self.errors = errors + message = f"Encountered {len(errors)} error(s) during sync:\n" + "\n".join( + f" - {e}" for e in errors + ) + super().__init__(message) + + +class DirectorySyncer: + """Synchronizes a working destination directory from a source directory. + + Supports concurrent SHA-aware incremental updates (via sync()) for worker + mode and concurrent full directory copying (via copytree()) for non-worker + mode, ensuring physical file materialization to prevent relative + cross-reference resolution failures. + """ + + def __init__( + self, + srcdir: pathlib.Path, + destdir: pathlib.Path, + max_workers: typing.Optional[int] = None, + ): + self._srcdir = srcdir + self._destdir = destdir + self._max_workers = max_workers or min(32, (os.cpu_count() or 4) + 4) + self._current_shas: typing.Dict[str, str] = {} + self._lock = threading.Lock() + self._finished_cond = threading.Condition(self._lock) + self._remaining = 0 + self._errors: typing.List[BaseException] = [] + self._executor: typing.Optional[concurrent.futures.ThreadPoolExecutor] = None + + def _reset_state(self) -> None: + with self._lock: + self._errors.clear() + self._remaining = 0 + + def _wait_for_completion(self) -> None: + with self._lock: + while self._remaining > 0: + self._finished_cond.wait() + if self._errors: + raise DirectorySyncerError(list(self._errors)) + + def _submit_task(self, fn, *args) -> None: + with self._lock: + self._remaining += 1 + future = self._executor.submit(fn, *args) + future.add_done_callback(self._handle_task_done) + + def _handle_task_done(self, future: concurrent.futures.Future) -> None: + exc = future.exception() + if exc: + with self._lock: + self._errors.append(exc) + + def _task_finished(self) -> None: + with self._lock: + self._remaining -= 1 + if self._remaining == 0: + self._finished_cond.notify_all() + + @contextlib.contextmanager + def _create_executor(self): + with concurrent.futures.ThreadPoolExecutor( + max_workers=self._max_workers + ) as executor: + self._executor = executor + try: + yield + finally: + self._executor = None + + def copytree(self) -> None: + """Concurrently copies srcdir to destdir without SHA tracking.""" + self._reset_state() + shutil.rmtree(self._destdir, ignore_errors=True) + with self._create_executor(): + self._submit_task(self._copy_dir, self._srcdir, self._destdir) + self._wait_for_completion() + + def sync(self, entries: typing.Dict[str, str]) -> None: + """Synchronizes destdir to match entries {relative_path: sha} concurrently.""" + self._reset_state() + + to_remove = set(self._current_shas.keys()) - set(entries.keys()) + to_copy = { + path: sha + for path, sha in entries.items() + if self._current_shas.get(path) != sha + } + + if not to_remove and not to_copy: + self._current_shas = dict(entries) + return + + with self._create_executor(): + # 1. Submit stale path removals concurrently ASAP + for rel_path in to_remove: + dest_path = self._destdir / rel_path + self._submit_task(self._remove_path, dest_path) + + # 2. Submit created/updated item copies concurrently ASAP + for rel_path in to_copy: + src_path = self._srcdir / rel_path + dest_path = self._destdir / rel_path + if src_path.is_dir(): + self._submit_task(self._copy_dir, src_path, dest_path) + else: + self._submit_task(self._copy_file, src_path, dest_path) + + self._wait_for_completion() + + self._current_shas = dict(entries) + + def _remove_path(self, dest_path: pathlib.Path) -> None: + try: + if dest_path.is_dir() and not dest_path.is_symlink(): + shutil.rmtree(dest_path) + else: + dest_path.unlink(missing_ok=True) + except BaseException as e: + e.add_note(f"Failed removing path: dest_path={dest_path}") + raise + finally: + self._task_finished() + + def _copy_file(self, src: pathlib.Path, dest: pathlib.Path) -> None: + try: + dest.parent.mkdir(parents=True, exist_ok=True) + shutil.copy2(src, dest) + mode = dest.stat().st_mode + dest.chmod(mode | stat.S_IWUSR) + except BaseException as e: + e.add_note(f"Failed copying file: src={src}, dest={dest}") + raise + finally: + self._task_finished() + + def _copy_dir(self, src: pathlib.Path, dest: pathlib.Path) -> None: + """Recursively creates destination directory and submits tasks for its entries.""" + try: + dest.mkdir(parents=True, exist_ok=True) + with os.scandir(src) as scanner: + for entry in scanner: + c_dest = dest / entry.name + c_src = pathlib.Path(entry.path) + if entry.is_dir(): + self._submit_task(self._copy_dir, c_src, c_dest) + else: + self._submit_task(self._copy_file, c_src, c_dest) + except BaseException as e: + e.add_note(f"Failed copying directory: src={src}, dest={dest}") + raise + finally: + self._task_finished() + + +class Worker: + """A Bazel persistent worker for Sphinx builds.""" + + def __init__( + self, instream: "typing.TextIO", outstream: "typing.TextIO", exec_root: str + ): + # NOTE: Sphinx performs its own logging re-configuration, so any + # logging config we do isn't respected by Sphinx. Controlling where + # stdout and stderr goes are the main mechanisms. Recall that + # Bazel send worker stderr to the worker log file. + # outputBase=$(bazel info output_base) + # find $outputBase/bazel-workers/ -type f -printf '%T@ %p\n' | sort -n | tail -1 | awk '{print $2}' + logging.basicConfig(level=logging.WARN) + logger.info("Initializing worker") + + # The directory that paths are relative to. + self._exec_root = exec_root + # Where requests are read from. + self._instream = instream + # Where responses are written to. + self._outstream = outstream + + # dict[str srcdir, dict[str path, str digest]] + self._digests = {} + self._syncers: typing.Dict[pathlib.Path, DirectorySyncer] = {} + + # Internal output directories the worker gives to Sphinx that need + # to be cleaned up upon exit. + # set[str path] + self._worker_outdirs = set() + self._extension = BazelWorkerExtension() + + sys.modules[_WORKER_SPHINX_EXT_MODULE_NAME] = self._extension + sphinx.application.builtin_extensions += (_WORKER_SPHINX_EXT_MODULE_NAME,) + + def __enter__(self): + return self + + def __exit__(self, exc_type, exc_val, exc_tb): + for worker_outdir in self._worker_outdirs: + shutil.rmtree(worker_outdir, ignore_errors=True) + + def run(self) -> None: + logger.info("Worker started") + try: + while True: + request = None + try: + request = self._get_next_request() + if request is None: + logger.info("Empty request: exiting") + break + response = self._process_request(request) + if response: + self._send_response(response) + except SphinxMainError as e: + logger.error( + "Sphinx main returned failure: exit_code=%s request=%s", + request, + e.exit_code, + ) + request_id = 0 if not request else request.get("requestId", 0) + self._send_response( + { + "exitCode": e.exit_code, + "output": str(e), + "requestId": request_id, + } + ) + except Exception: + logger.exception("Unhandled error: request=%s", request) + output = ( + f"Unhandled error:\nRequest id: {request.get('id')}\n" + + traceback.format_exc() + ) + request_id = 0 if not request else request.get("requestId", 0) + self._send_response( + { + "exitCode": 3, + "output": output, + "requestId": request_id, + } + ) + finally: + logger.info("Worker shutting down") + + def _get_next_request(self) -> "object | None": + line = self._instream.readline() + if not line: + return None + return json.loads(line) + + def _send_response(self, response: "WorkResponse") -> None: + self._outstream.write(json.dumps(response) + "\n") + self._outstream.flush() + + def _prepare_sphinx(self, request): + sphinx_args = request["arguments"] + srcdir = pathlib.Path(sphinx_args[0]) + destdir = pathlib.Path(f"{srcdir}.worker-in.d") + + incoming_digests = {} + current_digests = self._digests.setdefault(str(srcdir), {}) + is_first_request = not current_digests + changed_paths = [] + request_info = {"exec_root": self._exec_root, "inputs": request["inputs"]} + srcdir_prefix = str(srcdir) + "/" + for entry in request["inputs"]: + path = entry["path"] + # In persistent worker mode, request["inputs"] includes action-level + # tools (e.g. sphinx-build, sphinx_build.py) and params files that + # live outside srcdir. Only synchronize documentation sources inside srcdir. + if not path.startswith(srcdir_prefix): + continue + digest = entry["digest"] + # Make the path srcdir-relative so Sphinx understands it. + path = path.removeprefix(srcdir_prefix) + incoming_digests[path] = digest + + if path not in current_digests: + logger.info("path %s new", path) + changed_paths.append(path) + elif current_digests[path] != digest: + logger.info("path %s changed", path) + changed_paths.append(path) + + self._digests[str(srcdir)] = incoming_digests + self._extension.changed_paths = changed_paths + request_info["changed_sources"] = changed_paths + + bazel_outdir = sphinx_args[1] + worker_outdir = bazel_outdir + ".worker-out.d" + # The doctree dir deliberately lives outside the declared outputs so + # it survives between invocations (that is what makes worker builds + # incremental). A new worker has no digest history: it reports every + # file as changed and re-reads all docs. Doing that against the stale + # Sphinx environment of a previous worker produces spurious warnings + # (e.g. duplicate labels), which --fail-on-warning turns into build + # failures. So on the first request start from a clean slate. + if is_first_request: + shutil.rmtree(worker_outdir, ignore_errors=True) + shutil.rmtree(destdir, ignore_errors=True) + for arg in sphinx_args: + if arg.startswith("--doctree-dir="): + shutil.rmtree(arg.partition("=")[2], ignore_errors=True) + self._worker_outdirs.add(worker_outdir) + sphinx_args[1] = worker_outdir + + if srcdir not in self._syncers: + self._syncers[srcdir] = DirectorySyncer(srcdir, destdir) + syncer = self._syncers[srcdir] + syncer.sync(incoming_digests) + + sphinx_args[0] = str(destdir) + request_info_path = os.path.join( + sphinx_args[0], "_bazel_worker_request_info.json" + ) + with open(request_info_path, "w") as fp: + json.dump(request_info, fp) + sphinx_args.append(f"--define={_REQUEST_INFO_CONFIG_NAME}={request_info_path}") + + return worker_outdir, bazel_outdir, sphinx_args + + @contextlib.contextmanager + def _redirect_streams(self): + stdout = io.StringIO() + stderr = io.StringIO() + with contextlib.redirect_stdout(stdout), contextlib.redirect_stderr(stderr): + yield stdout, stderr + + def _process_request(self, request: "WorkRequest") -> "WorkResponse | None": + logger.info("Request: %s", json.dumps(request, sort_keys=True, indent=2)) + if request.get("cancel"): + return None + + worker_outdir, bazel_outdir, sphinx_args = self._prepare_sphinx(request) + + # Prevent anything from going to stdout because it breaks the worker + # protocol. We have limited control over where Sphinx sends output. + with self._redirect_streams() as (stdout, stderr): + logger.info("main args: %s", sphinx_args) + exit_code = main(sphinx_args) + # Running Sphinx multiple times in a process can give spurious + # errors. An invocation after an error seems to work, though. + if exit_code == 2: + logger.warning("Sphinx main() returned exit_code=2, retrying...") + # Reset streams to capture output of the retry cleanly + stdout.seek(0) + stdout.truncate(0) + stderr.seek(0) + stderr.truncate(0) + # If Sphinx cache (`--doctree-dir`) becomes corrupted across incremental + # updates or branch checkouts, exit code 2 is returned. Wiping out the cached + # doctrees before retrying allows Sphinx to recover cleanly from scratch. + for arg in sphinx_args: + if arg.startswith("--doctree-dir="): + shutil.rmtree(arg.split("=", 1)[1], ignore_errors=True) + exit_code = main(sphinx_args) + + if exit_code: + stdout_output = stdout.getvalue().strip() + stderr_output = stderr.getvalue().strip() + if stdout_output: + stdout_output = ( + "========== STDOUT START ==========\n" + + stdout_output + + "\n" + + "========== STDOUT END ==========\n" + ) + else: + stdout_output = "========== STDOUT EMPTY ==========\n" + if stderr_output: + stderr_output = ( + "========== STDERR START ==========\n" + + stderr_output + + "\n" + + "========== STDERR END ==========\n" + ) + else: + stderr_output = "========== STDERR EMPTY ==========\n" + + message = ( + "Sphinx main() returned failure: " + + f" exit code: {exit_code}\n" + + stdout_output + + stderr_output + ) + raise SphinxMainError(message, exit_code) + + # Copying is unfortunately necessary because Bazel doesn't know to + # implicily bring along what the symlinks point to. + shutil.copytree(worker_outdir, bazel_outdir, dirs_exist_ok=True) + + # Include both stdout and stderr in the response output so that Sphinx + # warnings or diagnostic messages written to stderr are reported to the + # Bazel console even when the build succeeds. + stdout_output = stdout.getvalue() + stderr_output = stderr.getvalue() + if stderr_output: + output = f"--- STDOUT ---\n{stdout_output}\n--- STDERR ---\n{stderr_output}" + else: + output = stdout_output + + response = { + "requestId": request.get("requestId", 0), + "output": output, + "exitCode": 0, + } + return response + + +class BazelWorkerExtension: + """A Sphinx extension implemented as a class acting like a module.""" + + def __init__(self): + # Make it look like a Module object + self.__name__ = _WORKER_SPHINX_EXT_MODULE_NAME + # set[str] of src-dir relative path names + self.changed_paths = set() + + def setup(self, app): + app.add_config_value(_REQUEST_INFO_CONFIG_NAME, "", "") + app.connect("env-get-outdated", self._handle_env_get_outdated) + return {"parallel_read_safe": True, "parallel_write_safe": True} + + def _handle_env_get_outdated(self, app, env, added, changed, removed): + changed_docs = set() + for p in self.changed_paths: + # Try multiple path resolutions because depending on how Sphinx and Bazel + # represent inputs (`p`), `env.path2doc` may require relative, srcdir-joined, + # or absolute paths to successfully resolve the document name. + doc = ( + env.path2doc(p) + or env.path2doc(os.path.join(env.srcdir, p)) + or env.path2doc(os.path.abspath(os.path.join(env.srcdir, p))) + ) + if doc: + changed_docs.add(doc) + + # When documents are added or removed across incremental builds or branch checkouts, + # parent documents whose `toctree` includes them (especially via glob patterns or + # explicit references to removed docs) must be invalidated and re-read. Otherwise, + # Sphinx retains stale table of contents entries or throws unresolvable reference errors. + if added or removed: + glob_toctrees = getattr(env, "glob_toctrees", set()) + for doc, includes in getattr(env, "toctree_includes", {}).items(): + if doc in glob_toctrees or not removed.isdisjoint(includes): + changed_docs.add(doc) + + logger.info("changed docs: %s", changed_docs) + return changed_docs + + +def _worker_main(stdin, stdout, exec_root): + with Worker(stdin, stdout, exec_root) as worker: + return worker.run() + + +def _non_worker_main(): + args = [] + for arg in sys.argv: + if arg.startswith("@"): + with open(arg.removeprefix("@")) as fp: + lines = [line.strip() for line in fp if line.strip()] + args.extend(lines) + else: + args.append(arg) + if len(args) > 1: + srcdir = pathlib.Path(args[1]) + destdir = pathlib.Path(f"{srcdir}.worker-in.d") + DirectorySyncer(srcdir, destdir).copytree() + args[1] = str(destdir) + sys.argv[:] = args + return main() + + +if __name__ == "__main__": + if "--persistent_worker" in sys.argv: + sys.exit(_worker_main(sys.stdin, sys.stdout, os.getcwd())) + else: + sys.exit(_non_worker_main()) diff --git a/sphinxdocs/private/sphinx_docs_library.bzl b/sphinxdocs/sphinxdocs/private/sphinx_docs_library.bzl similarity index 100% rename from sphinxdocs/private/sphinx_docs_library.bzl rename to sphinxdocs/sphinxdocs/private/sphinx_docs_library.bzl diff --git a/sphinxdocs/private/sphinx_docs_library_info.bzl b/sphinxdocs/sphinxdocs/private/sphinx_docs_library_info.bzl similarity index 100% rename from sphinxdocs/private/sphinx_docs_library_info.bzl rename to sphinxdocs/sphinxdocs/private/sphinx_docs_library_info.bzl diff --git a/sphinxdocs/private/sphinx_docs_library_macro.bzl b/sphinxdocs/sphinxdocs/private/sphinx_docs_library_macro.bzl similarity index 70% rename from sphinxdocs/private/sphinx_docs_library_macro.bzl rename to sphinxdocs/sphinxdocs/private/sphinx_docs_library_macro.bzl index 095b3769ca..e93a2f5bf6 100644 --- a/sphinxdocs/private/sphinx_docs_library_macro.bzl +++ b/sphinxdocs/sphinxdocs/private/sphinx_docs_library_macro.bzl @@ -1,6 +1,6 @@ """Implementation of sphinx_docs_library macro.""" -load("//python/private:util.bzl", "add_tag") # buildifier: disable=bzl-visibility +load("//sphinxdocs/private:util.bzl", "add_tag") # buildifier: disable=bzl-visibility load(":sphinx_docs_library.bzl", _sphinx_docs_library = "sphinx_docs_library") def sphinx_docs_library(**kwargs): @@ -9,5 +9,5 @@ def sphinx_docs_library(**kwargs): Args: **kwargs: Args passed onto underlying {bzl:rule}`sphinx_docs_library` rule """ - add_tag(kwargs, "@rules_python//sphinxdocs:sphinx_docs_library") + add_tag(kwargs, "//sphinxdocs:sphinx_docs_library") _sphinx_docs_library(**kwargs) diff --git a/sphinxdocs/private/sphinx_run_template.sh b/sphinxdocs/sphinxdocs/private/sphinx_run_template.sh similarity index 96% rename from sphinxdocs/private/sphinx_run_template.sh rename to sphinxdocs/sphinxdocs/private/sphinx_run_template.sh index 4a1f1e4410..aa83757c1b 100644 --- a/sphinxdocs/private/sphinx_run_template.sh +++ b/sphinxdocs/sphinxdocs/private/sphinx_run_template.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash declare -a args %SETUP_ARGS% diff --git a/sphinxdocs/private/sphinx_server.py b/sphinxdocs/sphinxdocs/private/sphinx_server.py similarity index 86% rename from sphinxdocs/private/sphinx_server.py rename to sphinxdocs/sphinxdocs/private/sphinx_server.py index 1f4fae86de..5d26eaf728 100644 --- a/sphinxdocs/private/sphinx_server.py +++ b/sphinxdocs/sphinxdocs/private/sphinx_server.py @@ -5,22 +5,26 @@ import time from http import server +from python.runfiles import Runfiles + def main(argv): - build_workspace_directory = os.environ["BUILD_WORKSPACE_DIRECTORY"] - docs_directory = argv[1] - serve_directory = os.path.join(build_workspace_directory, docs_directory) + r = Runfiles.Create() + serve_directory = r.Rlocation(argv[1]) + if not serve_directory: + print(f"Error: could not find runfile for '{argv[1]}'", file=sys.stderr) + return 1 class DirectoryHandler(server.SimpleHTTPRequestHandler): def __init__(self, *args, **kwargs): super().__init__(directory=serve_directory, *args, **kwargs) - address = ("0.0.0.0", 8000) + address = ("0.0.0.0", 8000) # noqa: F841 # with server.ThreadingHTTPServer(address, DirectoryHandler) as (ip, port, httpd): with _start_server(DirectoryHandler, "0.0.0.0", 8000) as (ip, port, httpd): def _print_server_info(): - print(f"Serving...") + print("Serving...") print(f" Address: http://{ip}:{port}") print(f" Serving directory: {serve_directory}") print(f" url: file://{serve_directory}") diff --git a/sphinxdocs/private/sphinx_stardoc.bzl b/sphinxdocs/sphinxdocs/private/sphinx_stardoc.bzl similarity index 96% rename from sphinxdocs/private/sphinx_stardoc.bzl rename to sphinxdocs/sphinxdocs/private/sphinx_stardoc.bzl index d5869b0bc4..75a4daf4d0 100644 --- a/sphinxdocs/private/sphinx_stardoc.bzl +++ b/sphinxdocs/sphinxdocs/private/sphinx_stardoc.bzl @@ -19,8 +19,8 @@ load("@bazel_skylib//lib:paths.bzl", "paths") load("@bazel_skylib//lib:types.bzl", "types") load("@bazel_skylib//rules:build_test.bzl", "build_test") load("@io_bazel_stardoc//stardoc:stardoc.bzl", "stardoc") -load("//python/private:util.bzl", "add_tag", "copy_propagating_kwargs") # buildifier: disable=bzl-visibility load("//sphinxdocs/private:sphinx_docs_library_macro.bzl", "sphinx_docs_library") +load("//sphinxdocs/private:util.bzl", "add_tag", "copy_propagating_kwargs") # buildifier: disable=bzl-visibility _StardocInputHelperInfo = provider( doc = "Extracts the single source file from a bzl library.", @@ -73,7 +73,7 @@ def sphinx_stardocs( **kwargs: Additional kwargs to pass onto each `sphinx_stardoc` target """ internal_name = "_{}".format(name) - add_tag(kwargs, "@rules_python//sphinxdocs:sphinx_stardocs") + add_tag(kwargs, "//sphinxdocs:sphinx_stardocs") common_kwargs = copy_propagating_kwargs(kwargs) common_kwargs["target_compatible_with"] = kwargs.get("target_compatible_with") @@ -160,7 +160,7 @@ def sphinx_stardoc( **kwargs: {type}`dict` common args passed onto rules. """ internal_name = "_{}".format(name.lstrip("_")) - add_tag(kwargs, "@rules_python//sphinxdocs:sphinx_stardoc") + add_tag(kwargs, "//sphinxdocs:sphinx_stardoc") common_kwargs = copy_propagating_kwargs(kwargs) common_kwargs["target_compatible_with"] = kwargs.get("target_compatible_with") @@ -171,7 +171,7 @@ def sphinx_stardoc( **common_kwargs ) - stardoc_name = internal_name + "_stardoc" + stardoc_name = internal_name + "__stardoc" # NOTE: The .binaryproto suffix is an optimization. It makes the stardoc() # call avoid performing a copy of the output to the desired name. @@ -186,7 +186,7 @@ def sphinx_stardoc( **common_kwargs ) - pb2md_name = internal_name + "_pb2md" + pb2md_name = internal_name + "__pb2md" _stardoc_proto_to_markdown( name = pb2md_name, src = stardoc_pb, diff --git a/sphinxdocs/sphinxdocs/private/util.bzl b/sphinxdocs/sphinxdocs/private/util.bzl new file mode 100644 index 0000000000..f285a83c13 --- /dev/null +++ b/sphinxdocs/sphinxdocs/private/util.bzl @@ -0,0 +1,58 @@ +"""Utility functions used by sphinxdocs.""" + +load("@bazel_skylib//lib:types.bzl", "types") + +# When bzlmod is enabled, canonical repos names have @@ in them, while under +# workspace builds, there is never a @@ in labels. +BZLMOD_ENABLED = "@@" in str(Label("//sphinxdocs:unused")) + +def copy_propagating_kwargs(from_kwargs, into_kwargs = None): + """Copies args that must be compatible between two targets with a dependency relationship. + + This is intended for when one target depends on another, so they must have + compatible settings such as `testonly` and `compatible_with`. This usually + happens when a macro generates multiple targets, some of which depend + on one another, so their settings must be compatible. + + Args: + from_kwargs: keyword args dict whose common kwarg will be copied. + into_kwargs: optional keyword args dict that the values from `from_kwargs` + will be copied into. The values in this dict will take precedence + over the ones in `from_kwargs` (i.e., if this has `testonly` already + set, then it won't be overwritten). + NOTE: THIS WILL BE MODIFIED IN-PLACE. + + Returns: + Keyword args to use for the depender target derived from the dependency + target. If `into_kwargs` was passed in, then that same object is + returned; this is to facilitate easy `**` expansion. + """ + if into_kwargs == None: + into_kwargs = {} + + # Include tags because people generally expect tags to propagate. + for attr in ("testonly", "tags", "compatible_with", "restricted_to", "target_compatible_with"): + if attr in from_kwargs and attr not in into_kwargs: + into_kwargs[attr] = from_kwargs[attr] + return into_kwargs + +def add_tag(attrs, tag): + """Adds `tag` to `attrs["tags"]`. + + Args: + attrs: dict of keyword args. It is modified in place. + tag: str, the tag to add. + """ + if "tags" in attrs and attrs["tags"] != None: + tags = attrs["tags"] + + # Preserve the input type: this allows a test verifying the underlying + # rule can accept the tuple for the tags argument. + if types.is_tuple(tags): + attrs["tags"] = tags + (tag,) + else: + # List concatenation is necessary because the original value + # may be a frozen list. + attrs["tags"] = tags + [tag] + else: + attrs["tags"] = [tag] diff --git a/sphinxdocs/sphinxdocs/private/visibility.bzl b/sphinxdocs/sphinxdocs/private/visibility.bzl new file mode 100644 index 0000000000..706be45a3b --- /dev/null +++ b/sphinxdocs/sphinxdocs/private/visibility.bzl @@ -0,0 +1,7 @@ +"""Shared code for use with visibility specs.""" + +# Use when a target isn't actually public, but needs public +# visibility to keep Bazel happy. +# Such cases are typically for defaults of rule attributes or macro args that +# get used outside of sphinxdocs itself. +NOT_ACTUALLY_PUBLIC = ["//visibility:public"] diff --git a/sphinxdocs/readthedocs.bzl b/sphinxdocs/sphinxdocs/readthedocs.bzl similarity index 100% rename from sphinxdocs/readthedocs.bzl rename to sphinxdocs/sphinxdocs/readthedocs.bzl diff --git a/sphinxdocs/sphinx.bzl b/sphinxdocs/sphinxdocs/sphinx.bzl similarity index 100% rename from sphinxdocs/sphinx.bzl rename to sphinxdocs/sphinxdocs/sphinx.bzl diff --git a/sphinxdocs/sphinx_docs_library.bzl b/sphinxdocs/sphinxdocs/sphinx_docs_library.bzl similarity index 100% rename from sphinxdocs/sphinx_docs_library.bzl rename to sphinxdocs/sphinxdocs/sphinx_docs_library.bzl diff --git a/sphinxdocs/sphinx_stardoc.bzl b/sphinxdocs/sphinxdocs/sphinx_stardoc.bzl similarity index 100% rename from sphinxdocs/sphinx_stardoc.bzl rename to sphinxdocs/sphinxdocs/sphinx_stardoc.bzl diff --git a/sphinxdocs/src/sphinx_bzl/BUILD.bazel b/sphinxdocs/sphinxdocs/src/sphinx_bzl/BUILD.bazel similarity index 69% rename from sphinxdocs/src/sphinx_bzl/BUILD.bazel rename to sphinxdocs/sphinxdocs/src/sphinx_bzl/BUILD.bazel index 8830315bc3..2dd25e09b3 100644 --- a/sphinxdocs/src/sphinx_bzl/BUILD.bazel +++ b/sphinxdocs/sphinxdocs/src/sphinx_bzl/BUILD.bazel @@ -1,7 +1,7 @@ -load("//python:py_library.bzl", "py_library") +load("@rules_python//python:py_library.bzl", "py_library") package( - default_visibility = ["//:__subpackages__"], + default_visibility = ["//sphinxdocs:__subpackages__"], ) # NOTE: This provides the library on its own, not its dependencies. diff --git a/sphinxdocs/sphinxdocs/src/sphinx_bzl/__init__.py b/sphinxdocs/sphinxdocs/src/sphinx_bzl/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/sphinxdocs/src/sphinx_bzl/bzl.py b/sphinxdocs/sphinxdocs/src/sphinx_bzl/bzl.py similarity index 91% rename from sphinxdocs/src/sphinx_bzl/bzl.py rename to sphinxdocs/sphinxdocs/src/sphinx_bzl/bzl.py index 90fb109614..c115737ba5 100644 --- a/sphinxdocs/src/sphinx_bzl/bzl.py +++ b/sphinxdocs/sphinxdocs/src/sphinx_bzl/bzl.py @@ -22,18 +22,25 @@ from typing import Callable, Iterable, TypeVar from docutils import nodes as docutils_nodes -from docutils.parsers.rst import directives as docutils_directives -from docutils.parsers.rst import states -from sphinx import addnodes, builders -from sphinx import directives as sphinx_directives -from sphinx import domains, environment, roles +from docutils.parsers.rst import directives as docutils_directives, states +from sphinx import ( + addnodes, + builders, + directives as sphinx_directives, + domains, + environment, + roles, +) from sphinx.highlighting import lexer_classes from sphinx.locale import _ -from sphinx.util import docfields -from sphinx.util import docutils as sphinx_docutils -from sphinx.util import inspect, logging -from sphinx.util import nodes as sphinx_nodes -from sphinx.util import typing as sphinx_typing +from sphinx.util import ( + docfields, + docutils as sphinx_docutils, + inspect, + logging, + nodes as sphinx_nodes, + typing as sphinx_typing, +) from typing_extensions import TypeAlias, override _logger = logging.getLogger(__name__) @@ -57,7 +64,7 @@ def _log_debug(message, *args): # NOTE: Non-warning log messages go to stdout and are only # visible when -q isn't passed to Sphinx. Note that the sphinx_docs build - # rule passes -q by default; use --//sphinxdocs:quiet=false to disable it. + # rule passes -q by default; use --//:quiet=false to disable it. _logger.debug("%s" + message, _LOG_PREFIX, *args) @@ -390,8 +397,12 @@ def _make_xrefs_for_arg_attr( descr=index_description, ), ), - # This allows referencing an arg as e.g `funcname.argname` - alt_names=[anchor_id], + alt_names=[ + # This allows referencing an arg as e.g `funcname.argname` + anchor_id, + # This allows referencing an arg as simply `argname` + arg_name, + ], ) # Two changes to how arg xrefs are created: @@ -451,7 +462,7 @@ def make_field( field_text = item[1][0].astext() parts = [p.strip() for p in field_text.split(",")] field_body = docutils_nodes.field_body() - for _, is_last, part in _position_iter(parts): + for _, is_last, part in _position_iter(parts): # noqa: F402 node = self.make_xref( self.bodyrolename, self._body_domain or domain, @@ -498,7 +509,55 @@ def run(self) -> list[docutils_nodes.Node]: self.env.ref_context["bzl:file"] = file_label self.env.ref_context["bzl:object_id_stack"] = [] self.env.ref_context["bzl:doc_id_stack"] = [] - return [] + + package_label, _, basename = file_label.partition(":") + + # Transform //foo/bar:BUILD.bazel into "bar" + # This allows referencing "bar" as itself + extra_alt_names = [] + if basename in ("BUILD.bazel", "BUILD"): + # Allow xref //foo + extra_alt_names.append(package_label) + basename = os.path.basename(package_label) + # Handle //:BUILD.bazel + if not basename: + # There isn't a convention for referring to the root package + # besides `//:`, which is already the file_label. So just + # use some obvious value + basename = "__ROOT_BAZEL_PACKAGE__" + + index_description = f"File {label}" + absolute_label = repo + label + self.env.get_domain("bzl").add_object( + _ObjectEntry( + full_id=absolute_label, + display_name=absolute_label, + object_type="obj", + search_priority=1, + index_entry=domains.IndexEntry( + name=basename, + subtype=_INDEX_SUBTYPE_NORMAL, + docname=self.env.docname, + anchor="", + extra="", + qualifier="", + descr=index_description, + ), + ), + alt_names=[ + # Allow xref //foo:bar.bzl + file_label, + # Allow xref bar.bzl + basename, + ] + + extra_alt_names, + ) + index_node = addnodes.index( + entries=[ + _index_node_tuple("single", f"File; {label}", ""), + ] + ) + return [index_node] class _BzlAttrInfo(sphinx_docutils.SphinxDirective): @@ -556,8 +615,10 @@ def first_child_with_class_name( root, class_name ) -> typing.Union[None, docutils_nodes.Element]: matches = root.findall( - lambda node: isinstance(node, docutils_nodes.Element) - and class_name in node["classes"] + lambda node: ( + isinstance(node, docutils_nodes.Element) + and class_name in node["classes"] + ) ) found = next(matches, None) return found @@ -714,7 +775,7 @@ def make_xref(name, title=None): return obj_id def _signature_add_object_type(self, sig_node: addnodes.desc_signature): - if sig_object_type := self._get_signature_object_type(): + if sig_object_type := self._get_signature_object_type(): # noqa: F841 sig_node += addnodes.desc_annotation("", self._get_signature_object_type()) sig_node += addnodes.desc_sig_space() @@ -1156,10 +1217,10 @@ class _BzlTagClass(_BzlCallable): doc_field_types = [ _BzlGroupedField( - "arg", + "attr", label=_("Attributes"), names=["attr"], - rolename="arg", + rolename="attr", can_collapse=False, ), ] @@ -1463,6 +1524,8 @@ class _BzlDomain(domains.Domain): # :obj:. # NOTE: We also use these object types for categorizing things in the # generated index page. + # NOTE: The object type keys control what object types are recognized + # in inventory files. object_types = { "arg": domains.ObjType("arg", "arg", "obj"), # macro/function arg "aspect": domains.ObjType("aspect", "aspect", "obj"), @@ -1486,6 +1549,8 @@ class _BzlDomain(domains.Domain): # types are objects that have a constructor and methods/attrs "type": domains.ObjType("type", "type", "obj"), "typedef": domains.ObjType("typedef", "typedef", "type", "obj"), + # generic objs usually come from inventories + "obj": domains.ObjType("object", "obj"), } # This controls: @@ -1657,6 +1722,12 @@ def add_object(self, entry: _ObjectEntry, alt_names=None) -> None: ) if entry.full_id in self.data["objects"]: existing = self.data["objects"][entry.full_id] + # If the objects are identical, then it's fine. This can happen + # when a doc is re-parsed (e.g. during a query) or if a symbol + # is documented multiple times in the same file. + if existing.to_get_objects_tuple() == entry.to_get_objects_tuple(): + return + raise Exception( f"Object {entry.full_id} already registered: " + f"existing={existing}, incoming={entry}" @@ -1684,6 +1755,27 @@ def add_object(self, entry: _ObjectEntry, alt_names=None) -> None: self.data["doc_names"].setdefault(docname, {}) self.data["doc_names"][docname][base_name] = entry + @override + def clear_doc(self, docname: str) -> None: + if docname not in self.data["doc_names"]: + return + for base_name, entry in self.data["doc_names"][docname].items(): + if entry.full_id in self.data["objects"]: + del self.data["objects"][entry.full_id] + + if entry.full_id in self.data["objects_by_type"].get(entry.object_type, {}): + del self.data["objects_by_type"][entry.object_type][entry.full_id] + + # We can't easily reverse the mapping for alt_names, so we have + # to iterate over all of them. This is potentially slow, but + # clear_doc isn't called often. + for alt_name, entries in list(self.data["alt_names"].items()): + if entry.full_id in entries: + del entries[entry.full_id] + if not entries: + del self.data["alt_names"][alt_name] + del self.data["doc_names"][docname] + def merge_domaindata( self, docnames: list[str], otherdata: dict[str, typing.Any] ) -> None: @@ -1702,14 +1794,44 @@ def merge_domaindata( def _on_missing_reference(app, env: environment.BuildEnvironment, node, contnode): + """Handle missing references + + There are two main cases this is designed to handle: + 1. Psuedo-types, like None + 2. External references that aren't exact matches, e.g. using + `--stamp=true` to refer to the Bazel builtin stamp flag. + """ + target = node["reftarget"] if node["refdomain"] != "bzl": return None - if node["reftype"] != "type": + if node["reftype"] not in ("type", "flag", "obj"): return None # There's no Bazel docs for None, so prevent missing xrefs warning - if node["reftarget"] == "None": + if target == "None": + return contnode + + # Useful psuedo-types + if target in ("Any", "object", "collection"): return contnode + + original_target = target + new_target = target + + # Normalize --foo flag references + new_target = new_target.lstrip("-") + # Allow foo() style references + new_target, _, _ = new_target.partition("(") + # Allow --foo=bar references + new_target, _, _ = new_target.partition("=") + + if new_target != original_target: + # Access the intersphinx extension's internal mapping + # we try to resolve the reference again with the stripped name + from sphinx.ext.intersphinx import missing_reference + + node["reftarget"] = new_target + return missing_reference(app, env, node, contnode) return None diff --git a/sphinxdocs/tests/__init__.py b/sphinxdocs/tests/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/sphinxdocs/tests/proto_to_markdown/BUILD.bazel b/sphinxdocs/tests/proto_to_markdown/BUILD.bazel index 09f537472c..632d6d946f 100644 --- a/sphinxdocs/tests/proto_to_markdown/BUILD.bazel +++ b/sphinxdocs/tests/proto_to_markdown/BUILD.bazel @@ -12,13 +12,11 @@ # See the License for the specific language governing permissions and # limitations under the License. -load("//python:py_test.bzl", "py_test") -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility +load("@rules_python//python:py_test.bzl", "py_test") py_test( name = "proto_to_markdown_test", srcs = ["proto_to_markdown_test.py"], - target_compatible_with = [] if IS_BAZEL_7_OR_HIGHER else ["@platforms//:incompatible"], deps = [ "//sphinxdocs/private:proto_to_markdown_lib", "@dev_pip//absl_py", diff --git a/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py b/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py index 66e3224b20..d88d2bf127 100644 --- a/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py +++ b/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py @@ -13,13 +13,11 @@ # limitations under the License. import io -import re from absl.testing import absltest from google.protobuf import text_format -from stardoc.proto import stardoc_output_pb2 - from sphinxdocs.private import proto_to_markdown +from stardoc.proto import stardoc_output_pb2 _EVERYTHING_MODULE = """\ module_docstring: "MODULE_DOC_STRING" @@ -82,6 +80,14 @@ default_value: "[BZLMOD_EXT_TAG_A_ATTRIBUTE_1_DEFAULT_VALUE]" } } + tag_class: { + tag_name: "bzlmod_ext_tag_no_doc" + attribute: { + name: "bzlmod_ext_tag_a_attribute_2", + type: STRING_LIST + default_value: "[BZLMOD_EXT_TAG_A_ATTRIBUTE_2_DEFAULT_VALUE]" + } + } } repository_rule_info: { rule_name: "repository_rule", @@ -151,6 +157,9 @@ def test_basic_rendering_everything(self): self.assertRegex(actual, "bzlmod_ext_tag_a_attribute_1") self.assertRegex(actual, "BZLMOD_EXT_TAG_A_ATTRIBUTE_1_DOC_STRING") self.assertRegex(actual, "BZLMOD_EXT_TAG_A_ATTRIBUTE_1_DEFAULT_VALUE") + self.assertRegex(actual, "{bzl:tag-class} bzlmod_ext_tag_no_doc") + self.assertRegex(actual, "bzlmod_ext_tag_a_attribute_2") + self.assertRegex(actual, "BZLMOD_EXT_TAG_A_ATTRIBUTE_2_DEFAULT_VALUE") self.assertRegex(actual, "{bzl:repo-rule} repository_rule") self.assertRegex(actual, "REPOSITORY_RULE_DOC_STRING") @@ -261,6 +270,41 @@ def test_render_module_extension(self): ::::: +:::::: +""" + self.assertIn(expected, actual) + + def test_render_repo_rule(self): + proto_text = """ +file: "@repo//pkg:foo.bzl" +repository_rule_info: { + rule_name: "repository_rule", + doc_string: "REPOSITORY_RULE_DOC_STRING" + attribute: { + name: "repository_rule_attribute_a", + doc_string: "REPOSITORY_RULE_ATTRIBUTE_A_DOC_STRING" + type: BOOLEAN + default_value: "True" + } + environ: "ENV_VAR_A" +} +""" + actual = self._render(proto_text) + expected = """ +::::::{bzl:repo-rule} repository_rule(repository_rule_attribute_a=True) + +REPOSITORY_RULE_DOC_STRING + +:attr repository_rule_attribute_a: + {bzl:default-value}`True` + {type}`bool` + REPOSITORY_RULE_ATTRIBUTE_A_DOC_STRING + :::{bzl:attr-info} Info + ::: + + +:envvars: ENV_VAR_A + :::::: """ self.assertIn(expected, actual) diff --git a/sphinxdocs/tests/sphinx_build/BUILD.bazel b/sphinxdocs/tests/sphinx_build/BUILD.bazel new file mode 100644 index 0000000000..b9e77220df --- /dev/null +++ b/sphinxdocs/tests/sphinx_build/BUILD.bazel @@ -0,0 +1,11 @@ +load("@rules_python//python:py_test.bzl", "py_test") + +py_test( + name = "directory_syncer_test", + srcs = ["directory_syncer_test.py"], + deps = [ + "//sphinxdocs/private:sphinx_build_lib", + "@dev_pip//absl_py", + "@dev_pip//sphinx", + ], +) diff --git a/sphinxdocs/tests/sphinx_build/directory_syncer_test.py b/sphinxdocs/tests/sphinx_build/directory_syncer_test.py new file mode 100644 index 0000000000..1365e2b3ce --- /dev/null +++ b/sphinxdocs/tests/sphinx_build/directory_syncer_test.py @@ -0,0 +1,133 @@ +import pathlib +import shutil +import stat +import tempfile + +from absl.testing import absltest +from sphinxdocs.private.sphinx_build import DirectorySyncer, DirectorySyncerError + + +class DirectorySyncerTest(absltest.TestCase): + def setUp(self): + super().setUp() + self.test_dir = pathlib.Path(tempfile.mkdtemp()) + self.addCleanup(shutil.rmtree, self.test_dir, ignore_errors=True) + self.srcdir = self.test_dir / "src" + self.destdir = self.test_dir / "dest" + self.srcdir.mkdir() + + def _write_src(self, rel_path, content, mode=None): + path = self.srcdir / rel_path + path.parent.mkdir(parents=True, exist_ok=True) + if path.exists(): + path.chmod(stat.S_IRUSR | stat.S_IWUSR | stat.S_IRGRP | stat.S_IROTH) + path.write_text(content) + if mode is not None: + path.chmod(mode) + return path + + def assert_dest_equals(self, rel_path, expected_content): + dest_file = self.destdir / rel_path + self.assertTrue(dest_file.exists(), f"Expected {dest_file} to exist") + self.assertEqual(expected_content, dest_file.read_text()) + + def assert_dest_not_exists(self, rel_path): + dest_file = self.destdir / rel_path + self.assertFalse(dest_file.exists(), f"Expected {dest_file} to not exist") + + def test_copytree(self): + self._write_src("file1.txt", "hello") + self._write_src("sub/file2.txt", "world") + + syncer = DirectorySyncer(self.srcdir, self.destdir) + syncer.copytree() + + self.assert_dest_equals("file1.txt", "hello") + self.assert_dest_equals("sub/file2.txt", "world") + + def test_sync_initial_and_incremental(self): + self._write_src("doc1.md", "v1") + self._write_src("doc2.md", "v1") + self._write_src("doc3.md", "v1") + + syncer = DirectorySyncer(self.srcdir, self.destdir) + # Initial sync + syncer.sync( + { + "doc1.md": "sha-doc1-v1", + "doc2.md": "sha-doc2-v1", + "doc3.md": "sha-doc3-v1", + } + ) + + self.assert_dest_equals("doc1.md", "v1") + self.assert_dest_equals("doc2.md", "v1") + self.assert_dest_equals("doc3.md", "v1") + + # Incremental sync: + # - doc1.md: unchanged SHA + # - doc2.md: updated SHA & content + # - doc3.md: removed from entries + # - doc4.md: newly created file + self._write_src("doc2.md", "v2") + self._write_src("doc4.md", "v1") + + syncer.sync( + { + "doc1.md": "sha-doc1-v1", + "doc2.md": "sha-doc2-v2", + "doc4.md": "sha-doc4-v1", + } + ) + + self.assert_dest_equals("doc1.md", "v1") + self.assert_dest_equals("doc2.md", "v2") + self.assert_dest_not_exists("doc3.md") + self.assert_dest_equals("doc4.md", "v1") + + def test_read_only_file_becomes_writable(self): + read_only_mode = stat.S_IRUSR | stat.S_IRGRP | stat.S_IROTH + self._write_src("readonly.txt", "version 1", mode=read_only_mode) + syncer = DirectorySyncer(self.srcdir, self.destdir) + syncer.sync({"readonly.txt": "sha-v1"}) + + dest_file = self.destdir / "readonly.txt" + self.assert_dest_equals("readonly.txt", "version 1") + self.assertTrue( + dest_file.stat().st_mode & stat.S_IWUSR, + "Destination file should be writable", + ) + + # Ensure subsequent incremental updates can overwrite the file without permission error + self._write_src("readonly.txt", "version 2", mode=read_only_mode) + syncer.sync({"readonly.txt": "sha-v2"}) + self.assert_dest_equals("readonly.txt", "version 2") + + def test_sync_directory_artifact(self): + dir_artifact = self.srcdir / "tree_art" + dir_artifact.mkdir() + (dir_artifact / "page.md").write_text("content") + + syncer = DirectorySyncer(self.srcdir, self.destdir) + syncer.sync( + { + "tree_art": "sha-dir-art", + } + ) + + self.assert_dest_equals("tree_art/page.md", "content") + + def test_errors_bubble_up(self): + syncer = DirectorySyncer(self.srcdir, self.destdir) + with self.assertRaises(DirectorySyncerError) as cm: + syncer.sync( + { + "non_existent_1.txt": "sha1", + "non_existent_2.txt": "sha2", + } + ) + self.assertGreaterEqual(len(cm.exception.errors), 2) + + +if __name__ == "__main__": + absltest.main() diff --git a/sphinxdocs/tests/sphinx_docs/BUILD.bazel b/sphinxdocs/tests/sphinx_docs/BUILD.bazel index 1a05db0ea3..4bbaf90691 100644 --- a/sphinxdocs/tests/sphinx_docs/BUILD.bazel +++ b/sphinxdocs/tests/sphinx_docs/BUILD.bazel @@ -1,5 +1,5 @@ load("@bazel_skylib//rules:build_test.bzl", "build_test") -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility +load("@rules_python//python:py_test.bzl", "py_test") load("//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs") load(":defs.bzl", "gen_directory") @@ -13,7 +13,7 @@ _TARGET_COMPATIBLE_WITH = select({ "@platforms//os:linux": [], "@platforms//os:macos": [], "//conditions:default": ["@platforms//:incompatible"], -}) if IS_BAZEL_7_OR_HIGHER else ["@platforms//:incompatible"] +}) sphinx_docs( name = "docs", @@ -22,10 +22,20 @@ sphinx_docs( ], config = "conf.py", formats = ["html"], + renamed_srcs = { + ":gen_binary_asset": "binary_asset.bin", + }, sphinx = ":sphinx-build", + strip_prefix = package_name() + "/", target_compatible_with = _TARGET_COMPATIBLE_WITH, ) +genrule( + name = "gen_binary_asset", + outs = ["binary_asset.bin"], + cmd = "printf '\\x00\\xff\\xfe\\xfd\\x80\\x90\\x00\\x01\\x02\\x03\\x04' > $@", +) + gen_directory( name = "generated_directory", ) @@ -40,6 +50,13 @@ sphinx_build_binary( ) build_test( - name = "build_tests", + name = "docs_build_test", targets = [":docs"], ) + +py_test( + name = "sphinx_docs_output_test", + srcs = ["sphinx_docs_output_test.py"], + data = [":docs"], + deps = ["@dev_pip//absl_py"], +) diff --git a/sphinxdocs/tests/sphinx_docs/defs.bzl b/sphinxdocs/tests/sphinx_docs/defs.bzl index 2e47ecc0f7..36fd1dba4f 100644 --- a/sphinxdocs/tests/sphinx_docs/defs.bzl +++ b/sphinxdocs/tests/sphinx_docs/defs.bzl @@ -6,7 +6,9 @@ def _gen_directory_impl(ctx): ctx.actions.run_shell( outputs = [out], command = """ -echo "# Hello" > {outdir}/index.md +printf '# Hello\\n' > {outdir}/index.md +printf '# Dir Page 1\\n\\n[Dir Page 2](dir_page2.md)\\n' > {outdir}/dir_page1.md +printf '# Dir Page 2\\n' > {outdir}/dir_page2.md """.format( outdir = out.path, ), diff --git a/sphinxdocs/tests/sphinx_docs/doc1.md b/sphinxdocs/tests/sphinx_docs/doc1.md new file mode 100644 index 0000000000..f6f70ba28c --- /dev/null +++ b/sphinxdocs/tests/sphinx_docs/doc1.md @@ -0,0 +1,3 @@ +# doc1 + +hello doc 1 diff --git a/sphinxdocs/tests/sphinx_docs/doc2.md b/sphinxdocs/tests/sphinx_docs/doc2.md new file mode 100644 index 0000000000..06eb76a596 --- /dev/null +++ b/sphinxdocs/tests/sphinx_docs/doc2.md @@ -0,0 +1,3 @@ +# doc 2 + +hello doc 3 diff --git a/sphinxdocs/tests/sphinx_docs/index.md b/sphinxdocs/tests/sphinx_docs/index.md index cdce641fa1..68a5fb38c3 100644 --- a/sphinxdocs/tests/sphinx_docs/index.md +++ b/sphinxdocs/tests/sphinx_docs/index.md @@ -3,6 +3,7 @@ :::{toctree} :glob: +generated_directory/dir_page1 ** genindex ::: diff --git a/sphinxdocs/tests/sphinx_docs/sphinx_docs_output_test.py b/sphinxdocs/tests/sphinx_docs/sphinx_docs_output_test.py new file mode 100644 index 0000000000..5d00817926 --- /dev/null +++ b/sphinxdocs/tests/sphinx_docs/sphinx_docs_output_test.py @@ -0,0 +1,28 @@ +import importlib.resources +import os +from xml.etree import ElementTree + +import tests.sphinx_docs as sphinx_docs +from absl.testing import absltest + + +class SphinxDocsOutputTest(absltest.TestCase): + def test_directory_artifact_relative_xref(self): + page1_path = importlib.resources.files(sphinx_docs).joinpath( + "docs/_build/html/generated_directory/dir_page1.html" + ) + self.assertTrue(os.path.exists(str(page1_path)), f"Not found at {page1_path}") + with open(str(page1_path)) as f: + xml = f.read() + doc_elem = ElementTree.fromstring(xml) + actual = None + for elem in doc_elem.iter(): + if "href" in elem.attrib: + if "".join(elem.itertext()).strip() == "Dir Page 2": + actual = elem.attrib["href"] + break + self.assertEqual("dir_page2.html", actual) + + +if __name__ == "__main__": + absltest.main() diff --git a/sphinxdocs/tests/sphinx_docs_conf_in_other_dir/BUILD.bazel b/sphinxdocs/tests/sphinx_docs_conf_in_other_dir/BUILD.bazel new file mode 100644 index 0000000000..eecbb90897 --- /dev/null +++ b/sphinxdocs/tests/sphinx_docs_conf_in_other_dir/BUILD.bazel @@ -0,0 +1,38 @@ +load("@bazel_skylib//rules:build_test.bzl", "build_test") +load("//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs") + +_TARGET_COMPATIBLE_WITH = select({ + "@platforms//os:linux": [], + "@platforms//os:macos": [], + "//conditions:default": ["@platforms//:incompatible"], +}) + +genrule( + name = "gen_conf", + srcs = ["src_conf.py"], + outs = ["other_dir/conf.py"], + cmd = "cp $(location src_conf.py) $@", +) + +sphinx_docs( + name = "docs", + srcs = ["index.md"], + config = ":gen_conf", + formats = ["html"], + sphinx = ":sphinx-build", + target_compatible_with = _TARGET_COMPATIBLE_WITH, +) + +sphinx_build_binary( + name = "sphinx-build", + tags = ["manual"], + deps = [ + "@dev_pip//myst_parser", + "@dev_pip//sphinx", + ], +) + +build_test( + name = "docs_build_test", + targets = [":docs"], +) diff --git a/sphinxdocs/tests/sphinx_docs_conf_in_other_dir/index.md b/sphinxdocs/tests/sphinx_docs_conf_in_other_dir/index.md new file mode 100644 index 0000000000..cd7d8b4188 --- /dev/null +++ b/sphinxdocs/tests/sphinx_docs_conf_in_other_dir/index.md @@ -0,0 +1,3 @@ +# Test Documentation + +This is a test document for reproducing conf.py in a separate directory. diff --git a/sphinxdocs/tests/sphinx_docs_conf_in_other_dir/src_conf.py b/sphinxdocs/tests/sphinx_docs_conf_in_other_dir/src_conf.py new file mode 100644 index 0000000000..d4f2f45d31 --- /dev/null +++ b/sphinxdocs/tests/sphinx_docs_conf_in_other_dir/src_conf.py @@ -0,0 +1,4 @@ +project = "Repro Test" +copyright = "2026, Test" +author = "Test" +extensions = ["myst_parser"] diff --git a/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel b/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel index 60a5e8d766..2cbc773f77 100644 --- a/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel +++ b/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel @@ -1,6 +1,6 @@ load("@bazel_skylib//:bzl_library.bzl", "bzl_library") -load("//python:py_test.bzl", "py_test") -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility +load("@bazel_skylib//rules:build_test.bzl", "build_test") +load("@rules_python//python:py_test.bzl", "py_test") load("//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs") load("//sphinxdocs:sphinx_stardoc.bzl", "sphinx_stardoc", "sphinx_stardocs") @@ -14,7 +14,7 @@ _TARGET_COMPATIBLE_WITH = select({ "@platforms//os:linux": [], "@platforms//os:macos": [], "//conditions:default": ["@platforms//:incompatible"], -}) if IS_BAZEL_7_OR_HIGHER else ["@platforms//:incompatible"] +}) sphinx_docs( name = "docs", @@ -40,6 +40,11 @@ sphinx_docs( ], ) +build_test( + name = "docs_build_test", + targets = [":docs"], +) + sphinx_stardocs( name = "simple_bzl_docs", srcs = [ @@ -64,7 +69,6 @@ sphinx_stardoc( deps = [":func_and_providers_bzl"], ) -# A bzl_library with multiple sources bzl_library( name = "func_and_providers_bzl", srcs = [ @@ -84,6 +88,7 @@ bzl_library( srcs = ["bzl_typedef.bzl"], ) +# A bzl_library with multiple sources sphinx_build_binary( name = "sphinx-build", tags = ["manual"], # Only needed as part of sphinx doc building diff --git a/sphinxdocs/tests/sphinx_stardoc/__init__.py b/sphinxdocs/tests/sphinx_stardoc/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/sphinxdocs/tests/sphinx_stardoc/index.md b/sphinxdocs/tests/sphinx_stardoc/index.md index 43ef14f55a..51a7a671ec 100644 --- a/sphinxdocs/tests/sphinx_stardoc/index.md +++ b/sphinxdocs/tests/sphinx_stardoc/index.md @@ -5,7 +5,7 @@ This is a set of documents to test the sphinx_stardoc extension. To build and view these docs, run: ``` -bazel run //sphinxdocs/tests/sphinx_stardoc:docs.serve +bazel run //tests/sphinx_stardoc:docs.serve ``` This will build the docs and start an HTTP server where they can be viewed. @@ -14,7 +14,7 @@ To aid the edit/debug cycle, `ibazel` can be used to automatically rebuild the HTML: ``` -ibazel build //sphinxdocs/tests/sphinx_stardoc:docs +ibazel build //tests/sphinx_stardoc:docs ``` :::{toctree} diff --git a/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py b/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py index 6d65c920e1..e6653b1656 100644 --- a/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py +++ b/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py @@ -3,7 +3,7 @@ from absl.testing import absltest, parameterized -from sphinxdocs.tests import sphinx_stardoc +import tests.sphinx_stardoc as sphinx_stardoc class SphinxOutputTest(parameterized.TestCase): @@ -13,7 +13,10 @@ def setUp(self): self._xmls = {} def assert_xref(self, doc, *, text, href): - match = self._doc_element(doc).find(f".//*[.='{text}']") + # Find an element with an 'href' attribute whose string content (including + # descendants like ) equals `text`. [@href] filters out ancestor + # elements like
  • or

    which also match [.='{text}']. + match = self._doc_element(doc).find(f".//*[@href][.='{text}']") if not match: self.fail(f"No element found with {text=}") actual = match.attrib.get("href", "") @@ -50,19 +53,70 @@ def _doc_element(self, doc): ("short_provider", "LangInfo", "provider.html#LangInfo"), ("short_tag_class", "myext.mytag", "module_extension.html#myext.mytag"), ("full_norepo_func", "//lang:function.bzl%myfunc", "function.html#myfunc"), - ("full_norepo_func_arg", "//lang:function.bzl%myfunc.arg1", "function.html#myfunc.arg1"), + ( + "full_norepo_func_arg", + "//lang:function.bzl%myfunc.arg1", + "function.html#myfunc.arg1", + ), ("full_norepo_rule", "//lang:rule.bzl%my_rule", "rule.html#my_rule"), - ("full_norepo_rule_attr", "//lang:rule.bzl%my_rule.ra1", "rule.html#my_rule.ra1"), - ("full_norepo_provider", "//lang:provider.bzl%LangInfo", "provider.html#LangInfo"), + ( + "full_norepo_rule_attr", + "//lang:rule.bzl%my_rule.ra1", + "rule.html#my_rule.ra1", + ), + ( + "full_norepo_provider", + "//lang:provider.bzl%LangInfo", + "provider.html#LangInfo", + ), ("full_norepo_aspect", "//lang:aspect.bzl%myaspect", "aspect.html#myaspect"), ("full_norepo_target", "//lang:relativetarget", "target.html#relativetarget"), - ("full_repo_func", "@testrepo//lang:function.bzl%myfunc", "function.html#myfunc"), - ("full_repo_func_arg", "@testrepo//lang:function.bzl%myfunc.arg1", "function.html#myfunc.arg1"), + ( + "full_repo_func", + "@testrepo//lang:function.bzl%myfunc", + "function.html#myfunc", + ), + ( + "full_repo_func_arg", + "@testrepo//lang:function.bzl%myfunc.arg1", + "function.html#myfunc.arg1", + ), ("full_repo_rule", "@testrepo//lang:rule.bzl%my_rule", "rule.html#my_rule"), - ("full_repo_rule_attr", "@testrepo//lang:rule.bzl%my_rule.ra1", "rule.html#my_rule.ra1"), - ("full_repo_provider", "@testrepo//lang:provider.bzl%LangInfo", "provider.html#LangInfo"), - ("full_repo_aspect", "@testrepo//lang:aspect.bzl%myaspect", "aspect.html#myaspect"), - ("full_repo_target", "@testrepo//lang:relativetarget", "target.html#relativetarget"), + ( + "full_repo_rule_attr", + "@testrepo//lang:rule.bzl%my_rule.ra1", + "rule.html#my_rule.ra1", + ), + ( + "full_repo_provider", + "@testrepo//lang:provider.bzl%LangInfo", + "provider.html#LangInfo", + ), + ( + "full_repo_aspect", + "@testrepo//lang:aspect.bzl%myaspect", + "aspect.html#myaspect", + ), + ( + "full_repo_target", + "@testrepo//lang:relativetarget", + "target.html#relativetarget", + ), + ( + "tag_class_attr_using_attr_role", + "myext.mytag.ta1", + "module_extension.html#myext.mytag.ta1", + ), + ( + "tag_class_attr_using_attr_role_just_attr_name", + "ta1", + "module_extension.html#myext.mytag.ta1", + ), + ("file_without_repo", "//lang:rule.bzl", "rule.html"), + ("file_with_repo", "@testrepo//lang:rule.bzl", "rule.html"), + ("package_absolute", "//lang", "target.html"), + ("package_basename", "lang", "target.html"), + ("relative_doc_link", "Rule documentation", "rule.html"), # fmt: on ) def test_xrefs(self, text, href): diff --git a/sphinxdocs/tests/sphinx_stardoc/xrefs.md b/sphinxdocs/tests/sphinx_stardoc/xrefs.md index 83f6869a48..e3a7d7b21e 100644 --- a/sphinxdocs/tests/sphinx_stardoc/xrefs.md +++ b/sphinxdocs/tests/sphinx_stardoc/xrefs.md @@ -7,6 +7,7 @@ Various tests of cross referencing support ## Short name +* [Rule documentation](rule.md) * function: {obj}`myfunc` * function arg: {obj}`myfunc.arg1` * rule: {obj}`my_rule` @@ -36,8 +37,23 @@ Various tests of cross referencing support ## Using origin keys -* provider using `{type}`: {type}`"@rules_python//sphinxdocs/tests/sphinx_stardoc:bzl_rule.bzl%GenericInfo"` +* provider using `{type}`: {type}`"//tests/sphinx_stardoc:bzl_rule.bzl%GenericInfo"` ## Any xref * {any}`LangInfo` + +## Tag class refs + +* tag class attribute using attr role: {attr}`myext.mytag.ta1` +* tag class attribute, just attr name, attr role: {attr}`ta1` + +## File refs + +* without repo {obj}`//lang:rule.bzl` +* with repo {obj}`@testrepo//lang:rule.bzl` + +## Package refs + +* absolute label {obj}`//lang` +* package basename {obj}`lang` diff --git a/tests/BUILD.bazel b/tests/BUILD.bazel index 0fb8e88135..927fc30566 100644 --- a/tests/BUILD.bazel +++ b/tests/BUILD.bazel @@ -1,6 +1,4 @@ load("@bazel_skylib//rules:build_test.bzl", "build_test") -load("@rules_shell//shell:sh_test.bzl", "sh_test") -load("//:version.bzl", "BAZEL_VERSION") package(default_visibility = ["//visibility:public"]) @@ -24,32 +22,7 @@ build_test( "//python:py_test_bzl", "//python/cc:py_cc_toolchain_bzl", "//python/cc:py_cc_toolchain_info_bzl", + "//python/cc:py_extension", "//python/entry_points:py_console_script_binary_bzl", ], ) - -genrule( - name = "assert_bazelversion", - srcs = ["//:.bazelversion"], - outs = ["assert_bazelversion_test.sh"], - cmd = """\ -set -o errexit -o nounset -o pipefail -current=$$(cat "$(execpath //:.bazelversion)") -cat > "$@" <&2 echo "ERROR: current bazel version '$${{current}}' is not the expected '{expected}'" - exit 1 -fi -EOF -""".format( - expected = BAZEL_VERSION, - ), - executable = True, -) - -sh_test( - name = "assert_bazelversion_test", - srcs = [":assert_bazelversion_test.sh"], -) diff --git a/tests/api/py_common/py_common_tests.bzl b/tests/api/py_common/py_common_tests.bzl index 572028b2a6..028da6cc37 100644 --- a/tests/api/py_common/py_common_tests.bzl +++ b/tests/api/py_common/py_common_tests.bzl @@ -13,7 +13,6 @@ # limitations under the License. """py_common tests.""" -load("@rules_python_internal//:rules_python_config.bzl", "config") load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:util.bzl", rt_util = "util") @@ -41,13 +40,11 @@ def _test_merge_py_infos_impl(env, target): py_common = _py_common.get(env.ctx) py1 = py_common.PyInfoBuilder() - if config.enable_pystar: - py1.direct_pyc_files.add(f1_pyc) + py1.direct_pyc_files.add(f1_pyc) py1.transitive_sources.add(f1_py) py2 = py_common.PyInfoBuilder() - if config.enable_pystar: - py1.direct_pyc_files.add(f2_pyc) + py1.direct_pyc_files.add(f2_pyc) py2.transitive_sources.add(f2_py) actual = py_info_subject( @@ -56,8 +53,7 @@ def _test_merge_py_infos_impl(env, target): ) actual.transitive_sources().contains_exactly([f1_py.path, f2_py.path]) - if config.enable_pystar: - actual.direct_pyc_files().contains_exactly([f1_pyc.path, f2_pyc.path]) + actual.direct_pyc_files().contains_exactly([f1_pyc.path, f2_pyc.path]) _tests.append(_test_merge_py_infos) diff --git a/tests/base_rules/base_tests.bzl b/tests/base_rules/base_tests.bzl index 3518e6f57a..a9fadd7564 100644 --- a/tests/base_rules/base_tests.bzl +++ b/tests/base_rules/base_tests.bzl @@ -16,7 +16,8 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:truth.bzl", "matching") load("@rules_testing//lib:util.bzl", "PREVENT_IMPLICIT_BUILDING_TAGS", rt_util = "util") -load("//python:defs.bzl", "PyInfo") +load("//python:py_info.bzl", "PyInfo") +load("//python:py_library.bzl", "py_library") load("//python/private:reexports.bzl", "BuiltinPyInfo") # buildifier: disable=bzl-visibility load("//tests/base_rules:util.bzl", pt_util = "util") load("//tests/support:py_info_subject.bzl", "py_info_subject") @@ -58,6 +59,50 @@ _not_produces_py_info = rule( implementation = _not_produces_py_info_impl, ) +def _test_py_info_populated(name, config): + rt_util.helper_target( + config.base_test_rule, + name = name + "_subject", + srcs = [name + "_subject.py"], + pyi_srcs = ["subject.pyi"], + pyi_deps = [name + "_lib2"], + ) + rt_util.helper_target( + py_library, + name = name + "_lib2", + srcs = ["lib2.py"], + pyi_srcs = ["lib2.pyi"], + ) + + analysis_test( + name = name, + target = name + "_subject", + impl = _test_py_info_populated_impl, + ) + +def _test_py_info_populated_impl(env, target): + info = env.expect.that_target(target).provider( + PyInfo, + factory = py_info_subject, + ) + info.direct_original_sources().contains_exactly([ + "{package}/test_py_info_populated_subject.py", + ]) + info.transitive_original_sources().contains_exactly([ + "{package}/test_py_info_populated_subject.py", + "{package}/lib2.py", + ]) + + info.direct_pyi_files().contains_exactly([ + "{package}/subject.pyi", + ]) + info.transitive_pyi_files().contains_exactly([ + "{package}/lib2.pyi", + "{package}/subject.pyi", + ]) + +_tests.append(_test_py_info_populated) + def _py_info_propagation_setup(name, config, produce_py_info_rule, test_impl): rt_util.helper_target( config.base_test_rule, diff --git a/tests/base_rules/precompile/precompile_tests.bzl b/tests/base_rules/precompile/precompile_tests.bzl index 895f2d3156..bff994aa1a 100644 --- a/tests/base_rules/precompile/precompile_tests.bzl +++ b/tests/base_rules/precompile/precompile_tests.bzl @@ -14,7 +14,6 @@ """Tests for precompiling behavior.""" -load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:truth.bzl", "matching") @@ -23,13 +22,11 @@ load("//python:py_binary.bzl", "py_binary") load("//python:py_info.bzl", "PyInfo") load("//python:py_library.bzl", "py_library") load("//python:py_test.bzl", "py_test") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//tests/support:py_info_subject.bzl", "py_info_subject") load( "//tests/support:support.bzl", - "ADD_SRCS_TO_RUNFILES", "CC_TOOLCHAIN", - "EXEC_TOOLS_TOOLCHAIN", - "PRECOMPILE", "PY_TOOLCHAINS", ) @@ -38,15 +35,12 @@ _COMMON_CONFIG_SETTINGS = { # it for conformity. "//command_line_option:allow_unresolved_symlinks": True, "//command_line_option:extra_toolchains": [PY_TOOLCHAINS, CC_TOOLCHAIN], - EXEC_TOOLS_TOOLCHAIN: "enabled", + labels.EXEC_TOOLS_TOOLCHAIN: "enabled", } _tests = [] def _test_executable_precompile_attr_enabled_setup(name, py_rule, **kwargs): - if not rp_config.enable_pystar: - rt_util.skip_test(name = name) - return rt_util.helper_target( py_rule, name = name + "_subject", @@ -114,9 +108,6 @@ def _test_precompile_enabled_py_test(name): _tests.append(_test_precompile_enabled_py_test) def _test_precompile_enabled_py_library_setup(name, impl, config_settings): - if not rp_config.enable_pystar: - rt_util.skip_test(name = name) - return rt_util.helper_target( py_library, name = name + "_subject", @@ -150,7 +141,7 @@ def _test_precompile_enabled_py_library_add_to_runfiles_disabled(name): name = name, impl = _test_precompile_enabled_py_library_add_to_runfiles_disabled_impl, config_settings = { - ADD_SRCS_TO_RUNFILES: "disabled", + labels.ADD_SRCS_TO_RUNFILES: "disabled", }, ) @@ -166,7 +157,7 @@ def _test_precompile_enabled_py_library_add_to_runfiles_enabled(name): name = name, impl = _test_precompile_enabled_py_library_add_to_runfiles_enabled_impl, config_settings = { - ADD_SRCS_TO_RUNFILES: "enabled", + labels.ADD_SRCS_TO_RUNFILES: "enabled", }, ) @@ -180,9 +171,6 @@ def _test_precompile_enabled_py_library_add_to_runfiles_enabled_impl(env, target _tests.append(_test_precompile_enabled_py_library_add_to_runfiles_enabled) def _test_pyc_only(name): - if not rp_config.enable_pystar: - rt_util.skip_test(name = name) - return rt_util.helper_target( py_binary, name = name + "_subject", @@ -203,7 +191,7 @@ def _test_pyc_only(name): name = name, impl = _test_pyc_only_impl, config_settings = _COMMON_CONFIG_SETTINGS | { - PRECOMPILE: "enabled", + labels.PRECOMPILE: "enabled", }, target = name + "_subject", ) @@ -233,9 +221,6 @@ def _test_pyc_only_impl(env, target): ) def _test_precompiler_action(name): - if not rp_config.enable_pystar: - rt_util.skip_test(name = name) - return rt_util.helper_target( py_binary, name = name + "_subject", @@ -310,7 +295,7 @@ def _setup_precompile_flag_pyc_collection_attr_interaction( impl = test_impl, target = name + "_bin", config_settings = _COMMON_CONFIG_SETTINGS | { - PRECOMPILE: precompile_flag, + labels.PRECOMPILE: precompile_flag, }, ) @@ -327,9 +312,6 @@ def _verify_runfiles(contains_patterns, not_contains_patterns): return _verify_runfiles_impl def _test_precompile_flag_enabled_pyc_collection_attr_include_pyc(name): - if not rp_config.enable_pystar: - rt_util.skip_test(name = name) - return _setup_precompile_flag_pyc_collection_attr_interaction( name = name, precompile_flag = "enabled", @@ -353,9 +335,6 @@ def _test_precompile_flag_enabled_pyc_collection_attr_disabled(name): """Verify that a binary can opt-out of using implicit pycs even when precompiling is enabled by default. """ - if not rp_config.enable_pystar: - rt_util.skip_test(name = name) - return _setup_precompile_flag_pyc_collection_attr_interaction( name = name, precompile_flag = "enabled", @@ -378,9 +357,6 @@ _tests.append(_test_precompile_flag_enabled_pyc_collection_attr_disabled) def _test_precompile_flag_disabled_pyc_collection_attr_include_pyc(name): """Verify that a binary can opt-in to using pycs even when precompiling is disabled by default.""" - if not rp_config.enable_pystar: - rt_util.skip_test(name = name) - return _setup_precompile_flag_pyc_collection_attr_interaction( name = name, precompile_flag = "disabled", @@ -400,9 +376,6 @@ def _test_precompile_flag_disabled_pyc_collection_attr_include_pyc(name): _tests.append(_test_precompile_flag_disabled_pyc_collection_attr_include_pyc) def _test_precompile_flag_disabled_pyc_collection_attr_disabled(name): - if not rp_config.enable_pystar: - rt_util.skip_test(name = name) - return _setup_precompile_flag_pyc_collection_attr_interaction( name = name, precompile_flag = "disabled", @@ -426,9 +399,6 @@ def _test_pyc_collection_disabled_library_omit_source(name): """Verify that, when a binary doesn't include implicit pyc files, libraries that set omit_source still have the py source file included. """ - if not rp_config.enable_pystar: - rt_util.skip_test(name = name) - return rt_util.helper_target( py_binary, name = name + "_subject", @@ -471,9 +441,6 @@ def _test_pyc_collection_disabled_library_omit_source_impl(env, target): _tests.append(_test_pyc_collection_disabled_library_omit_source) def _test_pyc_collection_include_dep_omit_source(name): - if not rp_config.enable_pystar: - rt_util.skip_test(name = name) - return rt_util.helper_target( py_binary, name = name + "_subject", @@ -515,9 +482,6 @@ def _test_pyc_collection_include_dep_omit_source_impl(env, target): _tests.append(_test_pyc_collection_include_dep_omit_source) def _test_precompile_attr_inherit_pyc_collection_disabled_precompile_flag_enabled(name): - if not rp_config.enable_pystar: - rt_util.skip_test(name = name) - return rt_util.helper_target( py_binary, name = name + "_subject", @@ -531,7 +495,7 @@ def _test_precompile_attr_inherit_pyc_collection_disabled_precompile_flag_enable impl = _test_precompile_attr_inherit_pyc_collection_disabled_precompile_flag_enabled_impl, target = name + "_subject", config_settings = _COMMON_CONFIG_SETTINGS | { - PRECOMPILE: "enabled", + labels.PRECOMPILE: "enabled", }, ) diff --git a/tests/base_rules/py_binary/py_binary_tests.bzl b/tests/base_rules/py_binary/py_binary_tests.bzl index 571955d3c6..86a9548f79 100644 --- a/tests/base_rules/py_binary/py_binary_tests.bzl +++ b/tests/base_rules/py_binary/py_binary_tests.bzl @@ -13,7 +13,7 @@ # limitations under the License. """Tests for py_binary.""" -load("//python:defs.bzl", "py_binary") +load("//python:py_binary.bzl", "py_binary") load( "//tests/base_rules:py_executable_base_tests.bzl", "create_executable_tests", diff --git a/tests/base_rules/py_executable_base_tests.bzl b/tests/base_rules/py_executable_base_tests.bzl index 3cc6dfb702..f6b3c9bb60 100644 --- a/tests/base_rules/py_executable_base_tests.bzl +++ b/tests/base_rules/py_executable_base_tests.bzl @@ -19,20 +19,20 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:truth.bzl", "matching") load("@rules_testing//lib:util.bzl", rt_util = "util") load("//python:py_executable_info.bzl", "PyExecutableInfo") +load("//python:py_info.bzl", "PyInfo") +load("//python:py_library.bzl", "py_library") +load("//python/private:common.bzl", "maybe_builtin_build_python_zip") # buildifier: disable=bzl-visibility +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:reexports.bzl", "BuiltinPyRuntimeInfo") # buildifier: disable=bzl-visibility -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility load("//tests/base_rules:base_tests.bzl", "create_base_tests") load("//tests/base_rules:util.bzl", "WINDOWS_ATTR", pt_util = "util") load("//tests/support:py_executable_info_subject.bzl", "PyExecutableInfoSubject") -load("//tests/support:support.bzl", "CC_TOOLCHAIN", "CROSSTOOL_TOP", "LINUX_X86_64", "WINDOWS_X86_64") +load("//tests/support:support.bzl", "CC_TOOLCHAIN", "CROSSTOOL_TOP") +load("//tests/support/platforms:platforms.bzl", "platform_targets") _tests = [] def _test_basic_windows(name, config): - if rp_config.enable_pystar: - target_compatible_with = [] - else: - target_compatible_with = ["@platforms//:incompatible"] rt_util.helper_target( config.rule, name = name + "_subject", @@ -44,24 +44,20 @@ def _test_basic_windows(name, config): impl = _test_basic_windows_impl, target = name + "_subject", config_settings = { - # NOTE: The default for this flag is based on the Bazel host OS, not - # the target platform. For windows, it defaults to true, so force - # it to that to match behavior when this test runs on other - # platforms. - "//command_line_option:build_python_zip": "true", "//command_line_option:cpu": "windows_x86_64", "//command_line_option:crosstool_top": CROSSTOOL_TOP, + "//command_line_option:extra_execution_platforms": [platform_targets.WINDOWS_X86_64], "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], - "//command_line_option:platforms": [WINDOWS_X86_64], + "//command_line_option:platforms": [platform_targets.WINDOWS_X86_64], }, - attr_values = {"target_compatible_with": target_compatible_with}, + attr_values = {}, ) def _test_basic_windows_impl(env, target): target = env.expect.that_target(target) target.executable().path().contains(".exe") target.runfiles().contains_predicate(matching.str_endswith( - target.meta.format_str("/{name}.zip"), + target.meta.format_str("/{name}"), )) target.runfiles().contains_predicate(matching.str_endswith( target.meta.format_str("/{name}.exe"), @@ -70,14 +66,11 @@ def _test_basic_windows_impl(env, target): _tests.append(_test_basic_windows) def _test_basic_zip(name, config): - if rp_config.enable_pystar: - target_compatible_with = select({ - # Disable the new test on windows because we have _test_basic_windows. - "@platforms//os:windows": ["@platforms//:incompatible"], - "//conditions:default": [], - }) - else: - target_compatible_with = ["@platforms//:incompatible"] + target_compatible_with = select({ + # Disable the new test on windows because we have _test_basic_windows. + "@platforms//os:windows": ["@platforms//:incompatible"], + "//conditions:default": [], + }) rt_util.helper_target( config.rule, name = name + "_subject", @@ -93,12 +86,15 @@ def _test_basic_zip(name, config): # the target platform. For windows, it defaults to true, so force # it to that to match behavior when this test runs on other # platforms. - "//command_line_option:build_python_zip": "true", + # Pass value to both native and starlark versions of the flag until + # the native one is removed. + labels.BUILD_PYTHON_ZIP: True, "//command_line_option:cpu": "linux_x86_64", "//command_line_option:crosstool_top": CROSSTOOL_TOP, + "//command_line_option:extra_execution_platforms": [platform_targets.LINUX_X86_64], "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], - "//command_line_option:platforms": [LINUX_X86_64], - }, + "//command_line_option:platforms": [platform_targets.LINUX_X86_64], + } | maybe_builtin_build_python_zip("true"), attr_values = {"target_compatible_with": target_compatible_with}, ) @@ -113,6 +109,52 @@ def _test_basic_zip_impl(env, target): _tests.append(_test_basic_zip) +def _test_config_settings_extra_toolchains(name, config): + rt_util.helper_target( + config.rule, + name = name + "_subject", + srcs = ["main.py"], + main = "main.py", + config_settings = { + "//command_line_option:extra_toolchains": "{tc},{tc}".format(tc = CC_TOOLCHAIN), + }, + ) + analysis_test( + name = name, + impl = _test_config_settings_extra_toolchains_impl, + target = name + "_subject", + ) + +def _test_config_settings_extra_toolchains_impl(env, target): + # If we got here, it means analysis succeeded, which implies the transition + # successfully parsed the CSV string into a list. + env.expect.that_target(target).has_provider(PyInfo) + +_tests.append(_test_config_settings_extra_toolchains) + +def _test_cross_compile_to_unix(name, config): + rt_util.helper_target( + config.rule, + name = name + "_subject", + main_module = "dummy", + ) + analysis_test( + name = name, + impl = _test_cross_compile_to_unix_impl, + target = name + "_subject", + # Cross-compilation of py_test fails since the default test toolchain + # requires an execution platform that matches the target platform. + config_settings = { + "//command_line_option:platforms": [platform_targets.EXOTIC_UNIX], + } if rp_config.bazel_9_or_later and not "py_test" in str(config.rule) else {}, + expect_failure = True, + ) + +def _test_cross_compile_to_unix_impl(_env, _target): + pass + +_tests.append(_test_cross_compile_to_unix) + def _test_executable_in_runfiles(name, config): rt_util.helper_target( config.rule, @@ -137,14 +179,99 @@ def _test_executable_in_runfiles_impl(env, target): "{workspace}/{package}/{test_name}_subject" + exe, ]) - if rp_config.enable_pystar: - py_exec_info = env.expect.that_target(target).provider(PyExecutableInfo, factory = PyExecutableInfoSubject.new) - py_exec_info.main().path().contains("_subject.py") - py_exec_info.interpreter_path().contains("python") - py_exec_info.runfiles_without_exe().contains_none_of([ - "{workspace}/{package}/{test_name}_subject" + exe, - "{workspace}/{package}/{test_name}_subject", - ]) + py_exec_info = env.expect.that_target(target).provider(PyExecutableInfo, factory = PyExecutableInfoSubject.new) + py_exec_info.main().path().contains("_subject.py") + py_exec_info.interpreter_path().contains("python") + py_exec_info.runfiles_without_exe().contains_none_of([ + "{workspace}/{package}/{test_name}_subject" + exe, + "{workspace}/{package}/{test_name}_subject", + ]) + +def _test_debugger(name, config): + # Using imports + rt_util.helper_target( + py_library, + name = name + "_debugger", + imports = ["."], + srcs = [rt_util.empty_file(name + "_debugger.py")], + ) + + rt_util.helper_target( + config.rule, + name = name + "_subject", + srcs = [rt_util.empty_file(name + "_subject.py")], + config_settings = { + # config_settings requires a fully qualified label + labels.DEBUGGER: "//{}:{}_debugger".format(native.package_name(), name), + }, + ) + + # Using venv + rt_util.helper_target( + py_library, + name = name + "_debugger_venv", + imports = ["site-packages"], + experimental_venvs_site_packages = "@rules_python//python/config_settings:venvs_site_packages", + srcs = [rt_util.empty_file("site-packages/" + name + "_debugger_venv.py")], + ) + + rt_util.helper_target( + config.rule, + name = name + "_subject_venv", + srcs = [rt_util.empty_file(name + "_subject_venv.py")], + config_settings = { + # config_settings requires a fully qualified label + labels.DEBUGGER: "//{}:{}_debugger_venv".format(native.package_name(), name), + }, + ) + + analysis_test( + name = name, + impl = _test_debugger_impl, + targets = { + "exec_target": name + "_subject", + "target": name + "_subject", + "target_venv": name + "_subject_venv", + }, + attrs = { + "exec_target": attr.label(cfg = "exec"), + }, + config_settings = { + labels.VENVS_SITE_PACKAGES: "yes", + labels.PYTHON_VERSION: "3.13", + }, + ) + +_tests.append(_test_debugger) + +def _test_debugger_impl(env, targets): + # 1. Subject + + # Check the file from debugger dep is injected. + env.expect.that_target(targets.target).runfiles().contains_at_least([ + "{workspace}/{package}/{test_name}_debugger.py", + ]) + + # #3481: Ensure imports are setup correcty. + meta = env.expect.meta.derive(format_str_kwargs = {"package": targets.target.label.package}) + env.expect.that_target(targets.target).has_provider(PyInfo) + imports = targets.target[PyInfo].imports.to_list() + env.expect.that_collection(imports).contains(meta.format_str("{workspace}/{package}")) + + # 2. Subject venv + + # #3481: Ensure that venv site-packages is setup correctly, if the dependency is coming + # from pip integration. + env.expect.that_target(targets.target_venv).runfiles().contains_predicate( + matching.str_endswith("site-packages/test_debugger_debugger_venv.py"), + ) + + # 3. Subject exec + + # Ensure that tools don't inherit debugger. + env.expect.that_target(targets.exec_target).runfiles().not_contains( + "{workspace}/{package}/{test_name}_debugger.py", + ) def _test_default_main_can_be_generated(name, config): rt_util.helper_target( @@ -185,11 +312,6 @@ def _test_default_main_can_have_multiple_path_segments_impl(env, target): ) def _test_default_main_must_be_in_srcs(name, config): - # Bazel 5 will crash with a Java stacktrace when the native Python - # rules have an error. - if not pt_util.is_bazel_6_or_higher(): - rt_util.skip_test(name = name) - return rt_util.helper_target( config.rule, name = name + "_subject", @@ -210,11 +332,6 @@ def _test_default_main_must_be_in_srcs_impl(env, target): ) def _test_default_main_cannot_be_ambiguous(name, config): - # Bazel 5 will crash with a Java stacktrace when the native Python - # rules have an error. - if not pt_util.is_bazel_6_or_higher(): - rt_util.skip_test(name = name) - return rt_util.helper_target( config.rule, name = name + "_subject", @@ -257,11 +374,6 @@ def _test_explicit_main_impl(env, target): ) def _test_explicit_main_cannot_be_ambiguous(name, config): - # Bazel 5 will crash with a Java stacktrace when the native Python - # rules have an error. - if not pt_util.is_bazel_6_or_higher(): - rt_util.skip_test(name = name) - return rt_util.helper_target( config.rule, name = name + "_subject", @@ -282,7 +394,7 @@ def _test_explicit_main_cannot_be_ambiguous_impl(env, target): matching.str_matches("foo.py*matches multiple"), ) -def _test_files_to_build(name, config): +def _test_default_outputs(name, config): rt_util.helper_target( config.rule, name = name + "_subject", @@ -290,14 +402,14 @@ def _test_files_to_build(name, config): ) analysis_test( name = name, - impl = _test_files_to_build_impl, + impl = _test_default_outputs_impl, target = name + "_subject", attrs = WINDOWS_ATTR, ) -_tests.append(_test_files_to_build) +_tests.append(_test_default_outputs) -def _test_files_to_build_impl(env, target): +def _test_default_outputs_impl(env, target): default_outputs = env.expect.that_target(target).default_outputs() if pt_util.is_windows(env): default_outputs.contains("{package}/{test_name}_subject.exe") @@ -307,22 +419,16 @@ def _test_files_to_build_impl(env, target): "{package}/{test_name}_subject.py", ]) - if IS_BAZEL_7_OR_HIGHER: - # As of Bazel 7, the first default output is the executable, so - # verify that is the case. rules_testing - # DepsetFileSubject.contains_exactly doesn't provide an in_order() - # call, nor access to the underlying depset, so we have to do things - # manually. - first_default_output = target[DefaultInfo].files.to_list()[0] - executable = target[DefaultInfo].files_to_run.executable - env.expect.that_file(first_default_output).equals(executable) + # As of Bazel 7, the first default output is the executable, so + # verify that is the case. rules_testing + # DepsetFileSubject.contains_exactly doesn't provide an in_order() + # call, nor access to the underlying depset, so we have to do things + # manually. + first_default_output = target[DefaultInfo].files.to_list()[0] + executable = target[DefaultInfo].files_to_run.executable + env.expect.that_file(first_default_output).equals(executable) def _test_name_cannot_end_in_py(name, config): - # Bazel 5 will crash with a Java stacktrace when the native Python - # rules have an error. - if not pt_util.is_bazel_6_or_higher(): - rt_util.skip_test(name = name) - return rt_util.helper_target( config.rule, name = name + "_subject.py", @@ -342,6 +448,54 @@ def _test_name_cannot_end_in_py_impl(env, target): matching.str_matches("name must not end in*.py"), ) +def _test_main_module_bootstrap_system_python(name, config): + rt_util.helper_target( + config.rule, + name = name + "_subject", + main_module = "dummy", + ) + analysis_test( + name = name, + impl = _test_main_module_bootstrap_system_python_impl, + target = name + "_subject", + config_settings = { + labels.BOOTSTRAP_IMPL: "system_python", + "//command_line_option:extra_execution_platforms": ["@bazel_tools//tools:host_platform", platform_targets.LINUX_X86_64], + "//command_line_option:platforms": [platform_targets.LINUX_X86_64], + }, + ) + +def _test_main_module_bootstrap_system_python_impl(env, target): + env.expect.that_target(target).default_outputs().contains( + "{package}/{test_name}_subject", + ) + +_tests.append(_test_main_module_bootstrap_system_python) + +def _test_main_module_bootstrap_script(name, config): + rt_util.helper_target( + config.rule, + name = name + "_subject", + main_module = "dummy", + ) + analysis_test( + name = name, + impl = _test_main_module_bootstrap_script_impl, + target = name + "_subject", + config_settings = { + labels.BOOTSTRAP_IMPL: "script", + "//command_line_option:extra_execution_platforms": ["@bazel_tools//tools:host_platform", platform_targets.LINUX_X86_64], + "//command_line_option:platforms": [platform_targets.LINUX_X86_64], + }, + ) + +def _test_main_module_bootstrap_script_impl(env, target): + env.expect.that_target(target).default_outputs().contains( + "{package}/{test_name}_subject", + ) + +_tests.append(_test_main_module_bootstrap_script) + def _test_py_runtime_info_provided(name, config): rt_util.helper_target( config.rule, @@ -365,28 +519,73 @@ def _test_py_runtime_info_provided_impl(env, target): _tests.append(_test_py_runtime_info_provided) -# Can't test this -- mandatory validation happens before analysis test -# can intercept it -# TODO(#1069): Once re-implemented in Starlark, modify rule logic to make this -# testable. -# def _test_srcs_is_mandatory(name, config): -# rt_util.helper_target( -# config.rule, -# name = name + "_subject", -# ) -# analysis_test( -# name = name, -# impl = _test_srcs_is_mandatory, -# target = name + "_subject", -# expect_failure = True, -# ) -# -# _tests.append(_test_srcs_is_mandatory) -# -# def _test_srcs_is_mandatory_impl(env, target): -# env.expect.that_target(target).failures().contains_predicate( -# matching.str_matches("mandatory*srcs"), -# ) +def _test_venv_output_prefix_with_path_separators(name, config): + rt_util.helper_target( + config.rule, + name = name + "/foo/tool", + srcs = ["main.py"], + main = "main.py", + ) + rt_util.helper_target( + config.rule, + name = name + "/bar/tool", + srcs = ["main.py"], + main = "main.py", + ) + rt_util.helper_target( + config.rule, + name = name + "/foo_tool", + srcs = ["main.py"], + main = "main.py", + ) + analysis_test( + name = name, + impl = _test_venv_output_prefix_with_path_separators_impl, + targets = { + "bar": name + "/bar/tool", + "foo": name + "/foo/tool", + "foo_underscore": name + "/foo_tool", + }, + ) + +def _test_venv_output_prefix_with_path_separators_impl(env, targets): + for target in [targets.foo, targets.bar, targets.foo_underscore]: + target = env.expect.that_target(target) + venv_file = "*/_{}.venv/*site-packages/bazel.pth".format( + target.meta.format_str("{name}"), + ) + target.runfiles().contains_predicate(matching.str_matches(venv_file)) + +_tests.append(_test_venv_output_prefix_with_path_separators) + +def _test_windows_target_with_path_separators(name, config): + rt_util.helper_target( + config.rule, + name = name + "/nested_subject", + srcs = ["main.py"], + main = "main.py", + ) + analysis_test( + name = name, + impl = _test_windows_target_with_path_separators_impl, + target = name + "/nested_subject", + config_settings = { + "//command_line_option:cpu": "windows_x86_64", + "//command_line_option:crosstool_top": CROSSTOOL_TOP, + "//command_line_option:extra_execution_platforms": [platform_targets.WINDOWS_X86_64], + "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], + "//command_line_option:platforms": [platform_targets.WINDOWS_X86_64], + }, + attr_values = {}, + ) + +def _test_windows_target_with_path_separators_impl(env, target): + target = env.expect.that_target(target) + target.runfiles().contains_predicate(matching.str_endswith( + target.meta.format_str("/{name}"), + )) + +_tests.append(_test_windows_target_with_path_separators) # ===== # You were gonna add a test at the end, weren't you? diff --git a/tests/base_rules/py_info/py_info_tests.bzl b/tests/base_rules/py_info/py_info_tests.bzl index 4067a59d24..0e1a96548c 100644 --- a/tests/base_rules/py_info/py_info_tests.bzl +++ b/tests/base_rules/py_info/py_info_tests.bzl @@ -13,33 +13,35 @@ # limitations under the License. """Tests for py_info.""" -load("@rules_python_internal//:rules_python_config.bzl", "config") load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:util.bzl", rt_util = "util") -load("//python:py_info.bzl", "PyInfo") +load("//python:py_info.bzl", "PyInfo", "VenvSymlinkKind") load("//python/private:py_info.bzl", "PyInfoBuilder") # buildifier: disable=bzl-visibility load("//python/private:reexports.bzl", "BuiltinPyInfo") # buildifier: disable=bzl-visibility load("//tests/support:py_info_subject.bzl", "py_info_subject") def _provide_py_info_impl(ctx): kwargs = { + "direct_original_sources": depset(ctx.files.direct_original_sources), "direct_pyc_files": depset(ctx.files.direct_pyc_files), + "direct_pyi_files": depset(ctx.files.direct_pyi_files), "imports": depset(ctx.attr.imports), + "transitive_original_sources": depset(ctx.files.transitive_original_sources), "transitive_pyc_files": depset(ctx.files.transitive_pyc_files), + "transitive_pyi_files": depset(ctx.files.transitive_pyi_files), "transitive_sources": depset(ctx.files.transitive_sources), } if ctx.attr.has_py2_only_sources != -1: kwargs["has_py2_only_sources"] = bool(ctx.attr.has_py2_only_sources) if ctx.attr.has_py3_only_sources != -1: - kwargs["has_py2_only_sources"] = bool(ctx.attr.has_py2_only_sources) + kwargs["has_py3_only_sources"] = bool(ctx.attr.has_py3_only_sources) providers = [] - if config.enable_pystar: - providers.append(PyInfo(**kwargs)) + providers.append(PyInfo(**kwargs)) # Handle Bazel 6 or if Bazel autoloading is enabled - if not config.enable_pystar or (BuiltinPyInfo and PyInfo != BuiltinPyInfo): + if BuiltinPyInfo and PyInfo != BuiltinPyInfo: providers.append(BuiltinPyInfo(**{ k: kwargs[k] for k in ( @@ -56,11 +58,15 @@ def _provide_py_info_impl(ctx): provide_py_info = rule( implementation = _provide_py_info_impl, attrs = { + "direct_original_sources": attr.label_list(allow_files = True), "direct_pyc_files": attr.label_list(allow_files = True), + "direct_pyi_files": attr.label_list(allow_files = True), "has_py2_only_sources": attr.int(default = -1), "has_py3_only_sources": attr.int(default = -1), "imports": attr.string_list(), + "transitive_original_sources": attr.label_list(allow_files = True), "transitive_pyc_files": attr.label_list(allow_files = True), + "transitive_pyi_files": attr.label_list(allow_files = True), "transitive_sources": attr.label_list(allow_files = True), }, ) @@ -87,10 +93,8 @@ def _test_py_info_create_impl(env, target): imports = depset(["import-path"]), transitive_sources = depset([trans_py]), uses_shared_libraries = True, - **(dict( - direct_pyc_files = depset([direct_pyc]), - transitive_pyc_files = depset([trans_pyc]), - ) if config.enable_pystar else {}) + direct_pyc_files = depset([direct_pyc]), + transitive_pyc_files = depset([trans_pyc]), ) subject = py_info_subject(actual, meta = env.expect.meta) @@ -99,9 +103,8 @@ def _test_py_info_create_impl(env, target): subject.has_py3_only_sources().equals(True) subject.transitive_sources().contains_exactly(["tests/base_rules/py_info/trans.py"]) subject.imports().contains_exactly(["import-path"]) - if config.enable_pystar: - subject.direct_pyc_files().contains_exactly(["tests/base_rules/py_info/direct.pyc"]) - subject.transitive_pyc_files().contains_exactly(["tests/base_rules/py_info/trans.pyc"]) + subject.direct_pyc_files().contains_exactly(["tests/base_rules/py_info/direct.pyc"]) + subject.transitive_pyc_files().contains_exactly(["tests/base_rules/py_info/trans.pyc"]) _tests.append(_test_py_info_create) @@ -109,7 +112,15 @@ def _test_py_info_builder(name): rt_util.helper_target( native.filegroup, name = name + "_misc", - srcs = ["trans.py", "direct.pyc", "trans.pyc"], + srcs = [ + "trans.py", + "direct.pyc", + "trans.pyc", + "original.py", + "trans-original.py", + "direct.pyi", + "trans.pyi", + ], ) py_info_targets = {} @@ -123,6 +134,10 @@ def _test_py_info_builder(name): direct_pyc_files = ["py{}-direct.pyc".format(n)], imports = ["py{}import".format(n)], transitive_pyc_files = ["py{}-trans.pyc".format(n)], + direct_original_sources = ["py{}-original-direct.py".format(n)], + transitive_original_sources = ["py{}-original-trans.py".format(n)], + direct_pyi_files = ["py{}-direct.pyi".format(n)], + transitive_pyi_files = ["py{}-trans.pyi".format(n)], ) analysis_test( name = name, @@ -133,16 +148,34 @@ def _test_py_info_builder(name): ) def _test_py_info_builder_impl(env, targets): - trans, direct_pyc, trans_pyc = targets.misc[DefaultInfo].files.to_list() - builder = PyInfoBuilder() + ( + trans, + direct_pyc, + trans_pyc, + original_py, + trans_original_py, + direct_pyi, + trans_pyi, + ) = targets.misc[DefaultInfo].files.to_list() + builder = PyInfoBuilder.new() builder.direct_pyc_files.add(direct_pyc) + builder.direct_original_sources.add(original_py) + builder.direct_pyi_files.add(direct_pyi) builder.merge_has_py2_only_sources(True) builder.merge_has_py3_only_sources(True) builder.imports.add("import-path") builder.transitive_pyc_files.add(trans_pyc) + builder.transitive_pyi_files.add(trans_pyi) + builder.transitive_original_sources.add(trans_original_py) builder.transitive_sources.add(trans) builder.merge_uses_shared_libraries(True) + symlink = builder.add_venv_symlink() + symlink.set_kind(VenvSymlinkKind.LIB) + symlink.set_venv_path("test_path") + symlink.set_link_to_path("test_target") + symlink.files.add(trans) + builder.merge_target(targets.py1) builder.merge_targets([targets.py2]) @@ -174,6 +207,8 @@ def _test_py_info_builder_impl(env, targets): "py5import", "py6import", ]) + + # Checks for non-Bazel builtin PyInfo if hasattr(actual, "direct_pyc_files"): subject.direct_pyc_files().contains_exactly([ "tests/base_rules/py_info/direct.pyc", @@ -189,7 +224,50 @@ def _test_py_info_builder_impl(env, targets): "tests/base_rules/py_info/py5-trans.pyc", "tests/base_rules/py_info/py6-trans.pyc", ]) + subject.direct_original_sources().contains_exactly([ + "tests/base_rules/py_info/original.py", + "tests/base_rules/py_info/py4-original-direct.py", + "tests/base_rules/py_info/py6-original-direct.py", + ]) + subject.transitive_original_sources().contains_exactly([ + "tests/base_rules/py_info/trans-original.py", + "tests/base_rules/py_info/py1-original-trans.py", + "tests/base_rules/py_info/py2-original-trans.py", + "tests/base_rules/py_info/py3-original-trans.py", + "tests/base_rules/py_info/py4-original-trans.py", + "tests/base_rules/py_info/py5-original-trans.py", + "tests/base_rules/py_info/py6-original-trans.py", + ]) + subject.direct_pyi_files().contains_exactly([ + "tests/base_rules/py_info/direct.pyi", + "tests/base_rules/py_info/py4-direct.pyi", + "tests/base_rules/py_info/py6-direct.pyi", + ]) + subject.transitive_pyi_files().contains_exactly([ + "tests/base_rules/py_info/trans.pyi", + "tests/base_rules/py_info/py1-trans.pyi", + "tests/base_rules/py_info/py2-trans.pyi", + "tests/base_rules/py_info/py3-trans.pyi", + "tests/base_rules/py_info/py4-trans.pyi", + "tests/base_rules/py_info/py5-trans.pyi", + "tests/base_rules/py_info/py6-trans.pyi", + ]) + + if hasattr(actual, "venv_symlinks"): + entries = actual.venv_symlinks.to_list() + env.expect.that_int(len(entries)).equals(1) + entry = entries[0] + env.expect.that_str(entry.venv_path).equals("test_path") + env.expect.that_str(entry.link_to_path).equals("test_target") + env.expect.that_str(entry.kind).equals(VenvSymlinkKind.LIB) + env.expect.that_collection(entry.files.to_list()).contains_exactly([trans]) + env.expect.that_bool(entry.package == None).equals(True) + env.expect.that_bool(entry.version == None).equals(True) + env.expect.that_bool(entry.link_to_file == None).equals(True) + + check(builder.build()) + # Call build() again to verify it doesn't duplicate/leak state check(builder.build()) if BuiltinPyInfo != None: check(builder.build_builtin_py_info()) diff --git a/tests/base_rules/py_library/py_library_tests.bzl b/tests/base_rules/py_library/py_library_tests.bzl index 526735af71..c375e72794 100644 --- a/tests/base_rules/py_library/py_library_tests.bzl +++ b/tests/base_rules/py_library/py_library_tests.bzl @@ -3,9 +3,12 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:truth.bzl", "matching") load("@rules_testing//lib:util.bzl", rt_util = "util") -load("//python:defs.bzl", "PyRuntimeInfo", "py_library") +load("//python:py_info.bzl", "PyInfo") +load("//python:py_library.bzl", "py_library") +load("//python:py_runtime_info.bzl", "PyRuntimeInfo") load("//tests/base_rules:base_tests.bzl", "create_base_tests") load("//tests/base_rules:util.bzl", pt_util = "util") +load("//tests/support:py_info_subject.bzl", "py_info_subject") _tests = [] @@ -26,7 +29,7 @@ def _test_py_runtime_info_not_present_impl(env, target): _tests.append(_test_py_runtime_info_not_present) -def _test_files_to_build(name, config): +def _test_default_outputs(name, config): rt_util.helper_target( config.rule, name = name + "_subject", @@ -35,15 +38,15 @@ def _test_files_to_build(name, config): analysis_test( name = name, target = name + "_subject", - impl = _test_files_to_build_impl, + impl = _test_default_outputs_impl, ) -def _test_files_to_build_impl(env, target): +def _test_default_outputs_impl(env, target): env.expect.that_target(target).default_outputs().contains_exactly([ "{package}/lib.py", ]) -_tests.append(_test_files_to_build) +_tests.append(_test_default_outputs) def _test_srcs_can_contain_rule_generating_py_and_nonpy_files(name, config): rt_util.helper_target( @@ -140,6 +143,63 @@ def _test_files_to_compile_impl(env, target): _tests.append(_test_files_to_compile) +def _test_pyi_srcs_not_propagated_via_srcs(name, config): + rt_util.helper_target( + config.rule, + name = name + "_lib", + srcs = ["lib.py"], + pyi_srcs = ["lib.pyi"], + ) + rt_util.helper_target( + config.rule, + name = name + "_subject", + srcs = [name + "_lib"], + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_pyi_srcs_not_propagated_via_srcs_impl, + ) + +def _test_pyi_srcs_not_propagated_via_srcs_impl(env, target): + info = env.expect.that_target(target).provider( + PyInfo, + factory = py_info_subject, + ) + info.transitive_pyi_files().contains_exactly([]) + +_tests.append(_test_pyi_srcs_not_propagated_via_srcs) + +def _test_pyi_srcs_propagated_via_deps(name, config): + rt_util.helper_target( + config.rule, + name = name + "_lib", + srcs = ["lib.py"], + pyi_srcs = ["lib.pyi"], + ) + rt_util.helper_target( + config.rule, + name = name + "_subject", + srcs = [name + "_lib"], + deps = [name + "_lib"], + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_pyi_srcs_propagated_via_deps_impl, + ) + +def _test_pyi_srcs_propagated_via_deps_impl(env, target): + info = env.expect.that_target(target).provider( + PyInfo, + factory = py_info_subject, + ) + info.transitive_pyi_files().contains_exactly([ + "{package}/lib.pyi", + ]) + +_tests.append(_test_pyi_srcs_propagated_via_deps) + def py_library_test_suite(name): config = struct(rule = py_library, base_test_rule = py_library) native.test_suite( diff --git a/tests/base_rules/py_test/py_test_tests.bzl b/tests/base_rules/py_test/py_test_tests.bzl index 6bd31ed3f9..e16204e9f2 100644 --- a/tests/base_rules/py_test/py_test_tests.bzl +++ b/tests/base_rules/py_test/py_test_tests.bzl @@ -15,13 +15,15 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:util.bzl", rt_util = "util") -load("//python:defs.bzl", "py_test") +load("//python:py_test.bzl", "py_test") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load( "//tests/base_rules:py_executable_base_tests.bzl", "create_executable_tests", ) load("//tests/base_rules:util.bzl", pt_util = "util") -load("//tests/support:support.bzl", "CC_TOOLCHAIN", "CROSSTOOL_TOP", "LINUX_X86_64", "MAC_X86_64") +load("//tests/support:support.bzl", "CC_TOOLCHAIN", "CROSSTOOL_TOP", "PY_TOOLCHAINS") +load("//tests/support/platforms:platforms.bzl", "platform_targets") # The Windows CI currently runs as root, which breaks when # the analysis tests try to install (but not use, because @@ -39,14 +41,6 @@ _SKIP_WINDOWS = { _tests = [] def _test_mac_requires_darwin_for_execution(name, config): - # Bazel 5.4 has a bug where every access of testing.ExecutionInfo is - # a different object that isn't equal to any other, which prevents - # rules_testing from detecting it properly and fails with an error. - # This is fixed in Bazel 6+. - if not pt_util.is_bazel_6_or_higher(): - rt_util.skip_test(name = name) - return - rt_util.helper_target( config.rule, name = name + "_subject", @@ -59,8 +53,9 @@ def _test_mac_requires_darwin_for_execution(name, config): config_settings = { "//command_line_option:cpu": "darwin_x86_64", "//command_line_option:crosstool_top": CROSSTOOL_TOP, - "//command_line_option:extra_toolchains": CC_TOOLCHAIN, - "//command_line_option:platforms": [MAC_X86_64], + "//command_line_option:extra_execution_platforms": [platform_targets.MAC_X86_64], + "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], + "//command_line_option:platforms": [platform_targets.MAC_X86_64], }, attr_values = _SKIP_WINDOWS, ) @@ -73,13 +68,6 @@ def _test_mac_requires_darwin_for_execution_impl(env, target): _tests.append(_test_mac_requires_darwin_for_execution) def _test_non_mac_doesnt_require_darwin_for_execution(name, config): - # Bazel 5.4 has a bug where every access of testing.ExecutionInfo is - # a different object that isn't equal to any other, which prevents - # rules_testing from detecting it properly and fails with an error. - # This is fixed in Bazel 6+. - if not pt_util.is_bazel_6_or_higher(): - rt_util.skip_test(name = name) - return rt_util.helper_target( config.rule, name = name + "_subject", @@ -92,8 +80,9 @@ def _test_non_mac_doesnt_require_darwin_for_execution(name, config): config_settings = { "//command_line_option:cpu": "k8", "//command_line_option:crosstool_top": CROSSTOOL_TOP, - "//command_line_option:extra_toolchains": CC_TOOLCHAIN, - "//command_line_option:platforms": [LINUX_X86_64], + "//command_line_option:extra_execution_platforms": [platform_targets.LINUX_X86_64], + "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], + "//command_line_option:platforms": [platform_targets.LINUX_X86_64], }, attr_values = _SKIP_WINDOWS, ) @@ -108,6 +97,61 @@ def _test_non_mac_doesnt_require_darwin_for_execution_impl(env, target): _tests.append(_test_non_mac_doesnt_require_darwin_for_execution) +_VALIDATE_TEST_MAIN_CONFIG_SETTINGS = { + "//command_line_option:extra_toolchains": [PY_TOOLCHAINS, CC_TOOLCHAIN], + labels.EXEC_TOOLS_TOOLCHAIN: "enabled", +} + +def _test_validate_test_main_enabled(name, config): + rt_util.helper_target( + config.rule, + name = name + "_subject", + srcs = [name + "_subject.py"], + ) + analysis_test( + name = name, + impl = _test_validate_test_main_enabled_impl, + target = name + "_subject", + config_settings = _VALIDATE_TEST_MAIN_CONFIG_SETTINGS | { + labels.VALIDATE_TEST_MAIN: "enabled", + }, + attr_values = _SKIP_WINDOWS, + ) + +def _test_validate_test_main_enabled_impl(env, target): + mnemonics = [a.mnemonic for a in target.actions] + env.expect.that_collection(mnemonics).contains("PyValidateTestMain") + env.expect.that_bool( + hasattr(target[OutputGroupInfo], "_validation"), + ).equals(True) + +_tests.append(_test_validate_test_main_enabled) + +def _test_validate_test_main_disabled(name, config): + rt_util.helper_target( + config.rule, + name = name + "_subject", + srcs = [name + "_subject.py"], + ) + analysis_test( + name = name, + impl = _test_validate_test_main_disabled_impl, + target = name + "_subject", + config_settings = _VALIDATE_TEST_MAIN_CONFIG_SETTINGS | { + labels.VALIDATE_TEST_MAIN: "disabled", + }, + attr_values = _SKIP_WINDOWS, + ) + +def _test_validate_test_main_disabled_impl(env, target): + mnemonics = [a.mnemonic for a in target.actions] + env.expect.that_collection(mnemonics).not_contains("PyValidateTestMain") + env.expect.that_bool( + hasattr(target[OutputGroupInfo], "_validation"), + ).equals(False) + +_tests.append(_test_validate_test_main_disabled) + def py_test_test_suite(name): config = struct(rule = py_test) native.test_suite( diff --git a/tests/base_rules/util.bzl b/tests/base_rules/util.bzl index a02cafa992..9fb66d7eb3 100644 --- a/tests/base_rules/util.bzl +++ b/tests/base_rules/util.bzl @@ -14,7 +14,6 @@ """Helpers and utilities multiple tests re-use.""" load("@bazel_skylib//lib:structs.bzl", "structs") -load("//python/private:util.bzl", "IS_BAZEL_6_OR_HIGHER") # buildifier: disable=bzl-visibility # Use this with is_windows() WINDOWS_ATTR = {"windows": attr.label(default = "@platforms//os:windows")} @@ -53,9 +52,6 @@ def _struct_with(s, **kwargs): struct_dict.update(kwargs) return struct(**struct_dict) -def _is_bazel_6_or_higher(): - return IS_BAZEL_6_OR_HIGHER - def _is_windows(env): """Tell if the target platform is windows. @@ -72,6 +68,5 @@ def _is_windows(env): util = struct( create_tests = _create_tests, struct_with = _struct_with, - is_bazel_6_or_higher = _is_bazel_6_or_higher, is_windows = _is_windows, ) diff --git a/tests/bootstrap_impls/BUILD.bazel b/tests/bootstrap_impls/BUILD.bazel index 2fb1f38ff0..89cd682a6a 100644 --- a/tests/bootstrap_impls/BUILD.bazel +++ b/tests/bootstrap_impls/BUILD.bazel @@ -11,46 +11,110 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. - -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility -load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test", "sh_py_run_test") +load("@rules_pkg//pkg:tar.bzl", "pkg_tar") +load("@rules_shell//shell:sh_test.bzl", "sh_test") +load("//python:py_test.bzl", "py_test") +load("//tests/support:py_reconfig.bzl", "py_reconfig_binary", "py_reconfig_test") +load("//tests/support:sh_py_run_test.bzl", "sh_py_run_test") +load("//tests/support:support.bzl", "SUPPORTS_BOOTSTRAP_SCRIPT") load(":venv_relative_path_tests.bzl", "relative_path_test_suite") -_SUPPORTS_BOOTSTRAP_SCRIPT = select({ - "@platforms//os:windows": ["@platforms//:incompatible"], - "//conditions:default": [], -}) if IS_BAZEL_7_OR_HIGHER else ["@platforms//:incompatible"] +py_reconfig_binary( + name = "bootstrap_script_zipapp_bin", + srcs = ["bin.py"], + bootstrap_impl = "script", + # Force it to not be self-executable + build_python_zip = False, + main = "bin.py", + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, +) + +filegroup( + name = "bootstrap_script_zipapp_zip", + testonly = 1, + srcs = [":bootstrap_script_zipapp_bin"], + output_group = "python_zip_file", +) + +sh_test( + name = "bootstrap_script_zipapp_test", + srcs = ["bootstrap_script_zipapp_test.sh"], + data = [":bootstrap_script_zipapp_zip"], + env = { + "ZIP_RLOCATION": "$(rlocationpaths :bootstrap_script_zipapp_zip)".format(), + }, + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, + deps = [ + "@bazel_tools//tools/bash/runfiles", + ], +) sh_py_run_test( name = "run_binary_zip_no_test", - build_python_zip = "no", + build_python_zip = False, py_src = "bin.py", sh_src = "run_binary_zip_no_test.sh", ) sh_py_run_test( name = "run_binary_zip_yes_test", - build_python_zip = "yes", + build_python_zip = True, py_src = "bin.py", sh_src = "run_binary_zip_yes_test.sh", ) +sh_py_run_test( + name = "run_binary_venvs_use_declare_symlink_no_test", + bootstrap_impl = "script", + py_src = "bin.py", + sh_src = "run_binary_venvs_use_declare_symlink_no_test.sh", + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, + venvs_use_declare_symlink = "no", +) + +sh_py_run_test( + name = "run_binary_find_runfiles_test", + py_src = "bin.py", + sh_src = "run_binary_find_runfiles_test.sh", + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, +) + sh_py_run_test( name = "run_binary_bootstrap_script_zip_yes_test", bootstrap_impl = "script", - build_python_zip = "yes", + build_python_zip = True, py_src = "bin.py", sh_src = "run_binary_zip_yes_test.sh", - target_compatible_with = _SUPPORTS_BOOTSTRAP_SCRIPT, + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, ) sh_py_run_test( name = "run_binary_bootstrap_script_zip_no_test", bootstrap_impl = "script", - build_python_zip = "no", + build_python_zip = False, py_src = "bin.py", sh_src = "run_binary_zip_no_test.sh", - target_compatible_with = _SUPPORTS_BOOTSTRAP_SCRIPT, + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, +) + +sh_py_run_test( + name = "run_binary_bootstrap_script_find_runfiles_test", + bootstrap_impl = "script", + py_src = "bin.py", + sh_src = "run_binary_find_runfiles_test.sh", + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, +) + +py_reconfig_test( + name = "bazel_tools_importable_system_python_test", + srcs = ["bazel_tools_importable_test.py"], + bootstrap_impl = "system_python", + # Necessary because bazel_tools doesn't have __init__.py files. + legacy_create_init = True, + main = "bazel_tools_importable_test.py", + deps = [ + "@bazel_tools//tools/python/runfiles", + ], ) py_reconfig_test( @@ -60,7 +124,7 @@ py_reconfig_test( env = {"BOOTSTRAP": "script"}, imports = ["./USER_IMPORT/site-packages"], main = "sys_path_order_test.py", - target_compatible_with = _SUPPORTS_BOOTSTRAP_SCRIPT, + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, ) py_reconfig_test( @@ -72,12 +136,47 @@ py_reconfig_test( main = "sys_path_order_test.py", ) +genrule( + name = "stdlib_shadowing_outputs", + outs = [ + "shutil.py", + "types.py", + ], + cmd = """ +cat > $(@D)/shutil.py <<'PY' +raise RuntimeError("target output shutil.py shadowed the stdlib shutil module") +PY +cat > $(@D)/types.py <<'PY' +raise RuntimeError("target output types.py shadowed the stdlib types module") +PY +""", +) + +py_reconfig_test( + name = "stdlib_shadowing_system_python_test", + srcs = [ + "stdlib_shadowing_test.py", + ":stdlib_shadowing_outputs", + ], + bootstrap_impl = "system_python", + main = "stdlib_shadowing_test.py", +) + +py_reconfig_test( + name = "main_module_test", + srcs = ["main_module.py"], + bootstrap_impl = "script", + imports = ["."], + main_module = "tests.bootstrap_impls.main_module", + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, +) + sh_py_run_test( name = "inherit_pythonsafepath_env_test", bootstrap_impl = "script", py_src = "bin.py", sh_src = "inherit_pythonsafepath_env_test.sh", - target_compatible_with = _SUPPORTS_BOOTSTRAP_SCRIPT, + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, ) sh_py_run_test( @@ -86,7 +185,44 @@ sh_py_run_test( imports = ["./MARKER"], py_src = "call_sys_exe.py", sh_src = "sys_executable_inherits_sys_path_test.sh", - target_compatible_with = _SUPPORTS_BOOTSTRAP_SCRIPT, + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, +) + +py_reconfig_test( + name = "interpreter_args_test", + srcs = ["interpreter_args_test.py"], + bootstrap_impl = "script", + interpreter_args = ["-XSPECIAL=1"], + main = "interpreter_args_test.py", + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, +) + +pkg_tar( + name = "external_binary", + testonly = True, + srcs = ["@other//:external_main"], + include_runfiles = True, + tags = ["manual"], # Don't build as part of wildcards +) + +sh_test( + name = "external_binary_test", + srcs = ["external_binary_test.sh"], + data = [":external_binary"], + # For now, skip this test on Windows because it fails for reasons + # other than the code path being tested. + target_compatible_with = select({ + "@platforms//os:windows": ["@platforms//:incompatible"], + "//conditions:default": [], + }), +) + +py_test( + name = "system_python_nodeps_test", + srcs = ["system_python_nodeps_test.py"], + config_settings = { + "//python/config_settings:bootstrap_impl": "system_python", + }, ) relative_path_test_suite(name = "relative_path_tests") diff --git a/tests/bootstrap_impls/a/b/c/BUILD.bazel b/tests/bootstrap_impls/a/b/c/BUILD.bazel index 8ffcbcd479..1c4b1e7b6b 100644 --- a/tests/bootstrap_impls/a/b/c/BUILD.bazel +++ b/tests/bootstrap_impls/a/b/c/BUILD.bazel @@ -1,10 +1,9 @@ -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility -load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test") +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") _SUPPORTS_BOOTSTRAP_SCRIPT = select({ "@platforms//os:windows": ["@platforms//:incompatible"], "//conditions:default": [], -}) if IS_BAZEL_7_OR_HIGHER else ["@platforms//:incompatible"] +}) py_reconfig_test( name = "nested_dir_test", diff --git a/tests/bootstrap_impls/bazel_tools_importable_test.py b/tests/bootstrap_impls/bazel_tools_importable_test.py new file mode 100644 index 0000000000..c374dd5dcf --- /dev/null +++ b/tests/bootstrap_impls/bazel_tools_importable_test.py @@ -0,0 +1,20 @@ +import sys +import unittest + + +class BazelToolsImportableTest(unittest.TestCase): + def test_bazel_tools_importable(self): + try: + import bazel_tools + import bazel_tools.tools.python + import bazel_tools.tools.python.runfiles # noqa: F401 + except ImportError as exc: + raise AssertionError( + "Failed to import bazel_tools.python.runfiles\n" + + "sys.path:\n" + + "\n".join(f"{i}: {v}" for i, v in enumerate(sys.path)) + ) from exc + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/bootstrap_impls/bin.py b/tests/bootstrap_impls/bin.py index c46e43adc8..3d467dcf29 100644 --- a/tests/bootstrap_impls/bin.py +++ b/tests/bootstrap_impls/bin.py @@ -22,3 +22,5 @@ print("PYTHONSAFEPATH:", os.environ.get("PYTHONSAFEPATH", "UNSET") or "EMPTY") print("sys.flags.safe_path:", sys.flags.safe_path) print("file:", __file__) +print("sys.executable:", sys.executable) +print("sys._base_executable:", sys._base_executable) diff --git a/tests/bootstrap_impls/bin_calls_bin/BUILD.bazel b/tests/bootstrap_impls/bin_calls_bin/BUILD.bazel new file mode 100644 index 0000000000..b5ba68f987 --- /dev/null +++ b/tests/bootstrap_impls/bin_calls_bin/BUILD.bazel @@ -0,0 +1,95 @@ +load("@rules_shell//shell:sh_test.bzl", "sh_test") +load("//python:py_library.bzl", "py_library") +load("//tests/support:py_reconfig.bzl", "py_reconfig_binary") +load("//tests/support:support.bzl", "NOT_WINDOWS", "SUPPORTS_BOOTSTRAP_SCRIPT") + +py_library( + name = "inner_lib", + srcs = ["inner_lib.py"], + tags = ["manual"], +) + +# ===== +# bootstrap_impl=system_python testing +# ===== +py_reconfig_binary( + name = "outer_bootstrap_system_python", + srcs = ["outer.py"], + bootstrap_impl = "system_python", + main = "outer.py", + tags = ["manual"], +) + +py_reconfig_binary( + name = "inner_bootstrap_system_python", + srcs = ["inner.py"], + bootstrap_impl = "system_python", + main = "inner.py", + tags = ["manual"], + deps = [":inner_lib"], +) + +genrule( + name = "outer_calls_inner_system_python", + outs = ["outer_calls_inner_system_python.out"], + cmd = "RULES_PYTHON_TESTING_TELL_RUNFILES_ROOT=1 $(location :outer_bootstrap_system_python) $(location :inner_bootstrap_system_python) > $@", + tags = ["manual"], + tools = [ + ":inner_bootstrap_system_python", + ":outer_bootstrap_system_python", + ], +) + +sh_test( + name = "bootstrap_system_python_test", + srcs = ["verify_system_python.sh"], + data = [ + "verify.sh", + ":outer_calls_inner_system_python", + ], + # The way verify_system_python.sh loads verify.sh doesn't work + # with Windows for some annoying reason. Just skip windows for now; + # the logic being test isn't OS-specific, so this should be fine. + target_compatible_with = NOT_WINDOWS, +) + +# ===== +# bootstrap_impl=script testing +# ===== +py_reconfig_binary( + name = "inner_bootstrap_script", + srcs = ["inner.py"], + bootstrap_impl = "script", + main = "inner.py", + tags = ["manual"], + deps = [":inner_lib"], +) + +py_reconfig_binary( + name = "outer_bootstrap_script", + srcs = ["outer.py"], + bootstrap_impl = "script", + main = "outer.py", + tags = ["manual"], +) + +genrule( + name = "outer_calls_inner_script_python", + outs = ["outer_calls_inner_script_python.out"], + cmd = "RULES_PYTHON_TESTING_TELL_RUNFILES_ROOT=1 $(location :outer_bootstrap_script) $(location :inner_bootstrap_script) > $@", + tags = ["manual"], + tools = [ + ":inner_bootstrap_script", + ":outer_bootstrap_script", + ], +) + +sh_test( + name = "bootstrap_script_python_test", + srcs = ["verify_script_python.sh"], + data = [ + "verify.sh", + ":outer_calls_inner_script_python", + ], + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, +) diff --git a/tests/bootstrap_impls/bin_calls_bin/inner.py b/tests/bootstrap_impls/bin_calls_bin/inner.py new file mode 100644 index 0000000000..74a70b5f0d --- /dev/null +++ b/tests/bootstrap_impls/bin_calls_bin/inner.py @@ -0,0 +1,15 @@ +import os + +runfiles_root = os.environ.get("RULES_PYTHON_TESTING_RUNFILES_ROOT") +runfiles_dir = os.environ.get("RUNFILES_DIR") +runfiles_manifest_file = os.environ.get("RUNFILES_MANIFEST_FILE") +print(f"inner: RULES_PYTHON_TESTING_RUNFILES_ROOT='{runfiles_root}'") +print(f"inner: RUNFILES_DIR='{runfiles_dir}'") +print(f"inner: RUNFILES_MANIFEST_FILE='{runfiles_manifest_file}'") + +try: + import tests.bootstrap_impls.bin_calls_bin.inner_lib as inner_lib + + print(f"inner: import_result='{inner_lib.confirm()}'") +except ImportError as e: + print(f"inner: import_result='{e}'") diff --git a/tests/bootstrap_impls/bin_calls_bin/inner_lib.py b/tests/bootstrap_impls/bin_calls_bin/inner_lib.py new file mode 100644 index 0000000000..5815b4f41b --- /dev/null +++ b/tests/bootstrap_impls/bin_calls_bin/inner_lib.py @@ -0,0 +1,3 @@ +# Rather than having a completely empty file... +def confirm(): + return "success" diff --git a/tests/bootstrap_impls/bin_calls_bin/outer.py b/tests/bootstrap_impls/bin_calls_bin/outer.py new file mode 100644 index 0000000000..a4432dff59 --- /dev/null +++ b/tests/bootstrap_impls/bin_calls_bin/outer.py @@ -0,0 +1,18 @@ +import os +import subprocess +import sys + +if __name__ == "__main__": + runfiles_root = os.environ.get("RULES_PYTHON_TESTING_RUNFILES_ROOT") + print(f"outer: RULES_PYTHON_TESTING_RUNFILES_ROOT='{runfiles_root}'") + + inner_binary_path = sys.argv[1] + result = subprocess.run( + [inner_binary_path], + capture_output=True, + text=True, + ) + print(result.stdout, end="") + if result.stderr: + print(result.stderr, end="", file=sys.stderr) + sys.exit(result.returncode) diff --git a/tests/bootstrap_impls/bin_calls_bin/verify.sh b/tests/bootstrap_impls/bin_calls_bin/verify.sh new file mode 100755 index 0000000000..bbe4252cc1 --- /dev/null +++ b/tests/bootstrap_impls/bin_calls_bin/verify.sh @@ -0,0 +1,50 @@ +#!/bin/bash +set -euo pipefail + +verify_output() { + local OUTPUT_FILE=$1 + + # Extract the RULES_PYTHON_TESTING_RUNFILES_ROOT values + local OUTER_RUNFILES_ROOT=$(grep "outer: RULES_PYTHON_TESTING_RUNFILES_ROOT" "$OUTPUT_FILE" | sed "s/outer: RULES_PYTHON_TESTING_RUNFILES_ROOT='\(.*\)'/\1/") + local INNER_RUNFILES_ROOT=$(grep "inner: RULES_PYTHON_TESTING_RUNFILES_ROOT" "$OUTPUT_FILE" | sed "s/inner: RULES_PYTHON_TESTING_RUNFILES_ROOT='\(.*\)'/\1/") + + echo "Outer runfiles root: $OUTER_RUNFILES_ROOT" + echo "Inner runfiles root: $INNER_RUNFILES_ROOT" + + # Extract the inner runfiles values + local INNER_RUNFILES_DIR=$(grep "inner: RUNFILES_DIR" "$OUTPUT_FILE" | sed "s/inner: RUNFILES_DIR='\(.*\)'/\1/") + local INNER_RUNFILES_MANIFEST_FILE=$(grep "inner: RUNFILES_MANIFEST_FILE" "$OUTPUT_FILE" | sed "s/inner: RUNFILES_MANIFEST_FILE='\(.*\)'/\1/") + + echo "Inner runfiles dir: $INNER_RUNFILES_DIR" + echo "Inner runfiles manifest file: $INNER_RUNFILES_MANIFEST_FILE" + + # Extract the inner lib import result + local INNER_LIB_IMPORT=$(grep "inner: import_result" "$OUTPUT_FILE" | sed "s/inner: import_result='\(.*\)'/\1/") + echo "Inner lib import result: $INNER_LIB_IMPORT" + + + # Check 1: The two values are different + if [ "$OUTER_RUNFILES_ROOT" == "$INNER_RUNFILES_ROOT" ]; then + echo "Error: Outer and Inner runfiles roots are the same." + exit 1 + fi + + # Check 2: Inner is not a subdirectory of Outer + case "$INNER_RUNFILES_ROOT" in + "$OUTER_RUNFILES_ROOT"/*) + echo "Error: Inner runfiles root is a subdirectory of Outer's." + exit 1 + ;; + *) + # This is the success case + ;; + esac + + # Check 3: inner_lib was imported + if [ "$INNER_LIB_IMPORT" != "success" ]; then + echo "Error: Inner lib was not successfully imported." + exit 1 + fi + + echo "Verification successful." +} diff --git a/tests/bootstrap_impls/bin_calls_bin/verify_script_python.sh b/tests/bootstrap_impls/bin_calls_bin/verify_script_python.sh new file mode 100755 index 0000000000..012daee05b --- /dev/null +++ b/tests/bootstrap_impls/bin_calls_bin/verify_script_python.sh @@ -0,0 +1,5 @@ +#!/bin/bash +set -euo pipefail + +source "$(dirname "$0")/verify.sh" +verify_output "$(dirname "$0")/outer_calls_inner_script_python.out" diff --git a/tests/bootstrap_impls/bin_calls_bin/verify_system_python.sh b/tests/bootstrap_impls/bin_calls_bin/verify_system_python.sh new file mode 100755 index 0000000000..460769fd04 --- /dev/null +++ b/tests/bootstrap_impls/bin_calls_bin/verify_system_python.sh @@ -0,0 +1,5 @@ +#!/bin/bash +set -euo pipefail + +source "$(dirname "$0")/verify.sh" +verify_output "$(dirname "$0")/outer_calls_inner_system_python.out" diff --git a/tests/bootstrap_impls/bootstrap_script_zipapp_test.sh b/tests/bootstrap_impls/bootstrap_script_zipapp_test.sh new file mode 100755 index 0000000000..558ca970d6 --- /dev/null +++ b/tests/bootstrap_impls/bootstrap_script_zipapp_test.sh @@ -0,0 +1,47 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# --- begin runfiles.bash initialization v3 --- +# Copy-pasted from the Bazel Bash runfiles library v3. +set -uo pipefail; set +e; f=bazel_tools/tools/bash/runfiles/runfiles.bash +source "${RUNFILES_DIR:-/dev/null}/$f" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "${RUNFILES_MANIFEST_FILE:-/dev/null}" | cut -f2- -d' ')" 2>/dev/null || \ + source "$0.runfiles/$f" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "$0.runfiles_manifest" | cut -f2- -d' ')" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "$0.exe.runfiles_manifest" | cut -f2- -d' ')" 2>/dev/null || \ + { echo>&2 "ERROR: cannot find $f"; exit 1; }; f=; set -e +# --- end runfiles.bash initialization v3 --- +set +e + +bin=$(rlocation $ZIP_RLOCATION) +if [[ -z "$bin" ]]; then + echo "Unable to locate test binary: $ZIP_RLOCATION" + exit 1 +fi +set -x +actual=$(python3 $bin) + +# How we detect if a zip file was executed from depends on which bootstrap +# is used. +# bootstrap_impl=script outputs RULES_PYTHON_ZIP_DIR= +# bootstrap_impl=system_python outputs file:.*Bazel.runfiles +expected_pattern="Hello" +if ! (echo "$actual" | grep "$expected_pattern" ) >/dev/null; then + echo "Test case failed: $1" + echo "expected output to match: $expected_pattern" + echo "but got:\n$actual" + exit 1 +fi + +exit 0 diff --git a/tests/bootstrap_impls/call_sys_exe.py b/tests/bootstrap_impls/call_sys_exe.py index 0c6157048c..c431145386 100644 --- a/tests/bootstrap_impls/call_sys_exe.py +++ b/tests/bootstrap_impls/call_sys_exe.py @@ -12,7 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. -import os import subprocess import sys diff --git a/tests/bootstrap_impls/external_binary_test.sh b/tests/bootstrap_impls/external_binary_test.sh new file mode 100755 index 0000000000..92799354d6 --- /dev/null +++ b/tests/bootstrap_impls/external_binary_test.sh @@ -0,0 +1,9 @@ +#!/usr/bin/env bash +set -euxo pipefail + +tmpdir="${TEST_TMPDIR}/external_binary" +mkdir -p "${tmpdir}" +tar xf "tests/bootstrap_impls/external_binary.tar" -C "${tmpdir}" +test -x "${tmpdir}/external_main" +output="$("${tmpdir}/external_main")" +test "$output" = "token" diff --git a/tests/bootstrap_impls/interpreter_args_test.py b/tests/bootstrap_impls/interpreter_args_test.py new file mode 100644 index 0000000000..27744c647f --- /dev/null +++ b/tests/bootstrap_impls/interpreter_args_test.py @@ -0,0 +1,25 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import sys +import unittest + + +class InterpreterArgsTest(unittest.TestCase): + def test_interpreter_args(self): + self.assertEqual(sys._xoptions, {"SPECIAL": "1"}) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/bootstrap_impls/main_module.py b/tests/bootstrap_impls/main_module.py new file mode 100644 index 0000000000..afb1ff6ba8 --- /dev/null +++ b/tests/bootstrap_impls/main_module.py @@ -0,0 +1,17 @@ +import sys +import unittest + + +class MainModuleTest(unittest.TestCase): + def test_run_as_module(self): + self.assertIsNotNone(__spec__, "__spec__ was none") + # If not run as a module, __spec__ is None + self.assertNotEqual(__name__, __spec__.name) + self.assertEqual(__spec__.name, "tests.bootstrap_impls.main_module") + + +if __name__ == "__main__": + unittest.main() +else: + # Guard against running it as a module in a non-main way. + sys.exit(f"__name__ should be __main__, got {__name__}") diff --git a/tests/bootstrap_impls/run_binary_find_runfiles_test.sh b/tests/bootstrap_impls/run_binary_find_runfiles_test.sh new file mode 100755 index 0000000000..a6c1b565db --- /dev/null +++ b/tests/bootstrap_impls/run_binary_find_runfiles_test.sh @@ -0,0 +1,59 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# --- begin runfiles.bash initialization v3 --- +# Copy-pasted from the Bazel Bash runfiles library v3. +set -uo pipefail; set +e; f=bazel_tools/tools/bash/runfiles/runfiles.bash +source "${RUNFILES_DIR:-/dev/null}/$f" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "${RUNFILES_MANIFEST_FILE:-/dev/null}" | cut -f2- -d' ')" 2>/dev/null || \ + source "$0.runfiles/$f" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "$0.runfiles_manifest" | cut -f2- -d' ')" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "$0.exe.runfiles_manifest" | cut -f2- -d' ')" 2>/dev/null || \ + { echo>&2 "ERROR: cannot find $f"; exit 1; }; f=; set -e +# --- end runfiles.bash initialization v3 --- +set +e + +bin=$(rlocation $BIN_RLOCATION) +if [[ -z "$bin" ]]; then + echo "Unable to locate test binary: $BIN_RLOCATION" + exit 1 +fi + +bin_link_layer_1=$TEST_TMPDIR/link1 +ln -s "$bin" "$bin_link_layer_1" +bin_link_layer_2=$TEST_TMPDIR/link2 +ln -s "$bin_link_layer_1" "$bin_link_layer_2" + +result=$(RUNFILES_DIR='' RUNFILES_MANIFEST_FILE='' $bin) +result_link_layer_1=$(RUNFILES_DIR='' RUNFILES_MANIFEST_FILE='' $bin_link_layer_1) +result_link_layer_2=$(RUNFILES_DIR='' RUNFILES_MANIFEST_FILE='' $bin_link_layer_2) + +if [[ "$result" != "$result_link_layer_1" ]]; then + echo "Output from test does not match output when invoked via a link;" + echo "Output from test:" + echo "$result" + echo "Output when invoked via a link:" + echo "$result_link_layer_1" + exit 1 +fi +if [[ "$result" != "$result_link_layer_2" ]]; then + echo "Output from test does not match output when invoked via a link to a link;" + echo "Output from test:" + echo "$result" + echo "Output when invoked via a link to a link:" + echo "$result_link_layer_2" + exit 1 +fi + +exit 0 diff --git a/tests/bootstrap_impls/run_binary_venvs_use_declare_symlink_no_test.sh b/tests/bootstrap_impls/run_binary_venvs_use_declare_symlink_no_test.sh new file mode 100755 index 0000000000..d4840116f9 --- /dev/null +++ b/tests/bootstrap_impls/run_binary_venvs_use_declare_symlink_no_test.sh @@ -0,0 +1,56 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# --- begin runfiles.bash initialization v3 --- +# Copy-pasted from the Bazel Bash runfiles library v3. +set -uo pipefail; set +e; f=bazel_tools/tools/bash/runfiles/runfiles.bash +source "${RUNFILES_DIR:-/dev/null}/$f" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "${RUNFILES_MANIFEST_FILE:-/dev/null}" | cut -f2- -d' ')" 2>/dev/null || \ + source "$0.runfiles/$f" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "$0.runfiles_manifest" | cut -f2- -d' ')" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "$0.exe.runfiles_manifest" | cut -f2- -d' ')" 2>/dev/null || \ + { echo>&2 "ERROR: cannot find $f"; exit 1; }; f=; set -e +# --- end runfiles.bash initialization v3 --- +set +e + +bin=$(rlocation $BIN_RLOCATION) +if [[ -z "$bin" ]]; then + echo "Unable to locate test binary: $BIN_RLOCATION" + exit 1 +fi +actual=$($bin) + +function expect_match() { + local expected_pattern=$1 + local actual=$2 + if ! (echo "$actual" | grep "$expected_pattern" ) >/dev/null; then + echo "expected to match: $expected_pattern" + echo "===== actual START =====" + echo "$actual" + echo "===== actual END =====" + echo + touch EXPECTATION_FAILED + return 1 + fi +} + +expect_match "sys.executable:.*tmp.*python3" "$actual" + +# Now test that using a custom location for the bootstrap files works +venvs_root=$(mktemp -d) +actual=$(RULES_PYTHON_EXTRACT_ROOT=$venvs_root $bin) +expect_match "sys.executable:.*$venvs_root" "$actual" + +# Exit if any of the expects failed +[[ ! -e EXPECTATION_FAILED ]] diff --git a/tests/bootstrap_impls/run_binary_zip_yes_test.sh b/tests/bootstrap_impls/run_binary_zip_yes_test.sh index ca278083dd..77fe4d3609 100755 --- a/tests/bootstrap_impls/run_binary_zip_yes_test.sh +++ b/tests/bootstrap_impls/run_binary_zip_yes_test.sh @@ -34,8 +34,8 @@ actual=$($bin) # How we detect if a zip file was executed from depends on which bootstrap # is used. # bootstrap_impl=script outputs RULES_PYTHON_ZIP_DIR: -# bootstrap_impl=system_python outputs file:.*Bazel.runfiles -expected_pattern="RULES_PYTHON_ZIP_DIR:/\|file:.*Bazel.runfiles" +# bootstrap_impl=system_python outputs file:.*Bazel.runfiles (or .exe.runfiles on Windows) +expected_pattern="RULES_PYTHON_ZIP_DIR:/\|file:.*Bazel.runfiles\|file:.*\.exe\.runfiles" if ! (echo "$actual" | grep "$expected_pattern" ) >/dev/null; then echo "expected output to match: $expected_pattern" echo "but got: $actual" diff --git a/tests/bootstrap_impls/stdlib_shadowing_test.py b/tests/bootstrap_impls/stdlib_shadowing_test.py new file mode 100644 index 0000000000..fc913857db --- /dev/null +++ b/tests/bootstrap_impls/stdlib_shadowing_test.py @@ -0,0 +1,20 @@ +# Copyright 2026 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Verifies stage 1 bootstrap stdlib imports cannot be shadowed.""" + + +def test_bootstrap_reached_main(): + # If stage 1 imports target outputs such as shutil.py instead of stdlib + # modules, the process fails before this test module is executed. + pass diff --git a/tests/bootstrap_impls/sys_path_order_test.py b/tests/bootstrap_impls/sys_path_order_test.py index 97c62a6be5..a9018c39ce 100644 --- a/tests/bootstrap_impls/sys_path_order_test.py +++ b/tests/bootstrap_impls/sys_path_order_test.py @@ -33,9 +33,13 @@ def test_sys_path_order(self): # error messages are more informative. categorized_paths = [] for i, value in enumerate(sys.path): - # The runtime's root repo may be added to sys.path, but it - # counts as a user directory, not stdlib directory. - if value in (sys.prefix, sys.base_prefix): + # On Windows, the `pythonXY.zip` entry shows up as `$venv/Scripts/pythonXY.zip` + # While it's technically part of the venv, it's considered the stdlib. + if os.name == "nt" and re.search("python.*[.]zip$", value): + category = "stdlib" + elif value in (sys.prefix, sys.base_prefix): + # The runtime's root repo may be added to sys.path, but it + # counts as a user directory, not stdlib directory. category = "user" elif value.startswith(sys.base_prefix): # The runtime's site-package directory might be called @@ -73,25 +77,15 @@ def test_sys_path_order(self): + f"for sys.path:\n{sys_path_str}" ) - if os.environ["BOOTSTRAP"] == "script": - self.assertTrue( - last_stdlib < first_user < first_runtime_site, - "Expected overall order to be (stdlib, user imports, runtime site) " - + f"with {last_stdlib=} < {first_user=} < {first_runtime_site=}\n" - + f"for sys.prefix={sys.prefix}\n" - + f"for sys.exec_prefix={sys.exec_prefix}\n" - + f"for sys.base_prefix={sys.base_prefix}\n" - + f"for sys.path:\n{sys_path_str}", - ) - else: - self.assertTrue( - first_user < last_stdlib < first_runtime_site, - f"Expected {first_user=} < {last_stdlib=} < {first_runtime_site=}\n" - + f"for sys.prefix={sys.prefix}\n" - + f"for sys.exec_prefix={sys.exec_prefix}\n" - + f"for sys.base_prefix={sys.base_prefix}\n" - + f"for sys.path:\n{sys_path_str}", - ) + self.assertTrue( + last_stdlib < first_user < first_runtime_site, + "Expected overall order to be (stdlib, user imports, runtime site) " + + f"with {last_stdlib=} < {first_user=} < {first_runtime_site=}\n" + + f"for sys.prefix={sys.prefix}\n" + + f"for sys.exec_prefix={sys.exec_prefix}\n" + + f"for sys.base_prefix={sys.base_prefix}\n" + + f"for sys.path:\n{sys_path_str}", + ) if __name__ == "__main__": diff --git a/tests/bootstrap_impls/system_python_nodeps_test.py b/tests/bootstrap_impls/system_python_nodeps_test.py new file mode 100644 index 0000000000..1affaab497 --- /dev/null +++ b/tests/bootstrap_impls/system_python_nodeps_test.py @@ -0,0 +1,11 @@ +print("Hello, world") + +# Verify py code from the stdlib can be imported. +import pathlib # noqa: E402 + +print(pathlib) + +# Verify a C-implemented module can be imported. +# Socket isn't implement in C, but requires `_socket`, +# which is implemented in C +import socket # noqa: E402, F401 diff --git a/tests/bootstrap_impls/venv_relative_path_tests.bzl b/tests/bootstrap_impls/venv_relative_path_tests.bzl index b21f220205..72b5012809 100644 --- a/tests/bootstrap_impls/venv_relative_path_tests.bzl +++ b/tests/bootstrap_impls/venv_relative_path_tests.bzl @@ -15,7 +15,7 @@ "Unit tests for relative_path computation" load("@rules_testing//lib:test_suite.bzl", "test_suite") -load("//python/private:py_executable_bazel.bzl", "relative_path") # buildifier: disable=bzl-visibility +load("//python/private:common.bzl", "relative_path") # buildifier: disable=bzl-visibility _tests = [] diff --git a/tests/build_data/BUILD.bazel b/tests/build_data/BUILD.bazel new file mode 100644 index 0000000000..64db005f51 --- /dev/null +++ b/tests/build_data/BUILD.bazel @@ -0,0 +1,25 @@ +load("//python:py_binary.bzl", "py_binary") +load("//python:py_test.bzl", "py_test") + +py_test( + name = "build_data_test", + srcs = ["build_data_test.py"], + data = [ + ":tool_build_data.txt", + ], + stamp = 1, + deps = ["//python/runfiles"], +) + +py_binary( + name = "print_build_data", + srcs = ["print_build_data.py"], + deps = ["//python/runfiles"], +) + +genrule( + name = "tool_build_data", + outs = ["tool_build_data.txt"], + cmd = "$(location :print_build_data) > $(OUTS)", + tools = [":print_build_data"], +) diff --git a/tests/build_data/build_data_test.py b/tests/build_data/build_data_test.py new file mode 100644 index 0000000000..6be4e52c84 --- /dev/null +++ b/tests/build_data/build_data_test.py @@ -0,0 +1,29 @@ +import unittest + +from python.runfiles import runfiles + + +class BuildDataTest(unittest.TestCase): + def test_target_build_data(self): + import bazel_binary_info + + self.assertIn("build_data.txt", bazel_binary_info.BUILD_DATA_FILE) + + build_data = bazel_binary_info.get_build_data() + self.assertIn("TARGET ", build_data) + self.assertIn("BUILD_HOST ", build_data) + self.assertIn("BUILD_USER ", build_data) + self.assertIn("BUILD_TIMESTAMP ", build_data) + self.assertIn("FORMATTED_DATE ", build_data) + self.assertIn("STAMPED TRUE", build_data) + + def test_tool_build_data(self): + rf = runfiles.Create() + path = rf.Rlocation("rules_python/tests/build_data/tool_build_data.txt") + with open(path) as fp: + build_data = fp.read() + + self.assertIn("STAMPED FALSE", build_data) + + +unittest.main() diff --git a/tests/build_data/print_build_data.py b/tests/build_data/print_build_data.py new file mode 100644 index 0000000000..0af77d72be --- /dev/null +++ b/tests/build_data/print_build_data.py @@ -0,0 +1,3 @@ +import bazel_binary_info + +print(bazel_binary_info.get_build_data()) diff --git a/tests/builders/BUILD.bazel b/tests/builders/BUILD.bazel index 3ad0c3e80c..f963cb0131 100644 --- a/tests/builders/BUILD.bazel +++ b/tests/builders/BUILD.bazel @@ -12,6 +12,42 @@ # See the License for the specific language governing permissions and # limitations under the License. +load(":attr_builders_tests.bzl", "attr_builders_test_suite") load(":builders_tests.bzl", "builders_test_suite") +load(":rule_builders_tests.bzl", "rule_builders_test_suite") builders_test_suite(name = "builders_test_suite") + +rule_builders_test_suite(name = "rule_builders_test_suite") + +attr_builders_test_suite(name = "attr_builders_test_suite") + +toolchain_type(name = "tct_1") + +toolchain_type(name = "tct_2") + +toolchain_type(name = "tct_3") + +toolchain_type(name = "tct_4") + +toolchain_type(name = "tct_5") + +filegroup(name = "empty") + +toolchain( + name = "tct_3_toolchain", + toolchain = "//tests/support/empty_toolchain:empty", + toolchain_type = "//tests/builders:tct_3", +) + +toolchain( + name = "tct_4_toolchain", + toolchain = "//tests/support/empty_toolchain:empty", + toolchain_type = ":tct_4", +) + +toolchain( + name = "tct_5_toolchain", + toolchain = "//tests/support/empty_toolchain:empty", + toolchain_type = ":tct_5", +) diff --git a/tests/builders/attr_builders_tests.bzl b/tests/builders/attr_builders_tests.bzl new file mode 100644 index 0000000000..3a771afde5 --- /dev/null +++ b/tests/builders/attr_builders_tests.bzl @@ -0,0 +1,470 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Tests for attr_builders.""" + +load("@rules_testing//lib:analysis_test.bzl", "analysis_test") +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("@rules_testing//lib:truth.bzl", "truth") +load("//python/private:attr_builders.bzl", "attrb") # buildifier: disable=bzl-visibility +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility + +def _expect_cfg_defaults(expect, cfg): + expect.where(expr = "cfg.outputs").that_collection(cfg.outputs()).contains_exactly([]) + expect.where(expr = "cfg.inputs").that_collection(cfg.inputs()).contains_exactly([]) + expect.where(expr = "cfg.implementation").that_bool(cfg.implementation()).equals(None) + expect.where(expr = "cfg.target").that_bool(cfg.target()).equals(True) + expect.where(expr = "cfg.exec_group").that_str(cfg.exec_group()).equals(None) + expect.where(expr = "cfg.which_cfg").that_str(cfg.which_cfg()).equals("target") + +_some_aspect = aspect(implementation = lambda target, ctx: None) +_SomeInfo = provider("MyInfo", fields = []) + +_tests = [] + +def _report_failures(name, env): + failures = env.failures + + def _report_failures_impl(env, target): + _ = target # @unused + env._failures.extend(failures) + + analysis_test( + name = name, + target = labels.NONE, + impl = _report_failures_impl, + ) + +# Calling attr.xxx() outside of the loading phase is an error, but rules_testing +# creates the expect/truth helpers during the analysis phase. To make the truth +# helpers available during the loading phase, fake out the ctx just enough to +# satify rules_testing. +def _loading_phase_expect(test_name): + env = struct( + ctx = struct( + workspace_name = "bogus", + label = Label(test_name), + attr = struct( + _impl_name = test_name, + ), + ), + failures = [], + ) + return env, truth.expect(env) + +def _expect_builds(expect, builder, attribute_type): + expect.that_str(str(builder.build())).contains(attribute_type) + +def _test_cfg_arg(name): + env, _ = _loading_phase_expect(name) + + def build_cfg(cfg): + attrb.Label(cfg = cfg).build() + + build_cfg(None) + build_cfg("target") + build_cfg("exec") + build_cfg(dict(exec_group = "eg")) + build_cfg(dict(implementation = (lambda settings, attr: None))) + build_cfg(config.exec()) + build_cfg(transition( + implementation = (lambda settings, attr: None), + inputs = [], + outputs = [], + )) + + # config.target is Bazel 8+ + if hasattr(config, "target"): + build_cfg(config.target()) + + # config.none is Bazel 8+ + if hasattr(config, "none"): + build_cfg("none") + build_cfg(config.none()) + + _report_failures(name, env) + +_tests.append(_test_cfg_arg) + +def _test_bool(name): + env, expect = _loading_phase_expect(name) + subject = attrb.Bool() + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.default()).equals(False) + expect.that_bool(subject.mandatory()).equals(False) + _expect_builds(expect, subject, "attr.bool") + + subject.set_default(True) + subject.set_mandatory(True) + subject.set_doc("doc") + + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.default()).equals(True) + expect.that_bool(subject.mandatory()).equals(True) + _expect_builds(expect, subject, "attr.bool") + + _report_failures(name, env) + +_tests.append(_test_bool) + +def _test_int(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.Int() + expect.that_int(subject.default()).equals(0) + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + expect.that_collection(subject.values()).contains_exactly([]) + _expect_builds(expect, subject, "attr.int") + + subject.set_default(42) + subject.set_doc("doc") + subject.set_mandatory(True) + subject.values().append(42) + + expect.that_int(subject.default()).equals(42) + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + expect.that_collection(subject.values()).contains_exactly([42]) + _expect_builds(expect, subject, "attr.int") + + _report_failures(name, env) + +_tests.append(_test_int) + +def _test_int_list(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.IntList() + expect.that_bool(subject.allow_empty()).equals(True) + expect.that_collection(subject.default()).contains_exactly([]) + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + _expect_builds(expect, subject, "attr.int_list") + + subject.default().append(99) + subject.set_doc("doc") + subject.set_mandatory(True) + + expect.that_collection(subject.default()).contains_exactly([99]) + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + _expect_builds(expect, subject, "attr.int_list") + + _report_failures(name, env) + +_tests.append(_test_int_list) + +def _test_label(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.Label() + + expect.that_str(subject.default()).equals(None) + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + expect.that_bool(subject.executable()).equals(False) + expect.that_bool(subject.allow_files()).equals(None) + expect.that_bool(subject.allow_single_file()).equals(None) + expect.that_collection(subject.providers()).contains_exactly([]) + expect.that_collection(subject.aspects()).contains_exactly([]) + _expect_cfg_defaults(expect, subject.cfg) + _expect_builds(expect, subject, "attr.label") + + subject.set_default("//foo:bar") + subject.set_doc("doc") + subject.set_mandatory(True) + subject.set_executable(True) + subject.add_allow_files(".txt") + subject.cfg.set_target() + subject.providers().append(_SomeInfo) + subject.aspects().append(_some_aspect) + subject.cfg.outputs().append(Label("//some:output")) + subject.cfg.inputs().append(Label("//some:input")) + impl = lambda: None + subject.cfg.set_implementation(impl) + + expect.that_str(subject.default()).equals("//foo:bar") + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + expect.that_bool(subject.executable()).equals(True) + expect.that_collection(subject.allow_files()).contains_exactly([".txt"]) + expect.that_bool(subject.allow_single_file()).equals(None) + expect.that_collection(subject.providers()).contains_exactly([_SomeInfo]) + expect.that_collection(subject.aspects()).contains_exactly([_some_aspect]) + expect.that_collection(subject.cfg.outputs()).contains_exactly([Label("//some:output")]) + expect.that_collection(subject.cfg.inputs()).contains_exactly([Label("//some:input")]) + expect.that_bool(subject.cfg.implementation()).equals(impl) + _expect_builds(expect, subject, "attr.label") + + _report_failures(name, env) + +_tests.append(_test_label) + +def _test_label_keyed_string_dict(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.LabelKeyedStringDict() + + expect.that_dict(subject.default()).contains_exactly({}) + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + expect.that_bool(subject.allow_files()).equals(False) + expect.that_collection(subject.providers()).contains_exactly([]) + expect.that_collection(subject.aspects()).contains_exactly([]) + _expect_cfg_defaults(expect, subject.cfg) + _expect_builds(expect, subject, "attr.label_keyed_string_dict") + + subject.default()["key"] = "//some:label" + subject.set_doc("doc") + subject.set_mandatory(True) + subject.set_allow_files(True) + subject.cfg.set_target() + subject.providers().append(_SomeInfo) + subject.aspects().append(_some_aspect) + subject.cfg.outputs().append("//some:output") + subject.cfg.inputs().append("//some:input") + impl = lambda: None + subject.cfg.set_implementation(impl) + + expect.that_dict(subject.default()).contains_exactly({"key": "//some:label"}) + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + expect.that_bool(subject.allow_files()).equals(True) + expect.that_collection(subject.providers()).contains_exactly([_SomeInfo]) + expect.that_collection(subject.aspects()).contains_exactly([_some_aspect]) + expect.that_collection(subject.cfg.outputs()).contains_exactly(["//some:output"]) + expect.that_collection(subject.cfg.inputs()).contains_exactly(["//some:input"]) + expect.that_bool(subject.cfg.implementation()).equals(impl) + + _expect_builds(expect, subject, "attr.label_keyed_string_dict") + + subject.add_allow_files(".txt") + expect.that_collection(subject.allow_files()).contains_exactly([".txt"]) + _expect_builds(expect, subject, "attr.label_keyed_string_dict") + + _report_failures(name, env) + +_tests.append(_test_label_keyed_string_dict) + +def _test_label_list(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.LabelList() + + expect.that_collection(subject.default()).contains_exactly([]) + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + expect.that_bool(subject.allow_files()).equals(False) + expect.that_collection(subject.providers()).contains_exactly([]) + expect.that_collection(subject.aspects()).contains_exactly([]) + _expect_cfg_defaults(expect, subject.cfg) + _expect_builds(expect, subject, "attr.label_list") + + subject.default().append("//some:label") + subject.set_doc("doc") + subject.set_mandatory(True) + subject.set_allow_files([".txt"]) + subject.providers().append(_SomeInfo) + subject.aspects().append(_some_aspect) + + expect.that_collection(subject.default()).contains_exactly(["//some:label"]) + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + expect.that_collection(subject.allow_files()).contains_exactly([".txt"]) + expect.that_collection(subject.providers()).contains_exactly([_SomeInfo]) + expect.that_collection(subject.aspects()).contains_exactly([_some_aspect]) + + _expect_builds(expect, subject, "attr.label_list") + + _report_failures(name, env) + +_tests.append(_test_label_list) + +def _test_output(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.Output() + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + _expect_builds(expect, subject, "attr.output") + + subject.set_doc("doc") + subject.set_mandatory(True) + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + _expect_builds(expect, subject, "attr.output") + + _report_failures(name, env) + +_tests.append(_test_output) + +def _test_output_list(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.OutputList() + expect.that_bool(subject.allow_empty()).equals(True) + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + _expect_builds(expect, subject, "attr.output_list") + + subject.set_allow_empty(False) + subject.set_doc("doc") + subject.set_mandatory(True) + expect.that_bool(subject.allow_empty()).equals(False) + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + _expect_builds(expect, subject, "attr.output_list") + + _report_failures(name, env) + +_tests.append(_test_output_list) + +def _test_string(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.String() + expect.that_str(subject.default()).equals("") + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + expect.that_collection(subject.values()).contains_exactly([]) + _expect_builds(expect, subject, "attr.string") + + subject.set_doc("doc") + subject.set_mandatory(True) + subject.values().append("green") + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + expect.that_collection(subject.values()).contains_exactly(["green"]) + _expect_builds(expect, subject, "attr.string") + + _report_failures(name, env) + +_tests.append(_test_string) + +def _test_string_dict(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.StringDict() + + expect.that_dict(subject.default()).contains_exactly({}) + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + expect.that_bool(subject.allow_empty()).equals(True) + _expect_builds(expect, subject, "attr.string_dict") + + subject.default()["key"] = "value" + subject.set_doc("doc") + subject.set_mandatory(True) + subject.set_allow_empty(False) + + expect.that_dict(subject.default()).contains_exactly({"key": "value"}) + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + expect.that_bool(subject.allow_empty()).equals(False) + _expect_builds(expect, subject, "attr.string_dict") + + _report_failures(name, env) + +_tests.append(_test_string_dict) + +def _test_string_keyed_label_dict(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.StringKeyedLabelDict() + + expect.that_dict(subject.default()).contains_exactly({}) + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + expect.that_bool(subject.allow_files()).equals(False) + expect.that_collection(subject.providers()).contains_exactly([]) + expect.that_collection(subject.aspects()).contains_exactly([]) + _expect_cfg_defaults(expect, subject.cfg) + _expect_builds(expect, subject, "attr.string_keyed_label_dict") + + subject.default()["key"] = "//some:label" + subject.set_doc("doc") + subject.set_mandatory(True) + subject.set_allow_files([".txt"]) + subject.providers().append(_SomeInfo) + subject.aspects().append(_some_aspect) + + expect.that_dict(subject.default()).contains_exactly({"key": "//some:label"}) + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + expect.that_collection(subject.allow_files()).contains_exactly([".txt"]) + expect.that_collection(subject.providers()).contains_exactly([_SomeInfo]) + expect.that_collection(subject.aspects()).contains_exactly([_some_aspect]) + + _expect_builds(expect, subject, "attr.string_keyed_label_dict") + + _report_failures(name, env) + +_tests.append(_test_string_keyed_label_dict) + +def _test_string_list(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.StringList() + + expect.that_collection(subject.default()).contains_exactly([]) + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + expect.that_bool(subject.allow_empty()).equals(True) + _expect_builds(expect, subject, "attr.string_list") + + subject.set_doc("doc") + subject.set_mandatory(True) + subject.default().append("blue") + subject.set_allow_empty(False) + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + expect.that_bool(subject.allow_empty()).equals(False) + expect.that_collection(subject.default()).contains_exactly(["blue"]) + _expect_builds(expect, subject, "attr.string_list") + + _report_failures(name, env) + +_tests.append(_test_string_list) + +def _test_string_list_dict(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.StringListDict() + + expect.that_dict(subject.default()).contains_exactly({}) + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + expect.that_bool(subject.allow_empty()).equals(True) + _expect_builds(expect, subject, "attr.string_list_dict") + + subject.set_doc("doc") + subject.set_mandatory(True) + subject.default()["key"] = ["red"] + subject.set_allow_empty(False) + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + expect.that_bool(subject.allow_empty()).equals(False) + expect.that_dict(subject.default()).contains_exactly({"key": ["red"]}) + _expect_builds(expect, subject, "attr.string_list_dict") + + _report_failures(name, env) + +_tests.append(_test_string_list_dict) + +def attr_builders_test_suite(name): + test_suite( + name = name, + tests = _tests, + ) diff --git a/tests/builders/rule_builders_tests.bzl b/tests/builders/rule_builders_tests.bzl new file mode 100644 index 0000000000..a8ac31f4bf --- /dev/null +++ b/tests/builders/rule_builders_tests.bzl @@ -0,0 +1,257 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Tests for rule_builders.""" + +load("@rules_testing//lib:analysis_test.bzl", "analysis_test") +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("@rules_testing//lib:util.bzl", "TestingAspectInfo") +load("//python/private:attr_builders.bzl", "attrb") # buildifier: disable=bzl-visibility +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility +load("//python/private:rule_builders.bzl", "ruleb") # buildifier: disable=bzl-visibility + +RuleInfo = provider(doc = "test provider", fields = []) + +_tests = [] # analysis-phase tests +_basic_tests = [] # loading-phase tests + +fruit = ruleb.Rule( + implementation = lambda ctx: [RuleInfo()], + attrs = { + "color": attrb.String(default = "yellow"), + "fertilizers": attrb.LabelList( + allow_files = True, + ), + "flavors": attrb.StringList(), + "nope": attr.label( + # config.none is Bazel 8+ + cfg = config.none() if hasattr(config, "none") else None, + ), + "organic": lambda: attrb.Bool(), + "origin": lambda: attrb.Label(), + "size": lambda: attrb.Int(default = 10), + }, +).build() + +def _test_fruit_rule(name): + fruit( + name = name + "_subject", + flavors = ["spicy", "sweet"], + organic = True, + size = 5, + origin = labels.NONE, + fertilizers = [ + "nitrogen.txt", + "phosphorus.txt", + ], + ) + + analysis_test( + name = name, + target = name + "_subject", + impl = _test_fruit_rule_impl, + ) + +def _test_fruit_rule_impl(env, target): + attrs = target[TestingAspectInfo].attrs + env.expect.that_str(attrs.color).equals("yellow") + env.expect.that_collection(attrs.flavors).contains_exactly(["spicy", "sweet"]) + env.expect.that_bool(attrs.organic).equals(True) + env.expect.that_int(attrs.size).equals(5) + + # //python:none is an alias to //python/private:sentinel; we see the + # resolved value, not the intermediate alias + env.expect.that_target(attrs.origin).label().equals(Label("//python/private:sentinel")) + + env.expect.that_collection(attrs.fertilizers).transform( + desc = "target.label", + map_each = lambda t: t.label, + ).contains_exactly([ + Label(":nitrogen.txt"), + Label(":phosphorus.txt"), + ]) + +_tests.append(_test_fruit_rule) + +# NOTE: `Rule.build()` can't be called because it's not during the top-level +# bzl evaluation. +def _test_rule_api(env): + subject = ruleb.Rule() + expect = env.expect + + expect.that_dict(subject.attrs.map).contains_exactly({}) + expect.that_collection(subject.cfg.outputs()).contains_exactly([]) + expect.that_collection(subject.cfg.inputs()).contains_exactly([]) + expect.that_bool(subject.cfg.implementation()).equals(None) + expect.that_str(subject.doc()).equals("") + expect.that_dict(subject.exec_groups()).contains_exactly({}) + expect.that_bool(subject.executable()).equals(False) + expect.that_collection(subject.fragments()).contains_exactly([]) + expect.that_bool(subject.implementation()).equals(None) + expect.that_collection(subject.provides()).contains_exactly([]) + expect.that_bool(subject.test()).equals(False) + expect.that_collection(subject.toolchains()).contains_exactly([]) + + subject.attrs.update({ + "builder": attrb.String(), + "factory": lambda: attrb.String(), + }) + subject.attrs.put("put_factory", lambda: attrb.Int()) + subject.attrs.put("put_builder", attrb.Int()) + + expect.that_dict(subject.attrs.map).keys().contains_exactly([ + "factory", + "builder", + "put_factory", + "put_builder", + ]) + expect.that_collection(subject.attrs.map.values()).transform( + desc = "type() of attr value", + map_each = type, + ).contains_exactly(["struct", "struct", "struct", "struct"]) + + subject.set_doc("doc") + expect.that_str(subject.doc()).equals("doc") + + subject.exec_groups()["eg"] = ruleb.ExecGroup() + expect.that_dict(subject.exec_groups()).keys().contains_exactly(["eg"]) + + subject.set_executable(True) + expect.that_bool(subject.executable()).equals(True) + + subject.fragments().append("frag") + expect.that_collection(subject.fragments()).contains_exactly(["frag"]) + + impl = lambda: None + subject.set_implementation(impl) + expect.that_bool(subject.implementation()).equals(impl) + + subject.provides().append(RuleInfo) + expect.that_collection(subject.provides()).contains_exactly([RuleInfo]) + + subject.set_test(True) + expect.that_bool(subject.test()).equals(True) + + subject.toolchains().append(ruleb.ToolchainType()) + expect.that_collection(subject.toolchains()).has_size(1) + + expect.that_collection(subject.cfg.outputs()).contains_exactly([]) + expect.that_collection(subject.cfg.inputs()).contains_exactly([]) + expect.that_bool(subject.cfg.implementation()).equals(None) + + subject.cfg.set_implementation(impl) + expect.that_bool(subject.cfg.implementation()).equals(impl) + subject.cfg.add_inputs(Label("//some:input")) + expect.that_collection(subject.cfg.inputs()).contains_exactly([ + str(Label("//some:input")), + ]) + subject.cfg.add_outputs(Label("//some:output")) + expect.that_collection(subject.cfg.outputs()).contains_exactly([ + str(Label("//some:output")), + ]) + +_basic_tests.append(_test_rule_api) + +def _test_exec_group(env): + subject = ruleb.ExecGroup() + + env.expect.that_collection(subject.toolchains()).contains_exactly([]) + env.expect.that_collection(subject.exec_compatible_with()).contains_exactly([]) + env.expect.that_str(str(subject.build())).contains("ExecGroup") + + subject.toolchains().append(ruleb.ToolchainType(labels.NONE)) + subject.exec_compatible_with().append("//some:constraint") + env.expect.that_str(str(subject.build())).contains("ExecGroup") + +_basic_tests.append(_test_exec_group) + +def _test_toolchain_type(env): + subject = ruleb.ToolchainType() + + env.expect.that_str(subject.name()).equals(None) + env.expect.that_bool(subject.mandatory()).equals(True) + subject.set_name("//some:toolchain_type") + env.expect.that_str(str(subject.build())).contains("ToolchainType") + + subject.set_name("//some:toolchain_type") + subject.set_mandatory(False) + env.expect.that_str(subject.name()).equals("//some:toolchain_type") + env.expect.that_bool(subject.mandatory()).equals(False) + env.expect.that_str(str(subject.build())).contains("ToolchainType") + +_basic_tests.append(_test_toolchain_type) + +rule_with_toolchains = ruleb.Rule( + implementation = lambda ctx: [], + toolchains = [ + ruleb.ToolchainType("//tests/builders:tct_1", mandatory = False), + lambda: ruleb.ToolchainType("//tests/builders:tct_2", mandatory = False), + "//tests/builders:tct_3", + Label("//tests/builders:tct_4"), + ], + exec_groups = { + "eg1": ruleb.ExecGroup( + toolchains = [ + ruleb.ToolchainType("//tests/builders:tct_1", mandatory = False), + lambda: ruleb.ToolchainType("//tests/builders:tct_2", mandatory = False), + "//tests/builders:tct_3", + Label("//tests/builders:tct_4"), + ], + ), + "eg2": lambda: ruleb.ExecGroup(), + }, +).build() + +def _test_rule_with_toolchains(name): + rule_with_toolchains( + name = name + "_subject", + tags = ["manual"], # Can't be built without extra_toolchains set + ) + + analysis_test( + name = name, + impl = lambda env, target: None, + target = name + "_subject", + config_settings = { + "//command_line_option:extra_toolchains": [ + Label("//tests/builders:all"), + ], + }, + ) + +_tests.append(_test_rule_with_toolchains) + +rule_with_immutable_attrs = ruleb.Rule( + implementation = lambda ctx: [], + attrs = { + "foo": attr.string(), + }, +).build() + +def _test_rule_with_immutable_attrs(name): + rule_with_immutable_attrs(name = name + "_subject") + analysis_test( + name = name, + target = name + "_subject", + impl = lambda env, target: None, + ) + +_tests.append(_test_rule_with_immutable_attrs) + +def rule_builders_test_suite(name): + test_suite( + name = name, + basic_tests = _basic_tests, + tests = _tests, + ) diff --git a/tests/cc/current_py_cc_headers/BUILD.bazel b/tests/cc/current_py_cc_headers/BUILD.bazel index e2d6a1b521..21723b59c4 100644 --- a/tests/cc/current_py_cc_headers/BUILD.bazel +++ b/tests/cc/current_py_cc_headers/BUILD.bazel @@ -12,6 +12,30 @@ # See the License for the specific language governing permissions and # limitations under the License. +load("@rules_cc//cc:cc_binary.bzl", "cc_binary") +load("//python:py_test.bzl", "py_test") load(":current_py_cc_headers_tests.bzl", "current_py_cc_headers_test_suite") current_py_cc_headers_test_suite(name = "current_py_cc_headers_tests") + +cc_binary( + name = "bin_abi3", + srcs = ["bin_abi3.cc"], + defines = ["Py_LIMITED_API=0x030A0000"], + linkshared = True, + deps = [ + "//python/cc:current_py_cc_headers_abi3", + ], +) + +py_test( + name = "abi3_headers_linkage_test_py", + srcs = ["abi3_headers_linkage_test.py"], + data = [":bin_abi3"], + main = "abi3_headers_linkage_test.py", + target_compatible_with = ["@platforms//os:windows"], + deps = [ + "//python/runfiles", + "@dev_pip//pefile", + ], +) diff --git a/tests/cc/current_py_cc_headers/abi3_headers_linkage_test.py b/tests/cc/current_py_cc_headers/abi3_headers_linkage_test.py new file mode 100644 index 0000000000..2d64828278 --- /dev/null +++ b/tests/cc/current_py_cc_headers/abi3_headers_linkage_test.py @@ -0,0 +1,28 @@ +import sys +import unittest + +import pefile + +from python.runfiles import runfiles + + +class CheckLinkageTest(unittest.TestCase): + @unittest.skipUnless(sys.platform.startswith("win"), "requires windows") + def test_linkage_windows(self): + rf = runfiles.Create() + dll_path = rf.Rlocation( + "rules_python/tests/cc/current_py_cc_headers/bin_abi3.dll" + ) + pe = pefile.PE(dll_path) + if not hasattr(pe, "DIRECTORY_ENTRY_IMPORT"): + self.fail("No import directory found.") + + imported_dlls = [ + entry.dll.decode("utf-8").lower() for entry in pe.DIRECTORY_ENTRY_IMPORT + ] + python_dlls = [dll for dll in imported_dlls if dll.startswith("python3")] + self.assertEqual(python_dlls, ["python3.dll"]) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/cc/current_py_cc_headers/bin_abi3.cc b/tests/cc/current_py_cc_headers/bin_abi3.cc new file mode 100644 index 0000000000..0211dbbfc6 --- /dev/null +++ b/tests/cc/current_py_cc_headers/bin_abi3.cc @@ -0,0 +1,17 @@ +#include + +int SomeFunction() { + // Early return to prevent the broken code below from running. + if (true) { + return 0; + } + + // The below code won't actually run. We just reference some Python + // symbols so the compiler and linker do some work to verify they are + // able to resolve the symbols. + // To make it actually run, more custom initialization is necessary. + // See https://docs.python.org/3/c-api/intro.html#embedding-python + Py_Initialize(); + Py_Finalize(); + return 0; +} diff --git a/tests/cc/current_py_cc_headers/current_py_cc_headers_tests.bzl b/tests/cc/current_py_cc_headers/current_py_cc_headers_tests.bzl index d07d08ac61..f52d93e75f 100644 --- a/tests/cc/current_py_cc_headers/current_py_cc_headers_tests.bzl +++ b/tests/cc/current_py_cc_headers/current_py_cc_headers_tests.bzl @@ -22,39 +22,29 @@ load("//tests/support:support.bzl", "CC_TOOLCHAIN") _tests = [] -def _test_current_toolchain_headers(name): - analysis_test( - name = name, - impl = _test_current_toolchain_headers_impl, - target = "//python/cc:current_py_cc_headers", - config_settings = { - "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], - }, - attrs = { - "header": attr.label( - default = "//tests/support/cc_toolchains:fake_header.h", - allow_single_file = True, - ), - }, - ) - -def _test_current_toolchain_headers_impl(env, target): +def _verify_headers_target(env, target): # Check that the forwarded CcInfo looks vaguely correct. compilation_context = env.expect.that_target(target).provider( CcInfo, factory = cc_info_subject, ).compilation_context() - compilation_context.direct_headers().contains_exactly([ - env.ctx.file.header, - ]) - compilation_context.direct_public_headers().contains_exactly([ - env.ctx.file.header, - ]) + compilation_context.direct_headers().contains_exactly( + env.ctx.files.header_files, + ) + compilation_context.direct_public_headers().contains_exactly( + env.ctx.files.header_files, + ) + + # NOTE: Bazel 8 and lower put cc_library.includes into `.system_includes`, + # while Bazel 9 put it in `.includes`. Both result in the includes being + # added as system includes, so either is acceptable for the expected + # `#include ` to work. + includes = compilation_context.actual.includes.to_list() + compilation_context.actual.system_includes.to_list() # NOTE: The include dir gets added twice, once for the source path, # and once for the config-specific path. - compilation_context.system_includes().contains_at_least_predicates([ - matching.str_matches("*/fake_include"), + env.expect.that_collection(includes).contains_at_least_predicates([ + matching.str_matches("*/py_include"), ]) # Check that the forward DefaultInfo looks correct @@ -62,6 +52,27 @@ def _test_current_toolchain_headers_impl(env, target): matching.str_matches("*/cc_toolchains/data.txt"), ) +def _test_current_toolchain_headers(name): + analysis_test( + name = name, + impl = _test_current_toolchain_headers_impl, + target = "//python/cc:current_py_cc_headers", + config_settings = { + "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], + }, + attrs = { + "header_files": attr.label_list( + default = [ + "//tests/support/cc_toolchains:py_headers_files", + ], + allow_files = True, + ), + }, + ) + +def _test_current_toolchain_headers_impl(env, target): + _verify_headers_target(env, target) + _tests.append(_test_current_toolchain_headers) def _test_toolchain_is_registered_by_default(name): @@ -76,6 +87,29 @@ def _test_toolchain_is_registered_by_default_impl(env, target): _tests.append(_test_toolchain_is_registered_by_default) +def _test_current_toolchain_headers_abi3(name): + analysis_test( + name = name, + impl = _test_current_toolchain_headers_abi3_impl, + target = "//python/cc:current_py_cc_headers_abi3", + config_settings = { + "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], + }, + attrs = { + "header_files": attr.label_list( + default = [ + "//tests/support/cc_toolchains:py_headers_abi3_files", + ], + allow_files = True, + ), + }, + ) + +def _test_current_toolchain_headers_abi3_impl(env, target): + _verify_headers_target(env, target) + +_tests.append(_test_current_toolchain_headers_abi3) + def current_py_cc_headers_test_suite(name): test_suite( name = name, diff --git a/tests/cc/current_py_cc_libs/BUILD.bazel b/tests/cc/current_py_cc_libs/BUILD.bazel index 9f335990e6..6b4e80fc6a 100644 --- a/tests/cc/current_py_cc_libs/BUILD.bazel +++ b/tests/cc/current_py_cc_libs/BUILD.bazel @@ -12,22 +12,14 @@ # See the License for the specific language governing permissions and # limitations under the License. +load("@rules_cc//cc:cc_test.bzl", "cc_test") load(":current_py_cc_libs_tests.bzl", "current_py_cc_libs_test_suite") current_py_cc_libs_test_suite(name = "current_py_cc_libs_tests") -# buildifier: disable=native-cc cc_test( name = "python_libs_linking_test", srcs = ["python_libs_linking_test.cc"], - # Windows fails with linking errors, but its not clear why; someone - # with more C + Windows experience will have to figure it out. - # - rickeylev@ - target_compatible_with = select({ - "@platforms//os:linux": [], - "@platforms//os:osx": [], - "//conditions:default": ["@platforms//:incompatible"], - }), deps = [ "@rules_python//python/cc:current_py_cc_headers", "@rules_python//python/cc:current_py_cc_libs", @@ -39,15 +31,12 @@ cc_test( # the expected Windows libraries are all present in the expected location. # Since we define the Py_LIMITED_API macro, we expect the linker to go search # for libs/python3.lib. -# buildifier: disable=native-cc cc_test( - name = "python_libs_linking_windows_test", + name = "python_abi3_libs_linking_test", srcs = ["python_libs_linking_test.cc"], defines = ["Py_LIMITED_API=0x030A0000"], - env = {"HELLO": "world"}, - target_compatible_with = ["@platforms//os:windows"], deps = [ - "@rules_python//python/cc:current_py_cc_headers", + "@rules_python//python/cc:current_py_cc_headers_abi3", "@rules_python//python/cc:current_py_cc_libs", ], ) diff --git a/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl b/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl index 0419a04a45..5b6f8e151d 100644 --- a/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl +++ b/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl @@ -28,9 +28,13 @@ def _test_py_cc_toolchain(name): impl = _test_py_cc_toolchain_impl, target = "//tests/support/cc_toolchains:fake_py_cc_toolchain_impl", attrs = { - "header": attr.label( - default = "//tests/support/cc_toolchains:fake_header.h", - allow_single_file = True, + "header_abi3_files": attr.label_list( + default = ["//tests/support/cc_toolchains:py_headers_abi3_files"], + allow_files = True, + ), + "header_files": attr.label_list( + default = ["//tests/support/cc_toolchains:py_headers_files"], + allow_files = True, ), }, ) @@ -43,24 +47,32 @@ def _test_py_cc_toolchain_impl(env, target): meta = env.expect.meta.derive(expr = "py_cc_toolchain_info"), ) toolchain.python_version().equals("3.999") + toolchain.abi_flags().equals("") + # ===== Verify headers info ===== headers_providers = toolchain.headers().providers_map() headers_providers.keys().contains_exactly(["CcInfo", "DefaultInfo"]) cc_info = headers_providers.get("CcInfo", factory = cc_info_subject) compilation_context = cc_info.compilation_context() - compilation_context.direct_headers().contains_exactly([ - env.ctx.file.header, - ]) - compilation_context.direct_public_headers().contains_exactly([ - env.ctx.file.header, - ]) + compilation_context.direct_headers().contains_exactly( + env.ctx.files.header_files, + ) + compilation_context.direct_public_headers().contains_exactly( + env.ctx.files.header_files, + ) + + # NOTE: Bazel 8 and lower put cc_library.includes into `.system_includes`, + # while Bazel 9 put it in `.includes`. Both result in the includes being + # added as system includes, so either is acceptable for the expected + # `#include ` to work. + includes = compilation_context.actual.includes.to_list() + compilation_context.actual.system_includes.to_list() # NOTE: The include dir gets added twice, once for the source path, - # and once for the config-specific path, but we don't care about that. - compilation_context.system_includes().contains_at_least_predicates([ - matching.str_matches("*/fake_include"), + # and once for the config-specific path. + env.expect.that_collection(includes).contains_at_least_predicates([ + matching.str_matches("*/py_include"), ]) default_info = headers_providers.get("DefaultInfo", factory = subjects.default_info) @@ -68,6 +80,32 @@ def _test_py_cc_toolchain_impl(env, target): matching.str_matches("*/cc_toolchains/data.txt"), ) + # ===== Verify headers_abi3 info ===== + headers_abi3_providers = toolchain.headers_abi3().providers_map() + headers_abi3_providers.keys().contains_exactly(["CcInfo", "DefaultInfo"]) + + cc_info = headers_abi3_providers.get("CcInfo", factory = cc_info_subject) + + compilation_context = cc_info.compilation_context() + compilation_context.direct_headers().contains_exactly( + env.ctx.files.header_abi3_files, + ) + compilation_context.direct_public_headers().contains_exactly( + env.ctx.files.header_abi3_files, + ) + + # NOTE: Bazel 8 and lower put cc_library.includes into `.system_includes`, + # while Bazel 9 put it in `.includes`. Both result in the includes being + # added as system includes, so either is acceptable for the expected + # `#include ` to work. + includes = compilation_context.actual.includes.to_list() + compilation_context.actual.system_includes.to_list() + + default_info = headers_abi3_providers.get("DefaultInfo", factory = subjects.default_info) + default_info.runfiles().contains_predicate( + matching.str_matches("*/cc_toolchains/data.txt"), + ) + + # ===== Verify libs info ===== libs_providers = toolchain.libs().providers_map() libs_providers.keys().contains_exactly(["CcInfo", "DefaultInfo"]) @@ -81,13 +119,45 @@ def _test_py_cc_toolchain_impl(env, target): matching.str_matches("/libpython3."), ) + # ===== Verify PEP 508 platform markers & SOABI ===== + toolchain.sys_platform().equals("linux") + toolchain.platform_machine().equals("x86_64") + toolchain.platform_tag().equals("x86_64-linux-gnu") + toolchain.soabi().equals("cpython-3999-x86_64-linux-gnu") + _tests.append(_test_py_cc_toolchain) +def _test_custom_pep508_markers(name): + py_cc_toolchain( + name = name + "_subject", + headers = "//tests/support/cc_toolchains:py_headers", + platform_machine = "arm64", + python_version = "3.11", + sys_platform = "darwin", + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_custom_pep508_markers_impl, + ) + +def _test_custom_pep508_markers_impl(env, target): + toolchain = PyCcToolchainInfoSubject.new( + target[platform_common.ToolchainInfo].py_cc_toolchain, + meta = env.expect.meta.derive(expr = "py_cc_toolchain_info"), + ) + toolchain.sys_platform().equals("darwin") + toolchain.platform_machine().equals("arm64") + toolchain.platform_tag().equals("darwin") + toolchain.soabi().equals("cpython-311-darwin") + +_tests.append(_test_custom_pep508_markers) + def _test_libs_optional(name): py_cc_toolchain( name = name + "_subject", libs = None, - headers = "//tests/support/cc_toolchains:fake_headers", + headers = "//tests/support/cc_toolchains:py_headers", python_version = "4.5", ) analysis_test( diff --git a/tests/cc/py_extension/BUILD.bazel b/tests/cc/py_extension/BUILD.bazel new file mode 100644 index 0000000000..e6edde36e6 --- /dev/null +++ b/tests/cc/py_extension/BUILD.bazel @@ -0,0 +1,194 @@ +load("@rules_cc//cc:cc_library.bzl", "cc_library") +load("@rules_cc//cc:cc_shared_library.bzl", "cc_shared_library") +load("//python:py_test.bzl", "py_test") + +# buildifier: disable=bzl-visibility +load("//python/cc:py_extension.bzl", "py_extension") +load(":py_extension_build_test.bzl", "py_extension_build_test") + +package( + default_visibility = ["//tests/cc/py_extension:__subpackages__"], +) + +licenses(["notice"]) + +##### + +py_extension_build_test( + # An extension defined solely by source files, with no deps + name = "ext_source", + srcs = ["ext_source.c"], +) + +##### + +py_extension_build_test( + # A python extension that explicitly includes current_py_cc_headers in deps. + # Verifies that explicit current_py_cc_headers does not cause a duplicate label error. + name = "ext_explicit_headers", + srcs = ["ext_source.c"], + deps = [ + "@rules_python//python/cc:current_py_cc_headers", + ], +) + +##### + +py_extension_build_test( + # A python extension that includes current_py_cc_headers inside a select() in deps. + # Verifies that select() in deps works cleanly without duplicate label or type error. + name = "ext_explicit_headers_select", + srcs = ["ext_source.c"], + deps = select({ + "//conditions:default": ["@rules_python//python/cc:current_py_cc_headers"], + }), +) + +##### + +py_extension_build_test( + # A python extension that gets its code from a statically-linked library + name = "ext_static", + deps = [":static_dep"], +) + +cc_library( + name = "static_dep", + srcs = ["static_dep.c"], + hdrs = ["static_dep.h"], +) + +##### + +py_extension_build_test( + # An extension that also depends on a data file + name = "ext_with_data", + data = ["some_data.txt"], + deps = [":static_dep"], +) + +##### + +py_extension_build_test( + # A python extension that dynamically links to another shared library + name = "ext_shared", + dynamic_deps = [ + ":add_one_shared", + ], + imports = ["."], + deps = [ + # + ":ext_shared_impl", + ], +) + +cc_library( + name = "ext_shared_impl", + srcs = ["ext_shared.c"], + copts = [ + # Gemini says PIC is needed + "-fPIC", + "-fvisibility=hidden", + ], + deps = [ + #":add_one_headers", + # todo: if we put this here, we statically link add_one into the + # extension. + ":add_one_impl", + "@rules_python//python/cc:current_py_cc_headers", + ], +) + +cc_shared_library( + name = "add_one_shared", + user_link_flags = select({ + "@platforms//os:windows": ["/EXPORT:add_one"], + "//conditions:default": [], + }), + deps = [":add_one_impl"], +) + +cc_library( + name = "add_one_headers", + hdrs = ["add_one.h"], +) + +cc_library( + name = "add_one_impl", + srcs = ["add_one.c"], + deps = [ + ":add_one_headers", + ":add_one_helper", + ], +) + +cc_library( + name = "add_one_helper", + srcs = ["add_one_helper.c"], + hdrs = ["add_one_helper.h"], +) + +##### + +py_extension_build_test( + # An extension that uses the Python limited API + name = "ext_limited", + py_limited_api = "3.8", + deps = [":ext_limited_impl"], +) + +cc_library( + name = "ext_limited_impl", + srcs = ["ext_limited.c"], + copts = [ + "-fPIC", + "-fvisibility=hidden", + ], + defines = ["Py_LIMITED_API=0x3080000"], + deps = [ + "@rules_python//python/cc:current_py_cc_headers", + ], +) + +##### + +py_extension_build_test( + # An extension with its PyInit_* function in a static dependency + name = "ext_init_in_dep", + deps = [":ext_init_in_dep_impl"], +) + +cc_library( + name = "ext_init_in_dep_impl", + srcs = ["ext_init_in_dep.c"], + deps = [ + "@rules_python//python/cc:current_py_cc_headers", + ], +) + +##### + +py_test( + name = "py_extension_test", + srcs = ["py_extension_test.py"], + deps = [ + ":ext_shared", + "@dev_pip//pyelftools", + "@rules_python//python/runfiles", + ], +) + +##### + +py_extension( + name = "ext_pkg_test", + srcs = ["ext_pkg_test.c"], +) + +py_test( + name = "py_extension_pkg_test", + srcs = ["py_extension_pkg_test.py"], + deps = [ + ":ext_pkg_test", + ], +) diff --git a/tests/cc/py_extension/add_one.c b/tests/cc/py_extension/add_one.c new file mode 100644 index 0000000000..7da8f6796e --- /dev/null +++ b/tests/cc/py_extension/add_one.c @@ -0,0 +1,7 @@ + +#include "add_one_helper.h" + +int add_one(int x) { + x = add_one_helper(x); + return x + 1; +} diff --git a/tests/cc/py_extension/add_one.h b/tests/cc/py_extension/add_one.h new file mode 100644 index 0000000000..eda0abf7e5 --- /dev/null +++ b/tests/cc/py_extension/add_one.h @@ -0,0 +1,6 @@ +#ifndef TESTS_CC_PY_EXTENSION_DYN_DEP_A_H_ +#define TESTS_CC_PY_EXTENSION_DYN_DEP_A_H_ + +int add_one(int x); + +#endif // TESTS_CC_PY_EXTENSION_DYN_DEP_A_H_ diff --git a/tests/cc/py_extension/add_one_helper.c b/tests/cc/py_extension/add_one_helper.c new file mode 100644 index 0000000000..21d19a5823 --- /dev/null +++ b/tests/cc/py_extension/add_one_helper.c @@ -0,0 +1,7 @@ + + +#include "add_one_helper.h" + +int add_one_helper(int i) { + return i + 1; +} diff --git a/tests/cc/py_extension/add_one_helper.h b/tests/cc/py_extension/add_one_helper.h new file mode 100644 index 0000000000..5524d3077f --- /dev/null +++ b/tests/cc/py_extension/add_one_helper.h @@ -0,0 +1,2 @@ + +int add_one_helper(int i); diff --git a/tests/cc/py_extension/dependency_graph/BUILD.bazel b/tests/cc/py_extension/dependency_graph/BUILD.bazel new file mode 100644 index 0000000000..47bb5062a0 --- /dev/null +++ b/tests/cc/py_extension/dependency_graph/BUILD.bazel @@ -0,0 +1,12 @@ +load(":dependency_graph_tests.bzl", "dependency_graph_test_suite") + +package( + default_testonly = True, + default_visibility = ["//visibility:private"], +) + +licenses(["notice"]) + +dependency_graph_test_suite( + name = "dependency_graph_tests", +) diff --git a/tests/cc/py_extension/dependency_graph/dependency_graph_tests.bzl b/tests/cc/py_extension/dependency_graph/dependency_graph_tests.bzl new file mode 100644 index 0000000000..269c29eacb --- /dev/null +++ b/tests/cc/py_extension/dependency_graph/dependency_graph_tests.bzl @@ -0,0 +1,236 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Parity tests comparing cc_shared_library and py_extension behavior.""" + +load("@rules_cc//cc:cc_library.bzl", "cc_library") +load("@rules_cc//cc:cc_shared_library.bzl", "cc_shared_library") +load("@rules_cc//cc/common:cc_shared_library_info.bzl", "CcSharedLibraryInfo") +load("@rules_testing//lib:analysis_test.bzl", "analysis_test", "test_suite") +load("@rules_testing//lib:util.bzl", "util") + +_tests = [] + +# For tests 1 and 2 +def _create_dynamic_deps_helpers(name): + util.helper_target( + cc_library, + name = name + "_libC", + srcs = ["test_lib_c.c"], + hdrs = ["test_symbols.h"], + copts = ["-fPIC"], + ) + util.helper_target( + cc_shared_library, + name = name + "_cslC", + deps = [":" + name + "_libC"], + ) + util.helper_target( + cc_library, + name = name + "_libB", + srcs = ["test_lib_b.c"], + hdrs = ["test_symbols.h"], + copts = ["-fPIC"], + deps = [":" + name + "_libC"], + ) + util.helper_target( + cc_shared_library, + name = name + "_cslB", + deps = [":" + name + "_libB"], + dynamic_deps = [":" + name + "_cslC"], + ) + util.helper_target( + cc_library, + name = name + "_libA", + srcs = ["test_lib_a.c"], + hdrs = ["test_symbols.h"], + copts = ["-fPIC"], + deps = [":" + name + "_libB", ":" + name + "_libC"], + ) + +# Test 1: CSL A -> CSL B -> CSL C (Dynamic deps) +def _test_csl_dynamic_deps_top(name): + _create_dynamic_deps_helpers(name) + util.helper_target( + cc_shared_library, + name = name + "_cslA", + deps = [":" + name + "_libA"], + dynamic_deps = [":" + name + "_cslB", ":" + name + "_cslC"], + ) + analysis_test( + name = name, + target = name + "_cslA", + impl = _csl_dynamic_deps_test_impl, + ) + +_tests.append(_test_csl_dynamic_deps_top) + +def _csl_dynamic_deps_test_impl(env, target): + env.expect.that_target(target).has_provider(CcSharedLibraryInfo) + csl_info = target[CcSharedLibraryInfo] + + # Derive labels + test_name = target.label.name[:-5] # remove "_cslA" + lib_a_label = target.label.same_package_label(test_name + "_libA") + lib_b_label = target.label.same_package_label(test_name + "_libB") + lib_c_label = target.label.same_package_label(test_name + "_libC") + + env.expect.that_collection([str(e) for e in csl_info.exports]).contains_exactly([str(lib_a_label)]) + static_libs = [str(lbl) for lbl in csl_info.link_once_static_libs] + env.expect.that_collection(static_libs).contains(str(lib_a_label)) + env.expect.that_collection(static_libs).contains_none_of([str(lib_b_label), str(lib_c_label)]) + +# Test 2: py_extension A -> CSL B -> CSL C (Dynamic deps) +def _test_pyext_dynamic_deps_cslB(name): + _create_dynamic_deps_helpers(name) + analysis_test( + name = name, + target = name + "_cslB", + impl = _cslB_deps_test_impl, + ) + +_tests.append(_test_pyext_dynamic_deps_cslB) + +def _test_pyext_dynamic_deps_cslC(name): + _create_dynamic_deps_helpers(name) + analysis_test( + name = name, + target = name + "_cslC", + impl = _cslC_deps_test_impl, + ) + +_tests.append(_test_pyext_dynamic_deps_cslC) + +def _cslC_deps_test_impl(env, target): + env.expect.that_target(target).has_provider(CcSharedLibraryInfo) + csl_info = target[CcSharedLibraryInfo] + + # Derive labels + test_name = target.label.name[:-5] # remove "_cslC" + lib_c_label = target.label.same_package_label(test_name + "_libC") + + env.expect.that_collection([str(e) for e in csl_info.exports]).contains_exactly([str(lib_c_label)]) + env.expect.that_collection([str(lbl) for lbl in csl_info.link_once_static_libs]).contains_exactly([str(lib_c_label)]) + +def _cslB_deps_test_impl(env, target): + env.expect.that_target(target).has_provider(CcSharedLibraryInfo) + csl_info = target[CcSharedLibraryInfo] + + # Derive labels + test_name = target.label.name[:-5] # remove "_cslB" + lib_b_label = target.label.same_package_label(test_name + "_libB") + lib_c_label = target.label.same_package_label(test_name + "_libC") + csl_c_label = target.label.same_package_label(test_name + "_cslC") + + env.expect.that_collection([str(e) for e in csl_info.exports]).contains_exactly([str(lib_b_label)]) + static_libs = [str(lbl) for lbl in csl_info.link_once_static_libs] + env.expect.that_collection(static_libs).contains(str(lib_b_label)) + env.expect.that_collection(static_libs).contains_none_of([str(lib_c_label)]) + + dynamic_deps = [str(d.linker_input.owner) for d in csl_info.dynamic_deps.to_list()] + env.expect.that_collection(dynamic_deps).contains(str(csl_c_label)) + +# For tests 3 and 4 +def _create_static_sharing_helpers(name): + util.helper_target( + cc_library, + name = name + "_libC", + srcs = ["test_lib_c.c"], + hdrs = ["test_symbols.h"], + copts = ["-fPIC"], + ) + util.helper_target( + cc_library, + name = name + "_libB", + srcs = ["test_lib_b.c"], + hdrs = ["test_symbols.h"], + copts = ["-fPIC"], + deps = [":" + name + "_libC"], + ) + util.helper_target( + cc_shared_library, + name = name + "_cslB", + deps = [":" + name + "_libB", ":" + name + "_libC"], + ) + util.helper_target( + cc_library, + name = name + "_libA", + srcs = ["test_lib_a.c"], + hdrs = ["test_symbols.h"], + copts = ["-fPIC"], + deps = [":" + name + "_libB", ":" + name + "_libC"], + ) + +# Test 3: CSL A -> CSL B, CL C (Static sharing) +def _test_csl_static_sharing_top(name): + _create_static_sharing_helpers(name) + util.helper_target( + cc_shared_library, + name = name + "_cslA", + deps = [":" + name + "_libA"], + dynamic_deps = [":" + name + "_cslB"], + ) + analysis_test( + name = name, + target = name + "_cslA", + impl = _csl_static_sharing_test_impl, + ) + +_tests.append(_test_csl_static_sharing_top) + +def _csl_static_sharing_test_impl(env, target): + env.expect.that_target(target).has_provider(CcSharedLibraryInfo) + csl_info = target[CcSharedLibraryInfo] + + # Derive labels + test_name = target.label.name[:-5] # remove "_cslA" + lib_a_label = target.label.same_package_label(test_name + "_libA") + lib_b_label = target.label.same_package_label(test_name + "_libB") + lib_c_label = target.label.same_package_label(test_name + "_libC") + + env.expect.that_collection([str(e) for e in csl_info.exports]).contains_exactly([str(lib_a_label)]) + static_libs = [str(lbl) for lbl in csl_info.link_once_static_libs] + env.expect.that_collection(static_libs).contains(str(lib_a_label)) + env.expect.that_collection(static_libs).contains_none_of([str(lib_b_label), str(lib_c_label)]) + +# Test 4: Same as 3, but A is py_extension + +def _test_pyext_static_sharing_cslB(name): + _create_static_sharing_helpers(name) + analysis_test( + name = name, + target = name + "_cslB", + impl = _cslB_static_sharing_test_impl, + ) + +_tests.append(_test_pyext_static_sharing_cslB) + +def _cslB_static_sharing_test_impl(env, target): + env.expect.that_target(target).has_provider(CcSharedLibraryInfo) + csl_info = target[CcSharedLibraryInfo] + + # Derive labels + test_name = target.label.name[:-5] # remove "_cslB" + lib_b_label = target.label.same_package_label(test_name + "_libB") + lib_c_label = target.label.same_package_label(test_name + "_libC") + + env.expect.that_collection([str(e) for e in csl_info.exports]).contains_exactly([str(lib_b_label), str(lib_c_label)]) + static_libs = [str(lbl) for lbl in csl_info.link_once_static_libs] + env.expect.that_collection(static_libs).contains_exactly([str(lib_b_label), str(lib_c_label)]) + +def dependency_graph_test_suite(name): + test_suite( + name = name, + tests = _tests, + ) diff --git a/tests/cc/py_extension/dependency_graph/test_lib_a.c b/tests/cc/py_extension/dependency_graph/test_lib_a.c new file mode 100644 index 0000000000..19e2f489bf --- /dev/null +++ b/tests/cc/py_extension/dependency_graph/test_lib_a.c @@ -0,0 +1,6 @@ +#include "test_symbols.h" + +void fnA() { + fnB(); + fnC(); +} diff --git a/tests/cc/py_extension/dependency_graph/test_lib_b.c b/tests/cc/py_extension/dependency_graph/test_lib_b.c new file mode 100644 index 0000000000..3621587c1c --- /dev/null +++ b/tests/cc/py_extension/dependency_graph/test_lib_b.c @@ -0,0 +1,5 @@ +#include "test_symbols.h" + +void fnB() { + fnC(); +} diff --git a/tests/cc/py_extension/dependency_graph/test_lib_c.c b/tests/cc/py_extension/dependency_graph/test_lib_c.c new file mode 100644 index 0000000000..99941576dd --- /dev/null +++ b/tests/cc/py_extension/dependency_graph/test_lib_c.c @@ -0,0 +1,6 @@ +#include "test_symbols.h" +#include + +void fnC() { + printf("fnC\n"); +} diff --git a/tests/cc/py_extension/dependency_graph/test_symbols.h b/tests/cc/py_extension/dependency_graph/test_symbols.h new file mode 100644 index 0000000000..59ab3b02e7 --- /dev/null +++ b/tests/cc/py_extension/dependency_graph/test_symbols.h @@ -0,0 +1,8 @@ +#ifndef TEST_SYMBOLS_H +#define TEST_SYMBOLS_H + +void fnC(); +void fnB(); +void fnA(); + +#endif // TEST_SYMBOLS_H diff --git a/tests/cc/py_extension/ext_init_in_dep.c b/tests/cc/py_extension/ext_init_in_dep.c new file mode 100644 index 0000000000..a6d32ba0b2 --- /dev/null +++ b/tests/cc/py_extension/ext_init_in_dep.c @@ -0,0 +1,20 @@ + +#include + +// No methods defined; we're just testing the init function. +static PyMethodDef ModuleMethods[] = { + {NULL, NULL, 0, NULL} /* Sentinel */ +}; + +static struct PyModuleDef ext_init_in_dep_module = { + PyModuleDef_HEAD_INIT, + "ext_init_in_dep", /* name of module */ + NULL, /* module documentation, may be NULL */ + -1, /* size of per-interpreter state of the module, + or -1 if the module keeps state in global variables. */ + ModuleMethods +}; + +PyMODINIT_FUNC PyInit_ext_init_in_dep(void) { + return PyModule_Create(&ext_init_in_dep_module); +} diff --git a/tests/cc/py_extension/ext_limited.c b/tests/cc/py_extension/ext_limited.c new file mode 100644 index 0000000000..f3622c5824 --- /dev/null +++ b/tests/cc/py_extension/ext_limited.c @@ -0,0 +1,22 @@ +#include + +static PyObject* get_limited_api_version(PyObject* self, PyObject* args) { + return PyUnicode_FromFormat("0x%08x", Py_LIMITED_API); +} + +static PyMethodDef ModuleMethods[] = { + {"get_limited_api_version", get_limited_api_version, METH_NOARGS, "Get the version of the limited API this extension was compiled against."}, + {NULL, NULL, 0, NULL} +}; + +static struct PyModuleDef ext_limited_module = { + PyModuleDef_HEAD_INIT, + "ext_limited", + NULL, + -1, + ModuleMethods +}; + +PyMODINIT_FUNC PyInit_ext_limited(void) { + return PyModule_Create(&ext_limited_module); +} diff --git a/tests/cc/py_extension/ext_pkg_test.c b/tests/cc/py_extension/ext_pkg_test.c new file mode 100644 index 0000000000..c151f0162b --- /dev/null +++ b/tests/cc/py_extension/ext_pkg_test.c @@ -0,0 +1,22 @@ +#include + +static PyObject* get_magic_number(PyObject* self, PyObject* args) { + return PyLong_FromLong(42); +} + +static PyMethodDef ModuleMethods[] = { + {"get_magic_number", get_magic_number, METH_NOARGS, "Returns 42."}, + {NULL, NULL, 0, NULL} +}; + +static struct PyModuleDef ext_pkg_test_module = { + PyModuleDef_HEAD_INIT, + "ext_pkg_test", + NULL, + -1, + ModuleMethods +}; + +PyMODINIT_FUNC PyInit_ext_pkg_test(void) { + return PyModule_Create(&ext_pkg_test_module); +} diff --git a/tests/cc/py_extension/ext_shared.c b/tests/cc/py_extension/ext_shared.c new file mode 100644 index 0000000000..4b79a6da2f --- /dev/null +++ b/tests/cc/py_extension/ext_shared.c @@ -0,0 +1,33 @@ +#include + +#include "tests/cc/py_extension/add_one.h" + +// A simple function that returns a Python integer. +static PyObject* do_alpha(PyObject* self, PyObject* args) { + return PyLong_FromLong(add_one(41)); +} + +// Method definition object for this extension, these are the functions +// that will be available in the module. +static PyMethodDef ModuleMethods[] = { + {"do_alpha", do_alpha, METH_NOARGS, "A simple C function."}, + {NULL, NULL, 0, NULL} /* Sentinel */ +}; + +// Module definition +// The arguments of this structure tell Python what to call your extension, +// what its methods are and where to look for its method definitions. +static struct PyModuleDef ext_shared_module = { + PyModuleDef_HEAD_INIT, + "ext_shared", /* name of module */ + NULL, /* module documentation, may be NULL */ + -1, /* size of per-interpreter state of the module, + or -1 if the module keeps state in global variables. */ + ModuleMethods +}; + +// The module init function. This must be exported and retained in the +// shared library output. +PyMODINIT_FUNC PyInit_ext_shared(void) { + return PyModule_Create(&ext_shared_module); +} diff --git a/tests/cc/py_extension/ext_source.c b/tests/cc/py_extension/ext_source.c new file mode 100644 index 0000000000..df0239df3a --- /dev/null +++ b/tests/cc/py_extension/ext_source.c @@ -0,0 +1,33 @@ + +#include + + +static PyObject* calc_one_plus_two(PyObject* self, PyObject* args) { + return PyLong_FromLong(1 + 2); +} + + +// Method definition object for this extension, these are the functions +// that will be available in the module. +static PyMethodDef ModuleMethods[] = { + {"calc_one_plus_two", calc_one_plus_two, METH_NOARGS, "A simple C function."}, + {NULL, NULL, 0, NULL} /* Sentinel */ +}; + +// Module definition +// The arguments of this structure tell Python what to call your extension, +// what its methods are and where to look for its method definitions. +static struct PyModuleDef ext_source_module = { + PyModuleDef_HEAD_INIT, + "ext_source", /* name of module */ + NULL, /* module documentation, may be NULL */ + -1, /* size of per-interpreter state of the module, + or -1 if the module keeps state in global variables. */ + ModuleMethods +}; + +// The module init function. This must be exported and retained in the +// shared library output. +PyMODINIT_FUNC PyInit_ext_source(void) { + return PyModule_Create(&ext_source_module); +} diff --git a/tests/cc/py_extension/ext_static.c b/tests/cc/py_extension/ext_static.c new file mode 100644 index 0000000000..500c52db00 --- /dev/null +++ b/tests/cc/py_extension/ext_static.c @@ -0,0 +1 @@ +/* A no-op C extension for static linking tests. */ diff --git a/tests/cc/py_extension/py_extension/BUILD.bazel b/tests/cc/py_extension/py_extension/BUILD.bazel new file mode 100644 index 0000000000..0d64c2bf9d --- /dev/null +++ b/tests/cc/py_extension/py_extension/BUILD.bazel @@ -0,0 +1,12 @@ +load(":py_extension_tests.bzl", "py_extension_analysis_test_suite") + +package( + default_testonly = True, + default_visibility = ["//visibility:private"], +) + +licenses(["notice"]) + +py_extension_analysis_test_suite( + name = "py_extension_analysis_tests", +) diff --git a/tests/cc/py_extension/py_extension/py_extension_tests.bzl b/tests/cc/py_extension/py_extension/py_extension_tests.bzl new file mode 100644 index 0000000000..ef2a2d8b6e --- /dev/null +++ b/tests/cc/py_extension/py_extension/py_extension_tests.bzl @@ -0,0 +1,102 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Tests for py_extension.""" + +load("@rules_testing//lib:analysis_test.bzl", "analysis_test", "test_suite") +load("@rules_testing//lib:truth.bzl", "matching") +load("//python/private:py_info.bzl", "PyInfo") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_static_deps_impl(env, target): + env.expect.that_target(target).has_provider(PyInfo) + py_info = target[PyInfo] + + # The extension should be in PyInfo + env.expect.that_collection(py_info.transitive_sources.to_list()).has_size(1) + env.expect.that_depset_of_files(py_info.transitive_sources).contains_predicate( + matching.file_path_matches("ext_static.*311-*"), + ) + +def _test_static_deps(name): + analysis_test( + name = name, + impl = _test_static_deps_impl, + target = "//tests/cc/py_extension:ext_static", + ) + +_tests.append(_test_static_deps) + +def _test_data_deps_impl(env, target): + env.expect.that_target(target).has_provider(PyInfo) + + # Check that data file is in runfiles + default_info = target[DefaultInfo] + env.expect.that_depset_of_files(default_info.default_runfiles.files).contains_predicate( + matching.file_basename_equals("some_data.txt"), + ) + +def _test_data_deps(name): + analysis_test( + name = name, + impl = _test_data_deps_impl, + target = "//tests/cc/py_extension:ext_with_data", + ) + +_tests.append(_test_data_deps) + +def _test_dynamic_deps_impl(env, target): + env.expect.that_target(target).has_provider(PyInfo) + py_info = target[PyInfo] + + # The extension should be in PyInfo + env.expect.that_collection(py_info.transitive_sources.to_list()).has_size(1) + env.expect.that_depset_of_files(py_info.transitive_sources).contains_predicate( + matching.file_path_matches("ext_shared.*311-*"), + ) + +def _test_dynamic_deps(name): + analysis_test( + name = name, + impl = _test_dynamic_deps_impl, + target = "//tests/cc/py_extension:ext_shared", + ) + +_tests.append(_test_dynamic_deps) + +def _test_musl_platform_impl(env, target): + env.expect.that_target(target).has_provider(PyInfo) + py_info = target[PyInfo] + env.expect.that_depset_of_files(py_info.transitive_sources).contains_predicate( + matching.file_path_matches("ext_static.*311-*"), + ) + +def _test_musl_platform(name): + analysis_test( + name = name, + impl = _test_musl_platform_impl, + target = "//tests/cc/py_extension:ext_static", + config_settings = { + str(Label("//python/config_settings:py_linux_libc")): "musl", + }, + ) + +_tests.append(_test_musl_platform) + +def py_extension_analysis_test_suite(name): + test_suite( + name = name, + tests = _tests, + ) diff --git a/tests/cc/py_extension/py_extension_build_test.bzl b/tests/cc/py_extension/py_extension_build_test.bzl new file mode 100644 index 0000000000..18cca2204b --- /dev/null +++ b/tests/cc/py_extension/py_extension_build_test.bzl @@ -0,0 +1,15 @@ +"""Macro helper for py_extension build testing.""" + +load("@bazel_skylib//rules:build_test.bzl", "build_test") +load("//python/cc:py_extension.bzl", "py_extension") + +def py_extension_build_test(name, **kwargs): + """Creates a py_extension target and a build_test verifying it builds.""" + py_extension( + name = name, + **kwargs + ) + build_test( + name = name + "_build_test", + targets = [":" + name], + ) diff --git a/tests/cc/py_extension/py_extension_pkg_test.py b/tests/cc/py_extension/py_extension_pkg_test.py new file mode 100644 index 0000000000..e3176d6a6c --- /dev/null +++ b/tests/cc/py_extension/py_extension_pkg_test.py @@ -0,0 +1,16 @@ +import unittest + +from tests.cc.py_extension import ext_pkg_test + + +class PyExtensionPkgTest(unittest.TestCase): + def test_import_via_package(self): + self.assertEqual(ext_pkg_test.get_magic_number(), 42) + + def test_direct_import(self): + with self.assertRaises(ModuleNotFoundError): + import ext_pkg_test # buildifier: disable=g-import-not-at-top # noqa: F401 + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/cc/py_extension/py_extension_test.py b/tests/cc/py_extension/py_extension_test.py new file mode 100644 index 0000000000..d82fe22bcc --- /dev/null +++ b/tests/cc/py_extension/py_extension_test.py @@ -0,0 +1,47 @@ +import os +import sys +import unittest + +import ext_shared +from elftools.elf.dynamic import DynamicSection +from elftools.elf.elffile import ELFFile + + +class PyExtensionTest(unittest.TestCase): + @unittest.skipIf( + sys.platform != "linux", "ELF inspection is only supported on Linux" + ) + def test_inspect_elf(self): + ext_path = ext_shared.__file__ + self.assertTrue( + os.path.exists(ext_path), f"Could not find ext_shared.so at {ext_path}" + ) + + with open(ext_path, "rb") as f: + elf = ELFFile(f) + + # Check for DT_NEEDED entry for the dynamic library + dynamic_section = elf.get_section_by_name(".dynamic") + self.assertIsNotNone(dynamic_section) + self.assertTrue(isinstance(dynamic_section, DynamicSection)) + + needed_libs = [ + tag.needed + for tag in dynamic_section.iter_tags() + if tag.entry.d_tag == "DT_NEEDED" + ] + self.assertIn("libadd_one_shared.so", needed_libs) + + # Check for the PyInit symbol + dynsym_section = elf.get_section_by_name(".dynsym") + self.assertIsNotNone(dynsym_section) + + symbols = [s.name for s in dynsym_section.iter_symbols()] + self.assertIn("PyInit_ext_shared", symbols) + + def test_import_and_call(self): + self.assertEqual(ext_shared.do_alpha(), 43) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/cc/py_extension/py_limited_api/BUILD.bazel b/tests/cc/py_extension/py_limited_api/BUILD.bazel new file mode 100644 index 0000000000..f548f3040f --- /dev/null +++ b/tests/cc/py_extension/py_limited_api/BUILD.bazel @@ -0,0 +1,12 @@ +load(":py_limited_api_tests.bzl", "py_limited_api_test_suite") + +package( + default_testonly = True, + default_visibility = ["//visibility:private"], +) + +licenses(["notice"]) + +py_limited_api_test_suite( + name = "py_limited_api_tests", +) diff --git a/tests/cc/py_extension/py_limited_api/py_limited_api_tests.bzl b/tests/cc/py_extension/py_limited_api/py_limited_api_tests.bzl new file mode 100644 index 0000000000..fbe92b936a --- /dev/null +++ b/tests/cc/py_extension/py_limited_api/py_limited_api_tests.bzl @@ -0,0 +1,140 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Tests for the py_limited_api attribute for py_extension.""" + +load("@rules_cc//cc:cc_library.bzl", "cc_library") +load("@rules_testing//lib:analysis_test.bzl", "analysis_test", "test_suite") +load("@rules_testing//lib:truth.bzl", "matching") +load("@rules_testing//lib:util.bzl", "util") +load("//python/cc:py_extension.bzl", "py_extension") + +def _test_limited_pass_impl(env, target): + env.expect.that_depset_of_files(target[DefaultInfo].files).contains_predicate( + matching.file_path_matches("tests/cc/py_extension/py_limited_api/{}.abi3.*".format(target.label.name)), + ) + +def _test_limited_same_version(name): + util.helper_target( + cc_library, + name = name + "_csl", + defines = ["Py_LIMITED_API=0x03080000"], + deps = [ + "@rules_python//python/cc:current_py_cc_headers", + ], + ) + py_extension( + name = name + "_pyext", + deps = [":" + name + "_csl"], + py_limited_api = "3.8", + ) + analysis_test( + name = name, + target = name + "_pyext", + impl = _test_limited_pass_impl, + ) + +def _test_limited_older_dep(name): + util.helper_target( + cc_library, + name = name + "_csl", + defines = ["Py_LIMITED_API=0x03080000"], # 3.8 + deps = [ + "@rules_python//python/cc:current_py_cc_headers", + ], + ) + py_extension( + name = name + "_pyext", + deps = [":" + name + "_csl"], + py_limited_api = "3.9", # 3.9 + ) + analysis_test( + name = name, + target = name + "_pyext", + impl = _test_limited_pass_impl, + ) + +def _test_no_limited_api(name): + util.helper_target( + cc_library, + name = name + "_csl", + deps = [ + "@rules_python//python/cc:current_py_cc_headers", + ], + ) + py_extension( + name = name + "_pyext", + deps = [":" + name + "_csl"], + ) + analysis_test( + name = name, + target = name + "_pyext", + impl = _test_no_limited_api_impl, + ) + +def _test_no_limited_api_impl(env, target): + # Should pass, nothing to assert on filename since it is platform-specific + _ = env # @unused + _ = target # @unused + +def _test_no_limited_api_dep_has_limited(name): + util.helper_target( + cc_library, + name = name + "_csl", + defines = ["Py_LIMITED_API=0x03080000"], + deps = [ + "@rules_python//python/cc:current_py_cc_headers", + ], + ) + py_extension( + name = name + "_pyext", + deps = [":" + name + "_csl"], + ) + analysis_test( + name = name, + target = name + "_pyext", + impl = _test_no_limited_api_dep_has_limited_impl, + ) + +def _test_no_limited_api_dep_has_limited_impl(env, target): + _ = env # @unused + _ = target # @unused + +def _test_limited_api_dep_has_no_python(name): + util.helper_target( + cc_library, + name = name + "_csl", + ) + py_extension( + name = name + "_pyext", + deps = [":" + name + "_csl"], + py_limited_api = "3.8", + ) + analysis_test( + name = name, + target = name + "_pyext", + impl = _test_limited_pass_impl, + ) + +def py_limited_api_test_suite(name): + test_suite( + name = name, + tests = [ + _test_limited_same_version, + _test_limited_older_dep, + _test_no_limited_api, + _test_no_limited_api_dep_has_limited, + _test_limited_api_dep_has_no_python, + ], + ) diff --git a/tests/cc/py_extension/some_data.txt b/tests/cc/py_extension/some_data.txt new file mode 100644 index 0000000000..4b5dc1d64c --- /dev/null +++ b/tests/cc/py_extension/some_data.txt @@ -0,0 +1 @@ +This is a data file diff --git a/tests/cc/py_extension/static_dep.c b/tests/cc/py_extension/static_dep.c new file mode 100644 index 0000000000..fa95e4dacc --- /dev/null +++ b/tests/cc/py_extension/static_dep.c @@ -0,0 +1,5 @@ +#include "static_dep.h" + +int my_lib_func() { + return 42; +} diff --git a/tests/cc/py_extension/static_dep.h b/tests/cc/py_extension/static_dep.h new file mode 100644 index 0000000000..d0f272abd7 --- /dev/null +++ b/tests/cc/py_extension/static_dep.h @@ -0,0 +1 @@ +int my_lib_func(); diff --git a/tests/config_settings/construct_config_settings_tests.bzl b/tests/config_settings/construct_config_settings_tests.bzl index 087efbbc70..1d21a8680d 100644 --- a/tests/config_settings/construct_config_settings_tests.bzl +++ b/tests/config_settings/construct_config_settings_tests.bzl @@ -47,7 +47,7 @@ def _test_minor_version_matching(name): } minor_cpu_matches = { str(Label(":is_python_3.11_aarch64")): "matched-3.11-aarch64", - str(Label(":is_python_3.11_ppc")): "matched-3.11-ppc", + str(Label(":is_python_3.11_ppc64le")): "matched-3.11-ppc64le", str(Label(":is_python_3.11_s390x")): "matched-3.11-s390x", str(Label(":is_python_3.11_x86_64")): "matched-3.11-x86_64", } @@ -58,7 +58,7 @@ def _test_minor_version_matching(name): } minor_os_cpu_matches = { str(Label(":is_python_3.11_linux_aarch64")): "matched-3.11-linux-aarch64", - str(Label(":is_python_3.11_linux_ppc")): "matched-3.11-linux-ppc", + str(Label(":is_python_3.11_linux_ppc64le")): "matched-3.11-linux-ppc64le", str(Label(":is_python_3.11_linux_s390x")): "matched-3.11-linux-s390x", str(Label(":is_python_3.11_linux_x86_64")): "matched-3.11-linux-x86_64", str(Label(":is_python_3.11_osx_aarch64")): "matched-3.11-osx-aarch64", @@ -171,7 +171,7 @@ def construct_config_settings_test_suite(name): # buildifier: disable=function- }, ) - for cpu in ["s390x", "ppc", "x86_64", "aarch64"]: + for cpu in ["s390x", "ppc", "ppc64le", "x86_64", "aarch64"]: native.config_setting( name = "is_python_3.11_" + cpu, constraint_values = [ @@ -185,6 +185,7 @@ def construct_config_settings_test_suite(name): # buildifier: disable=function- for (os, cpu) in [ ("linux", "aarch64"), ("linux", "ppc"), + ("linux", "ppc64le"), ("linux", "s390x"), ("linux", "x86_64"), ("osx", "aarch64"), diff --git a/tests/config_settings/transition/BUILD.bazel b/tests/config_settings/transition/BUILD.bazel index 19d4958669..093de50959 100644 --- a/tests/config_settings/transition/BUILD.bazel +++ b/tests/config_settings/transition/BUILD.bazel @@ -1,6 +1,3 @@ load(":multi_version_tests.bzl", "multi_version_test_suite") -load(":py_args_tests.bzl", "py_args_test_suite") - -py_args_test_suite(name = "py_args_tests") multi_version_test_suite(name = "multi_version_tests") diff --git a/tests/config_settings/transition/multi_version_tests.bzl b/tests/config_settings/transition/multi_version_tests.bzl index 5805bab32d..2b4f73a225 100644 --- a/tests/config_settings/transition/multi_version_tests.bzl +++ b/tests/config_settings/transition/multi_version_tests.bzl @@ -13,14 +13,18 @@ # limitations under the License. """Tests for py_test.""" +load("@pythons_hub//:versions.bzl", "DEFAULT_PYTHON_VERSION") load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") -load("@rules_testing//lib:util.bzl", "TestingAspectInfo", rt_util = "util") +load("@rules_testing//lib:util.bzl", rt_util = "util") +load("//python:py_binary.bzl", "py_binary") load("//python:py_info.bzl", "PyInfo") -load("//python/config_settings:transition.bzl", py_binary_transitioned = "py_binary", py_test_transitioned = "py_test") +load("//python:py_test.bzl", "py_test") +load("//python/private:common.bzl", "maybe_builtin_build_python_zip") # buildifier: disable=bzl-visibility +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:reexports.bzl", "BuiltinPyInfo") # buildifier: disable=bzl-visibility -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility load("//tests/support:support.bzl", "CC_TOOLCHAIN") +load("//tests/support/platforms:platforms.bzl", "platform_targets") # NOTE @aignas 2024-06-04: we are using here something that is registered in the MODULE.Bazel # and if you find tests failing, it could be because of the toolchain resolution issues here. @@ -28,13 +32,13 @@ load("//tests/support:support.bzl", "CC_TOOLCHAIN") # If the toolchain is not resolved then you will have a weird message telling # you that your transition target does not have a PyRuntime provider, which is # caused by there not being a toolchain detected for the target. -_PYTHON_VERSION = "3.11" +_PYTHON_VERSION = DEFAULT_PYTHON_VERSION _tests = [] def _test_py_test_with_transition(name): rt_util.helper_target( - py_test_transitioned, + py_test, name = name + "_subject", srcs = [name + "_subject.py"], python_version = _PYTHON_VERSION, @@ -56,7 +60,7 @@ _tests.append(_test_py_test_with_transition) def _test_py_binary_with_transition(name): rt_util.helper_target( - py_binary_transitioned, + py_binary, name = name + "_subject", srcs = [name + "_subject.py"], python_version = _PYTHON_VERSION, @@ -78,7 +82,7 @@ _tests.append(_test_py_binary_with_transition) def _setup_py_binary_windows(name, *, impl, build_python_zip): rt_util.helper_target( - py_binary_transitioned, + py_binary, name = name + "_subject", srcs = [name + "_subject.py"], python_version = _PYTHON_VERSION, @@ -89,66 +93,32 @@ def _setup_py_binary_windows(name, *, impl, build_python_zip): target = name + "_subject", impl = impl, config_settings = { - "//command_line_option:build_python_zip": build_python_zip, + labels.BUILD_PYTHON_ZIP: build_python_zip, "//command_line_option:extra_toolchains": CC_TOOLCHAIN, - "//command_line_option:platforms": str(Label("//tests/support:windows_x86_64")), - }, + "//command_line_option:platforms": str(platform_targets.WINDOWS_X86_64), + } | maybe_builtin_build_python_zip(str(build_python_zip)), ) def _test_py_binary_windows_build_python_zip_false(name): _setup_py_binary_windows( name, - build_python_zip = "false", + build_python_zip = False, impl = _test_py_binary_windows_build_python_zip_false_impl, ) def _test_py_binary_windows_build_python_zip_false_impl(env, target): default_outputs = env.expect.that_target(target).default_outputs() - if IS_BAZEL_7_OR_HIGHER: - # TODO: These outputs aren't correct. The outputs shouldn't - # have the "_" prefix on them (those are coming from the underlying - # wrapped binary). - env.expect.that_target(target).default_outputs().contains_exactly([ - "{package}/_{test_name}_subject", - "{package}/_{test_name}_subject.exe", - "{package}/{test_name}_subject", - "{package}/{test_name}_subject.py", - ]) - else: - inner_exe = target[TestingAspectInfo].attrs.target[DefaultInfo].files_to_run.executable - default_outputs.contains_at_least([ - inner_exe.short_path, - ]) -_tests.append(_test_py_binary_windows_build_python_zip_false) + # TODO: These outputs aren't correct. The outputs shouldn't + # have the "_" prefix on them (those are coming from the underlying + # wrapped binary). + default_outputs.contains_exactly([ + "{package}/{test_name}_subject.exe", + "{package}/{test_name}_subject", + "{package}/{test_name}_subject.py", + ]) -def _test_py_binary_windows_build_python_zip_true(name): - _setup_py_binary_windows( - name, - build_python_zip = "true", - impl = _test_py_binary_windows_build_python_zip_true_impl, - ) - -def _test_py_binary_windows_build_python_zip_true_impl(env, target): - default_outputs = env.expect.that_target(target).default_outputs() - if IS_BAZEL_7_OR_HIGHER: - # TODO: These outputs aren't correct. The outputs shouldn't - # have the "_" prefix on them (those are coming from the underlying - # wrapped binary). - default_outputs.contains_exactly([ - "{package}/_{test_name}_subject.exe", - "{package}/_{test_name}_subject.zip", - "{package}/{test_name}_subject.py", - "{package}/{test_name}_subject.zip", - ]) - else: - inner_exe = target[TestingAspectInfo].attrs.target[DefaultInfo].files_to_run.executable - default_outputs.contains_at_least([ - "{package}/{test_name}_subject.zip", - inner_exe.short_path, - ]) - -_tests.append(_test_py_binary_windows_build_python_zip_true) +_tests.append(_test_py_binary_windows_build_python_zip_false) def multi_version_test_suite(name): test_suite( diff --git a/tests/config_settings/transition/py_args_tests.bzl b/tests/config_settings/transition/py_args_tests.bzl deleted file mode 100644 index 4538c88a5c..0000000000 --- a/tests/config_settings/transition/py_args_tests.bzl +++ /dev/null @@ -1,68 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"" - -load("@rules_testing//lib:test_suite.bzl", "test_suite") -load("//python/config_settings/private:py_args.bzl", "py_args") # buildifier: disable=bzl-visibility - -_tests = [] - -def _test_py_args_default(env): - actual = py_args("foo", {}) - - want = { - "args": None, - "data": None, - "deps": None, - "env": None, - "main": "foo.py", - "srcs": None, - } - env.expect.that_dict(actual).contains_exactly(want) - -_tests.append(_test_py_args_default) - -def _test_kwargs_get_consumed(env): - kwargs = { - "args": ["some", "args"], - "data": ["data"], - "deps": ["deps"], - "env": {"key": "value"}, - "main": "__main__.py", - "srcs": ["__main__.py"], - "visibility": ["//visibility:public"], - } - actual = py_args("bar_bin", kwargs) - - want = { - "args": ["some", "args"], - "data": ["data"], - "deps": ["deps"], - "env": {"key": "value"}, - "main": "__main__.py", - "srcs": ["__main__.py"], - } - env.expect.that_dict(actual).contains_exactly(want) - env.expect.that_dict(kwargs).keys().contains_exactly(["visibility"]) - -_tests.append(_test_kwargs_get_consumed) - -def py_args_test_suite(name): - """Create the test suite. - - Args: - name: the name of the test suite - """ - test_suite(name = name, basic_tests = _tests) diff --git a/tests/coverage_deps/BUILD.bazel b/tests/coverage_deps/BUILD.bazel new file mode 100644 index 0000000000..8ec6025902 --- /dev/null +++ b/tests/coverage_deps/BUILD.bazel @@ -0,0 +1,17 @@ +# Copyright 2026 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +load(":coverage_deps_test.bzl", "coverage_deps_test_suite") + +coverage_deps_test_suite(name = "coverage_deps_tests") diff --git a/tests/coverage_deps/coverage_deps_test.bzl b/tests/coverage_deps/coverage_deps_test.bzl new file mode 100644 index 0000000000..ddb82cd6c5 --- /dev/null +++ b/tests/coverage_deps/coverage_deps_test.bzl @@ -0,0 +1,52 @@ +# Copyright 2026 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"Tests for coverage_dep's handling of platforms with no bundled wheel." + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private:coverage_deps.bzl", "coverage_dep") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_windows_platform_returns_none(env): + # Windows is intentionally unsupported: the upstream coverage wrapper is + # written in shell. + result = coverage_dep( + name = "unused_for_test", + python_version = "3.10", + platform = "x86_64-pc-windows-msvc", + visibility = ["//visibility:public"], + ) + env.expect.that_bool(result == None).equals(True) + +_tests.append(_test_windows_platform_returns_none) + +# NOTE: there is intentionally no unit test for the supported-wheel path +# (where coverage_dep returns a non-None label and emits no warning). +# That path calls `maybe(http_archive, ...)`, which calls +# `native.existing_rule()`. `native.existing_rule()` is only valid during +# BUILD file, legacy macro, or rule finalizer evaluation -- not during +# rule analysis, which is the phase rules_testing analysis tests run in. +# Calling coverage_dep with supported args from here therefore fails with +# "existing_rule() can only be used while evaluating a BUILD file, ...". +# The supported-wheel path is exercised end-to-end by `bazel coverage` +# against a real py_test target during ordinary use of the toolchain. + +def coverage_deps_test_suite(name): + """Create the test suite. + + Args: + name: the name of the test suite. + """ + test_suite(name = name, basic_tests = _tests) diff --git a/tests/deprecated/BUILD.bazel b/tests/deprecated/BUILD.bazel new file mode 100644 index 0000000000..4b920679f1 --- /dev/null +++ b/tests/deprecated/BUILD.bazel @@ -0,0 +1,96 @@ +load("@bazel_skylib//rules:build_test.bzl", "build_test") +load( + "@python//3.11:defs.bzl", + hub_compile_pip_requirements = "compile_pip_requirements", + hub_py_binary = "py_binary", + hub_py_console_script_binary = "py_console_script_binary", + hub_py_test = "py_test", +) +load( + "@python_3_11//:defs.bzl", + versioned_compile_pip_requirements = "compile_pip_requirements", + versioned_py_binary = "py_binary", + versioned_py_console_script_binary = "py_console_script_binary", + versioned_py_test = "py_test", +) +load("//python/config_settings:transition.bzl", transition_py_binary = "py_binary", transition_py_test = "py_test") + +# TODO @aignas 2025-01-22: remove the referenced symbols when releasing v2 + +transition_py_binary( + name = "transition_py_binary", + srcs = ["dummy.py"], + main = "dummy.py", + python_version = "3.11", +) + +transition_py_test( + name = "transition_py_test", + srcs = ["dummy.py"], + main = "dummy.py", + python_version = "3.11", +) + +versioned_py_binary( + name = "versioned_py_binary", + srcs = ["dummy.py"], + main = "dummy.py", +) + +versioned_py_test( + name = "versioned_py_test", + srcs = ["dummy.py"], + main = "dummy.py", +) + +versioned_py_console_script_binary( + name = "versioned_py_console_script_binary", + pkg = "@rules_python_publish_deps//twine", + script = "twine", +) + +versioned_compile_pip_requirements( + name = "versioned_compile_pip_requirements", + src = "requirements.in", + requirements_txt = "requirements.txt", +) + +hub_py_binary( + name = "hub_py_binary", + srcs = ["dummy.py"], + main = "dummy.py", +) + +hub_py_test( + name = "hub_py_test", + srcs = ["dummy.py"], + main = "dummy.py", +) + +hub_py_console_script_binary( + name = "hub_py_console_script_binary", + pkg = "@rules_python_publish_deps//twine", + script = "twine", +) + +hub_compile_pip_requirements( + name = "hub_compile_pip_requirements", + src = "requirements.in", + requirements_txt = "requirements_hub.txt", +) + +build_test( + name = "build_test", + targets = [ + "transition_py_binary", + "transition_py_test", + "versioned_py_binary", + "versioned_py_test", + "versioned_py_console_script_binary", + "versioned_compile_pip_requirements", + "hub_py_binary", + "hub_py_test", + "hub_py_console_script_binary", + "hub_compile_pip_requirements", + ], +) diff --git a/tests/deprecated/dummy.py b/tests/deprecated/dummy.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/deprecated/requirements.in b/tests/deprecated/requirements.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/deprecated/requirements.txt b/tests/deprecated/requirements.txt new file mode 100644 index 0000000000..4d53f7c4e3 --- /dev/null +++ b/tests/deprecated/requirements.txt @@ -0,0 +1,6 @@ +# +# This file is autogenerated by pip-compile with Python 3.11 +# by the following command: +# +# bazel run //tests/deprecated:versioned_compile_pip_requirements.update +# diff --git a/tests/deprecated/requirements_hub.txt b/tests/deprecated/requirements_hub.txt new file mode 100644 index 0000000000..444beb63a5 --- /dev/null +++ b/tests/deprecated/requirements_hub.txt @@ -0,0 +1,6 @@ +# +# This file is autogenerated by pip-compile with Python 3.11 +# by the following command: +# +# bazel run //tests/deprecated:hub_compile_pip_requirements.update +# diff --git a/tests/docs/BUILD.bazel b/tests/docs/BUILD.bazel new file mode 100644 index 0000000000..bdc99a290c --- /dev/null +++ b/tests/docs/BUILD.bazel @@ -0,0 +1,10 @@ +load("@bazel_skylib//rules:build_test.bzl", "build_test") + +licenses(["notice"]) + +build_test( + name = "docs_build_test", + targets = [ + "//docs:docs", + ], +) diff --git a/tests/entry_points/py_console_script_gen_test.py b/tests/entry_points/py_console_script_gen_test.py index a5fceb67f9..92fa42f167 100644 --- a/tests/entry_points/py_console_script_gen_test.py +++ b/tests/entry_points/py_console_script_gen_test.py @@ -47,6 +47,7 @@ def test_no_console_scripts_error(self): out=outfile, console_script=None, console_script_guess="", + shebang="", ) self.assertEqual( @@ -76,6 +77,7 @@ def test_no_entry_point_selected_error(self): out=outfile, console_script=None, console_script_guess="bar-baz", + shebang="", ) self.assertEqual( @@ -106,6 +108,7 @@ def test_incorrect_entry_point(self): out=outfile, console_script="baz", console_script_guess="", + shebang="", ) self.assertEqual( @@ -134,6 +137,7 @@ def test_a_single_entry_point(self): out=out, console_script=None, console_script_guess="foo", + shebang="", ) got = out.read_text() @@ -158,7 +162,7 @@ def test_a_single_entry_point(self): raise if __name__ == "__main__": - sys.exit(baz()) + sys.exit(baz()) # type: ignore """ ) self.assertEqual(want, got) @@ -185,12 +189,42 @@ def test_a_second_entry_point_class_method(self): out=out, console_script="bar", console_script_guess="", + shebang="", ) got = out.read_text() - self.assertRegex(got, "from foo\.baz import Bar") - self.assertRegex(got, "sys\.exit\(Bar\.baz\(\)\)") + self.assertRegex(got, r"from foo\.baz import Bar") + self.assertRegex(got, r"sys\.exit\(Bar\.baz\(\)\)") + + def test_shebang_included(self): + with tempfile.TemporaryDirectory() as tmpdir: + tmpdir = pathlib.Path(tmpdir) + given_contents = ( + textwrap.dedent( + """ + [console_scripts] + foo = foo.bar:baz + """ + ).strip() + + "\n" + ) + entry_points = tmpdir / "entry_points.txt" + entry_points.write_text(given_contents) + out = tmpdir / "foo.py" + + shebang = "#!/usr/bin/env python3" + run( + entry_points=entry_points, + out=out, + console_script=None, + console_script_guess="foo", + shebang=shebang, + ) + + got = out.read_text() + + self.assertTrue(got.startswith(shebang + "\n")) if __name__ == "__main__": diff --git a/tests/exec_toolchain_matching/exec_toolchain_matching_tests.bzl b/tests/exec_toolchain_matching/exec_toolchain_matching_tests.bzl index f6eae5ad5f..a26e4f5f6e 100644 --- a/tests/exec_toolchain_matching/exec_toolchain_matching_tests.bzl +++ b/tests/exec_toolchain_matching/exec_toolchain_matching_tests.bzl @@ -18,9 +18,9 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:util.bzl", rt_util = "util") load("//python:py_runtime.bzl", "py_runtime") load("//python:py_runtime_pair.bzl", "py_runtime_pair") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", "TARGET_TOOLCHAIN_TYPE") # buildifier: disable=bzl-visibility -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility -load("//tests/support:support.bzl", "LINUX", "MAC", "PYTHON_VERSION") +load("//tests/support/platforms:platforms.bzl", "platform_targets") _LookupInfo = provider() # buildifier: disable=provider-params @@ -126,10 +126,10 @@ def _test_exec_matches_target_python_version(name): target = name + "_subject", impl = _test_exec_matches_target_python_version_impl, config_settings = { - "//command_line_option:extra_execution_platforms": [str(MAC)], + "//command_line_option:extra_execution_platforms": [str(platform_targets.MAC)], "//command_line_option:extra_toolchains": ["//tests/exec_toolchain_matching:all"], - "//command_line_option:platforms": [str(LINUX)], - PYTHON_VERSION: "3.12", + "//command_line_option:platforms": [str(platform_targets.LINUX)], + labels.PYTHON_VERSION: "3.12", }, ) @@ -142,11 +142,10 @@ def _test_exec_matches_target_python_version_impl(env, target): env.expect.that_str(target_runtime.interpreter_path).equals("/linux/python3.12") env.expect.that_str(exec_runtime.interpreter_path).equals("/mac/python3.12") - if IS_BAZEL_7_OR_HIGHER: - target_version = target_runtime.interpreter_version_info - exec_version = exec_runtime.interpreter_version_info + target_version = target_runtime.interpreter_version_info + exec_version = exec_runtime.interpreter_version_info - env.expect.that_bool(target_version == exec_version) + env.expect.that_bool(target_version == exec_version) def exec_toolchain_matching_test_suite(name): test_suite(name = name, tests = _tests) diff --git a/tests/get_release_info/BUILD.bazel b/tests/get_release_info/BUILD.bazel new file mode 100644 index 0000000000..26517e6dec --- /dev/null +++ b/tests/get_release_info/BUILD.bazel @@ -0,0 +1,24 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +load(":get_release_info_tests.bzl", "get_release_info_test_suite") + +package( + default_testonly = True, + default_visibility = ["//:__subpackages__"], +) + +licenses(["notice"]) + +get_release_info_test_suite(name = "get_release_info") diff --git a/tests/get_release_info/get_release_info_tests.bzl b/tests/get_release_info/get_release_info_tests.bzl new file mode 100644 index 0000000000..0b1b60adc3 --- /dev/null +++ b/tests/get_release_info/get_release_info_tests.bzl @@ -0,0 +1,112 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Tests for get_release_info.""" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python:versions.bzl", "get_release_info") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_file_url(env): + """Tests that a file:/// url is handled correctly.""" + tool_versions = { + "3.11.5": { + "sha256": { + "x86_64-unknown-linux-gnu": "fbed6f7694b2faae5d7c401a856219c945397f772eea5ca50c6eb825cbc9d1e1", + }, + "strip_prefix": "python", + "url": "file:///tmp/cpython-3.11.5.tar.gz", + }, + } + + expected_url = "file:///tmp/cpython-3.11.5.tar.gz" + expected_filename = "file:///tmp/cpython-3.11.5.tar.gz" + + filename, urls, strip_prefix, patches, patch_strip = get_release_info( + platform = "x86_64-unknown-linux-gnu", + python_version = "3.11.5", + tool_versions = tool_versions, + ) + + env.expect.that_str(filename).equals(expected_filename) + env.expect.that_collection(urls).contains_exactly([expected_url]) + env.expect.that_str(strip_prefix).equals("python") + env.expect.that_collection(patches).has_size(0) + env.expect.that_bool(patch_strip == None).equals(True) + +_tests.append(_test_file_url) + +def _test_astral_mirror(env): + """Tests that the releases.astral.sh mirror is added as a secondary URL.""" + tool_versions = { + "3.11.5": { + "sha256": { + "x86_64-unknown-linux-gnu": "fbed6f7694b2faae5d7c401a856219c945397f772eea5ca50c6eb825cbc9d1e1", + }, + "strip_prefix": "python", + "url": "20230826/cpython-{python_version}+20230826-{platform}-{build}.tar.gz", + }, + } + + expected_urls = [ + "https://github.com/astral-sh/python-build-standalone/releases/download/20230826/cpython-3.11.5+20230826-x86_64-unknown-linux-gnu-install_only.tar.gz", + "https://releases.astral.sh/github/python-build-standalone/releases/download/20230826/cpython-3.11.5+20230826-x86_64-unknown-linux-gnu-install_only.tar.gz", + "https://github.com/indygreg/python-build-standalone/releases/download/20230826/cpython-3.11.5+20230826-x86_64-unknown-linux-gnu-install_only.tar.gz", + ] + + _, urls, _, _, _ = get_release_info( + platform = "x86_64-unknown-linux-gnu", + python_version = "3.11.5", + tool_versions = tool_versions, + ) + + env.expect.that_collection(urls).contains_exactly(expected_urls) + +_tests.append(_test_astral_mirror) + +def _test_astral_mirror_legacy(env): + """Tests that the releases.astral.sh mirror is added for legacy indygreg URLs.""" + tool_versions = { + "3.11.5": { + "sha256": { + "x86_64-unknown-linux-gnu": "fbed6f7694b2faae5d7c401a856219c945397f772eea5ca50c6eb825cbc9d1e1", + }, + "strip_prefix": "python", + "url": "20230826/cpython-{python_version}+20230826-{platform}-{build}.tar.gz", + }, + } + + expected_urls = [ + "https://github.com/indygreg/python-build-standalone/releases/download/20230826/cpython-3.11.5+20230826-x86_64-unknown-linux-gnu-install_only.tar.gz", + "https://releases.astral.sh/github/python-build-standalone/releases/download/20230826/cpython-3.11.5+20230826-x86_64-unknown-linux-gnu-install_only.tar.gz", + ] + + _, urls, _, _, _ = get_release_info( + platform = "x86_64-unknown-linux-gnu", + python_version = "3.11.5", + base_urls = ["https://github.com/indygreg/python-build-standalone/releases/download"], + tool_versions = tool_versions, + ) + + env.expect.that_collection(urls).contains_exactly(expected_urls) + +_tests.append(_test_astral_mirror_legacy) + +def get_release_info_test_suite(name): + """Defines the test suite for get_release_info.""" + test_suite( + name = name, + basic_tests = _tests, + ) diff --git a/tests/implicit_namespace_packages/BUILD.bazel b/tests/implicit_namespace_packages/BUILD.bazel new file mode 100644 index 0000000000..b544c4d118 --- /dev/null +++ b/tests/implicit_namespace_packages/BUILD.bazel @@ -0,0 +1,12 @@ +load("//python:py_test.bzl", "py_test") +load("//tests/support:support.bzl", "SUPPORTS_BZLMOD") + +py_test( + name = "namespace_packages_test", + srcs = ["namespace_packages_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, + deps = [ + "@implicit_namespace_ns_sub1//:pkg", + "@implicit_namespace_ns_sub2//:pkg", + ], +) diff --git a/tests/implicit_namespace_packages/namespace_packages_test.py b/tests/implicit_namespace_packages/namespace_packages_test.py new file mode 100644 index 0000000000..a1ee27d71b --- /dev/null +++ b/tests/implicit_namespace_packages/namespace_packages_test.py @@ -0,0 +1,23 @@ +import unittest + + +class NamespacePackagesTest(unittest.TestCase): + def test_both_importable(self): + import nspkg + import nspkg.subpkg1 + import nspkg.subpkg1.subpkgmod + import nspkg.subpkg2.subpkgmod + + self.assertEqual("nspkg.subpkg1", nspkg.subpkg1.expected_name) + self.assertEqual( + "nspkg.subpkg1.subpkgmod", nspkg.subpkg1.subpkgmod.expected_name + ) + + self.assertEqual("nspkg.subpkg2", nspkg.subpkg2.expected_name) + self.assertEqual( + "nspkg.subpkg2.subpkgmod", nspkg.subpkg2.subpkgmod.expected_name + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/implicit_namespace_packages/testdata/ns-sub1/ns-sub1-1.0.dist-info/METADATA b/tests/implicit_namespace_packages/testdata/ns-sub1/ns-sub1-1.0.dist-info/METADATA new file mode 100644 index 0000000000..ecec6086ba --- /dev/null +++ b/tests/implicit_namespace_packages/testdata/ns-sub1/ns-sub1-1.0.dist-info/METADATA @@ -0,0 +1,2 @@ +Name: ns-sub1 +Version: 1.0 diff --git a/tests/implicit_namespace_packages/testdata/ns-sub1/ns-sub1-1.0.dist-info/RECORD b/tests/implicit_namespace_packages/testdata/ns-sub1/ns-sub1-1.0.dist-info/RECORD new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/implicit_namespace_packages/testdata/ns-sub1/ns-sub1-1.0.dist-info/WHEEL b/tests/implicit_namespace_packages/testdata/ns-sub1/ns-sub1-1.0.dist-info/WHEEL new file mode 100644 index 0000000000..a64521a1cc --- /dev/null +++ b/tests/implicit_namespace_packages/testdata/ns-sub1/ns-sub1-1.0.dist-info/WHEEL @@ -0,0 +1 @@ +Wheel-Version: 1.0 diff --git a/tests/implicit_namespace_packages/testdata/ns-sub1/nspkg/subpkg1/__init__.py b/tests/implicit_namespace_packages/testdata/ns-sub1/nspkg/subpkg1/__init__.py new file mode 100644 index 0000000000..6657257dc6 --- /dev/null +++ b/tests/implicit_namespace_packages/testdata/ns-sub1/nspkg/subpkg1/__init__.py @@ -0,0 +1 @@ +expected_name = "nspkg.subpkg1" diff --git a/tests/implicit_namespace_packages/testdata/ns-sub1/nspkg/subpkg1/subpkgmod.py b/tests/implicit_namespace_packages/testdata/ns-sub1/nspkg/subpkg1/subpkgmod.py new file mode 100644 index 0000000000..b03bf39642 --- /dev/null +++ b/tests/implicit_namespace_packages/testdata/ns-sub1/nspkg/subpkg1/subpkgmod.py @@ -0,0 +1 @@ +expected_name = "nspkg.subpkg1.subpkgmod" diff --git a/tests/implicit_namespace_packages/testdata/ns-sub2/ns_sub2-1.0.dist-info/METADATA b/tests/implicit_namespace_packages/testdata/ns-sub2/ns_sub2-1.0.dist-info/METADATA new file mode 100644 index 0000000000..92cbb8ec2e --- /dev/null +++ b/tests/implicit_namespace_packages/testdata/ns-sub2/ns_sub2-1.0.dist-info/METADATA @@ -0,0 +1,2 @@ +Name: ns-sub2 +Version: 1.0 diff --git a/tests/implicit_namespace_packages/testdata/ns-sub2/ns_sub2-1.0.dist-info/RECORD b/tests/implicit_namespace_packages/testdata/ns-sub2/ns_sub2-1.0.dist-info/RECORD new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/implicit_namespace_packages/testdata/ns-sub2/ns_sub2-1.0.dist-info/WHEEL b/tests/implicit_namespace_packages/testdata/ns-sub2/ns_sub2-1.0.dist-info/WHEEL new file mode 100644 index 0000000000..a64521a1cc --- /dev/null +++ b/tests/implicit_namespace_packages/testdata/ns-sub2/ns_sub2-1.0.dist-info/WHEEL @@ -0,0 +1 @@ +Wheel-Version: 1.0 diff --git a/tests/implicit_namespace_packages/testdata/ns-sub2/nspkg/subpkg2/__init__.py b/tests/implicit_namespace_packages/testdata/ns-sub2/nspkg/subpkg2/__init__.py new file mode 100644 index 0000000000..29bfb67066 --- /dev/null +++ b/tests/implicit_namespace_packages/testdata/ns-sub2/nspkg/subpkg2/__init__.py @@ -0,0 +1 @@ +expected_name = "nspkg.subpkg2" diff --git a/tests/implicit_namespace_packages/testdata/ns-sub2/nspkg/subpkg2/subpkgmod.py b/tests/implicit_namespace_packages/testdata/ns-sub2/nspkg/subpkg2/subpkgmod.py new file mode 100644 index 0000000000..45a28eb851 --- /dev/null +++ b/tests/implicit_namespace_packages/testdata/ns-sub2/nspkg/subpkg2/subpkgmod.py @@ -0,0 +1 @@ +expected_name = "nspkg.subpkg2.subpkgmod" diff --git a/tests/integration/BUILD.bazel b/tests/integration/BUILD.bazel index 289c85d38a..13b9c2e855 100644 --- a/tests/integration/BUILD.bazel +++ b/tests/integration/BUILD.bazel @@ -12,47 +12,19 @@ # See the License for the specific language governing permissions and # limitations under the License. -load("@bazel_binaries//:defs.bzl", "bazel_binaries") load("@rules_bazel_integration_test//bazel_integration_test:defs.bzl", "default_test_runner") +load("//python:py_binary.bzl", "py_binary") load("//python:py_library.bzl", "py_library") +load("//tests/support:support.bzl", "NOT_WINDOWS") load(":integration_test.bzl", "rules_python_integration_test") licenses(["notice"]) -_LEGACY_WORKSPACE_FLAGS = [ +_WORKSPACE_FLAGS = [ "--noenable_bzlmod", -] - -_WORKSPACE_FLAGS = _LEGACY_WORKSPACE_FLAGS + [ "--enable_workspace", ] -_WORKSPACE_GAZELLE_PLUGIN_FLAGS = [ - "--override_repository=rules_python_gazelle_plugin=../../../rules_python_gazelle_plugin", -] - -_GAZELLE_PLUGIN_FLAGS = [ - "--override_module=rules_python_gazelle_plugin=../../../rules_python_gazelle_plugin", -] - -default_test_runner( - name = "bazel_6_4_workspace_test_runner", - bazel_cmds = [ - "info {}".format(" ".join(_LEGACY_WORKSPACE_FLAGS)), - "test {} //...".format(" ".join(_LEGACY_WORKSPACE_FLAGS)), - ], - visibility = ["//visibility:public"], -) - -default_test_runner( - name = "bazel_6_4_workspace_test_runner_gazelle_plugin", - bazel_cmds = [ - "info {}".format(" ".join(_LEGACY_WORKSPACE_FLAGS + _WORKSPACE_GAZELLE_PLUGIN_FLAGS)), - "test {} //...".format(" ".join(_LEGACY_WORKSPACE_FLAGS + _WORKSPACE_GAZELLE_PLUGIN_FLAGS)), - ], - visibility = ["//visibility:public"], -) - default_test_runner( name = "workspace_test_runner", bazel_cmds = [ @@ -62,31 +34,27 @@ default_test_runner( visibility = ["//visibility:public"], ) -default_test_runner( - name = "workspace_test_runner_gazelle_plugin", - bazel_cmds = [ - "info {}".format(" ".join(_WORKSPACE_FLAGS + _WORKSPACE_GAZELLE_PLUGIN_FLAGS)), - "test {} //...".format(" ".join(_WORKSPACE_FLAGS + _WORKSPACE_GAZELLE_PLUGIN_FLAGS)), - ], - visibility = ["//visibility:public"], -) - default_test_runner( name = "test_runner", visibility = ["//visibility:public"], ) -default_test_runner( - name = "test_runner_gazelle_plugin", - bazel_cmds = [ - "info {}".format(" ".join(_GAZELLE_PLUGIN_FLAGS)), - "test {} //...".format(" ".join(_GAZELLE_PLUGIN_FLAGS)), +rules_python_integration_test( + name = "bzlmod_lockfile_test", + bazel_versions = ["9.1.0"], + py_main = "bzlmod_lockfile_test.py", +) + +test_suite( + name = "subset", + tags = ["manual"], + tests = [ + "bzlmod_lockfile_test_bazel_9.1.0", + "local_toolchains_test_bazel_self", + "uv_lock_test_bazel_self", ], - visibility = ["//visibility:public"], ) -# TODO: add compile_pip_requirements_test_from_external_repo - rules_python_integration_test( name = "compile_pip_requirements_test", ) @@ -98,26 +66,27 @@ rules_python_integration_test( ) rules_python_integration_test( - name = "ignore_root_user_error_test", + name = "local_toolchains_test", + env = { + "RULES_PYTHON_BZLMOD_DEBUG": "1", + }, ) rules_python_integration_test( - name = "ignore_root_user_error_workspace_test", + name = "local_toolchains_workspace_test", bzlmod = False, - workspace_path = "ignore_root_user_error", + env = { + "RULES_PYTHON_BZLMOD_DEBUG": "1", + }, + workspace_path = "local_toolchains", ) rules_python_integration_test( - name = "local_toolchains_test", - bazel_versions = [ - version - for version in bazel_binaries.versions.all - if not version.startswith("6.") - ], + name = "pip_parse_test", ) rules_python_integration_test( - name = "pip_parse_test", + name = "pip_parse_isolated_test", ) rules_python_integration_test( @@ -136,13 +105,69 @@ rules_python_integration_test( workspace_path = "py_cc_toolchain_registered", ) +rules_python_integration_test( + name = "runtime_manifests_test", +) + rules_python_integration_test( name = "custom_commands_test", py_main = "custom_commands_test.py", ) +rules_python_integration_test( + name = "toolchain_target_settings_test", + py_main = "toolchain_target_settings_test.py", +) + +rules_python_integration_test( + name = "validate_test_main_test", + py_main = "validate_test_main_test.py", +) + +rules_python_integration_test( + name = "unified_pypi_test", + py_main = "unified_pypi_test.py", +) + +rules_python_integration_test( + name = "uv_lock_test", + py_deps = [ + "@pypiserver//pypiserver", + ":uv_lock_pypi_server_lib", + ], + py_main = "uv_lock_test.py", +) + +rules_python_integration_test( + name = "whl_library_test", +) + py_library( name = "runner_lib", srcs = ["runner.py"], imports = ["../../"], ) + +py_library( + name = "uv_lock_pypi_server_lib", + srcs = ["uv_lock_pypi_server.py"], + imports = ["../../"], + # currently windows is not working due to + # https://github.com/pypiserver/pypiserver/blob/main/pypiserver/config.py#L123 + # + # class DEFAULTS: + # .... + # PACKAGE_DIRECTORIES = [pathlib.Path("~/packages").expanduser().resolve()] + # .... + # + # which is loaded through `__init__.py` even though it is not used and breaks because + # in a Windows sandbox one cannot resolve the home directory. + target_compatible_with = NOT_WINDOWS, + deps = ["@pypiserver//pypiserver"], +) + +py_binary( + name = "uv_lock_pypi_server", + srcs = ["uv_lock_pypi_server.py"], + deps = [":uv_lock_pypi_server_lib"], +) diff --git a/tests/integration/bazel_from_env b/tests/integration/bazel_from_env index 96780b8156..a372736f32 100755 --- a/tests/integration/bazel_from_env +++ b/tests/integration/bazel_from_env @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # # A simple wrapper so rules_bazel_integration_test can use the # bazel version inherited from the environment. diff --git a/tests/integration/bzlmod_lockfile/.bazelrc b/tests/integration/bzlmod_lockfile/.bazelrc new file mode 100644 index 0000000000..3687a11fae --- /dev/null +++ b/tests/integration/bzlmod_lockfile/.bazelrc @@ -0,0 +1,9 @@ +# Bazel configuration flags + +build --enable_runfiles +common --lockfile_mode=error + +common --experimental_isolated_extension_usages + +# https://docs.bazel.build/versions/main/best-practices.html#using-the-bazelrc-file +try-import %workspace%/user.bazelrc diff --git a/tests/integration/bzlmod_lockfile/.bazelversion b/tests/integration/bzlmod_lockfile/.bazelversion new file mode 100644 index 0000000000..47da986f86 --- /dev/null +++ b/tests/integration/bzlmod_lockfile/.bazelversion @@ -0,0 +1 @@ +9.1.0 diff --git a/tests/integration/bzlmod_lockfile/BUILD.bazel b/tests/integration/bzlmod_lockfile/BUILD.bazel new file mode 100644 index 0000000000..7e99a242ae --- /dev/null +++ b/tests/integration/bzlmod_lockfile/BUILD.bazel @@ -0,0 +1,10 @@ +load("@rules_python//python:py_test.bzl", "py_test") + +py_test( + name = "test_dummy", + srcs = ["test_dummy.py"], + deps = [ + "@pypi//pycparser", + "@pypi//six", + ], +) diff --git a/tests/integration/bzlmod_lockfile/MODULE.bazel b/tests/integration/bzlmod_lockfile/MODULE.bazel new file mode 100644 index 0000000000..bfd43cb895 --- /dev/null +++ b/tests/integration/bzlmod_lockfile/MODULE.bazel @@ -0,0 +1,19 @@ +module(name = "bzlmod_lockfile") + +bazel_dep(name = "rules_python") +local_path_override( + module_name = "rules_python", + path = "../../..", +) + +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.toolchain(python_version = "3.13") + +# TODO: This test module should also verify that isolate = True works, will do in a followup PR. +pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") +pip.parse( + hub_name = "pypi_lockfile", + python_version = "3.13", + requirements_lock = "//:requirements_lock.txt", +) +use_repo(pip, pypi = "pypi_lockfile") diff --git a/tests/integration/bzlmod_lockfile/MODULE.bazel.lock b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock new file mode 100644 index 0000000000..6b293283ae --- /dev/null +++ b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock @@ -0,0 +1,679 @@ +{ + "lockFileVersion": 26, + "registryFileHashes": { + "https://bcr.bazel.build/bazel_registry.json": "8a28e4aff06ee60aed2a8c281907fb8bcbf3b753c91fb5a5c57da3215d5b3497", + "https://bcr.bazel.build/modules/abseil-cpp/20210324.2/MODULE.bazel": "7cd0312e064fde87c8d1cd79ba06c876bd23630c83466e9500321be55c96ace2", + "https://bcr.bazel.build/modules/abseil-cpp/20211102.0/MODULE.bazel": "70390338f7a5106231d20620712f7cccb659cd0e9d073d1991c038eb9fc57589", + "https://bcr.bazel.build/modules/abseil-cpp/20230125.1/MODULE.bazel": "89047429cb0207707b2dface14ba7f8df85273d484c2572755be4bab7ce9c3a0", + "https://bcr.bazel.build/modules/abseil-cpp/20230802.0.bcr.1/MODULE.bazel": "1c8cec495288dccd14fdae6e3f95f772c1c91857047a098fad772034264cc8cb", + "https://bcr.bazel.build/modules/abseil-cpp/20230802.0/MODULE.bazel": "d253ae36a8bd9ee3c5955384096ccb6baf16a1b1e93e858370da0a3b94f77c16", + "https://bcr.bazel.build/modules/abseil-cpp/20230802.1/MODULE.bazel": "fa92e2eb41a04df73cdabeec37107316f7e5272650f81d6cc096418fe647b915", + "https://bcr.bazel.build/modules/abseil-cpp/20240116.1/MODULE.bazel": "37bcdb4440fbb61df6a1c296ae01b327f19e9bb521f9b8e26ec854b6f97309ed", + "https://bcr.bazel.build/modules/abseil-cpp/20240116.2/MODULE.bazel": "73939767a4686cd9a520d16af5ab440071ed75cec1a876bf2fcfaf1f71987a16", + "https://bcr.bazel.build/modules/abseil-cpp/20250127.1/MODULE.bazel": "c4a89e7ceb9bf1e25cf84a9f830ff6b817b72874088bf5141b314726e46a57c1", + "https://bcr.bazel.build/modules/abseil-cpp/20250512.1/MODULE.bazel": "d209fdb6f36ffaf61c509fcc81b19e81b411a999a934a032e10cd009a0226215", + "https://bcr.bazel.build/modules/abseil-cpp/20250814.1/MODULE.bazel": "51f2312901470cdab0dbdf3b88c40cd21c62a7ed58a3de45b365ddc5b11bcab2", + "https://bcr.bazel.build/modules/abseil-cpp/20250814.1/source.json": "cea3901d7e299da7320700abbaafe57a65d039f10d0d7ea601c4a66938ea4b0c", + "https://bcr.bazel.build/modules/apple_support/1.11.1/MODULE.bazel": "1843d7cd8a58369a444fc6000e7304425fba600ff641592161d9f15b179fb896", + "https://bcr.bazel.build/modules/apple_support/1.15.1/MODULE.bazel": "a0556fefca0b1bb2de8567b8827518f94db6a6e7e7d632b4c48dc5f865bc7c85", + "https://bcr.bazel.build/modules/apple_support/1.21.0/MODULE.bazel": "ac1824ed5edf17dee2fdd4927ada30c9f8c3b520be1b5fd02a5da15bc10bff3e", + "https://bcr.bazel.build/modules/apple_support/1.21.1/MODULE.bazel": "5809fa3efab15d1f3c3c635af6974044bac8a4919c62238cce06acee8a8c11f1", + "https://bcr.bazel.build/modules/apple_support/1.24.2/MODULE.bazel": "0e62471818affb9f0b26f128831d5c40b074d32e6dda5a0d3852847215a41ca4", + "https://bcr.bazel.build/modules/apple_support/1.24.2/source.json": "2c22c9827093250406c5568da6c54e6fdf0ef06238def3d99c71b12feb057a8d", + "https://bcr.bazel.build/modules/bazel_features/1.10.0/MODULE.bazel": "f75e8807570484a99be90abcd52b5e1f390362c258bcb73106f4544957a48101", + "https://bcr.bazel.build/modules/bazel_features/1.11.0/MODULE.bazel": "f9382337dd5a474c3b7d334c2f83e50b6eaedc284253334cf823044a26de03e8", + "https://bcr.bazel.build/modules/bazel_features/1.15.0/MODULE.bazel": "d38ff6e517149dc509406aca0db3ad1efdd890a85e049585b7234d04238e2a4d", + "https://bcr.bazel.build/modules/bazel_features/1.17.0/MODULE.bazel": "039de32d21b816b47bd42c778e0454217e9c9caac4a3cf8e15c7231ee3ddee4d", + "https://bcr.bazel.build/modules/bazel_features/1.18.0/MODULE.bazel": "1be0ae2557ab3a72a57aeb31b29be347bcdc5d2b1eb1e70f39e3851a7e97041a", + "https://bcr.bazel.build/modules/bazel_features/1.19.0/MODULE.bazel": "59adcdf28230d220f0067b1f435b8537dd033bfff8db21335ef9217919c7fb58", + "https://bcr.bazel.build/modules/bazel_features/1.21.0/MODULE.bazel": "675642261665d8eea09989aa3b8afb5c37627f1be178382c320d1b46afba5e3b", + "https://bcr.bazel.build/modules/bazel_features/1.23.0/MODULE.bazel": "fd1ac84bc4e97a5a0816b7fd7d4d4f6d837b0047cf4cbd81652d616af3a6591a", + "https://bcr.bazel.build/modules/bazel_features/1.27.0/MODULE.bazel": "621eeee06c4458a9121d1f104efb80f39d34deff4984e778359c60eaf1a8cb65", + "https://bcr.bazel.build/modules/bazel_features/1.28.0/MODULE.bazel": "4b4200e6cbf8fa335b2c3f43e1d6ef3e240319c33d43d60cc0fbd4b87ece299d", + "https://bcr.bazel.build/modules/bazel_features/1.3.0/MODULE.bazel": "cdcafe83ec318cda34e02948e81d790aab8df7a929cec6f6969f13a489ccecd9", + "https://bcr.bazel.build/modules/bazel_features/1.30.0/MODULE.bazel": "a14b62d05969a293b80257e72e597c2da7f717e1e69fa8b339703ed6731bec87", + "https://bcr.bazel.build/modules/bazel_features/1.33.0/MODULE.bazel": "8b8dc9d2a4c88609409c3191165bccec0e4cb044cd7a72ccbe826583303459f6", + "https://bcr.bazel.build/modules/bazel_features/1.4.1/MODULE.bazel": "e45b6bb2350aff3e442ae1111c555e27eac1d915e77775f6fdc4b351b758b5d7", + "https://bcr.bazel.build/modules/bazel_features/1.42.1/MODULE.bazel": "275a59b5406ff18c01739860aa70ad7ccb3cfb474579411decca11c93b951080", + "https://bcr.bazel.build/modules/bazel_features/1.42.1/source.json": "fcd4396b2df85f64f2b3bb436ad870793ecf39180f1d796f913cc9276d355309", + "https://bcr.bazel.build/modules/bazel_skylib/1.0.3/MODULE.bazel": "bcb0fd896384802d1ad283b4e4eb4d718eebd8cb820b0a2c3a347fb971afd9d8", + "https://bcr.bazel.build/modules/bazel_skylib/1.1.1/MODULE.bazel": "1add3e7d93ff2e6998f9e118022c84d163917d912f5afafb3058e3d2f1545b5e", + "https://bcr.bazel.build/modules/bazel_skylib/1.2.0/MODULE.bazel": "44fe84260e454ed94ad326352a698422dbe372b21a1ac9f3eab76eb531223686", + "https://bcr.bazel.build/modules/bazel_skylib/1.2.1/MODULE.bazel": "f35baf9da0efe45fa3da1696ae906eea3d615ad41e2e3def4aeb4e8bc0ef9a7a", + "https://bcr.bazel.build/modules/bazel_skylib/1.3.0/MODULE.bazel": "20228b92868bf5cfc41bda7afc8a8ba2a543201851de39d990ec957b513579c5", + "https://bcr.bazel.build/modules/bazel_skylib/1.4.1/MODULE.bazel": "a0dcb779424be33100dcae821e9e27e4f2901d9dfd5333efe5ac6a8d7ab75e1d", + "https://bcr.bazel.build/modules/bazel_skylib/1.4.2/MODULE.bazel": "3bd40978e7a1fac911d5989e6b09d8f64921865a45822d8b09e815eaa726a651", + "https://bcr.bazel.build/modules/bazel_skylib/1.5.0/MODULE.bazel": "32880f5e2945ce6a03d1fbd588e9198c0a959bb42297b2cfaf1685b7bc32e138", + "https://bcr.bazel.build/modules/bazel_skylib/1.6.1/MODULE.bazel": "8fdee2dbaace6c252131c00e1de4b165dc65af02ea278476187765e1a617b917", + "https://bcr.bazel.build/modules/bazel_skylib/1.7.0/MODULE.bazel": "0db596f4563de7938de764cc8deeabec291f55e8ec15299718b93c4423e9796d", + "https://bcr.bazel.build/modules/bazel_skylib/1.7.1/MODULE.bazel": "3120d80c5861aa616222ec015332e5f8d3171e062e3e804a2a0253e1be26e59b", + "https://bcr.bazel.build/modules/bazel_skylib/1.8.1/MODULE.bazel": "88ade7293becda963e0e3ea33e7d54d3425127e0a326e0d17da085a5f1f03ff6", + "https://bcr.bazel.build/modules/bazel_skylib/1.8.2/MODULE.bazel": "69ad6927098316848b34a9142bcc975e018ba27f08c4ff403f50c1b6e646ca67", + "https://bcr.bazel.build/modules/bazel_skylib/1.8.2/source.json": "34a3c8bcf233b835eb74be9d628899bb32999d3e0eadef1947a0a562a2b16ffb", + "https://bcr.bazel.build/modules/buildozer/8.5.1/MODULE.bazel": "a35d9561b3fc5b18797c330793e99e3b834a473d5fbd3d7d7634aafc9bdb6f8f", + "https://bcr.bazel.build/modules/buildozer/8.5.1/source.json": "e3386e6ff4529f2442800dee47ad28d3e6487f36a1f75ae39ae56c70f0cd2fbd", + "https://bcr.bazel.build/modules/google_benchmark/1.8.2/MODULE.bazel": "a70cf1bba851000ba93b58ae2f6d76490a9feb74192e57ab8e8ff13c34ec50cb", + "https://bcr.bazel.build/modules/googletest/1.11.0/MODULE.bazel": "3a83f095183f66345ca86aa13c58b59f9f94a2f81999c093d4eeaa2d262d12f4", + "https://bcr.bazel.build/modules/googletest/1.14.0.bcr.1/MODULE.bazel": "22c31a561553727960057361aa33bf20fb2e98584bc4fec007906e27053f80c6", + "https://bcr.bazel.build/modules/googletest/1.14.0/MODULE.bazel": "cfbcbf3e6eac06ef9d85900f64424708cc08687d1b527f0ef65aa7517af8118f", + "https://bcr.bazel.build/modules/googletest/1.15.2/MODULE.bazel": "6de1edc1d26cafb0ea1a6ab3f4d4192d91a312fd2d360b63adaa213cd00b2108", + "https://bcr.bazel.build/modules/googletest/1.17.0/MODULE.bazel": "dbec758171594a705933a29fcf69293d2468c49ec1f2ebca65c36f504d72df46", + "https://bcr.bazel.build/modules/googletest/1.17.0/source.json": "38e4454b25fc30f15439c0378e57909ab1fd0a443158aa35aec685da727cd713", + "https://bcr.bazel.build/modules/jsoncpp/1.9.5/MODULE.bazel": "31271aedc59e815656f5736f282bb7509a97c7ecb43e927ac1a37966e0578075", + "https://bcr.bazel.build/modules/jsoncpp/1.9.6/MODULE.bazel": "2f8d20d3b7d54143213c4dfc3d98225c42de7d666011528dc8fe91591e2e17b0", + "https://bcr.bazel.build/modules/jsoncpp/1.9.6/source.json": "a04756d367a2126c3541682864ecec52f92cdee80a35735a3cb249ce015ca000", + "https://bcr.bazel.build/modules/libpfm/4.11.0/MODULE.bazel": "45061ff025b301940f1e30d2c16bea596c25b176c8b6b3087e92615adbd52902", + "https://bcr.bazel.build/modules/nlohmann_json/3.6.1/MODULE.bazel": "6f7b417dcc794d9add9e556673ad25cb3ba835224290f4f848f8e2db1e1fca74", + "https://bcr.bazel.build/modules/nlohmann_json/3.6.1/source.json": "f448c6e8963fdfa7eb831457df83ad63d3d6355018f6574fb017e8169deb43a9", + "https://bcr.bazel.build/modules/package_metadata/0.0.7/MODULE.bazel": "7adb03933fc8401f495800cf4eafcff0edc6da0ff55c7db223ef69d19f689486", + "https://bcr.bazel.build/modules/package_metadata/0.0.7/source.json": "50639625e937b56115012674c797cca7a05a96b4878c87d803c13dc2b31de8a0", + "https://bcr.bazel.build/modules/platforms/0.0.10/MODULE.bazel": "8cb8efaf200bdeb2150d93e162c40f388529a25852b332cec879373771e48ed5", + "https://bcr.bazel.build/modules/platforms/0.0.11/MODULE.bazel": "0daefc49732e227caa8bfa834d65dc52e8cc18a2faf80df25e8caea151a9413f", + "https://bcr.bazel.build/modules/platforms/0.0.4/MODULE.bazel": "9b328e31ee156f53f3c416a64f8491f7eb731742655a47c9eec4703a71644aee", + "https://bcr.bazel.build/modules/platforms/0.0.5/MODULE.bazel": "5733b54ea419d5eaf7997054bb55f6a1d0b5ff8aedf0176fef9eea44f3acda37", + "https://bcr.bazel.build/modules/platforms/0.0.6/MODULE.bazel": "ad6eeef431dc52aefd2d77ed20a4b353f8ebf0f4ecdd26a807d2da5aa8cd0615", + "https://bcr.bazel.build/modules/platforms/0.0.7/MODULE.bazel": "72fd4a0ede9ee5c021f6a8dd92b503e089f46c227ba2813ff183b71616034814", + "https://bcr.bazel.build/modules/platforms/0.0.8/MODULE.bazel": "9f142c03e348f6d263719f5074b21ef3adf0b139ee4c5133e2aa35664da9eb2d", + "https://bcr.bazel.build/modules/platforms/0.0.9/MODULE.bazel": "4a87a60c927b56ddd67db50c89acaa62f4ce2a1d2149ccb63ffd871d5ce29ebc", + "https://bcr.bazel.build/modules/platforms/1.0.0/MODULE.bazel": "f05feb42b48f1b3c225e4ccf351f367be0371411a803198ec34a389fb22aa580", + "https://bcr.bazel.build/modules/platforms/1.0.0/source.json": "f4ff1fd412e0246fd38c82328eb209130ead81d62dcd5a9e40910f867f733d96", + "https://bcr.bazel.build/modules/protobuf/21.7/MODULE.bazel": "a5a29bb89544f9b97edce05642fac225a808b5b7be74038ea3640fae2f8e66a7", + "https://bcr.bazel.build/modules/protobuf/27.0/MODULE.bazel": "7873b60be88844a0a1d8f80b9d5d20cfbd8495a689b8763e76c6372998d3f64c", + "https://bcr.bazel.build/modules/protobuf/29.0-rc2/MODULE.bazel": "6241d35983510143049943fc0d57937937122baf1b287862f9dc8590fc4c37df", + "https://bcr.bazel.build/modules/protobuf/29.1/MODULE.bazel": "557c3457560ff49e122ed76c0bc3397a64af9574691cb8201b4e46d4ab2ecb95", + "https://bcr.bazel.build/modules/protobuf/3.19.0/MODULE.bazel": "6b5fbb433f760a99a22b18b6850ed5784ef0e9928a72668b66e4d7ccd47db9b0", + "https://bcr.bazel.build/modules/protobuf/32.1/MODULE.bazel": "89cd2866a9cb07fee9ff74c41ceace11554f32e0d849de4e23ac55515cfada4d", + "https://bcr.bazel.build/modules/protobuf/33.4/MODULE.bazel": "114775b816b38b6d0ca620450d6b02550c60ceedfdc8d9a229833b34a223dc42", + "https://bcr.bazel.build/modules/protobuf/33.4/source.json": "555f8686b4c7d6b5ba731fbea13bf656b4bfd9a7ff629c1d9d3f6e1d6155de79", + "https://bcr.bazel.build/modules/pybind11_bazel/2.11.1/MODULE.bazel": "88af1c246226d87e65be78ed49ecd1e6f5e98648558c14ce99176da041dc378e", + "https://bcr.bazel.build/modules/pybind11_bazel/2.12.0/MODULE.bazel": "e6f4c20442eaa7c90d7190d8dc539d0ab422f95c65a57cc59562170c58ae3d34", + "https://bcr.bazel.build/modules/pybind11_bazel/2.12.0/source.json": "6900fdc8a9e95866b8c0d4ad4aba4d4236317b5c1cd04c502df3f0d33afed680", + "https://bcr.bazel.build/modules/re2/2023-09-01/MODULE.bazel": "cb3d511531b16cfc78a225a9e2136007a48cf8a677e4264baeab57fe78a80206", + "https://bcr.bazel.build/modules/re2/2024-07-02.bcr.1/MODULE.bazel": "b4963dda9b31080be1905ef085ecd7dd6cd47c05c79b9cdf83ade83ab2ab271a", + "https://bcr.bazel.build/modules/re2/2024-07-02.bcr.1/source.json": "2ff292be6ef3340325ce8a045ecc326e92cbfab47c7cbab4bd85d28971b97ac4", + "https://bcr.bazel.build/modules/re2/2024-07-02/MODULE.bazel": "0eadc4395959969297cbcf31a249ff457f2f1d456228c67719480205aa306daa", + "https://bcr.bazel.build/modules/rules_android/0.1.1/MODULE.bazel": "48809ab0091b07ad0182defb787c4c5328bd3a278938415c00a7b69b50c4d3a8", + "https://bcr.bazel.build/modules/rules_android/0.1.1/source.json": "e6986b41626ee10bdc864937ffb6d6bf275bb5b9c65120e6137d56e6331f089e", + "https://bcr.bazel.build/modules/rules_apple/3.16.0/MODULE.bazel": "0d1caf0b8375942ce98ea944be754a18874041e4e0459401d925577624d3a54a", + "https://bcr.bazel.build/modules/rules_apple/4.1.0/MODULE.bazel": "76e10fd4a48038d3fc7c5dc6e63b7063bbf5304a2e3bd42edda6ec660eebea68", + "https://bcr.bazel.build/modules/rules_apple/4.1.0/source.json": "8ee81e1708756f81b343a5eb2b2f0b953f1d25c4ab3d4a68dc02754872e80715", + "https://bcr.bazel.build/modules/rules_cc/0.0.1/MODULE.bazel": "cb2aa0747f84c6c3a78dad4e2049c154f08ab9d166b1273835a8174940365647", + "https://bcr.bazel.build/modules/rules_cc/0.0.10/MODULE.bazel": "ec1705118f7eaedd6e118508d3d26deba2a4e76476ada7e0e3965211be012002", + "https://bcr.bazel.build/modules/rules_cc/0.0.13/MODULE.bazel": "0e8529ed7b323dad0775ff924d2ae5af7640b23553dfcd4d34344c7e7a867191", + "https://bcr.bazel.build/modules/rules_cc/0.0.15/MODULE.bazel": "6704c35f7b4a72502ee81f61bf88706b54f06b3cbe5558ac17e2e14666cd5dcc", + "https://bcr.bazel.build/modules/rules_cc/0.0.16/MODULE.bazel": "7661303b8fc1b4d7f532e54e9d6565771fea666fbdf839e0a86affcd02defe87", + "https://bcr.bazel.build/modules/rules_cc/0.0.17/MODULE.bazel": "2ae1d8f4238ec67d7185d8861cb0a2cdf4bc608697c331b95bf990e69b62e64a", + "https://bcr.bazel.build/modules/rules_cc/0.0.2/MODULE.bazel": "6915987c90970493ab97393024c156ea8fb9f3bea953b2f3ec05c34f19b5695c", + "https://bcr.bazel.build/modules/rules_cc/0.0.6/MODULE.bazel": "abf360251023dfe3efcef65ab9d56beefa8394d4176dd29529750e1c57eaa33f", + "https://bcr.bazel.build/modules/rules_cc/0.0.8/MODULE.bazel": "964c85c82cfeb6f3855e6a07054fdb159aced38e99a5eecf7bce9d53990afa3e", + "https://bcr.bazel.build/modules/rules_cc/0.0.9/MODULE.bazel": "836e76439f354b89afe6a911a7adf59a6b2518fafb174483ad78a2a2fde7b1c5", + "https://bcr.bazel.build/modules/rules_cc/0.1.1/MODULE.bazel": "2f0222a6f229f0bf44cd711dc13c858dad98c62d52bd51d8fc3a764a83125513", + "https://bcr.bazel.build/modules/rules_cc/0.1.2/MODULE.bazel": "557ddc3a96858ec0d465a87c0a931054d7dcfd6583af2c7ed3baf494407fd8d0", + "https://bcr.bazel.build/modules/rules_cc/0.2.0/MODULE.bazel": "b5c17f90458caae90d2ccd114c81970062946f49f355610ed89bebf954f5783c", + "https://bcr.bazel.build/modules/rules_cc/0.2.13/MODULE.bazel": "eecdd666eda6be16a8d9dc15e44b5c75133405e820f620a234acc4b1fdc5aa37", + "https://bcr.bazel.build/modules/rules_cc/0.2.17/MODULE.bazel": "1849602c86cb60da8613d2de887f9566a6d354a6df6d7009f9d04a14402f9a84", + "https://bcr.bazel.build/modules/rules_cc/0.2.17/source.json": "3832f45d145354049137c0090df04629d9c2b5493dc5c2bf46f1834040133a07", + "https://bcr.bazel.build/modules/rules_cc/0.2.8/MODULE.bazel": "f1df20f0bf22c28192a794f29b501ee2018fa37a3862a1a2132ae2940a23a642", + "https://bcr.bazel.build/modules/rules_foreign_cc/0.9.0/MODULE.bazel": "c9e8c682bf75b0e7c704166d79b599f93b72cfca5ad7477df596947891feeef6", + "https://bcr.bazel.build/modules/rules_fuzzing/0.5.2/MODULE.bazel": "40c97d1144356f52905566c55811f13b299453a14ac7769dfba2ac38192337a8", + "https://bcr.bazel.build/modules/rules_java/4.0.0/MODULE.bazel": "5a78a7ae82cd1a33cef56dc578c7d2a46ed0dca12643ee45edbb8417899e6f74", + "https://bcr.bazel.build/modules/rules_java/5.3.5/MODULE.bazel": "a4ec4f2db570171e3e5eb753276ee4b389bae16b96207e9d3230895c99644b86", + "https://bcr.bazel.build/modules/rules_java/6.5.2/MODULE.bazel": "1d440d262d0e08453fa0c4d8f699ba81609ed0e9a9a0f02cd10b3e7942e61e31", + "https://bcr.bazel.build/modules/rules_java/7.10.0/MODULE.bazel": "530c3beb3067e870561739f1144329a21c851ff771cd752a49e06e3dc9c2e71a", + "https://bcr.bazel.build/modules/rules_java/7.12.2/MODULE.bazel": "579c505165ee757a4280ef83cda0150eea193eed3bef50b1004ba88b99da6de6", + "https://bcr.bazel.build/modules/rules_java/7.2.0/MODULE.bazel": "06c0334c9be61e6cef2c8c84a7800cef502063269a5af25ceb100b192453d4ab", + "https://bcr.bazel.build/modules/rules_java/7.6.1/MODULE.bazel": "2f14b7e8a1aa2f67ae92bc69d1ec0fa8d9f827c4e17ff5e5f02e91caa3b2d0fe", + "https://bcr.bazel.build/modules/rules_java/8.6.1/MODULE.bazel": "f4808e2ab5b0197f094cabce9f4b006a27766beb6a9975931da07099560ca9c2", + "https://bcr.bazel.build/modules/rules_java/9.1.0/MODULE.bazel": "ee63f27e36a3fada80342869361182f120a9819c74320e8e65b1e04ba0cd7a9d", + "https://bcr.bazel.build/modules/rules_java/9.1.0/source.json": "da589573c1dee2c9ac4a568b301269a2e8191110ff0345c1a959fa7ea6c4dfd6", + "https://bcr.bazel.build/modules/rules_jvm_external/4.4.2/MODULE.bazel": "a56b85e418c83eb1839819f0b515c431010160383306d13ec21959ac412d2fe7", + "https://bcr.bazel.build/modules/rules_jvm_external/5.1/MODULE.bazel": "33f6f999e03183f7d088c9be518a63467dfd0be94a11d0055fe2d210f89aa909", + "https://bcr.bazel.build/modules/rules_jvm_external/5.2/MODULE.bazel": "d9351ba35217ad0de03816ef3ed63f89d411349353077348a45348b096615036", + "https://bcr.bazel.build/modules/rules_jvm_external/6.3/MODULE.bazel": "c998e060b85f71e00de5ec552019347c8bca255062c990ac02d051bb80a38df0", + "https://bcr.bazel.build/modules/rules_jvm_external/6.7/MODULE.bazel": "e717beabc4d091ecb2c803c2d341b88590e9116b8bf7947915eeb33aab4f96dd", + "https://bcr.bazel.build/modules/rules_jvm_external/6.7/source.json": "5426f412d0a7fc6b611643376c7e4a82dec991491b9ce5cb1cfdd25fe2e92be4", + "https://bcr.bazel.build/modules/rules_kotlin/1.9.6/MODULE.bazel": "d269a01a18ee74d0335450b10f62c9ed81f2321d7958a2934e44272fe82dcef3", + "https://bcr.bazel.build/modules/rules_kotlin/1.9.6/source.json": "2faa4794364282db7c06600b7e5e34867a564ae91bda7cae7c29c64e9466b7d5", + "https://bcr.bazel.build/modules/rules_license/0.0.3/MODULE.bazel": "627e9ab0247f7d1e05736b59dbb1b6871373de5ad31c3011880b4133cafd4bd0", + "https://bcr.bazel.build/modules/rules_license/0.0.7/MODULE.bazel": "088fbeb0b6a419005b89cf93fe62d9517c0a2b8bb56af3244af65ecfe37e7d5d", + "https://bcr.bazel.build/modules/rules_license/1.0.0/MODULE.bazel": "a7fda60eefdf3d8c827262ba499957e4df06f659330bbe6cdbdb975b768bb65c", + "https://bcr.bazel.build/modules/rules_license/1.0.0/source.json": "a52c89e54cc311196e478f8382df91c15f7a2bfdf4c6cd0e2675cc2ff0b56efb", + "https://bcr.bazel.build/modules/rules_pkg/0.7.0/MODULE.bazel": "df99f03fc7934a4737122518bb87e667e62d780b610910f0447665a7e2be62dc", + "https://bcr.bazel.build/modules/rules_pkg/1.0.1/MODULE.bazel": "5b1df97dbc29623bccdf2b0dcd0f5cb08e2f2c9050aab1092fd39a41e82686ff", + "https://bcr.bazel.build/modules/rules_pkg/1.0.1/source.json": "bd82e5d7b9ce2d31e380dd9f50c111d678c3bdaca190cb76b0e1c71b05e1ba8a", + "https://bcr.bazel.build/modules/rules_proto/4.0.0/MODULE.bazel": "a7a7b6ce9bee418c1a760b3d84f83a299ad6952f9903c67f19e4edd964894e06", + "https://bcr.bazel.build/modules/rules_proto/5.3.0-21.7/MODULE.bazel": "e8dff86b0971688790ae75528fe1813f71809b5afd57facb44dad9e8eca631b7", + "https://bcr.bazel.build/modules/rules_proto/6.0.2/MODULE.bazel": "ce916b775a62b90b61888052a416ccdda405212b6aaeb39522f7dc53431a5e73", + "https://bcr.bazel.build/modules/rules_proto/7.1.0/MODULE.bazel": "002d62d9108f75bb807cd56245d45648f38275cb3a99dcd45dfb864c5d74cb96", + "https://bcr.bazel.build/modules/rules_proto/7.1.0/source.json": "39f89066c12c24097854e8f57ab8558929f9c8d474d34b2c00ac04630ad8940e", + "https://bcr.bazel.build/modules/rules_shell/0.2.0/MODULE.bazel": "fda8a652ab3c7d8fee214de05e7a9916d8b28082234e8d2c0094505c5268ed3c", + "https://bcr.bazel.build/modules/rules_shell/0.3.0/MODULE.bazel": "de4402cd12f4cc8fda2354fce179fdb068c0b9ca1ec2d2b17b3e21b24c1a937b", + "https://bcr.bazel.build/modules/rules_shell/0.6.1/MODULE.bazel": "72e76b0eea4e81611ef5452aa82b3da34caca0c8b7b5c0c9584338aa93bae26b", + "https://bcr.bazel.build/modules/rules_shell/0.6.1/source.json": "20ec05cd5e592055e214b2da8ccb283c7f2a421ea0dc2acbf1aa792e11c03d0c", + "https://bcr.bazel.build/modules/rules_swift/1.16.0/MODULE.bazel": "4a09f199545a60d09895e8281362b1ff3bb08bbde69c6fc87aff5b92fcc916ca", + "https://bcr.bazel.build/modules/rules_swift/2.1.1/MODULE.bazel": "494900a80f944fc7aa61500c2073d9729dff0b764f0e89b824eb746959bc1046", + "https://bcr.bazel.build/modules/rules_swift/2.4.0/MODULE.bazel": "1639617eb1ede28d774d967a738b4a68b0accb40650beadb57c21846beab5efd", + "https://bcr.bazel.build/modules/rules_swift/3.1.2/MODULE.bazel": "72c8f5cf9d26427cee6c76c8e3853eb46ce6b0412a081b2b6db6e8ad56267400", + "https://bcr.bazel.build/modules/rules_swift/3.1.2/source.json": "e85761f3098a6faf40b8187695e3de6d97944e98abd0d8ce579cb2daf6319a66", + "https://bcr.bazel.build/modules/stardoc/0.5.1/MODULE.bazel": "1a05d92974d0c122f5ccf09291442580317cdd859f07a8655f1db9a60374f9f8", + "https://bcr.bazel.build/modules/stardoc/0.5.3/MODULE.bazel": "c7f6948dae6999bf0db32c1858ae345f112cacf98f174c7a8bb707e41b974f1c", + "https://bcr.bazel.build/modules/stardoc/0.7.0/MODULE.bazel": "05e3d6d30c099b6770e97da986c53bd31844d7f13d41412480ea265ac9e8079c", + "https://bcr.bazel.build/modules/swift_argument_parser/1.3.1.1/MODULE.bazel": "5e463fbfba7b1701d957555ed45097d7f984211330106ccd1352c6e0af0dcf91", + "https://bcr.bazel.build/modules/swift_argument_parser/1.3.1.2/MODULE.bazel": "75aab2373a4bbe2a1260b9bf2a1ebbdbf872d3bd36f80bff058dccd82e89422f", + "https://bcr.bazel.build/modules/swift_argument_parser/1.3.1.2/source.json": "5fba48bbe0ba48761f9e9f75f92876cafb5d07c0ce059cc7a8027416de94a05b", + "https://bcr.bazel.build/modules/toml.bzl/0.4.1/MODULE.bazel": "6bc0b938f03ade8d58c2fca0ad5c3fa12b4764e1e1927ad50b0c860286db2167", + "https://bcr.bazel.build/modules/toml.bzl/0.4.1/source.json": "86a90afd8b43c9b69ad31f5c03998c3adcf4b08175e621addb93e3a38eec538b", + "https://bcr.bazel.build/modules/upb/0.0.0-20220923-a547704/MODULE.bazel": "7298990c00040a0e2f121f6c32544bab27d4452f80d9ce51349b1a28f3005c43", + "https://bcr.bazel.build/modules/zlib/1.2.11/MODULE.bazel": "07b389abc85fdbca459b69e2ec656ae5622873af3f845e1c9d80fe179f3effa0", + "https://bcr.bazel.build/modules/zlib/1.3.1.bcr.5/MODULE.bazel": "eec517b5bbe5492629466e11dae908d043364302283de25581e3eb944326c4ca", + "https://bcr.bazel.build/modules/zlib/1.3.1.bcr.5/source.json": "22bc55c47af97246cfc093d0acf683a7869377de362b5d1c552c2c2e16b7a806", + "https://bcr.bazel.build/modules/zlib/1.3.1/MODULE.bazel": "751c9940dcfe869f5f7274e1295422a34623555916eb98c174c1e945594bf198" + }, + "selectedYankedVersions": {}, + "moduleExtensions": { + "@@pybind11_bazel+//:internal_configure.bzl%internal_configure_extension": { + "general": { + "bzlTransitiveDigest": "b+RP7Sgl8KN0VHamrgTqzGLuYPcQ/Mo4ptNkkHUIIlA=", + "usagesDigest": "D1r3lfzMuUBFxgG8V6o0bQTLMk3GkaGOaPzw53wrwyw=", + "recordedInputs": [ + "REPO_MAPPING:pybind11_bazel+,bazel_tools bazel_tools", + "FILE:@@pybind11_bazel+//MODULE.bazel e6f4c20442eaa7c90d7190d8dc539d0ab422f95c65a57cc59562170c58ae3d34" + ], + "generatedRepoSpecs": { + "pybind11": { + "repoRuleId": "@@bazel_tools//tools/build_defs/repo:http.bzl%http_archive", + "attributes": { + "build_file": "@@pybind11_bazel+//:pybind11-BUILD.bazel", + "strip_prefix": "pybind11-2.12.0", + "urls": [ + "https://github.com/pybind/pybind11/archive/v2.12.0.zip" + ] + } + } + } + } + }, + "@@rules_kotlin+//src/main/starlark/core/repositories:bzlmod_setup.bzl%rules_kotlin_extensions": { + "general": { + "bzlTransitiveDigest": "Ga4z8lQy1YQ5rAMy+dOl0dqcCEBnYNCXku8x3YQmDZI=", + "usagesDigest": "QI2z8ZUR+mqtbwsf2fLqYdJAkPOHdOV+tF2yVAUgRzw=", + "recordedInputs": [ + "REPO_MAPPING:rules_kotlin+,bazel_tools bazel_tools" + ], + "generatedRepoSpecs": { + "com_github_jetbrains_kotlin_git": { + "repoRuleId": "@@rules_kotlin+//src/main/starlark/core/repositories:compiler.bzl%kotlin_compiler_git_repository", + "attributes": { + "urls": [ + "https://github.com/JetBrains/kotlin/releases/download/v1.9.23/kotlin-compiler-1.9.23.zip" + ], + "sha256": "93137d3aab9afa9b27cb06a824c2324195c6b6f6179d8a8653f440f5bd58be88" + } + }, + "com_github_jetbrains_kotlin": { + "repoRuleId": "@@rules_kotlin+//src/main/starlark/core/repositories:compiler.bzl%kotlin_capabilities_repository", + "attributes": { + "git_repository_name": "com_github_jetbrains_kotlin_git", + "compiler_version": "1.9.23" + } + }, + "com_github_google_ksp": { + "repoRuleId": "@@rules_kotlin+//src/main/starlark/core/repositories:ksp.bzl%ksp_compiler_plugin_repository", + "attributes": { + "urls": [ + "https://github.com/google/ksp/releases/download/1.9.23-1.0.20/artifacts.zip" + ], + "sha256": "ee0618755913ef7fd6511288a232e8fad24838b9af6ea73972a76e81053c8c2d", + "strip_version": "1.9.23-1.0.20" + } + }, + "com_github_pinterest_ktlint": { + "repoRuleId": "@@bazel_tools//tools/build_defs/repo:http.bzl%http_file", + "attributes": { + "sha256": "01b2e0ef893383a50dbeb13970fe7fa3be36ca3e83259e01649945b09d736985", + "urls": [ + "https://github.com/pinterest/ktlint/releases/download/1.3.0/ktlint" + ], + "executable": true + } + }, + "rules_android": { + "repoRuleId": "@@bazel_tools//tools/build_defs/repo:http.bzl%http_archive", + "attributes": { + "sha256": "cd06d15dd8bb59926e4d65f9003bfc20f9da4b2519985c27e190cddc8b7a7806", + "strip_prefix": "rules_android-0.1.1", + "urls": [ + "https://github.com/bazelbuild/rules_android/archive/v0.1.1.zip" + ] + } + } + } + } + }, + "@@rules_python+//python/uv:uv.bzl%uv": { + "general": { + "bzlTransitiveDigest": "DafUArm1CsjJZcemT0FvUq9rUch+pq38+JqfOBJCpm8=", + "usagesDigest": "6yXGw7XDyXjOfqBL0SBu1YBEMMYPQzCE3jTzUCkxPgg=", + "recordedInputs": [ + "REPO_MAPPING:rules_python+,bazel_tools bazel_tools", + "REPO_MAPPING:rules_python+,platforms platforms" + ], + "generatedRepoSpecs": { + "uv": { + "repoRuleId": "@@rules_python+//python/uv/private:uv_toolchains_repo.bzl%uv_toolchains_repo", + "attributes": { + "toolchain_type": "'@@rules_python+//python/uv:uv_toolchain_type'", + "toolchain_names": [ + "none" + ], + "toolchain_implementations": { + "none": "'@@rules_python+//python:none'" + }, + "toolchain_compatible_with": { + "none": [ + "@platforms//:incompatible" + ] + }, + "toolchain_target_settings": {} + } + } + } + } + } + }, + "facts": { + "@@rules_python+//python/extensions:pip.bzl%pip": { + "dist_hashes": { + "https://pypi.org/simple": { + "backports-tarfile": { + "https://files.pythonhosted.org/packages/86/72/cd9b395f25e290e633655a100af28cb253e4393396264a98bd5f5951d50f/backports_tarfile-1.2.0.tar.gz": "sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991", + "https://files.pythonhosted.org/packages/b9/fa/123043af240e49752f1c4bd24da5053b6bd00cad78c2be53c0d1e8b975bc/backports.tarfile-1.2.0-py3-none-any.whl": "sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34" + }, + "certifi": { + "https://files.pythonhosted.org/packages/0b/a7/71ac2cff56fec219ed242bb11b8efb69fcc4bec75db06fb7bfe35de520e6/certifi-2026.7.22-py3-none-any.whl": "sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775", + "https://files.pythonhosted.org/packages/a3/c2/24167ea9858356b47a87a50d39908bfdb72ceeefe0041586e704e5376b3a/certifi-2026.7.22.tar.gz": "sha256:741e2c3b351ddf169a738da9f2c048608ff7f2c5cc02f1ebc6b118bb090d5d55" + }, + "cffi": { + "https://files.pythonhosted.org/packages/04/8c/b925975448cf20634a9fbd5efceb807219db452653648d2897c0989cab2d/cffi-2.1.0-cp311-cp311-musllinux_1_2_x86_64.whl": "sha256:89095c1968b4ba8285840e131bf2891b09ae137fe2146905acae0354fbce1b5e", + "https://files.pythonhosted.org/packages/05/ef/6cd4f8c671517162379dc79cfae5aea9106bc38abb89628d5c16adf6a838/cffi-2.1.0-cp315-cp315-win_arm64.whl": "sha256:8d35c139744adb3e727cd51b1a18324bbe44b8bd41bf8322bca4d41289f48eda", + "https://files.pythonhosted.org/packages/0f/6f/ade5ce9863a57992a6ea3d0d10d7e29b8749fc127204b3d493d667b2815f/cffi-2.1.0-cp314-cp314-win32.whl": "sha256:1854b724d00f6654c742097d5387569021be12d3a0f770eae1df8f8acfcc6acd", + "https://files.pythonhosted.org/packages/11/b6/12fc55092817a5faa26fb8c40c7f9d662e11a46ee248c137aafc42517d92/cffi-2.1.0-cp315-cp315t-macosx_10_15_x86_64.whl": "sha256:f9912624a0c0b834b7520d7769b3644453aabc0a7e1c839da7359f050750e9bc", + "https://files.pythonhosted.org/packages/14/d0/117dcd9209255ad8571fbc8c92ef32593a1d294dcec91ddc4e4db50606f2/cffi-2.1.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "sha256:eb4e8997a49aa2c08a3e43c9045d224448b8941d88e7ac163c7d383e560cbf98", + "https://files.pythonhosted.org/packages/14/f0/134c00ce0779ec86dea2aa1aac69339c2741a8045072676763512363a2ea/cffi-2.1.0-cp314-cp314t-macosx_10_15_x86_64.whl": "sha256:7ea6b3e2c4250ff1de21c630fe72d0f63eb95c2c32ffbf64a358cf4a8836d714", + "https://files.pythonhosted.org/packages/19/e5/d3cc82a4a0be7902af279c04181ad038449c096734464a5ae1de3e1401bd/cffi-2.1.0-cp315-cp315t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "sha256:0611e7ebf90573a535ebdc33ae9da222d037853983e13359f580fab781ca017f", + "https://files.pythonhosted.org/packages/1b/dc/5620cf930688be01f2d673804291de757a934c90b946dbdc3d84130c2ea4/cffi-2.1.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "sha256:b6422532152adf4e59b110cb2808cee7a033800952f5c036b4af047ee43199e7", + "https://files.pythonhosted.org/packages/1e/85/990925db5df586ec90beb97529c853497e7f85ba0234830447faf41c3057/cffi-2.1.0-cp312-cp312-macosx_10_15_x86_64.whl": "sha256:df2b82571a1b30f58a87bf4e5a9e78d2b1eff6c6ce8fd3aa3757221f93f0863f", + "https://files.pythonhosted.org/packages/20/71/7c8372d30e42415602ed9f268f7cfd66f1b855fed881ecd168bcb45dbc0b/cffi-2.1.0-cp314-cp314-macosx_10_15_x86_64.whl": "sha256:1ff3456eab0d889592d1936d6125bbfbc7ae4d3354a700f8bd80450a66445d4d", + "https://files.pythonhosted.org/packages/22/d7/1a74539db16d8bfd839ff1515948948efbb162e574650fd3d846896eea95/cffi-2.1.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "sha256:88023dfe18799507b73f1dbb0d14326a17465de1bc9c9c7655c22845e9ddc3a2", + "https://files.pythonhosted.org/packages/22/f0/a2fc43084c0433caf7f461bccc013e28f848d04ee1c5ed7fce71423cf4d9/cffi-2.1.0-cp311-cp311-musllinux_1_2_i686.whl": "sha256:7762faa47e8ff7eb80bd261d9a7d8eea2d8baa69de5e95b70c1f338bbe712f02", + "https://files.pythonhosted.org/packages/28/3b/fad54de07260b93ddeef4b96d0131d57ea900675df1d410ae1deee52d7a6/cffi-2.1.0-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.whl": "sha256:33eb1ad83ebe8f313e0df035c406227d55a79456704a863fad9842136af5ad7d", + "https://files.pythonhosted.org/packages/28/ed/c127d3ac36e899c965e3361357c3befacd6578c03f40125183e41c3b219e/cffi-2.1.0-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.whl": "sha256:6ca4919c6e4f89aa99c42510b42cf54596892c00b3f9077f6bdd1505e24b9c8d", + "https://files.pythonhosted.org/packages/2c/0e/fac738d73728c6cea2a88a2883dca54892496cbba88a1dc1f2909cb8a6f5/cffi-2.1.0-cp315-cp315-manylinux2014_s390x.manylinux_2_17_s390x.whl": "sha256:2b71d409cccee78310ab5dec549aed052aaea483346e282c7b02362596e01bb0", + "https://files.pythonhosted.org/packages/2c/d8/772b8259bf75749adffb1c546828978381fb516f60cf701f6c83daf60c85/cffi-2.1.0-cp315-cp315-win32.whl": "sha256:0a42c688d19fca6e095a53c6a6e2295a5b050a8b289f109adab02a9e61a25de6", + "https://files.pythonhosted.org/packages/2e/1a/cc6ae6c2913a03aab8898eee57963cf1035b8df5872ed8b9115fcc7e2be8/cffi-2.1.0-cp314-cp314-win_arm64.whl": "sha256:7d28dff1db6764108bc30788d85d61c876beff416d9a49cb9dd7c5a9f34f5804", + "https://files.pythonhosted.org/packages/2e/d2/065fcae1c73979fac8e054462478d0ff8a29c40cdc2ed7ea5676a061df53/cffi-2.1.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "sha256:276f20fffd7b396e12516ba8edf9509210ac248cbbc5acbc39cd512f9f59ebe6", + "https://files.pythonhosted.org/packages/2f/dd/afa2191fc6d57fedd26e5844a2fe2fcc0bbfa00961bbaa5a41e4921e7cca/cffi-2.1.0-cp315-cp315-win_amd64.whl": "sha256:bccbbb5ee76a61f9d99b5bf3846a51d7fca4b6a732fe46f89295610edaf41853", + "https://files.pythonhosted.org/packages/38/37/04f54b8e63a02f3d908332c9effbf8c366167c6f733ed8a3d4f79b7e2a1e/cffi-2.1.0-cp313-cp313-musllinux_1_2_aarch64.whl": "sha256:961be50688f7fba2fa65f63712d3b9b341a22311f5253460ce933f52f0de1c8c", + "https://files.pythonhosted.org/packages/38/66/04781a77b411f0bb5b234d62c1814754ab75ebe455ccff1b08e8d7aae98f/cffi-2.1.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "sha256:4d433a51f1870e43a13b6732f92aaf540ff77c2015097c78556f75a2d6c030e0", + "https://files.pythonhosted.org/packages/3a/a6/e879bb68cc23a2bc9ba8f4b7d8019f0c2694bad2ab6c4a3701d429439f58/cffi-2.1.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "sha256:ff067a8d8d880e7809e4ac88eb009bb848870115317b306666502ccad30b147f", + "https://files.pythonhosted.org/packages/3b/30/c806937ed5e4c2c7ac30d9d6b76b5dc57ff8b75d83800d9bb11a8253cf2a/cffi-2.1.0-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "sha256:a016194dbe13d14ee9556e734b772d8d67b947092b268d757fd4290e3ba2dfc2", + "https://files.pythonhosted.org/packages/41/aa/3c1409cdd26094efacd1c36c66e0a6eb9d4296e4fd4f9901b8b2042f4323/cffi-2.1.0-cp310-cp310-musllinux_1_2_i686.whl": "sha256:c5f5df567f6eb216de69be06ce55c8b714090fae02b18a3b40da8163b8c5fa9c", + "https://files.pythonhosted.org/packages/41/de/92b9eeed4ae4a21d6fd9b2a2c8505cbed573299902ea73981cc13f7ff62c/cffi-2.1.0-cp314-cp314-win_amd64.whl": "sha256:1b96bfe2c4bd825681b7d311ad6d9b7280a091f43e8f63da5729638083cd3bfb", + "https://files.pythonhosted.org/packages/45/ca/f91641185cdd90c36d317a9dc7f85e88ef8682d8b300977baff5e23c35d8/cffi-2.1.0-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.whl": "sha256:19c54ac121cad98450b4896fa9a43ee0180d57bc4bc911a33db6cab1efab6cd3", + "https://files.pythonhosted.org/packages/4b/92/e7bb136ad6b5352603732cf907ef862ca103f20f2031c1735a46300c20c9/cffi-2.1.0-cp312-cp312-macosx_11_0_arm64.whl": "sha256:78474632761faa0fb96f30b1c928c84ebcf68713cbb80d15bab09dfe61640fde", + "https://files.pythonhosted.org/packages/4b/a4/77b53abbf7a1e0beb9637edbef2a94d15f9c822f591e85d439ffd91519a6/cffi-2.1.0-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "sha256:46b1c8db8f6122420f32d02fffb924c2fe9bc772d228c7c711748fff56aabb2b", + "https://files.pythonhosted.org/packages/50/d8/3b86aba791cb610d24e8a3e1b2cd529e71fa15096b04e4d4e360049d4a4c/cffi-2.1.0-cp314-cp314t-macosx_11_0_arm64.whl": "sha256:6af371f3767faeffc6ac1ef57cdfd25844403e9d3f476c5537caee499de96376", + "https://files.pythonhosted.org/packages/55/c7/8c8c50cb11c6750051daf12164098a9a6f027ac4356967fd4d800a07f242/cffi-2.1.0-cp315-cp315-ios_13_0_arm64_iphoneos.whl": "sha256:2e9dabb9abcb7ad15938c7196ad5c1718a4e6d33cc79b4c0209bdb64c4a54a5c", + "https://files.pythonhosted.org/packages/57/5f/ff100cae70ebe9d8df1c01a00e510e45d9adb5c1fdda84791b199141de97/cffi-2.1.0.tar.gz": "sha256:efc1cdd798b1aaf39b4610bba7aad28c9bea9b910f25c784ccf9ec1fa719d1f9", + "https://files.pythonhosted.org/packages/58/0c/f528df19cc94b675087324d4760d9e6d5bfae97d6217aa4fac43de4f5fcc/cffi-2.1.0-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.whl": "sha256:d9fafc5aa2e2a39aaf7f8cc0c1f044a9b07fca12e558dca53a3cc5c654ad67a7", + "https://files.pythonhosted.org/packages/58/85/7ae00d5c8dd6266f4e944c3db630f3c5c9a98b61d469c714d848b1d8138a/cffi-2.1.0-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl": "sha256:a95b05f9baf29b91171b3a8bd2020b028835243e7b0ff6bb23e2a3c228518b1b", + "https://files.pythonhosted.org/packages/5a/02/d5e6c43ea85c41bda2a184a3418f195fe7cf602967a8d2b94e085b83deef/cffi-2.1.0-cp315-cp315-musllinux_1_2_x86_64.whl": "sha256:af5e2915d41fe6c961694d7bfdc8562942638200f3ce2765dfb8b745cf997629", + "https://files.pythonhosted.org/packages/5a/47/59eb7975cb0e4ef0afa764ea945b29a5bb4537a9f771cb7d6c8a5dd74c95/cffi-2.1.0-cp313-cp313-win_amd64.whl": "sha256:8e74a6135550c4748af665b1b1118b6aab33b1fc6a16f9aff630af107c3b4512", + "https://files.pythonhosted.org/packages/5a/67/9e6e09409336d9e515c58367e7cfcf4f89df06ad25252675595a58eb59d5/cffi-2.1.0-cp315-cp315t-manylinux2014_s390x.manylinux_2_17_s390x.whl": "sha256:762f99479dcb369f60ab9017ad4ab97a36a1dd7c1ee5a3b15db0f4b8659120cd", + "https://files.pythonhosted.org/packages/5a/af/34fee85c48f8d94efc8597bc09470c9dd274c145f1c12e0fbc6ab6d38d74/cffi-2.1.0-cp313-cp313-win_arm64.whl": "sha256:2282cd5e38aa8accd03e99d1256af8411c84cdbee6a89d841b563fdbd1f3e50f", + "https://files.pythonhosted.org/packages/5d/7c/b7379a5704c79eda57ce075869ba70a0368d1c850f803b3c0d078d39dcaf/cffi-2.1.0-cp315-cp315-musllinux_1_2_aarch64.whl": "sha256:8f9ec95b8a043d3dfbc74d9abc6f7baf524dd27a8dc160b0a32ff9cdab650c28", + "https://files.pythonhosted.org/packages/62/f2/c9522a81c32132799a1972c39f5c5f8b4c8b9f00488a23feaa6c06f07741/cffi-2.1.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "sha256:1e9f50d192a3e525b15a75ab5114e442d83d657b7ec29182a991bc9a88fd3a66", + "https://files.pythonhosted.org/packages/65/68/9f3ef890cf3c6ab97bd531c5677f67613d302165d16f8142b2811782a614/cffi-2.1.0-cp311-cp311-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "sha256:30b65779d598c370374fefabf138d456fd6f3216bfa7bedfab1ba82025b0cd93", + "https://files.pythonhosted.org/packages/68/5a/e536c528bc8057496c360c0978559a2dc45653f89dd6151078aa7d8fca1a/cffi-2.1.0-cp315-cp315t-win32.whl": "sha256:cb96698e3c7413d906ce83f8ffd245ec1bd94707541f299d0ce4d6b0193e982b", + "https://files.pythonhosted.org/packages/69/aa/24580a278de21fd7322635556334d9b535f1cbc00b0a3919447cdf464c65/cffi-2.1.0-cp310-cp310-macosx_11_0_arm64.whl": "sha256:164bff1657b2a74f0b6d54e11c9b375bc97b931f2ca9c43fcf875838da1570dd", + "https://files.pythonhosted.org/packages/6c/d0/47e338384ab6b1004241002fa616301020cea4fc95f283506565d252f276/cffi-2.1.0-cp314-cp314t-musllinux_1_2_aarch64.whl": "sha256:c16914df9fb7f500e440e6875fa23ff5e0b31db01fa9c06af98d59a91f0dc2e4", + "https://files.pythonhosted.org/packages/6e/28/bd53988b9833e8f8ad539d26f4c07a6b3f6bcb1e9e02e7ca038250b3428d/cffi-2.1.0-cp312-cp312-musllinux_1_2_aarch64.whl": "sha256:98fff996e983a36d3aa2eca83af40c5821202e7e6f32d13ae94e3d2286f10cfe", + "https://files.pythonhosted.org/packages/6f/08/f2e7d62c460faae0926f2d6e423694aa409ced3bc1fe2927a0a6e5f05416/cffi-2.1.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "sha256:799416bae98336e400981ff6e532d67d5c709cfb30afb79865a1315f94b0e224", + "https://files.pythonhosted.org/packages/70/25/65bd5b58ea4bfdfc15cde02cb5365f89ef8ab8b2adfb8fe5c4bd4233382f/cffi-2.1.0-cp314-cp314t-musllinux_1_2_x86_64.whl": "sha256:5ecbd0499275d57506d397eebe1981cee87b47fcd9ef5c22cab7ed7644a39a94", + "https://files.pythonhosted.org/packages/70/b6/9003c33a3e7d2c1306f5962e646457dcfe5a8cd8fce6bbe02d7af25db783/cffi-2.1.0-cp310-cp310-win32.whl": "sha256:9d72af0cf10a76a600a9690078fe31c63b9588c8e86bf9fd353f713c84b5db0f", + "https://files.pythonhosted.org/packages/79/99/0d0fd37f055224085f42bbb2c022d002e17dde4a97972822327b07d84101/cffi-2.1.0-cp312-cp312-musllinux_1_2_x86_64.whl": "sha256:379de10ce1ba048b1448599d1b37b24caee16309d1ac98d3982fc997f768700b", + "https://files.pythonhosted.org/packages/7d/95/8de304305cd9204974b0ca051b86d307cafca13aa575a0ef1b44d92c0d8c/cffi-2.1.0-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl": "sha256:702c436735fbe99d59ada02a1f65cfc0d31c0ee8b7290912f8fbc5cd1e4b16c3", + "https://files.pythonhosted.org/packages/84/4c/82f132cb4418ee6d953d982b19191e87e2a6372c8a4ce36e50b69d6ade4a/cffi-2.1.0-cp313-cp313-macosx_11_0_arm64.whl": "sha256:716ff8ec22f20b4d988b12884086bcef0fc99737043e503f7a3935a6be99b1ea", + "https://files.pythonhosted.org/packages/88/a9/02cae418ec4beb282ace11958d9d4737793439d561fadc7e6d56f2e2b354/cffi-2.1.0-cp310-cp310-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "sha256:c941bb58d5a6e1c3892d86e42927ed6c180302f07e6d395d08c416e594b98b46", + "https://files.pythonhosted.org/packages/88/f6/01890cfd63c08f8eb96a8319b0443690197d240a8bd6346048cf7bde9190/cffi-2.1.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "sha256:3b926723c13eba9f81d2ef3820d63aeceec3b2d4639906047bf675cb8a7a500d", + "https://files.pythonhosted.org/packages/8a/26/710688310447531c7a22f857c7f79d9855ec18b03e04494ced723fb37e2f/cffi-2.1.0-cp310-cp310-win_amd64.whl": "sha256:fb62edb5bb52cca65fab91a63afa7561607120d26090a7e8fda6fb9f064726da", + "https://files.pythonhosted.org/packages/8b/31/e115c985105dd7ffb32444505f18ceb874bb42d992af05d5dced7ecf1980/cffi-2.1.0-cp315-cp315t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "sha256:3681e031db29958a7502f5c0c9d6bbc4c36cb20f7b104086fa642d1799631ff8", + "https://files.pythonhosted.org/packages/8c/e9/45c3a76ad8d43ad9261f4c95436da61128d3ca545d72b9612c0ab5be0b1c/cffi-2.1.0-cp313-cp313-macosx_10_15_x86_64.whl": "sha256:15faec4adfff450819f3aee0e2e02c812de6edb88203aa58807955db2003472a", + "https://files.pythonhosted.org/packages/8d/2e/cdac88979f295fde5daa69622c7d2111e56e7ceb94f211357fbe452339e4/cffi-2.1.0-cp315-cp315t-macosx_11_0_arm64.whl": "sha256:df92f2aba50eb4d96718b68ef76f2e57a57b54f2fa62333496d16c6d585a85ca", + "https://files.pythonhosted.org/packages/96/88/a996879e2eeccb815f6e3a5967b12a308257412acec882039d386bd2aa7b/cffi-2.1.0-cp313-cp313-ios_13_0_arm64_iphoneos.whl": "sha256:10537b1df4967ca26d21e5072d7d54188354483b91dc75058968d3f0cf13fbda", + "https://files.pythonhosted.org/packages/99/e2/67680bf19a6b60d2bb7ff83baefa2a4c3d2d7dc0f3277034b802e1fc504c/cffi-2.1.0-cp315-cp315-ios_13_0_arm64_iphonesimulator.whl": "sha256:37f525a7e7e50c017fdebe58b787be310ad59357ae43a053943a6e1a6c526001", + "https://files.pythonhosted.org/packages/9e/4e/e8d7cb5783f1841a3c8fb3a7735838d7484d08ec08c9f984b14cac1ac0e9/cffi-2.1.0-cp311-cp311-win_arm64.whl": "sha256:35aaea0c7ee0e58a5cd8c2fd1a48fdf7ece0d2699b7ecdda08194e9ce5dd9b3d", + "https://files.pythonhosted.org/packages/a0/17/1073b53b68c9b5ca6914adf5f8bf55aacc2d3be102418c90700160ea8605/cffi-2.1.0-cp315-cp315t-win_arm64.whl": "sha256:cbb7640ce37159548d2147b5b8c241f962143d4c71231431820783f4dc78f210", + "https://files.pythonhosted.org/packages/a0/1c/4ed5a0e5bdca6cbc275556de3328dd1b76fd0c11cc13c88fe66d1d8715f2/cffi-2.1.0-cp313-cp313-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "sha256:63960549e4f8dc41e31accb97b975abaecfc44c03e396c093a6436763c2ea7db", + "https://files.pythonhosted.org/packages/a2/a5/d4fe77b589e5e82d43ebc809bf2e6474afe8e48e32ea050b9357645b6471/cffi-2.1.0-cp311-cp311-musllinux_1_2_aarch64.whl": "sha256:9d8272c0e483b024e1b9ad029821470ed8ec65631dbd90217469da0e7cd89f1c", + "https://files.pythonhosted.org/packages/a6/cf/2b684132056f438567b61e19d690dd31cd0921ace051e0a458be6074369e/cffi-2.1.0-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.whl": "sha256:47ff3a8bfd8cb9da1af7524b965127095055654c177fcfc7578debcb015eecd0", + "https://files.pythonhosted.org/packages/a8/eb/f636456ff21a83fc13c032b58cc5dde061691546ac79efa284b2989b7982/cffi-2.1.0-cp312-cp312-win_amd64.whl": "sha256:c97f080ea627e2863524c5af3836e2270b5f5dfff1f104392b959f8df0c5d384", + "https://files.pythonhosted.org/packages/a9/d6/c72eecca433cd3e681c65ed313ab4835d9d4a379704d0f628a6a05f51c2e/cffi-2.1.0-cp313-cp313-musllinux_1_2_x86_64.whl": "sha256:bf5c6cf48238b0eb4c086978c492ad1cbc22373fc5b2d7353b3a598ce6db887a", + "https://files.pythonhosted.org/packages/b0/80/c138990aa2a70b1a269f6e06348729836d733d6f970867943f61d367f8cc/cffi-2.1.0-cp312-cp312-win32.whl": "sha256:9b8f0f26ca4e7513c534d351eca551947d053fac438f2a04ac96d882909b0d3a", + "https://files.pythonhosted.org/packages/b3/c1/6dbd291ee2ae5a50a034aa057207081f545923bbf15dad4511e985aafff5/cffi-2.1.0-cp314-cp314-musllinux_1_2_x86_64.whl": "sha256:dbf7c7a88e2bac086f06d14577332760bdeecc42bdec8ac4077f6260557d9326", + "https://files.pythonhosted.org/packages/b5/9f/d4dc66ca651eb1145a133314cda721abf13cfac3d28c4a0402263ae6ad75/cffi-2.1.0-cp315-cp315t-musllinux_1_2_x86_64.whl": "sha256:ba00f661f8ba35d075c937174e27c2c421cec3942fd2e0ea3e66996757c0fdd9", + "https://files.pythonhosted.org/packages/b6/3d/f20f8b886b254e3ad10e15cd4186d3aed49f3e6a35ab37aab9f8f25f7c03/cffi-2.1.0-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "sha256:bf01d8c84cbea96b944c73b22182e6c7c432b3475632b8111dbfdc95ddad6e13", + "https://files.pythonhosted.org/packages/b9/26/d00496b22de4d4228f32dde94ad996f350c8aad676d63bcca0743c8dea4d/cffi-2.1.0-cp314-cp314t-win_amd64.whl": "sha256:0582a58f3051372229ca8e7f5f589f9e5632678208d8636fea3676711fdf7fe5", + "https://files.pythonhosted.org/packages/b9/65/b434abc97ce7cecc2c640fde160507c0ecc7e21544b483ba3325d2e2ea17/cffi-2.1.0-cp315-cp315t-musllinux_1_2_aarch64.whl": "sha256:86cf8755a791f72c85dc287128cc62d4f24d392e3f1e15837245623f4a33cccc", + "https://files.pythonhosted.org/packages/ba/b0/e131a9c41f10607926278453d9596163594fe1c4ebc46efe3b5e5b34eb84/cffi-2.1.0-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.whl": "sha256:a5781494d4d400a3f47f8f1da94b324f6e6b440a53387774002890a2a2f4b50f", + "https://files.pythonhosted.org/packages/c0/e9/6d7724983b3d5a0908dbf74f64038ade77c18646ff6636ec7894fd392ce1/cffi-2.1.0-cp310-cp310-macosx_10_15_x86_64.whl": "sha256:b65f590ef2a44640f9a05dbb548a429b4ade77913ce683ac8b1480777658a6c0", + "https://files.pythonhosted.org/packages/c3/c0/d1ec30ffb370f748f2fb54425972bfef9871e0132e82fb589c46b6676049/cffi-2.1.0-cp312-cp312-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "sha256:5972433ad71a9e46516584ef60a0fda12d9dc459938d1539c3ddecf9bdc1368d", + "https://files.pythonhosted.org/packages/c6/4b/e706f67279140f92939da3475ad610df18bfd52d50f14953a8e5fede71d5/cffi-2.1.0-cp313-cp313-win32.whl": "sha256:db3eb7d46527159a878ec3460e9d40615bc25ba337d477db681aea6e4f05c5d2", + "https://files.pythonhosted.org/packages/cc/82/3d5c705acb7abbba9bbd7d79b8e62e0f25b6120eb7ae6ac49f1b721722fe/cffi-2.1.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "sha256:ac0f1a2d0cfa7eea3f2aaf006ab6e70e8feeb16b75d65b7e5939982ca2f11056", + "https://files.pythonhosted.org/packages/cc/d7/97d3136f81db489ec8d1d67748c110d6c994268fd7528014aa9f2b085e4e/cffi-2.1.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "sha256:d53d10f7da99ae46f7373b9150393e9c5eab9b224909982b43832668de4779f5", + "https://files.pythonhosted.org/packages/d0/9a/bb1d5ed9c3fcae158e9f6391bf309c95d98c2ac37ed56573228471d0af5e/cffi-2.1.0-cp310-cp310-musllinux_1_2_aarch64.whl": "sha256:3d7f118b5adbfdfead90c25822690b02bc8074fba949bb7858bec4ebd55adb43", + "https://files.pythonhosted.org/packages/d3/0b/0ffe8b82d3875bced5fa1e7986a7a46b748262a40ab7f60b475eb9fb1bb3/cffi-2.1.0-cp315-cp315t-win_amd64.whl": "sha256:f146d154428a2523f9cc7936c02353c2459b8f6cf07d3cd1ee1c0a611109c5d5", + "https://files.pythonhosted.org/packages/d3/27/93195977168ee63aed233a1a0993a2178798654d1f4bddcdd321d6fd3b21/cffi-2.1.0-cp314-cp314-musllinux_1_2_aarch64.whl": "sha256:c351efb95e832a853a29361675f33a7ce53de1a109cd73fd47af0712213aa4ce", + "https://files.pythonhosted.org/packages/d3/67/85c89a59ba36a671e79638f44d466749f08179266a57e4f2ffdf92174072/cffi-2.1.0-cp311-cp311-macosx_10_15_x86_64.whl": "sha256:02cb7ff33ded4f1532476731f89ede53e2e488a8e6205515a82144246ffa7dcc", + "https://files.pythonhosted.org/packages/d5/dd/0c7dbf815a579ff005008a2d815a55d6bb047c349eef536d9dc53d3f0a8d/cffi-2.1.0-cp314-cp314t-win_arm64.whl": "sha256:510aeeeac94811b138077451da1fb18b308a5feab47dd2b603af55804155e1c8", + "https://files.pythonhosted.org/packages/d6/5c/584e626835f0375c928176c04137c96927165cb8733cdb3150ec04e5ee5e/cffi-2.1.0-cp314-cp314-macosx_11_0_arm64.whl": "sha256:c4165821e131d6d4ca444347c2b694e2311bcfa3fe5a861cc72968f28867beac", + "https://files.pythonhosted.org/packages/d8/f0/81478e482afa03f6d18dc8f2afb5edc45b3080853b634b5ed91961be0998/cffi-2.1.0-cp314-cp314-ios_13_0_arm64_iphoneos.whl": "sha256:d2117334c3af3bdcb9a88522b844a2bdb5efdc4f71c6c822df55486ae1c3347a", + "https://files.pythonhosted.org/packages/dc/78/aa01ac599a8a4322533d45a1f9bc93b338276d2d59dabbe7c6d92a775c81/cffi-2.1.0-cp314-cp314t-win32.whl": "sha256:7d034dcffa09e9a46c93fa3a3be402096cb5354ac6e41ab8e5cc9cd8b642ad76", + "https://files.pythonhosted.org/packages/dd/2c/400ea43e721727dca8a65c4521390e9196757caba4a45643acb2b63271b8/cffi-2.1.0-cp312-cp312-win_arm64.whl": "sha256:6d194185eabd279f1c05ebe3504265ddfc5ad2b58d0714f7db9f01da592e9eb6", + "https://files.pythonhosted.org/packages/e0/27/1d0b408497e41a74795af122d7b603c418c5fed0171450f899afd04e594f/cffi-2.1.0-cp315-cp315t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "sha256:0520e1f4c35f44e209cbbb421b67eec42e6a157f59444dfb6058874ff3610e5d", + "https://files.pythonhosted.org/packages/e5/4b/1f4c36ab273980d7aa75bb126ea4f8971f24a96108acad3a0a084028c57b/cffi-2.1.0-cp315-cp315-macosx_11_0_arm64.whl": "sha256:cdf2448aab5f661c9315308ec8b93f4e8a1a67a3c733f8631067a2b67d5913dc", + "https://files.pythonhosted.org/packages/e6/3f/0b04a700dd64f465c93020253a793a82c9b4dff9961f48facd0df945d9b8/cffi-2.1.0-cp315-cp315-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "sha256:7d3538f9c0e50670f4deb93dbb696576e60590369cae2faf7de681e597a8a1f1", + "https://files.pythonhosted.org/packages/ea/dd/e3b0baa2d3d6a857ac72b7efbf18e32e487c9cdafcc13049ad765495b15e/cffi-2.1.0-cp311-cp311-macosx_11_0_arm64.whl": "sha256:f5bce581e6b8c235e566a14768a943b172ada3ed73537bb0c0be1edee312d4e7", + "https://files.pythonhosted.org/packages/eb/d8/df4543cc087245044ed02ef3ad8e0a26619d0075ac7a77a12dc81177851b/cffi-2.1.0-cp315-cp315-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "sha256:6274dcb2d15cef48daa73ed1be5a40d501d74dccd0cd6db364776d12cb6ba022", + "https://files.pythonhosted.org/packages/eb/da/5c4918a2d61d86fa927d716cb3d8e4626ef8dc8f605a599d32f33897f59a/cffi-2.1.0-cp311-cp311-win32.whl": "sha256:64c753a0f87a256020004f37a1c8c02c480e725f910f0b2a0f3f07debd1b2479", + "https://files.pythonhosted.org/packages/ec/d1/9a5b7169499e8e8d8e636de70b97ac7c9447104d2ff1a2cd94790cea5162/cffi-2.1.0-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "sha256:0a96b74cda968eebbad56d973efe5098974f0a9fb323865bf99ea1fd24e3e64c", + "https://files.pythonhosted.org/packages/ed/a5/e8bbb1ce5b3ac2f53ad6a10bde44318a5a8d99d4f4a000d44a6e39aeb3e4/cffi-2.1.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "sha256:7d5980a3433d4b71a5e120f9dd551403d7824e31e2e67124fe2769c404c06913", + "https://files.pythonhosted.org/packages/ed/da/4bbe583a3b3a5c8c60892124fe17f3fa3656523faf0d3484eae90f091853/cffi-2.1.0-cp315-cp315-macosx_10_15_x86_64.whl": "sha256:95f2954c2c9473d892eca6e0409f3568b37ab62a8eedb122461f73cc273476e3", + "https://files.pythonhosted.org/packages/ef/c3/ad299dc38f3583f8d916b299f028af418a9ec98bc695fcbebeae7420691c/cffi-2.1.0-cp315-cp315-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "sha256:90bec57cf82089383bd06a605b3eb8daebf7e5a668520beaf6e327a83a947699", + "https://files.pythonhosted.org/packages/f9/c8/6c2de1d55cf35ef8b92885d5ef280790f0fb9634d87ea1cc315176aecd61/cffi-2.1.0-cp311-cp311-win_amd64.whl": "sha256:4f26194e3d95e06501b942642855aed4f953d55e95d7d01b7c4483db3ecff458", + "https://files.pythonhosted.org/packages/f9/cf/398272b8bbfd58aa314fda5a7f1cdbb26d1d78ae324a11211521315dd1f0/cffi-2.1.0-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "sha256:03e9810d18c646077e501f661b682fbf5dee4676048527ca3cffe66faa9960dd", + "https://files.pythonhosted.org/packages/fa/75/74dfb7c3fc6ebbd408038476bd4c1d7e925c62614e7b9c534ecc34218288/cffi-2.1.0-cp310-cp310-musllinux_1_2_x86_64.whl": "sha256:11b3fb55f4f8ad92274ed26705f65d8f91457de71f5380061eb6d125a768fecd", + "https://files.pythonhosted.org/packages/fb/d2/4398416cd699b35167947c6e22aca52c47e69ad5695073c9f1f2c52e04aa/cffi-2.1.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "sha256:aa7a1b53a2a4452ada2d1b5dade9960b2522f1e61293a811a077439e39029565" + }, + "charset-normalizer": { + "https://files.pythonhosted.org/packages/00/5e/17398df3a139985ba9d11ed072531986f408c8fca952835ef1ab1820c02b/charset_normalizer-3.4.9-cp314-cp314t-macosx_10_15_universal2.whl": "sha256:609b3ba8fcc0fb5ab7af00719d0fb6ad0cb518e48e7712d12fd68f1327951198", + "https://files.pythonhosted.org/packages/01/c4/4fa4c8b3097a11f3c5f09a35b72ed6855fb1d332469504962ab7bafcc702/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "sha256:5e226f6218febc71f6c1fc2fafb91c226f75bdc1d8fb12d66823716e891608fd", + "https://files.pythonhosted.org/packages/01/da/a44bd7a13d426e69e4894557106cd58669097bfad4a8681123b618fbfc5d/charset_normalizer-3.4.9-cp310-cp310-win_arm64.whl": "sha256:375b83ed0aecfce76c16d198fbc21f3b11b337d68662bea0a995046682a11419", + "https://files.pythonhosted.org/packages/0b/e3/85ec501f206fb049259288c1f3506e53876937fb00edb47009348e66756b/charset_normalizer-3.4.9-cp311-cp311-macosx_10_9_universal2.whl": "sha256:0e94703ec9684807f20cfb5eed95c70f67f2a8f21ad620146d7b5a13677b93e5", + "https://files.pythonhosted.org/packages/0c/74/2f62c8821b969ea3bd67cc2e6976834f48ca5d12664d2559ebcd9bcfbed7/charset_normalizer-3.4.9-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "sha256:871ff67ea1aad4dfd91736464934d56b32dac49f9fbe16cddba36198a7b3a0db", + "https://files.pythonhosted.org/packages/0c/e7/aaf6da33fc9f4691cda8f7efbc9f69179d3d39ec8a4799baf273ee1d8db0/charset_normalizer-3.4.9-cp310-cp310-musllinux_1_2_armv7l.whl": "sha256:65a7ff3f705e57d392f7261b6d0550fe137c3019477431f1c355e0db0a7d3e15", + "https://files.pythonhosted.org/packages/0e/42/6dbc00b8cd16011691203e33570fa42ed5746599a2e878112d16eab403a3/charset_normalizer-3.4.9-cp312-cp312-win32.whl": "sha256:78841cccf1af7b40f6f716338d50c0902dbe88d9f800b3c973b7a9a0a693a642", + "https://files.pythonhosted.org/packages/10/e0/47c079dd82d217c807479cd59ffd30af56307ea31c108b75758970459ad3/charset_normalizer-3.4.9-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "sha256:4d1c96a7a18b9690a4d46df09e3e3382406ae3213727cd1019ebade1c4a81917", + "https://files.pythonhosted.org/packages/14/cb/1db8b96547ee3186cd2dd7f2e59dd560a9b80748f3604171f3c153d62811/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "sha256:58150c9f9b9a552505912d182ccdf26f6396fb6094816ceebcbb20eecabaed94", + "https://files.pythonhosted.org/packages/17/6d/bff78a4bacc4891bc63ec5bdc6776d8c85e47fab93d0d5f6223068fad0a4/charset_normalizer-3.4.9-cp39-cp39-win32.whl": "sha256:93d59d504b230e83c7a843251681959a0b6a9cd76f6e146ce1b8a80eb8739af9", + "https://files.pythonhosted.org/packages/19/79/55c32d06d76ae4feafe053f061f3e3ab70bcf19f4007797ce8c3efda7830/charset_normalizer-3.4.9-cp314-cp314-manylinux_2_31_armv7l.whl": "sha256:f7fb7d750cfa0a070d2c24e831fd3481019a60dd317ea2b39acbcebc08b6ed81", + "https://files.pythonhosted.org/packages/1d/85/181c652953eb5276d198f375b1dd641047392050098100a3a02d6534f657/charset_normalizer-3.4.9-cp310-cp310-musllinux_1_2_aarch64.whl": "sha256:e9701d0049d92c16703a42771b98d560b95248949f23f8cf7b4eddd201814fb9", + "https://files.pythonhosted.org/packages/20/95/d75e82f8ce9fd323ebf059c16c9aadefb22a1ecde13b7840b35835e4886c/charset_normalizer-3.4.9-cp314-cp314-win_arm64.whl": "sha256:40a126142a56b2dfc0aacbad1de8310cbf60da7656db0e6b16eebd48e3e93519", + "https://files.pythonhosted.org/packages/28/e9/9fb6099b868c82a40698a748ae0fbd4f31ccc13844c176a07158ba2abbfd/charset_normalizer-3.4.9-cp39-cp39-win_arm64.whl": "sha256:476743fe6dfe14a2da12e3ac79125dc84a3b2cf8094369a47a1529b0cd8549fe", + "https://files.pythonhosted.org/packages/2b/f9/ef4a69ea338ad3c0deceea0f5f7d2380ae8b52132b06d652cb0d2cd86706/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "sha256:8a79d9f4d8001473a30c163556b3c3bfebec837495a412dde78b51672f6134f9", + "https://files.pythonhosted.org/packages/33/9a/895095b83e7907abd6d3d99aad3a38ad0d9686cc186cb0c94c24320fe63e/charset_normalizer-3.4.9-cp312-cp312-musllinux_1_2_aarch64.whl": "sha256:60f44ade2cf573dad7a277e6f8ca9a51a21dda572b13bd7d8539bb3cd5dbedde", + "https://files.pythonhosted.org/packages/36/31/a276bb2e66243072a3fd06fdcab9cbb61a305b02143d70d2bda21d888fa8/charset_normalizer-3.4.9-cp310-cp310-manylinux_2_31_armv7l.whl": "sha256:bcf74c1df76758a395bf0af608c04c82257523f55c9868b334f06270d0f2112b", + "https://files.pythonhosted.org/packages/37/8d/ca39a7559a4797505530d084fd3a49a2c959efbbbff146302fb7be4e3b35/charset_normalizer-3.4.9-cp310-cp310-win_amd64.whl": "sha256:8c041122946b7ba21bb32c45b1aa57b1be35527690aeb3c5c234521085632eee", + "https://files.pythonhosted.org/packages/3a/25/45f30093ae27dd7b92a793b61882a38685f993700113ca36e0c9c14965e1/charset_normalizer-3.4.9-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "sha256:a4fbdde9dd4a9ce5fd52c2b3a347bb50cc89483ef783f1cb00d408c13f7a96c0", + "https://files.pythonhosted.org/packages/3d/c6/eee9dca4439b1061f76373f06ea855678cc4a64c1c3c90b50e479edbb8eb/charset_normalizer-3.4.9-cp314-cp314t-win_arm64.whl": "sha256:19ac87f93086ce37b86e098888555c4b4bc48102279bae3350098c0ed664b501", + "https://files.pythonhosted.org/packages/3d/ca/ad1d7c7d3077dab873f539d3e1d083c0845a762cb0bafdfbe3ef93add598/charset_normalizer-3.4.9-cp39-cp39-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "sha256:920079c3f7456fa213e0829ed2073aaa727fd39d889ead5b4f35d0de5460d04f", + "https://files.pythonhosted.org/packages/3d/ef/d96ec496cfea0c21db43b0ad03891308b02388d054cc902cf0e5a1ad6a88/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf", + "https://files.pythonhosted.org/packages/42/ab/b9bc2e77d6b44a7e46ef62ec5cac1c9a6ba7b9135a5d560f002696ec9995/charset_normalizer-3.4.9-cp314-cp314-musllinux_1_2_aarch64.whl": "sha256:a4cfde78a9f2880208d16a93b795726a3017d5977e08d1e162a7a31322479c41", + "https://files.pythonhosted.org/packages/44/95/80282cce0fae9c3061203d723ee87da996aed79679e65d8935050ee7ca1f/charset_normalizer-3.4.9-cp311-cp311-manylinux_2_31_armv7l.whl": "sha256:c0323c9daef75ef2e5083624b4585018a0c9d5e3b40f607eed81a311270b934b", + "https://files.pythonhosted.org/packages/48/18/c8f397329c35e32f6a837e488986f4ae03bd2abebc453b48714991630c2f/charset_normalizer-3.4.9-cp313-cp313-musllinux_1_2_aarch64.whl": "sha256:416c229f77e5ea25b3dfd4b582f8d73d7e43c22320302b9ab128a2d3a0b38efe", + "https://files.pythonhosted.org/packages/49/ba/768fa3f36048d81c477a0ce61f813bc1454d80917ccfe550abd9f44f5e24/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "sha256:f840ed6d8ecba8255df8c42b87fadeda98ddfc6eeec05e2dc66e26d46dd6f58a", + "https://files.pythonhosted.org/packages/49/fd/a1d26144398c67486422a72bf5812cda22cb4ccfcd95a290fb41ceb4b8e2/charset_normalizer-3.4.9-cp314-cp314-win_amd64.whl": "sha256:16b65ea0f2465b6fb52aa22de5eca612aa964ddfec00a912e26f4656cbef890b", + "https://files.pythonhosted.org/packages/4b/4c/5361f9aa7f2cb58d94f2ab831b3d493f69efb1d239654b4744e3c09527cb/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "sha256:9104ed0bd76a429d46f9ec0dbc9b08ad1d2dcdf2b00a5a0daa1c145329b35b44", + "https://files.pythonhosted.org/packages/4e/6e/de0229a7ef40f6f9d28a837eebf4ec47bdca5dab4e900c84f22919af636a/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "sha256:4773092f8019072343a7447203308b176e10199920eb02d6195e81bbb3274c29", + "https://files.pythonhosted.org/packages/4f/8d/1569f4d0032d6ba2a4fe4591c35bf87868c600c41a71eb5c2e1ffa8464c2/charset_normalizer-3.4.9-cp311-cp311-win_arm64.whl": "sha256:1d22856ffbe153a602df38e4a5464f0b748a54002e0d69ac6d2ad0a197cc99ec", + "https://files.pythonhosted.org/packages/50/78/ce342ca4ff30b2eb49fe6d9578df85974f90c67d294113e94efdd9664cbd/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "sha256:7b86a2b16095d250c6f58b3d9b2eee6f4147754344f3dab0922f7c9bf7d226c9", + "https://files.pythonhosted.org/packages/52/94/af74dde74a3996bd959c350709bfe50e297823d70a8c1cbd54b838880863/charset_normalizer-3.4.9-cp314-cp314t-manylinux_2_31_armv7l.whl": "sha256:f86c6358749bd4fda175388691e3ba8c46e24c5347d0afd20f9b7edfc9faf07d", + "https://files.pythonhosted.org/packages/5e/be/7ee4453d7e88dfbc4104ccd34900b9f2c7c17dac22881865fe0e82424a25/charset_normalizer-3.4.9-cp310-cp310-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "sha256:b5314963fce9b0b12743891de876e724997864ee22aa496f903f426c7e2fa5b2", + "https://files.pythonhosted.org/packages/5f/c0/6eec7bdabe6cbbcc274ec04596f6d93865751a0541d33d60d1ce179bd372/charset_normalizer-3.4.9-cp39-cp39-musllinux_1_2_armv7l.whl": "sha256:ad41ba96094304aa090f5a30cb6e4fb3b3f1c264c523394b4c39bbacc4dc92ba", + "https://files.pythonhosted.org/packages/63/01/f2fb3bd3a73be48b173ee0c6aa8d2497af97d5663a8c4c4b491de4c62f7a/charset_normalizer-3.4.9-cp310-cp310-musllinux_1_2_x86_64.whl": "sha256:79580094b00d1789d1f93ea55bc43cb2f611910c72235b7657f3482ddcc1b22d", + "https://files.pythonhosted.org/packages/6a/05/c94d5cd23396289c54c93b02e0273b4dd8921641d9968c4828caf9bbaad9/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "sha256:df7276909358e5635ae203673ab7e509ddd224225a8d6b0790bf13eb2bde1cc5", + "https://files.pythonhosted.org/packages/6c/ef/2473d3c4d869155be4af1191111d59c4d5c4e0173026f7e85b176e23bf65/charset_normalizer-3.4.9-cp313-cp313-musllinux_1_2_x86_64.whl": "sha256:69b157c5d3292bcd443faca052f3096f637f1e074b98212a933c074ae23dc3b8", + "https://files.pythonhosted.org/packages/6d/46/79847edd07244a4a2d443c6655a7b6ee94203c21539414b059f32713c357/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "sha256:3c09a49d6cde137258beb3d551994a2927fd35ad5cf96aed573f61bbd67c5f84", + "https://files.pythonhosted.org/packages/6e/fb/d560d1d1555debbfe7849d9cac6145c1b537709d79576bf22557ed803b82/charset_normalizer-3.4.9-cp313-cp313-win_arm64.whl": "sha256:611057cc5d5c0afc743ba8be6bd828c17e0aaa8643f9d0a9b9bb7dea80eb8012", + "https://files.pythonhosted.org/packages/70/4a/ecbd131485c07fcdfad54e28946d513e3da22ef3b4bd854dcafae54ec739/charset_normalizer-3.4.9-cp312-cp312-macosx_10_13_universal2.whl": "sha256:45b0cc4e3556cd875e09102988d1ab8356c998b596c9fced84547c8138b487a0", + "https://files.pythonhosted.org/packages/78/ad/98aae8630ac71f16711968e38a5acfecce41b778bf2f0312851020f565a8/charset_normalizer-3.4.9-cp314-cp314t-musllinux_1_2_armv7l.whl": "sha256:cd6c3d4b783c556fa00bf540854e42f135e2f256abd29669fcd0da0f2dec79c2", + "https://files.pythonhosted.org/packages/7e/8d/496817fa0944239ecae662dd57ea765cfeaec6a735f9f025d4b7b72e7143/charset_normalizer-3.4.9-cp314-cp314-macosx_10_15_universal2.whl": "sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380", + "https://files.pythonhosted.org/packages/7f/f4/ffbb83546e1f198ecc70ecd372b65cf2b50f9068b380abd67640f17a8e18/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "sha256:16d10d789dd9bcca1173c95af82c58433122564b7bc39385124be735a35cbe99", + "https://files.pythonhosted.org/packages/80/33/6c99c1b3e6b8bf730e1bc809b9a2608f224145069114c479a2e9e1494346/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "sha256:c1225416b463483160e4af85d5fc3a9690ccb53fd4b1865a6437825f5ede3209", + "https://files.pythonhosted.org/packages/80/3f/bd97d3d9c613013d07cb7733d299385b41df37f0471310f5a73dc359f0b8/charset_normalizer-3.4.9-cp314-cp314t-win_amd64.whl": "sha256:9b8e0f3107e2200b76f6054de99016eac3ee6762713587b36baaa7e4bd2ae177", + "https://files.pythonhosted.org/packages/80/cc/f920afd1a23c58ccd53c1d36085a71893a4737ff5e66e0371efab6809850/charset_normalizer-3.4.9-cp312-cp312-win_amd64.whl": "sha256:4b3dac63058cc36820b0dd072f89898604e2d39686fe05321729d00d8ac185a0", + "https://files.pythonhosted.org/packages/83/d5/9096aa3cf532dfad237861544eb47a0f20d5adbf1039760fed8eaae935d9/charset_normalizer-3.4.9-cp311-cp311-win32.whl": "sha256:ac351b3b8014eead140e77e9717e2992c6bbe30b63bc3422422eb84865412e3d", + "https://files.pythonhosted.org/packages/83/dc/9b29fa4412b318bf3bfea985c35d67eb55e04b59a7c3f2237168b0e0be6f/charset_normalizer-3.4.9-cp312-cp312-musllinux_1_2_x86_64.whl": "sha256:03d07803992c6c7bbc976327f34b18b6160327fc81cb82c9d504720ac0be3b62", + "https://files.pythonhosted.org/packages/86/7e/5ce0bba863470fd1902d5e5843968951bddf38abe4742fc97116ef4598b3/charset_normalizer-3.4.9-cp313-cp313-musllinux_1_2_armv7l.whl": "sha256:75286256590a6320cf106a0d28970d3560aad9ee09aa7b34fb40524792436d35", + "https://files.pythonhosted.org/packages/87/3a/ad914516df7e358a81aae018caa5e0470ba827fa6d763b1d2e87d920a5f6/charset_normalizer-3.4.9-cp312-cp312-manylinux_2_31_armv7l.whl": "sha256:90c44bc373b7687f6948b693cceaea1348ae0975d7474746559494468e3c1d84", + "https://files.pythonhosted.org/packages/8c/e7/5ddfd76fc061eb52de219658a4aa431cbacadf0a0219c8854f00da50d289/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "sha256:33bdcc2a32c0a0e861f60841a512c8acc658c87c2ac59d89e3a46dacf7d866e4", + "https://files.pythonhosted.org/packages/92/8f/3a47a3667c83c2df9483d91644c6c107de3bf8874aa1793da9d3012eb986/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "sha256:e4fd89cc178bced6ad29cb3e6dd4aa63fa5017c3524dbd0b25998fb64a87cc8b", + "https://files.pythonhosted.org/packages/98/2b/f97f1c193fb855c345d678f5077d6926034db0722df74c8f057020e05a25/charset_normalizer-3.4.9-py3-none-any.whl": "sha256:68e5f26a1ad57ded6d1cfb85331d1c1a195314756471d97758c48498bb4dcdf5", + "https://files.pythonhosted.org/packages/9b/f2/c0d4b8508565a36bc5c624e88ed297f5b0b1095011034d7f5b83a69908b5/charset_normalizer-3.4.9-cp314-cp314-win32.whl": "sha256:c1c948747b03be832dceed96ca815cef7360de9aa19d37c730f8e3f6101aca48", + "https://files.pythonhosted.org/packages/a1/34/ef5c05f412f42520d7709b7d3784d19640839eb7366ded1755511585429f/charset_normalizer-3.4.9-cp312-cp312-musllinux_1_2_armv7l.whl": "sha256:a1786910334ed46ab1dd73222f2cd1e05c2c3bb39f6dddb4f8b36fc382058a39", + "https://files.pythonhosted.org/packages/a2/55/86048bde1c9d0352940bd7b87d825091a52aef67d01cde6c6f7342c5b552/charset_normalizer-3.4.9-cp39-cp39-win_amd64.whl": "sha256:ddf4af30b417d9fe16481e9b81c27ab2a7cde1ff7ba3e85653b02db7d145dc7b", + "https://files.pythonhosted.org/packages/a5/34/49b9060e8418b14fb5cba9cf6bfb383111e2538a03a1fb18e66a95aeb3d5/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "sha256:04ce310cb89c15df659582aee80a0603788732a5e017d5bd5c81158106ce249c", + "https://files.pythonhosted.org/packages/a5/ff/c946d63bc3786d5b84d960b0f7ab7e25b828486a946b5aa997625bcaf6a6/charset_normalizer-3.4.9-cp311-cp311-musllinux_1_2_armv7l.whl": "sha256:3d92613ec25e43b05f042302531ec0f00b8445190e43325880cbd6ab7c2581da", + "https://files.pythonhosted.org/packages/a6/ec/81e22253f4b7091eca6515bb3da5e45d05a663f7f567bb745695dc60f892/charset_normalizer-3.4.9-cp39-cp39-macosx_10_9_universal2.whl": "sha256:253a4a220747e8b5faf57ec320c4f5efb0cef05f647420bf267143ec15dba10a", + "https://files.pythonhosted.org/packages/ad/81/8e983840c6e5b93b33c2ba81aa3d52c2e42f0e9a690ce7607a2e61da4a5c/charset_normalizer-3.4.9-cp310-cp310-macosx_10_9_universal2.whl": "sha256:cd6280cf040f233bd7d3407b743b4b4c74f70e8e1c4199cb112a62c941c0772a", + "https://files.pythonhosted.org/packages/af/ba/5e5007c370702f85d2ef75791fac7943ed41e080364a673b20142e430e3e/charset_normalizer-3.4.9-cp311-cp311-musllinux_1_2_x86_64.whl": "sha256:280081916dc341820640489a66e4696049401ef1cf6dd672f672e70ad915aca3", + "https://files.pythonhosted.org/packages/b0/3e/faee8f9de92b14ee1198e9163252bb15efee7301b31256a3b6d9ebfdd0dd/charset_normalizer-3.4.9-cp313-cp313-manylinux_2_31_armv7l.whl": "sha256:5b10cd92fc5c498b35a8635df6d5a100207f88b63a4dc1de7ef9a548e1e2cd63", + "https://files.pythonhosted.org/packages/b1/27/693ee5e8a18191eb38647360c51cd505013e2bd3b366aa43fd5344c21e3c/charset_normalizer-3.4.9-cp314-cp314t-win32.whl": "sha256:0d861473f743244d349b50f850d10eb87aeb22bbdcc8e64f79273c94af5a8226", + "https://files.pythonhosted.org/packages/b2/06/97ec2aeae780b31d742b6352218b43841a6871e2564578ca522dce4a45c3/charset_normalizer-3.4.9-cp313-cp313-macosx_10_13_universal2.whl": "sha256:440eede837960000d74978f0eba527be106b5b9aee0daf779d395276ed0b0614", + "https://files.pythonhosted.org/packages/b3/46/03ddc7da576d814fe0a36dd1f0fd3258e95404b4b2e3c026b7923d7e133f/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "sha256:304b13570067b2547562e308af560b3963857b1fa90bd6afd978130130fe2d6a", + "https://files.pythonhosted.org/packages/bd/2a/23f34ec9d04624958e137efdc394888716353190e75f25dd22c7a2c7a8aa/charset_normalizer-3.4.9.tar.gz": "sha256:673611bbd43f0810bec0b0f028ddeaaa501190339cac411f347ac76917c3ae7b", + "https://files.pythonhosted.org/packages/c3/69/2a5385192e67175f7d8bd5ce4f57c24bc956439adeae5c13a99aa28a53d1/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "sha256:2a441ea71902098ffe78c5abe6c494f44160b4af614ed16c3d9a3b1d17fd8ee2", + "https://files.pythonhosted.org/packages/c4/02/c57a22739fe05246b0b5783b3bfb6afaac4eebb46f3ececdfb2f048f780e/charset_normalizer-3.4.9-cp310-cp310-win32.whl": "sha256:432786d3561e69aeeae6c7e8648964ce0ad05736120135601f87ac26b9c83381", + "https://files.pythonhosted.org/packages/c8/53/a8c042eb9eee4716f4d42a0f5a571eb32a09ec429be9fb0b8b9d765393ba/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "sha256:68ce9f4d6b26d5ccbf7fd4459bf75f74a0a146677ebba80597df60cbdb20e6f4", + "https://files.pythonhosted.org/packages/cb/32/2e64bd2be10e89c61e57ebe6a93fd98ae88eb7ebe414b5121f22c96c69eb/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "sha256:cc1b0fff8ead343dae06305f954eb8468ba0ec1a97881f42489d198e4ce3c632", + "https://files.pythonhosted.org/packages/cd/91/7253a32e86b7e1d1239b1b36ba6dd0f021a21107ab33054b53119cc083b9/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "sha256:51447e9aa2684679af07ca5021c3db526e0284347ebf4ffcec1154c3350cfe32", + "https://files.pythonhosted.org/packages/d0/39/8ff066c672434225f8d25f8b739f992af250944392173dcc88362681c9bf/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "sha256:21e764fd1e70b6a3e205a0e46f3051701f98a8cb3fad66eeb80e48bb502f8698", + "https://files.pythonhosted.org/packages/d0/a3/53ddae3db108a088156aa8ddfafd411ebbc1340f48c5573f697b27f69a39/charset_normalizer-3.4.9-cp313-cp313-win32.whl": "sha256:51307f5c71007673a2bf8232ad973483d281e74cb99c8c5a990af1eefa6277d9", + "https://files.pythonhosted.org/packages/d4/ce/9af95f7876194bd7a14e3dfe4a4de2e0bff02666a3910d72beafd06cc297/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "sha256:df115d4d83168fdf2cae48ef1ff6d1cb4c466364e30861b37121de0f3bf1b990", + "https://files.pythonhosted.org/packages/d7/74/3c12f9755717dfe5c5c87da63f35d765fa0c00382ec26bf23f7fae34f2ba/charset_normalizer-3.4.9-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "sha256:9cdef90ae47919cae358d8ab15797a800ed41da7aba5d72419fb510729e2ed4b", + "https://files.pythonhosted.org/packages/d9/8d/feabb82cb49fcad14515b1d7d1ca4787b0da7fc723a212bf89bc9e0fac52/charset_normalizer-3.4.9-cp311-cp311-musllinux_1_2_aarch64.whl": "sha256:67830fc78e67501f47bb950471b2dcb9b35b140084429318e862895a8e89c993", + "https://files.pythonhosted.org/packages/de/d1/b4319dc3229d8272fba305e206fc0a148e2de8d4087917ce62ae6382f359/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "sha256:aa99adc8f081b475a12843953db36831eaf83ec33eb46a90629ca6a5de45a616", + "https://files.pythonhosted.org/packages/e4/56/6c745619ac397e8871e2bcd3cea1eec86b877488f33888b3aef5c3ed506e/charset_normalizer-3.4.9-cp314-cp314t-musllinux_1_2_aarch64.whl": "sha256:83aed2c10721ddd90f68140685391b50811a880af20654c59af6b6c66c40513c", + "https://files.pythonhosted.org/packages/e8/5f/b98b8da398637b551e427e7be922bdec19177dc54d6811dcdaa503f23aac/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "sha256:9bb41182d93ea91f60b4bc8fbf4c820c69ef8a12ab2d917f3f1834f1acad07e8", + "https://files.pythonhosted.org/packages/e8/ef/6953a77c7cf2c2ff9998e6f575ab3e380119f100223381565a4f94c1f836/charset_normalizer-3.4.9-cp313-cp313-win_amd64.whl": "sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115", + "https://files.pythonhosted.org/packages/ea/f8/72eb13dcabe7257035cea8aefd922caad2f110d252bf9f67c4c2ca763aee/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "sha256:84fd18bcc17526fc2b3c1af7d2b9217d32c9c04448c16ec693b9b4f1985c3d33", + "https://files.pythonhosted.org/packages/eb/78/59344ff9a4a7b5f6530bf7bec2c980047cc42c3a616596cdbd8cb5c1a1af/charset_normalizer-3.4.9-cp39-cp39-musllinux_1_2_x86_64.whl": "sha256:43b9e366a31fdd1c87d0eb08f579b4a82b723ea54338f040d6b4e518a026ea29", + "https://files.pythonhosted.org/packages/ec/96/5d9364e3342d69f3a045e1777bc47c85c383e6e9466d561b33fdb419d1f9/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "sha256:9b2aff1c7b3884512b9512c3eaadd9bab39fb45042ffaaa1dd08ff2b9f8109d9", + "https://files.pythonhosted.org/packages/ec/b4/ef5a49b2e77c00deb43bb3256592b115ba9e4346016e82c516b8d215bf68/charset_normalizer-3.4.9-cp39-cp39-manylinux_2_31_armv7l.whl": "sha256:231ddcbb35e2ff8973e1365db41fe0572662893b99a05deb183b68ad4c0c8bd4", + "https://files.pythonhosted.org/packages/ed/61/710738687f90d01c06a04ed52d6ca1e62dd9b1d8cc2567098167c4691034/charset_normalizer-3.4.9-cp39-cp39-musllinux_1_2_aarch64.whl": "sha256:0fa1aec2d32bcc03c8fa0f6f1712caad1adc38509f31142112e5c9daf5b9c833", + "https://files.pythonhosted.org/packages/ed/a1/e29995109e455dc8eff8d0fac6ae509be39561318a7cfeac5d33ad029213/charset_normalizer-3.4.9-cp311-cp311-win_amd64.whl": "sha256:6366a16e1a25018694d6a5d784d09b046edc9eac40ea2b54065c3052672516a1", + "https://files.pythonhosted.org/packages/ee/f0/f1c4fe746c395922961b5916ed1d7d6e7d4c84851d19ed43cc89980ec953/charset_normalizer-3.4.9-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "sha256:32286a2c8d167e897177b673176c1e3e00d4057caf5d2b64eef9a3666b03018e", + "https://files.pythonhosted.org/packages/f0/e6/0386d43a261ff4e4b30c5857af7df877254b46bec7b9d1b74b6bf969a90b/charset_normalizer-3.4.9-cp312-cp312-win_arm64.whl": "sha256:78fa18e436a1a0e58dbd7e02fc4473f3f32cceb12df9dfca542d075961c307d2", + "https://files.pythonhosted.org/packages/f1/60/b22cdbee7e4013dab8b0d7647fc6181120fbbbc8f7025c226d15bd5a47fc/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "sha256:bd47ba7fc3ca94896759ea0109775132d3e7ab921fbf54038e1bab2e46c313c9", + "https://files.pythonhosted.org/packages/f1/78/c9c71d599f5aa2d42bcdd35cbbd46d7f535351a57e40ff7d8e5a7e219401/charset_normalizer-3.4.9-cp314-cp314-musllinux_1_2_armv7l.whl": "sha256:d4d6fcde76f94f5cb9e43e9e9a61f16dacefd228cbbf6f1a09bd9b219a92f1a1", + "https://files.pythonhosted.org/packages/f4/c4/b3e049d2aa3766180c78507110543d9d50894cc97f57de543f1be521dcdc/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "sha256:c25fe15c70c59eb7c5ce8c06a1f3fa1da0ecc5ea1e7a5922c40fd2fa9b0d5046", + "https://files.pythonhosted.org/packages/f6/39/c914445c321a845097ce4f6ac7de9a18228a77b766272125a1ce00d851eb/charset_normalizer-3.4.9-cp314-cp314-musllinux_1_2_x86_64.whl": "sha256:898f0e9068ca27d37f8e83a5b962821df851532e6c4a7d615c1c033f9da6eedf", + "https://files.pythonhosted.org/packages/f7/40/9593d54209765207a7f11073c06494c1721e4ca4a0a426c597679bf7f91e/charset_normalizer-3.4.9-cp314-cp314t-musllinux_1_2_x86_64.whl": "sha256:ee2f2a527e3c1a6e6411eb4209642e138b544a2d72fe5d0d76daf77b24063534" + }, + "cryptography": { + "https://files.pythonhosted.org/packages/09/41/3797cfaf69cae04a13ee78ebd83f0678d9c02b4779d21ce24445326f1a69/cryptography-49.0.0-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "sha256:36d1709f992593689b45bda411498d62c6e365f2ca00b84657d4dadd24de16db", + "https://files.pythonhosted.org/packages/11/2d/5e1fb307cb5931881516b464c98774b3f2c36b5d4bb9a2830253cf553cad/cryptography-49.0.0-cp314-cp314t-manylinux_2_34_x86_64.whl": "sha256:d8ecde755e2e91bf773fc94e8c9d730cd7f2007004cb492263a794ec3899a1c8", + "https://files.pythonhosted.org/packages/17/50/983e838c7fd0d87fd8c969bcdd328edaf5f756e38df5281637424c155873/cryptography-49.0.0-cp314-cp314t-manylinux_2_31_armv7l.whl": "sha256:07cab27cc7b7e0fd28e5e26bb9eeedde5c135c868b46de4a27845abe94af6122", + "https://files.pythonhosted.org/packages/19/2a/5bb823f5bedcf80718cea7fbc95ec5515cca3769633c4b01a32be7f30e7c/cryptography-49.0.0-cp39-abi3-macosx_11_0_arm64.whl": "sha256:ec5e529fb80935c94fe7b729f9972b50e351a0e6b50aa294fd5cabb109fcc29a", + "https://files.pythonhosted.org/packages/1f/09/f42b1d190c5ba75f72062a387f8030d1d75f6ab035788f1d9c4b01de6525/cryptography-49.0.0-cp311-abi3-win_amd64.whl": "sha256:e5dfc1e64de5677cec922ffa8da89c546d0415bf6efdf081842e5d44c84e1f0e", + "https://files.pythonhosted.org/packages/1f/99/d1c90d6041656cc6ee229dc99cd67fd0cd5aec3c5f7d72fffc27cc750054/cryptography-49.0.0.tar.gz": "sha256:f89660a348f4f78a92366240a61404e337586ef7f5909a2fef59ca88ef505493", + "https://files.pythonhosted.org/packages/20/2c/0622f20ff02b2ef32558733443805dc82fd4c275be01b2d19d14676f3a1b/cryptography-49.0.0-cp311-abi3-manylinux_2_28_x86_64.whl": "sha256:2afe9051da7ae7bd5905da5a949280c7d2bb75682e188f650a9d0f2756b834c6", + "https://files.pythonhosted.org/packages/24/01/186c825898477d77e2324d5360fefe622ff1d8d1963ec0554e2cada8ec77/cryptography-49.0.0-cp39-abi3-manylinux_2_28_ppc64le.whl": "sha256:9e82dcc8e56052715fb18b2429e3bca4823b1629136a2084fc45a9a5cecb9b64", + "https://files.pythonhosted.org/packages/2c/99/2d13299eb3dd27b02dcfaafcc91d6b5cb3329f7cbd6d8f51921acd566c1a/cryptography-49.0.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "sha256:35b151772baff2c74cba7fa290ceaff4c3b11c0c881eb93eb5dbc05a7cfbba18", + "https://files.pythonhosted.org/packages/3d/df/40577043ca124e17012f408ddddaeb213b856336ac82ddb3bc915f39e29f/cryptography-49.0.0-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "sha256:f78ff2c9ed8dc2d036b0f4d640e22522213d047c1b14e61205a7e55c80a494d4", + "https://files.pythonhosted.org/packages/47/f1/1d3eaa243bfc5de4a187b22aa8c048b3e4980bfbe830ac46e6bac2e66947/cryptography-49.0.0-cp314-cp314t-manylinux_2_28_ppc64le.whl": "sha256:f37d847238971164fdbc68ade6f6574aecc9c0af714190e2083429ff68f4ce9d", + "https://files.pythonhosted.org/packages/4a/91/01ce7303a4579e6d3a6abef01bd322848e9ea7a219adcabc5048b9033571/cryptography-49.0.0-cp311-abi3-manylinux_2_28_aarch64.whl": "sha256:53ecee2e23f7169b6117e99fc8a944e5e50f79e69758a83b52a00cb98ab2b2d2", + "https://files.pythonhosted.org/packages/4c/fe/93ecac273d3738939d023612ad12cca9a3740a5345d69fda04134c43fd96/cryptography-49.0.0-cp314-cp314t-win_amd64.whl": "sha256:33cd0565932807baddb67b96dbee92f2c374b5c89dee09fd74079aeb8c8dba61", + "https://files.pythonhosted.org/packages/4f/01/339573cf1023163a400b0b5d16f6d507de413b9f60be6fd1b77feeaf6737/cryptography-49.0.0-cp311-abi3-musllinux_1_2_aarch64.whl": "sha256:b87e65d263b3e5d3bb92a57e2a6638e2f31110fa7aa890c7b2dbba42248d0a3f", + "https://files.pythonhosted.org/packages/58/39/2d51306721330c486495853eda1c567880ff036de15a14c4b74f399934af/cryptography-49.0.0-cp314-cp314t-manylinux_2_28_x86_64.whl": "sha256:c2bc30226390d60ea19d9f82b19db005fe0452154a23c1c410c12ea801e43561", + "https://files.pythonhosted.org/packages/62/99/a2c95cf8293f07491e9e27c20cc4dcd18176d944e674679adeb1d0173fd6/cryptography-49.0.0-cp311-abi3-manylinux_2_28_ppc64le.whl": "sha256:2eda353d8a27bcbcaa4cbed18994a74ab4d19a2ca897db188ea269ab9b71419b", + "https://files.pythonhosted.org/packages/63/d3/4a83af35d65e3fad632c926fad684c193ea4398569ccb0bbbc7fe8f5dc9a/cryptography-49.0.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl": "sha256:fc1e275c2f1d97b1a6450b8b0ea3ebfa6e087a611c2b26cb2404d48588abab7b", + "https://files.pythonhosted.org/packages/67/d0/a5fcd3515f0bae49a7b6d0413cc1bdccdcc1fc0047037a0d480642cdc5d6/cryptography-49.0.0-cp39-abi3-manylinux_2_34_ppc64le.whl": "sha256:6fc361c34fb6aac015ce19435876635e5c6d21db31998b0920f675f131e043b8", + "https://files.pythonhosted.org/packages/68/28/8a3ad4653662c93fc44dc4e5d8fd374c25c42e07b34bbfbadf49cf57a5a8/cryptography-49.0.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "sha256:7abcee80084cda3f7691f3eb1ce480d8df49cec637b429aa35986c1de71738aa", + "https://files.pythonhosted.org/packages/6c/72/3e798c064bc39e471008075d0f9bc9daf77a80879c092e4a8e170c585ed4/cryptography-49.0.0-cp39-abi3-manylinux_2_31_armv7l.whl": "sha256:8c25ceb16df5b9435f3f6a9829204985b0e0cbee3b48aacd432c7d2c850b44d9", + "https://files.pythonhosted.org/packages/6c/a0/db537264e234f7273a73ec020873d6d6b39dfd8a53db78b550ca8320440e/cryptography-49.0.0-cp39-abi3-musllinux_1_2_aarch64.whl": "sha256:67e1d20ad9ef3a563c59ef22e7a8a0b8210bd26604369ea4a30a7c66aefe504e", + "https://files.pythonhosted.org/packages/6d/88/05563c7fe2e914e87d1a536d06fe83e66b4e1d95cb593e05aea375531da8/cryptography-49.0.0-cp311-abi3-manylinux_2_34_aarch64.whl": "sha256:ccac2bfebc306b862133e3bb71f3f6ee8bb525240089b2d952e4144b3a6d5da7", + "https://files.pythonhosted.org/packages/71/fd/577302e213a1be9468f92d1afef66fcf1ef83d516819d9992ca547f592bd/cryptography-49.0.0-cp311-abi3-musllinux_1_2_x86_64.whl": "sha256:66ec79c3904820572d7e987abdf304281f141d37ad9a489b8e97066e7b9b6459", + "https://files.pythonhosted.org/packages/86/12/c48a424f38db03027be9f7ed5c7dc5de9933dbee992865f98b13727a009d/cryptography-49.0.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "sha256:196ecd6a36e4e9aa10270393bb98d8df88fccee0bf1e5128b91ae4eb4375896d", + "https://files.pythonhosted.org/packages/93/77/8df9eb486495979bccecd1062e2eaf435250e84437040295b57d09048b0b/cryptography-49.0.0-cp39-abi3-musllinux_1_2_x86_64.whl": "sha256:42b0684e0e40cf26122427802486f6d93aea593612603a94fbf260c7eb1e9c1b", + "https://files.pythonhosted.org/packages/94/64/2923570ac1c0bd3a737aa366ac3abbbbde273042308b8cde95e2364a6e6a/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_34_aarch64.whl": "sha256:b47db11c2c3525083296069b98ac5221907455e989ae0c2e3008bde851921615", + "https://files.pythonhosted.org/packages/9b/22/adf66990e63584a68dfb50c24f48a125c07b1699899381c8151e63ed458c/cryptography-49.0.0-cp311-abi3-macosx_11_0_arm64.whl": "sha256:966fe0e9c67490071f14c0d2b1cb2dfb3023c5ce39457343931415f08382f2db", + "https://files.pythonhosted.org/packages/a0/84/84fe36f19caf857d61cb7fc9c63035a47ffabd84ea12d1d393148efa3615/cryptography-49.0.0-cp39-abi3-manylinux_2_34_x86_64.whl": "sha256:2400ef9c9e2299a25614eb1dea3db54a69b1349efd043bfac9c67630d136df36", + "https://files.pythonhosted.org/packages/a3/5b/c5246635d5fd3b64e0d45ae10e99fd32fe9676a79915ccfe5a61ba9af1a5/cryptography-49.0.0-cp311-abi3-manylinux_2_31_armv7l.whl": "sha256:0b82e28ee398a386f0807bba7884d30f25218855690f45115831bcce5d90822c", + "https://files.pythonhosted.org/packages/a5/4d/9c0cd02f95e2602dd5e563da149ee0830abef3537be8b34dc56281ebe27a/cryptography-49.0.0-cp39-abi3-manylinux_2_28_aarch64.whl": "sha256:0f21641cf4b30fca7aee061ced0ec7ad7b073518088b7c9969a297c0ae796c69", + "https://files.pythonhosted.org/packages/a7/f5/8f571d7e27c55bce9f76f026143bcb1e040a4233149ecca0bea5fa5dd5f7/cryptography-49.0.0-cp314-cp314t-manylinux_2_34_aarch64.whl": "sha256:b20133d204d2bb56ba047642199603876c872026ca53e79c35b83772ab2cc505", + "https://files.pythonhosted.org/packages/a8/b2/2193fc74f81aee4f9b62733133b73b5176718932ed8f2e4b03fa040480a6/cryptography-49.0.0-cp314-cp314t-manylinux_2_28_aarch64.whl": "sha256:4ae387c9cb68ea569ca17e490d66d8142b81c3cc814bf179974b7d146e490bbb", + "https://files.pythonhosted.org/packages/a9/3c/f3ad17eecc1a57b0ba236dc01f90e783c51f4a2f35f64777cc4f47a184b2/cryptography-49.0.0-cp311-abi3-manylinux_2_34_x86_64.whl": "sha256:cbc77da8c523d5abd028635ba850a6966fcee2c82e2bf65a41d1d8afe0f98be9", + "https://files.pythonhosted.org/packages/aa/50/a9caea39ad19c431c1a3f8a31114df65b260cdfe67786b6c7e7c040c4c44/cryptography-49.0.0-pp311-pypy311_pp73-win_amd64.whl": "sha256:be9fcb48a55f023493482827d4f459bd263cc20efde64f204b97c123201850c6", + "https://files.pythonhosted.org/packages/ab/f8/614dc7e051418cfe53d55173c1e24c6b0085e89996fe90508c2fdf769aef/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_34_x86_64.whl": "sha256:084ef1af862eb07ec46d25f68689f2102a9fc0e05ce7b80f14f5fe51e4eef0f6", + "https://files.pythonhosted.org/packages/b8/7b/62cbbab75d0659865bf0273790031544a0b16c8072d258f9428dcd8190dc/cryptography-49.0.0-cp39-abi3-manylinux_2_28_x86_64.whl": "sha256:6f2debedf9ca60cf1d5bd466475638af5130f89965605cd818484d19987d3a21", + "https://files.pythonhosted.org/packages/b9/26/814681d14248d95d73d5c3eea0c39a94eb8302df966f670a2c60de90974b/cryptography-49.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl": "sha256:32703d93296f5c1f4b53349ad3a250c2cae0fdecd3a3dd5d47e616d8d616af27", + "https://files.pythonhosted.org/packages/c2/e6/f60198ea8d9dfa15fff9ed4ca02ce362f6eadd9ba757dcc50634c4257b63/cryptography-49.0.0-cp39-abi3-win_amd64.whl": "sha256:026ac7423e6fa66872d3bf889be5974507da3944f866f704fa200eadacd00001", + "https://files.pythonhosted.org/packages/c4/b6/d7696e4e890d6ae1469935164c9e5215c557671cb78d6e3f458ccceaa632/cryptography-49.0.0-cp311-abi3-manylinux_2_34_ppc64le.whl": "sha256:d0527ce944105f257f605a827d6ebead966c752038b6e8656abb9c5edee6fc68", + "https://files.pythonhosted.org/packages/d6/a7/f9dac0ab7f80368c56993a7bf638ef9935f825c91902798481fac0898138/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_28_aarch64.whl": "sha256:c83782480a4a9da4d0feb51950131ba32e12e70813848b3343f6e18c28a66838", + "https://files.pythonhosted.org/packages/d7/70/2ba3769dd0ae167e2f33dfa9592d45db6ff9a61d62ca1a5b3d1bdd09068f/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_28_x86_64.whl": "sha256:b39efa323140595abd3ecca8529d321ae50f55f3aa3ba9cc81ea56a6011953d5", + "https://files.pythonhosted.org/packages/e4/c0/bff5a02ee731d207d6a1ed51732549d8c53d2bc8da1d10ec6f2844201d68/cryptography-49.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl": "sha256:e3fb64c420688e5319ae25113a354015abbd8dffbfbc41781a1ea66fc7622ac3", + "https://files.pythonhosted.org/packages/e6/8b/43011f7ebe515a8aa20d61f290a326cd890c2e738e16e59eaff8d9c3a412/cryptography-49.0.0-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "sha256:0e959b578856a3924bc0cbb710fc12c387b9412a951389f3ca61704a9e25f325", + "https://files.pythonhosted.org/packages/e7/84/0e27016a6fc5a0886f797018b26aa42f40c09a82332bff77822a451deaaa/cryptography-49.0.0-cp314-cp314t-manylinux_2_34_ppc64le.whl": "sha256:b970c6da94d5bb18629db453d14f2a1300f6bf59b61e9b82377931ef95504866", + "https://files.pythonhosted.org/packages/ec/9e/db72b3ae7fc9cfad53e630e56c6ae83b9b6ff0bf3718ffb8012d20b3aabf/cryptography-49.0.0-cp314-cp314t-macosx_11_0_arm64.whl": "sha256:73a205dce83953d131a4aa1e0fd917a2fd1c5b1eef251e9d7152efefcbf5caf7", + "https://files.pythonhosted.org/packages/f0/ee/6fca21d1ac73e06f8bef71940abfd4d2f6472b4bca284d770f32bd4086f6/cryptography-49.0.0-cp39-abi3-manylinux_2_34_aarch64.whl": "sha256:28d8b15e6275f12c8a207dc309dfa957903c927d08d0cc937ee3f63f200693cc" + }, + "docutils": { + "https://files.pythonhosted.org/packages/32/91/30151a39f7570f448ed84529390628a651d7f27c87d73c9b887f8189695e/docutils-0.23-py3-none-any.whl": "sha256:25d013af9bf23bc1c7b2b093dff4208166c53a94786c9e447808335ef1185fea", + "https://files.pythonhosted.org/packages/39/a4/5180d9afc57e8fca05601dd652bdff19604c218814037fe90ffc7625a50a/docutils-0.23.tar.gz": "sha256:746f5060322511280a1e50eb76846ed6bf2342984b2ac04dc42caa1a8d78799e" + }, + "id": { + "https://files.pythonhosted.org/packages/42/77/de194443bf38daed9452139e960c632b0ef9f9a5dd9ce605fdf18ca9f1b1/id-1.6.1-py3-none-any.whl": "sha256:f5ec41ed2629a508f5d0988eda142e190c9c6da971100612c4de9ad9f9b237ca", + "https://files.pythonhosted.org/packages/6d/04/c2156091427636080787aac190019dc64096e56a23b7364d3c1764ee3a06/id-1.6.1.tar.gz": "sha256:d0732d624fb46fd4e7bc4e5152f00214450953b9e772c182c1c22964def1a069" + }, + "idna": { + "https://files.pythonhosted.org/packages/1e/5e/d4e9f1a599fb8e573b7b87160658329fbf28d19eac2718f51fc3def3aa5a/idna-3.18-py3-none-any.whl": "sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2", + "https://files.pythonhosted.org/packages/cd/63/9496c57188a2ee585e0f1db071d75089a11e98aa86eb99d9d7618fc1edce/idna-3.18.tar.gz": "sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848" + }, + "importlib-metadata": { + "https://files.pythonhosted.org/packages/38/3d/2d244233ac4f76e38533cfcb2991c9eb4c7bf688ae0a036d30725b8faafe/importlib_metadata-9.0.0-py3-none-any.whl": "sha256:2d21d1cc5a017bd0559e36150c21c830ab1dc304dedd1b7ea85d20f45ef3edd7", + "https://files.pythonhosted.org/packages/a9/01/15bb152d77b21318514a96f43af312635eb2500c96b55398d020c93d86ea/importlib_metadata-9.0.0.tar.gz": "sha256:a4f57ab599e6a2e3016d7595cfd72eb4661a5106e787a95bcc90c7105b831efc" + }, + "jaraco-classes": { + "https://files.pythonhosted.org/packages/06/c0/ed4a27bc5571b99e3cff68f8a9fa5b56ff7df1c2251cc715a652ddd26402/jaraco.classes-3.4.0.tar.gz": "sha256:47a024b51d0239c0dd8c8540c6c7f484be3b8fcf0b2d85c13825780d3b3f3acd", + "https://files.pythonhosted.org/packages/7f/66/b15ce62552d84bbfcec9a4873ab79d993a1dd4edb922cbfccae192bd5b5f/jaraco.classes-3.4.0-py3-none-any.whl": "sha256:f662826b6bed8cace05e7ff873ce0f9283b5c924470fe664fff1c2f00f581790" + }, + "jaraco-context": { + "https://files.pythonhosted.org/packages/af/50/4763cd07e722bb6285316d390a164bc7e479db9d90daa769f22578f698b4/jaraco_context-6.1.2.tar.gz": "sha256:f1a6c9d391e661cc5b8d39861ff077a7dc24dc23833ccee564b234b81c82dfe3", + "https://files.pythonhosted.org/packages/f2/58/bc8954bda5fcda97bd7c19be11b85f91973d67a706ed4a3aec33e7de22db/jaraco_context-6.1.2-py3-none-any.whl": "sha256:bf8150b79a2d5d91ae48629d8b427a8f7ba0e1097dd6202a9059f29a36379535" + }, + "jaraco-functools": { + "https://files.pythonhosted.org/packages/02/36/ecc85bc96c273dc8a11273ed4782272975e6338d4a3e9228621175edf0e3/jaraco_functools-4.6.0-py3-none-any.whl": "sha256:99e3dc0060c5cbe8fcd1cdb36258e2a65ca40f1566b2033b12abb1bb44dd3c30", + "https://files.pythonhosted.org/packages/6c/1f/c23395957d41ccf27c4e535c3d334c4051e5395b3752057ba4cbaec35c56/jaraco_functools-4.6.0.tar.gz": "sha256:880c577ec9720b3a052d5bc611fb9f2269b3d87902ef42440df443b88e443280" + }, + "jeepney": { + "https://files.pythonhosted.org/packages/7b/6f/357efd7602486741aa73ffc0617fb310a29b588ed0fd69c2399acbb85b0c/jeepney-0.9.0.tar.gz": "sha256:cf0e9e845622b81e4a28df94c40345400256ec608d0e55bb8a3feaa9163f5732", + "https://files.pythonhosted.org/packages/b2/a3/e137168c9c44d18eff0376253da9f1e9234d0239e0ee230d2fee6cea8e55/jeepney-0.9.0-py3-none-any.whl": "sha256:97e5714520c16fc0a45695e5365a2e11b81ea79bba796e26f9f1d178cb182683" + }, + "keyring": { + "https://files.pythonhosted.org/packages/43/4b/674af6ef2f97d56f0ab5153bf0bfa28ccb6c3ed4d1babf4305449668807b/keyring-25.7.0.tar.gz": "sha256:fe01bd85eb3f8fb3dd0405defdeac9a5b4f6f0439edbb3149577f244a2e8245b", + "https://files.pythonhosted.org/packages/81/db/e655086b7f3a705df045bf0933bdd9c2f79bb3c97bfef1384598bb79a217/keyring-25.7.0-py3-none-any.whl": "sha256:be4a0b195f149690c166e850609a477c532ddbfbaed96a404d4e43f8d5e2689f" + }, + "markdown-it-py": { + "https://files.pythonhosted.org/packages/06/ff/7841249c247aa650a76b9ee4bbaeae59370dc8bfd2f6c01f3630c35eb134/markdown_it_py-4.2.0.tar.gz": "sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49", + "https://files.pythonhosted.org/packages/b3/81/4da04ced5a082363ecfa159c010d200ecbd959ae410c10c0264a38cac0f5/markdown_it_py-4.2.0-py3-none-any.whl": "sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a" + }, + "mdurl": { + "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl": "sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", + "https://files.pythonhosted.org/packages/d6/54/cfe61301667036ec958cb99bd3efefba235e65cdeb9c84d24a8293ba1d90/mdurl-0.1.2.tar.gz": "sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba" + }, + "more-itertools": { + "https://files.pythonhosted.org/packages/de/1d/f4da6f02cdffe04d6362210b807146a26044c88d839208aec273bb0d9184/more_itertools-11.1.0.tar.gz": "sha256:48e8f4d9e7e5878571ecf6f2b4e57634f93cd474cc8cfbd2376f2d11b396e30d", + "https://files.pythonhosted.org/packages/e8/3d/1087453384dbde46a8c7f9356eead2c58be8a7bf156bca40243377c85715/more_itertools-11.1.0-py3-none-any.whl": "sha256:4b65538ae22f6fed0ce4874efd317463a7489796a0939fa66824dd542125a192" + }, + "nh3": { + "https://files.pythonhosted.org/packages/09/a1/ea83abe738a3fbaa203dfdb836ca7cbab0e7e9609faaee4fe1d4652599c0/nh3-0.3.6-cp314-cp314t-musllinux_1_2_i686.whl": "sha256:edb2b4a1a27523e6cc7c417f8d21ce3d005243548b93e56b762b66b0c7f589f9", + "https://files.pythonhosted.org/packages/0a/13/6f1e302ca674ac74362e150848ad56a1be5145391204f74facdb8e94df12/nh3-0.3.6-cp38-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl": "sha256:455469a29951edc92bc48b47ac2281c3f2609e6c4f6a047056449f8c2c23facf", + "https://files.pythonhosted.org/packages/0e/24/a0d80182a18919665fefd19c1c06f1d1df1c9a6455d0252de40c034a0bc3/nh3-0.3.6-cp38-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl": "sha256:e6b7beece07525dc6e6b0fc2f104442de2ba328360ad00e50cbe2e1fd620447d", + "https://files.pythonhosted.org/packages/11/f9/3966c61455668c08853bf5e33b4bed93c421f3194ce4de896dc248d6f6ce/nh3-0.3.6-cp38-abi3-musllinux_1_2_armv7l.whl": "sha256:f5ed5fe84aee7f39db95c214a7421bf0499fbf500fec6d86a4e29bfc37971438", + "https://files.pythonhosted.org/packages/17/0c/6cdb5ee1e127be50dc8391e54bddc1f64e87bf4bfad0c55633320e2e02db/nh3-0.3.6-cp38-abi3-musllinux_1_2_x86_64.whl": "sha256:36d06341bd501240d320f5942481ed5e6846136b666e1ba4faf802b78ebc875f", + "https://files.pythonhosted.org/packages/19/d3/479cb4ae440424825735d60525b53e3c77fd60fd6e6afc0e984f00eb0178/nh3-0.3.6-cp38-abi3-musllinux_1_2_i686.whl": "sha256:082675ff87b9385ec430ffe6d5847ba7456cc39b73720cd4add472f9f4cffd56", + "https://files.pythonhosted.org/packages/25/bb/431615ba1d1d3eb63cde0f974f2114edf863a8a3f6049a12fed23fc241d3/nh3-0.3.6-cp38-abi3-manylinux_2_17_ppc64.manylinux2014_ppc64.whl": "sha256:44673b27010051ab5a5e438a86ec31bbda61d4a77d7e900af6b7be3037c1abae", + "https://files.pythonhosted.org/packages/30/a8/fb2c38845efb703a9173bffdfc745fc64d2b0e55cfc73a3647d2f028250c/nh3-0.3.6-cp38-abi3-manylinux_2_5_i686.manylinux1_i686.whl": "sha256:2f90d9a0cfdbee218994fdaaeeb5a0fde62d08f35e4eef0378ec1e2200172fd0", + "https://files.pythonhosted.org/packages/36/ea/5542f3c45da4c00290d9d67a65e996702e23e613c4b627de3e09cb9fe357/nh3-0.3.6-cp314-cp314t-win_amd64.whl": "sha256:4713502748f564fee0633b37b3403783ce0a3af3a3d148ad91025a5bdadb7bc6", + "https://files.pythonhosted.org/packages/3c/a6/bfaa00046e58603507dcfc266c4778e3ab7adf68a5dedd73b6274b8d9314/nh3-0.3.6-cp38-abi3-manylinux_2_31_riscv64.whl": "sha256:25c733bee928530556b1db0ea46c52cf5aa686146e38e60a6fc7cb801ef91cec", + "https://files.pythonhosted.org/packages/41/21/e1084ab18eb589506335c7c7576f2d4643e9a0c0e33983ef0e549a256b96/nh3-0.3.6-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl": "sha256:e1b160831c9cdb06a6c79c2f9cdb11386602938f9af260d1c457a85add4f6f69", + "https://files.pythonhosted.org/packages/49/09/0d8e3101636d9ad88cdefb2914e764cb8e876ebdbb4286bfc251277d9c67/nh3-0.3.6-cp314-cp314t-musllinux_1_2_armv7l.whl": "sha256:889932a97fb4abb6f95fef1914c0d269ebfb60011e67121c1163059b9449dbb4", + "https://files.pythonhosted.org/packages/4b/4a/526f199626bfcb496bc01a268051b44737962005553b158e985ed7e64865/nh3-0.3.6-cp314-cp314t-musllinux_1_2_aarch64.whl": "sha256:f2f14b7ae1fca99c4a66c981aac3974e7fbc1ca30a12673d223ae1df76680917", + "https://files.pythonhosted.org/packages/59/62/5b6108bedaef2b2637fed04c87bdbcb5967b9961758b41f0e466ef22a022/nh3-0.3.6-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl": "sha256:34d2b0d934156b87ee114f599a3ba9b8b9e17b5d79652ba3a13fa50903de965e", + "https://files.pythonhosted.org/packages/5b/67/314f6151bad77a93d751978a344033e1fc890822f05f0416079338e34231/nh3-0.3.6-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl": "sha256:905f877dc66dd7aea4a76e54bcb26acb5ff8216f720c0017ccf63e0e6035698e", + "https://files.pythonhosted.org/packages/5e/1b/ef84624f14954d270f74060a19fc550dd4f06656399447569afb584d8c06/nh3-0.3.6.tar.gz": "sha256:f3736c9dd3d1856f80cd031715b84ca75cda2bbb1ac802c3da26bfce590838d7", + "https://files.pythonhosted.org/packages/66/35/26bd47e6af5915a628281dccdac354ddf4e32f7397047894270acd8c9870/nh3-0.3.6-cp314-cp314t-win_arm64.whl": "sha256:69bbb92865a693d909db3a700d3c01537533844d0948c1e9323561ce06ecda41", + "https://files.pythonhosted.org/packages/66/69/0654482b8635012fbae67826bd6c381abb05d841ac7388b9b4666300fdad/nh3-0.3.6-cp314-cp314t-musllinux_1_2_x86_64.whl": "sha256:43bc1ed3fa0716295fabee29ba42b2667e4a51d140b0a68e092170a765474fa6", + "https://files.pythonhosted.org/packages/68/17/06e72a18ee9b572914447338237ca7eb164c0df901f141bc10d1282247a2/nh3-0.3.6-cp38-abi3-musllinux_1_2_aarch64.whl": "sha256:82ca5bf427ad1b216b65ede1a2e2d87dc49bec417ceba0f297213107d3cd9d78", + "https://files.pythonhosted.org/packages/7b/e5/7cafee2f0413ca4cb0ef3bd111e94d408a48810008b283ad8aee00dd1809/nh3-0.3.6-cp38-abi3-win_arm64.whl": "sha256:69f365963f63a1e9bff53bdbb3c542c7c2efed3e163c9d5d83a772a2ac468c21", + "https://files.pythonhosted.org/packages/82/fa/2b5d684e3edf1e81bfd02d298c78c3e3da77ca1d8a2be3183a79544a7548/nh3-0.3.6-cp38-abi3-win_amd64.whl": "sha256:f338ac7d594c067679f1e99b4f5ec3906842979560f9d8f15d6bdfa39a353b10", + "https://files.pythonhosted.org/packages/99/3e/6506aa4f23dc7b7993a2d0a45dca3ce864ec48380adfe15a173e643c63e8/nh3-0.3.6-cp314-cp314t-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl": "sha256:2411e8c3cee81a1ddd62c2a5d50585c28aa5566d373ad1db92536b95ddb24ef2", + "https://files.pythonhosted.org/packages/b0/94/f48d08e6f72a406300fa11d8acd929fea1a80d4bf750fa292cb10785f126/nh3-0.3.6-cp38-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl": "sha256:d14bf7982e7a77c0c775634c29c07ce08b38a046df73e1c1f139b3e82f18a38e", + "https://files.pythonhosted.org/packages/e3/e1/e96e7864a7a53bd6b6fab7e9632467382a2a2c1f3fed951918ad131542fb/nh3-0.3.6-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl": "sha256:e196fa70c2ff2eb4de7d3df3108f8f358c1d69dff20d45b11f20a5aa227ffb6d", + "https://files.pythonhosted.org/packages/e9/55/9de666ad975d6ccd77d799ea0add55ee2347aa81286ce21b2a97c070746b/nh3-0.3.6-cp38-abi3-win32.whl": "sha256:5276ef17bdba9ad8040575c74072008b13aae429436e9d0429e718bb5f90f4da", + "https://files.pythonhosted.org/packages/ed/a6/1f7285ffadc8307c4dbeb08d21b920536d5117785056d1079e998c4dfa44/nh3-0.3.6-cp314-cp314t-win32.whl": "sha256:597a8e843bea00b2eb5520658dc24a9bb032e7fc9e7c2c0c4cd29420220c9796", + "https://files.pythonhosted.org/packages/f3/ab/a7653bce9a3b204be6a6931767a9e23595807bb84790ce6685e4d7e5bd08/nh3-0.3.6-cp38-abi3-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl": "sha256:a43ebd7543555c3ac1bc353023d0794e75cb76f6f18f19c32e95441496c0cc25" + }, + "packaging": { + "https://files.pythonhosted.org/packages/d7/f1/e7a6dd94a8d4a5626c03e4e99c87f241ba9e350cd9e6d75123f992427270/packaging-26.2.tar.gz": "sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661", + "https://files.pythonhosted.org/packages/df/b2/87e62e8c3e2f4b32e5fe99e0b86d576da1312593b39f47d8ceef365e95ed/packaging-26.2-py3-none-any.whl": "sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e" + }, + "pycparser": { + "https://files.pythonhosted.org/packages/0c/c3/44f3fbbfa403ea2a7c779186dc20772604442dde72947e7d01069cbe98e3/pycparser-3.0-py3-none-any.whl": "sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992", + "https://files.pythonhosted.org/packages/1b/7d/92392ff7815c21062bea51aa7b87d45576f649f16458d78b7cf94b9ab2e6/pycparser-3.0.tar.gz": "sha256:600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29" + }, + "pygments": { + "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz": "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", + "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl": "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176" + }, + "pywin32-ctypes": { + "https://files.pythonhosted.org/packages/85/9f/01a1a99704853cb63f253eea009390c88e7131c67e66a0a02099a8c917cb/pywin32-ctypes-0.2.3.tar.gz": "sha256:d162dc04946d704503b2edc4d55f3dba5c1d539ead017afa00142c38b9885755", + "https://files.pythonhosted.org/packages/de/3d/8161f7711c017e01ac9f008dfddd9410dff3674334c233bde66e7ba65bbf/pywin32_ctypes-0.2.3-py3-none-any.whl": "sha256:8a1513379d709975552d202d942d9837758905c8d01eb82b8bcc30918929e7b8" + }, + "readme-renderer": { + "https://files.pythonhosted.org/packages/02/51/d3a6ea424652c60f05600d8c2e01a55c913755e7cdad64afabbd1aa16f44/readme_renderer-45.0.tar.gz": "sha256:030a8fac74904f8fba11ad1bb6964e3f76e896dc7e5e71f16af190c9056696d1", + "https://files.pythonhosted.org/packages/97/1b/295bf2fa3e740131778065e5ffa2c481f0e7210182d408e9a2c244ff5b0c/readme_renderer-45.0-py3-none-any.whl": "sha256:3385ed220117104a2bceb4a9dac8c5fdf6d1f96890d7ea2a9c7174fd5c84091f" + }, + "requests": { + "https://files.pythonhosted.org/packages/a0/f4/c67b0b3f1b9245e8d266f0f112c500d50e5b4e83cb6f3b71b6528104182a/requests-2.34.2-py3-none-any.whl": "sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0", + "https://files.pythonhosted.org/packages/ac/c3/e2a2b89f2d3e2179abd6d00ebd70bff6273f37fb3e0cc209f48b39d00cbf/requests-2.34.2.tar.gz": "sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed" + }, + "requests-toolbelt": { + "https://files.pythonhosted.org/packages/3f/51/d4db610ef29373b879047326cbf6fa98b6c1969d6f6dc423279de2b1be2c/requests_toolbelt-1.0.0-py2.py3-none-any.whl": "sha256:cccfdd665f0a24fcf4726e690f65639d272bb0637b9b92dfd91a5568ccf6bd06", + "https://files.pythonhosted.org/packages/f3/61/d7545dafb7ac2230c70d38d31cbfe4cc64f7144dc41f6e4e4b78ecd9f5bb/requests-toolbelt-1.0.0.tar.gz": "sha256:7681a0a3d047012b5bdc0ee37d7f8f07ebe76ab08caeccfc3921ce23c88d5bc6" + }, + "rfc3986": { + "https://files.pythonhosted.org/packages/85/40/1520d68bfa07ab5a6f065a186815fb6610c86fe957bc065754e47f7b0840/rfc3986-2.0.0.tar.gz": "sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c", + "https://files.pythonhosted.org/packages/ff/9a/9afaade874b2fa6c752c36f1548f718b5b83af81ed9b76628329dab81c1b/rfc3986-2.0.0-py2.py3-none-any.whl": "sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd" + }, + "rich": { + "https://files.pythonhosted.org/packages/82/3b/64d4899d73f91ba49a8c18a8ff3f0ea8f1c1d75481760df8c68ef5235bf5/rich-15.0.0-py3-none-any.whl": "sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb", + "https://files.pythonhosted.org/packages/c0/8f/0722ca900cc807c13a6a0c696dacf35430f72e0ec571c4275d2371fca3e9/rich-15.0.0.tar.gz": "sha256:edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36" + }, + "secretstorage": { + "https://files.pythonhosted.org/packages/1c/03/e834bcd866f2f8a49a85eaff47340affa3bfa391ee9912a952a1faa68c7b/secretstorage-3.5.0.tar.gz": "sha256:f04b8e4689cbce351744d5537bf6b1329c6fc68f91fa666f60a380edddcd11be", + "https://files.pythonhosted.org/packages/b7/46/f5af3402b579fd5e11573ce652019a67074317e18c1935cc0b4ba9b35552/secretstorage-3.5.0-py3-none-any.whl": "sha256:0ce65888c0725fcb2c5bc0fdb8e5438eece02c523557ea40ce0703c266248137" + }, + "six": { + "https://files.pythonhosted.org/packages/94/e7/b2c673351809dca68a0e064b6af791aa332cf192da575fd474ed7d6f16a2/six-1.17.0.tar.gz": "sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81", + "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl": "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274" + }, + "twine": { + "https://files.pythonhosted.org/packages/92/3c/58f808a359700f39a967dffede33efeac809262c03303fa3eec6afff8f49/twine-7.0.0.tar.gz": "sha256:85cdb29c518efef867360ae4acd4b0dfd61c8654a22fca08e6f8539f05022177", + "https://files.pythonhosted.org/packages/96/08/ddcdc06225eaad6de0e48e1002b06d919dbde20582d0662c7af51308e5d6/twine-7.0.0-py3-none-any.whl": "sha256:b854164df26db268af05f49aa5c0344b10e27a494343ff05b1e0bad3b135f5a7" + }, + "urllib3": { + "https://files.pythonhosted.org/packages/53/0c/06f8b233b8fd13b9e5ee11424ef85419ba0d8ba0b3138bf360be2ff56953/urllib3-2.7.0.tar.gz": "sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c", + "https://files.pythonhosted.org/packages/7f/3e/5db95bcf282c52709639744ca2a8b149baccf648e39c8cc87553df9eae0c/urllib3-2.7.0-py3-none-any.whl": "sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897" + }, + "zipp": { + "https://files.pythonhosted.org/packages/3a/13/547360d81e6d88d58492968ffda9f9542854f11310ee556fef14260cc886/zipp-4.1.0-py3-none-any.whl": "sha256:25ad4e16390cd314347dd8f1de67a2ac538ae658ed4ab9db16029c07c188e97f", + "https://files.pythonhosted.org/packages/b9/d8/eab98a517c14134c0b2eb4e2387bc5f457334293ec5d2dd3857ec2966802/zipp-4.1.0.tar.gz": "sha256:4cb57381f544315db7688e976e922a2b18cdb513d21cc194eb42232ba2a3e602" + } + } + }, + "fact_version": "v2" + } + } +} diff --git a/tests/integration/bzlmod_lockfile/README.md b/tests/integration/bzlmod_lockfile/README.md new file mode 100644 index 0000000000..42c941c4dc --- /dev/null +++ b/tests/integration/bzlmod_lockfile/README.md @@ -0,0 +1,6 @@ +# Pip parse isolation and lock file test + +Update the lock file with the following command: +``` +bazel mod deps --lockfile_mode=update +``` diff --git a/tests/integration/bzlmod_lockfile/WORKSPACE b/tests/integration/bzlmod_lockfile/WORKSPACE new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/integration/bzlmod_lockfile/requirements_lock.txt b/tests/integration/bzlmod_lockfile/requirements_lock.txt new file mode 100644 index 0000000000..d8dcf2d44e --- /dev/null +++ b/tests/integration/bzlmod_lockfile/requirements_lock.txt @@ -0,0 +1,8 @@ +six==1.17.0 \ + --hash=sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274 +# rules_python v2.0.0's own tools/publish/requirements_linux.txt pins pycparser==2.23. +# On macOS: lock gets pycparser-3.0 only (darwin publish requirements have no pycparser). +# On Linux: rules_python_publish_deps hub adds pycparser-2.23 to computed facts → mismatch. +pycparser==3.0 \ + --hash=sha256:600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29 \ + --hash=sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992 diff --git a/tests/integration/bzlmod_lockfile/test_dummy.py b/tests/integration/bzlmod_lockfile/test_dummy.py new file mode 100644 index 0000000000..110b1ba42f --- /dev/null +++ b/tests/integration/bzlmod_lockfile/test_dummy.py @@ -0,0 +1,17 @@ +""" +Verify that a dependency added using the pip extension can be imported. +See MODULE.bazel. +""" + +import unittest + +import six + + +class TestDummy(unittest.TestCase): + def test_import(self): + self.assertTrue(hasattr(six, "PY3")) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/integration/bzlmod_lockfile_test.py b/tests/integration/bzlmod_lockfile_test.py new file mode 100644 index 0000000000..650f132f0f --- /dev/null +++ b/tests/integration/bzlmod_lockfile_test.py @@ -0,0 +1,59 @@ +import difflib +import os +import pathlib +import unittest + +from tests.integration import runner + + +class BzlmodLockfileTest(runner.TestCase): + def test_bzlmod_lockfile(self): + lockfile_path = self.repo_root / "MODULE.bazel.lock" + self.assertTrue( + lockfile_path.exists(), + f"Expected lockfile at {lockfile_path}", + ) + original_lockfile = lockfile_path.read_text() + + res = self.run_bazel("test", "//...", check=False) + if res.exit_code == 0: + return + + # Generate updated lockfile to compare diff and export artifact + self.run_bazel("mod", "deps", "--lockfile_mode=update", check=False) + updated_lockfile = lockfile_path.read_text() if lockfile_path.exists() else "" + + undeclared_outputs_dir = os.environ.get("TEST_UNDECLARED_OUTPUTS_DIR") + if undeclared_outputs_dir: + out_dir = pathlib.Path(undeclared_outputs_dir) + (out_dir / "MODULE.bazel.lock").write_text(updated_lockfile) + + if original_lockfile != updated_lockfile: + diff_lines = list( + difflib.unified_diff( + original_lockfile.splitlines(keepends=True), + updated_lockfile.splitlines(keepends=True), + fromfile="MODULE.bazel.lock (checked-in)", + tofile="MODULE.bazel.lock (expected/updated)", + ) + ) + diff_str = "".join(diff_lines) + + if undeclared_outputs_dir: + (out_dir / "MODULE.bazel.lock.diff").write_text(diff_str) + + msg = ( + f"MODULE.bazel.lock is out of date.\n\n" + f"--- DIFF ---\n{diff_str}\n" + f"--- END DIFF ---\n\n" + f"To update the lockfile, run:\n" + f" bazel mod deps --lockfile_mode=update\n" + f"inside tests/integration/bzlmod_lockfile, or copy the updated MODULE.bazel.lock artifact.\n" + ) + self.fail(msg) + else: + self.fail(f"bazel test //... failed:\n{res.describe()}") + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/integration/compile_pip_requirements/BUILD.bazel b/tests/integration/compile_pip_requirements/BUILD.bazel index 6df46b8372..b1f398c4a8 100644 --- a/tests/integration/compile_pip_requirements/BUILD.bazel +++ b/tests/integration/compile_pip_requirements/BUILD.bazel @@ -29,6 +29,19 @@ compile_pip_requirements( requirements_txt = "requirements_lock.txt", ) +genquery( + name = "requirements_update_data", + expression = """ +some(labels(data, @compile_pip_requirements//:requirements.update) intersect @compile_pip_requirements//:requirements.in) union +some(labels(data, @compile_pip_requirements//:requirements.update) intersect @compile_pip_requirements//:requirements_extra.in) +""", + scope = [ + "@compile_pip_requirements//:requirements.update", + "@compile_pip_requirements//:requirements.in", + "@compile_pip_requirements//:requirements_extra.in", + ], +) + compile_pip_requirements( name = "requirements_nohashes", src = "requirements.txt", diff --git a/tests/integration/compile_pip_requirements/WORKSPACE b/tests/integration/compile_pip_requirements/WORKSPACE index 0eeab2067c..397d828261 100644 --- a/tests/integration/compile_pip_requirements/WORKSPACE +++ b/tests/integration/compile_pip_requirements/WORKSPACE @@ -3,6 +3,11 @@ local_repository( path = "../../..", ) +local_repository( + name = "compile_pip_requirements", + path = ".", +) + load("@rules_python//python:repositories.bzl", "py_repositories", "python_register_toolchains") py_repositories() diff --git a/tests/integration/compile_pip_requirements_test_from_external_repo/MODULE.bazel b/tests/integration/compile_pip_requirements_test_from_external_repo/MODULE.bazel index 596a0bcfc8..201cb4a1ab 100644 --- a/tests/integration/compile_pip_requirements_test_from_external_repo/MODULE.bazel +++ b/tests/integration/compile_pip_requirements_test_from_external_repo/MODULE.bazel @@ -19,7 +19,7 @@ local_path_override( pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") pip.parse( - hub_name = "pypi", + hub_name = "pypi_compile", python_version = "3.9", requirements_lock = "@compile_pip_requirements//:requirements_lock.txt", ) diff --git a/tests/integration/custom_commands_test.py b/tests/integration/custom_commands_test.py index f78ee468bd..336937ece0 100644 --- a/tests/integration/custom_commands_test.py +++ b/tests/integration/custom_commands_test.py @@ -12,16 +12,20 @@ # See the License for the specific language governing permissions and # limitations under the License. -import logging import unittest from tests.integration import runner class CustomCommandsTest(runner.TestCase): - # Regression test for https://github.com/bazelbuild/rules_python/issues/1840 + # Regression test for https://github.com/bazel-contrib/rules_python/issues/1840 def test_run_build_python_zip_false(self): - result = self.run_bazel("run", "--build_python_zip=false", "//:bin") + result = self.run_bazel( + "run", + "--build_python_zip=false", + "--@rules_python//python/config_settings:build_python_zip=false", + "//:bin", + ) self.assert_result_matches(result, "bazel-out") diff --git a/tests/integration/ignore_root_user_error/.bazelrc b/tests/integration/ignore_root_user_error/.bazelrc deleted file mode 100644 index bb7b5742cd..0000000000 --- a/tests/integration/ignore_root_user_error/.bazelrc +++ /dev/null @@ -1,7 +0,0 @@ -common --action_env=RULES_PYTHON_BZLMOD_DEBUG=1 -common --lockfile_mode=off -test --test_output=errors - -# Windows requires these for multi-python support: -build --enable_runfiles -common:bazel7.x --incompatible_python_disallow_native_rules diff --git a/tests/integration/ignore_root_user_error/.gitignore b/tests/integration/ignore_root_user_error/.gitignore deleted file mode 100644 index ac51a054d2..0000000000 --- a/tests/integration/ignore_root_user_error/.gitignore +++ /dev/null @@ -1 +0,0 @@ -bazel-* diff --git a/tests/integration/ignore_root_user_error/BUILD.bazel b/tests/integration/ignore_root_user_error/BUILD.bazel deleted file mode 100644 index 6e3b7b9d24..0000000000 --- a/tests/integration/ignore_root_user_error/BUILD.bazel +++ /dev/null @@ -1,32 +0,0 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -load("@rules_python//python:py_test.bzl", "py_test") -load("@rules_python//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility - -py_test( - name = "foo_test", - srcs = ["foo_test.py"], - visibility = ["//visibility:public"], -) - -py_test( - name = "bzlmod_test", - srcs = ["bzlmod_test.py"], - data = [ - "@rules_python//python/runfiles", - "@rules_python_bzlmod_debug//:debug_info.json", - ], - target_compatible_with = [] if BZLMOD_ENABLED else ["@platforms//:incompatible"], -) diff --git a/tests/integration/ignore_root_user_error/MODULE.bazel b/tests/integration/ignore_root_user_error/MODULE.bazel deleted file mode 100644 index 15c37c4388..0000000000 --- a/tests/integration/ignore_root_user_error/MODULE.bazel +++ /dev/null @@ -1,20 +0,0 @@ -module(name = "ignore_root_user_error") - -bazel_dep(name = "rules_python", version = "0.0.0") -local_path_override( - module_name = "rules_python", - path = "../../..", -) - -bazel_dep(name = "submodule") -local_path_override( - module_name = "submodule", - path = "submodule", -) - -python = use_extension("@rules_python//python/extensions:python.bzl", "python") -python.toolchain( - ignore_root_user_error = True, - python_version = "3.11", -) -use_repo(python, "rules_python_bzlmod_debug") diff --git a/tests/integration/ignore_root_user_error/README.md b/tests/integration/ignore_root_user_error/README.md deleted file mode 100644 index 47da5eb9ad..0000000000 --- a/tests/integration/ignore_root_user_error/README.md +++ /dev/null @@ -1,2 +0,0 @@ -# ignore_root_user_errors -There are cases when we have to run Python targets with root, e.g., in Docker containers, requiring setting `ignore_root_user_error = True` when registering Python toolchain. This test makes sure that rules_python works in this case. \ No newline at end of file diff --git a/tests/integration/ignore_root_user_error/WORKSPACE b/tests/integration/ignore_root_user_error/WORKSPACE deleted file mode 100644 index 0a25819ecd..0000000000 --- a/tests/integration/ignore_root_user_error/WORKSPACE +++ /dev/null @@ -1,29 +0,0 @@ -load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") - -local_repository( - name = "rules_python", - path = "../../..", -) - -load("@rules_python//python:repositories.bzl", "py_repositories", "python_register_toolchains") - -py_repositories() - -python_register_toolchains( - name = "python39", - ignore_root_user_error = True, - python_version = "3.9", -) - -http_archive( - name = "bazel_skylib", - sha256 = "c6966ec828da198c5d9adbaa94c05e3a1c7f21bd012a0b29ba8ddbccb2c93b0d", - urls = [ - "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.1.1/bazel-skylib-1.1.1.tar.gz", - "https://github.com/bazelbuild/bazel-skylib/releases/download/1.1.1/bazel-skylib-1.1.1.tar.gz", - ], -) - -load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace") - -bazel_skylib_workspace() diff --git a/tests/integration/ignore_root_user_error/bzlmod_test.py b/tests/integration/ignore_root_user_error/bzlmod_test.py deleted file mode 100644 index a1d6dc0630..0000000000 --- a/tests/integration/ignore_root_user_error/bzlmod_test.py +++ /dev/null @@ -1,40 +0,0 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -import json -import pathlib -import unittest - -from python.runfiles import runfiles - - -class BzlmodTest(unittest.TestCase): - def test_ignore_root_user_error_true_for_all_toolchains(self): - rf = runfiles.Create() - debug_path = pathlib.Path( - rf.Rlocation("rules_python_bzlmod_debug/debug_info.json") - ) - debug_info = json.loads(debug_path.read_bytes()) - actual = debug_info["toolchains_registered"] - # Because the root module set ignore_root_user_error=True, that should - # be the default for all other toolchains. - for entry in actual: - self.assertTrue( - entry["ignore_root_user_error"], - msg=f"Expected ignore_root_user_error=True, but got: {entry}", - ) - - -if __name__ == "__main__": - unittest.main() diff --git a/tests/integration/ignore_root_user_error/submodule/MODULE.bazel b/tests/integration/ignore_root_user_error/submodule/MODULE.bazel deleted file mode 100644 index f12870963c..0000000000 --- a/tests/integration/ignore_root_user_error/submodule/MODULE.bazel +++ /dev/null @@ -1,9 +0,0 @@ -module(name = "submodule") - -bazel_dep(name = "rules_python", version = "0.0.0") - -python = use_extension("@rules_python//python/extensions:python.bzl", "python") -python.toolchain( - ignore_root_user_error = False, - python_version = "3.10", -) diff --git a/tests/integration/integration_test.bzl b/tests/integration/integration_test.bzl index c437953319..f3d5cb6967 100644 --- a/tests/integration/integration_test.bzl +++ b/tests/integration/integration_test.bzl @@ -21,32 +21,22 @@ load( ) load("//python:py_test.bzl", "py_test") -def _test_runner(*, name, bazel_version, py_main, bzlmod, gazelle_plugin): +def _test_runner(*, name, bazel_version, py_main, bzlmod, py_deps): if py_main: test_runner = "{}_bazel_{}_py_runner".format(name, bazel_version) py_test( name = test_runner, srcs = [py_main], main = py_main, - deps = [":runner_lib"], + deps = [":runner_lib"] + py_deps, # Hide from ... patterns; should only be run as part # of the bazel integration test tags = ["manual"], ) return test_runner - if bazel_version.startswith("6") and not bzlmod: - if gazelle_plugin: - return "//tests/integration:bazel_6_4_workspace_test_runner_gazelle_plugin" - else: - return "//tests/integration:bazel_6_4_workspace_test_runner" - - if bzlmod and gazelle_plugin: - return "//tests/integration:test_runner_gazelle_plugin" - elif bzlmod: + if bzlmod: return "//tests/integration:test_runner" - elif gazelle_plugin: - return "//tests/integration:workspace_test_runner_gazelle_plugin" else: return "//tests/integration:workspace_test_runner" @@ -54,9 +44,9 @@ def rules_python_integration_test( name, workspace_path = None, bzlmod = True, - gazelle_plugin = False, tags = None, py_main = None, + py_deps = None, bazel_versions = None, **kwargs): """Runs a bazel-in-bazel integration test. @@ -67,11 +57,11 @@ def rules_python_integration_test( `_test` suffix. bzlmod: bool, default True. If true, run with bzlmod enabled, otherwise disable bzlmod. - gazelle_plugin: Whether the test uses the gazelle plugin. tags: Test tags. py_main: Optional `.py` file to run tests using. When specified, a python based test runner is used, and this source file is the main entry point and responsible for executing tests. + py_deps: Optional test runner deps to use for setup. bazel_versions: `list[str] | None`, the bazel versions to test. I not specified, defaults to all configured bazel versions. **kwargs: Passed to the upstream `bazel_integration_tests` rule. @@ -103,8 +93,8 @@ def rules_python_integration_test( name = name, bazel_version = bazel_version, py_main = py_main, + py_deps = py_deps or [], bzlmod = bzlmod, - gazelle_plugin = gazelle_plugin, ) bazel_integration_test( name = "{}_bazel_{}".format(name, bazel_version), diff --git a/tests/integration/local_toolchains/.bazelrc b/tests/integration/local_toolchains/.bazelrc index 39df41d9f4..0fbb7678d1 100644 --- a/tests/integration/local_toolchains/.bazelrc +++ b/tests/integration/local_toolchains/.bazelrc @@ -1,6 +1,10 @@ +startup --windows_enable_symlinks common --action_env=RULES_PYTHON_BZLMOD_DEBUG=1 +common --repo_env=RULES_PYTHON_BZLMOD_DEBUG=1 common --lockfile_mode=off test --test_output=errors # Windows requires these for multi-python support: build --enable_runfiles common:bazel7.x --incompatible_python_disallow_native_rules +build --//:py=local +common --announce_rc diff --git a/tests/integration/local_toolchains/BUILD.bazel b/tests/integration/local_toolchains/BUILD.bazel index 6fbf548901..ae99e7cf2f 100644 --- a/tests/integration/local_toolchains/BUILD.bazel +++ b/tests/integration/local_toolchains/BUILD.bazel @@ -12,9 +12,71 @@ # See the License for the specific language governing permissions and # limitations under the License. +load("@bazel_skylib//rules:common_settings.bzl", "string_flag") load("@rules_python//python:py_test.bzl", "py_test") +load("@rules_python//python/cc:py_extension.bzl", "py_extension") py_test( - name = "test", - srcs = ["test.py"], + name = "local_runtime_test", + srcs = ["local_runtime_test.py"], + config_settings = { + "//:py": "local", + }, + # Make this test better respect pyenv + env_inherit = [ + "PYENV_VERSION", + "PATH", + ], +) + +py_test( + name = "repo_runtime_test", + srcs = ["repo_runtime_test.py"], + config_settings = { + "//:py": "repo", + }, +) + +config_setting( + name = "is_py_local", + flag_values = { + ":py": "local", + }, +) + +config_setting( + name = "is_py_repo", + flag_values = { + ":py": "repo", + }, +) + +# Set `--//:py=local` to use the local toolchain +# (This is set in this example's .bazelrc) +string_flag( + name = "py", + build_setting_default = "", +) + +# Build rules to generate a python extension. +py_extension( + name = "echo_ext", + testonly = True, + srcs = ["echo_ext.cc"], + copts = select({ + "@rules_cc//cc/compiler:msvc-cl": [], + "//conditions:default": ["-fvisibility=hidden"], + }), + imports = ["."], +) + +py_test( + name = "echo_test", + srcs = ["echo_test.py"], + # Make this test better respect pyenv/local Python DLLs on Windows + env_inherit = [ + "PYENV_VERSION", + "PATH", + ], + deps = [":echo_ext"], ) diff --git a/tests/integration/local_toolchains/MODULE.bazel b/tests/integration/local_toolchains/MODULE.bazel index d4ef12e952..fe90fa235a 100644 --- a/tests/integration/local_toolchains/MODULE.bazel +++ b/tests/integration/local_toolchains/MODULE.bazel @@ -14,27 +14,87 @@ module(name = "module_under_test") bazel_dep(name = "rules_python", version = "0.0.0") +bazel_dep(name = "bazel_skylib", version = "1.7.1") +bazel_dep(name = "platforms", version = "0.0.11") +bazel_dep(name = "rules_cc", version = "0.1.5") + local_path_override( module_name = "rules_python", path = "../../..", ) -local_runtime_repo = use_repo_rule("@rules_python//python/private:local_runtime_repo.bzl", "local_runtime_repo") +# Step 1: Define the python runtime +local_runtime_repo = use_repo_rule("@rules_python//python/local_toolchains:repos.bzl", "local_runtime_repo") -local_runtime_toolchains_repo = use_repo_rule("@rules_python//python/private:local_runtime_toolchains_repo.bzl", "local_runtime_toolchains_repo") +local_runtime_toolchains_repo = use_repo_rule("@rules_python//python/local_toolchains:repos.bzl", "local_runtime_toolchains_repo") +# This will use `python3` from the environment local_runtime_repo( name = "local_python3", interpreter_path = "python3", on_failure = "fail", ) +pbs_archive = use_repo_rule("//:pbs_archive.bzl", "pbs_archive") + +# "pbs" means "python-build-standalone" +# This maps the different platform runtimes to URLS and SHAs +pbs_archive( + name = "pbs_runtime", + sha256 = { + "linux": "0a01bad99fd4a165a11335c29eb43015dfdb8bd5ba8e305538ebb54f3bf3146d", + "mac os x": "7f5ec658219bdb1d1142c6abab89680322166c78350a017fb0af3c869dceee41", + "windows": "005cb2abf4cfa4aaa48fb10ce4e33fe4335ea4d1f55202dbe4e20c852e45e0f9", + }, + urls = { + "linux": "https://github.com/astral-sh/python-build-standalone/releases/download/20250918/cpython-3.13.7+20250918-x86_64-unknown-linux-gnu-install_only.tar.gz", + "mac os x": "https://github.com/astral-sh/python-build-standalone/releases/download/20250918/cpython-3.13.7+20250918-aarch64-apple-darwin-install_only.tar.gz", + "windows": "https://github.com/astral-sh/python-build-standalone/releases/download/20250918/cpython-3.13.7+20250918-x86_64-pc-windows-msvc-install_only.tar.gz", + }, +) + +# This will use Python from the `pbs_runtime` repository. +# The pbs_runtime is just an example; the repo just needs to be a valid Python +# installation. +local_runtime_repo( + name = "repo_python3", + interpreter_target = "@pbs_runtime//:python/bin/python", + on_failure = "fail", +) + +# Step 2: Create toolchains for the runtimes +# Below, we configure them to only activate if the `//:py` flag has particular +# values. local_runtime_toolchains_repo( name = "local_toolchains", - runtimes = ["local_python3"], + runtimes = [ + "local_python3", + "repo_python3", + ], + target_compatible_with = { + "local_python3": [ + "HOST_CONSTRAINTS", + ], + "repo_python3": [ + "HOST_CONSTRAINTS", + ], + }, + target_settings = { + "local_python3": [ + "@//:is_py_local", + ], + "repo_python3": [ + "@//:is_py_repo", + ], + }, ) +config = use_extension("@rules_python//python/extensions:config.bzl", "config") +config.add_transition_setting(setting = "//:py") + python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.toolchain(python_version = "3.13") use_repo(python, "rules_python_bzlmod_debug") +# Step 3: Register the toolchains register_toolchains("@local_toolchains//:all") diff --git a/tests/integration/local_toolchains/WORKSPACE b/tests/integration/local_toolchains/WORKSPACE index e69de29bb2..159f16deab 100644 --- a/tests/integration/local_toolchains/WORKSPACE +++ b/tests/integration/local_toolchains/WORKSPACE @@ -0,0 +1,76 @@ +workspace( + name = "module_under_test", +) + +local_repository( + name = "rules_python", + path = "../../..", +) + +load("@rules_python//python:repositories.bzl", "py_repositories") + +py_repositories( + transition_settings = [ + "@//:py", + ], +) + +load("@rules_python//python/local_toolchains:repos.bzl", "local_runtime_repo", "local_runtime_toolchains_repo") + +# Step 1: Define the python runtime. +local_runtime_repo( + name = "local_python3", + interpreter_path = "python3", + on_failure = "fail", + # or interpreter_path = "C:\\path\\to\\python.exe" +) + +load("//:pbs_archive.bzl", "pbs_archive") + +pbs_archive( + name = "pbs_runtime", + sha256 = { + "linux": "0a01bad99fd4a165a11335c29eb43015dfdb8bd5ba8e305538ebb54f3bf3146d", + "mac os x": "4fb42ffc8aad2a42ca7646715b8926bc6b2e0d31f13d2fec25943dc236a6fd60", + "windows": "005cb2abf4cfa4aaa48fb10ce4e33fe4335ea4d1f55202dbe4e20c852e45e0f9", + }, + urls = { + "linux": "https://github.com/astral-sh/python-build-standalone/releases/download/20250918/cpython-3.13.7+20250918-x86_64-unknown-linux-gnu-install_only.tar.gz", + "mac os x": "https://github.com/astral-sh/python-build-standalone/releases/download/20250918/cpython-3.13.7+20250918-x86_64-apple-darwin-install_only.tar.gz", + "windows server 2022": "https://github.com/astral-sh/python-build-standalone/releases/download/20250918/cpython-3.13.7+20250918-x86_64-pc-windows-msvc-install_only.tar.gz", + }, +) + +local_runtime_repo( + name = "repo_python3", + interpreter_target = "@pbs_runtime//:python/bin/python3", + on_failure = "fail", +) + +# Step 2: Create toolchains for the runtimes +local_runtime_toolchains_repo( + name = "local_toolchains", + runtimes = [ + "local_python3", + "repo_python3", + ], + target_compatible_with = { + "local_python3": [ + "HOST_CONSTRAINTS", + ], + "repo_python3": [ + "HOST_CONSTRAINTS", + ], + }, + target_settings = { + "local_python3": [ + "@//:is_py_local", + ], + "repo_python3": [ + "@//:is_py_repo", + ], + }, +) + +# Step 3: Register the toolchains +register_toolchains("@local_toolchains//:all") diff --git a/tests/integration/local_toolchains/echo_ext.cc b/tests/integration/local_toolchains/echo_ext.cc new file mode 100644 index 0000000000..367d1a13b3 --- /dev/null +++ b/tests/integration/local_toolchains/echo_ext.cc @@ -0,0 +1,21 @@ +#include + +static PyObject *echoArgs(PyObject *self, PyObject *args) { return args; } + +static PyMethodDef echo_methods[] = { + { "echo", echoArgs, METH_VARARGS, "Returns a tuple of the input args" }, + { NULL, NULL, 0, NULL }, +}; + +extern "C" { + +PyMODINIT_FUNC PyInit_echo_ext(void) { + static struct PyModuleDef echo_module_def = { + // Module definition + PyModuleDef_HEAD_INIT, "echo_ext", "'echo_ext' module", -1, echo_methods + }; + + return PyModule_Create(&echo_module_def); +} + +} // extern "C" diff --git a/tests/integration/local_toolchains/echo_test.py b/tests/integration/local_toolchains/echo_test.py new file mode 100644 index 0000000000..655205fe29 --- /dev/null +++ b/tests/integration/local_toolchains/echo_test.py @@ -0,0 +1,8 @@ +import unittest + +import echo_ext + + +class ExtensionTest(unittest.TestCase): + def test_echo_extension(self): + self.assertEqual(echo_ext.echo(42, "str"), (42, "str")) diff --git a/tests/integration/local_toolchains/local_runtime_test.py b/tests/integration/local_toolchains/local_runtime_test.py new file mode 100644 index 0000000000..220ceaead4 --- /dev/null +++ b/tests/integration/local_toolchains/local_runtime_test.py @@ -0,0 +1,75 @@ +import os.path +import shutil +import subprocess +import sys +import tempfile +import unittest + + +class LocalToolchainTest(unittest.TestCase): + maxDiff = None + + def test_python_from_path_used(self): + # NOTE: This is a bit brittle. It assumes the environment during the + # repo-phase and when the test is run are roughly the same. It's + # easy to violate this condition if there are shell-local changes + # that wouldn't be reflected when sub-shells are run later. + shell_path = shutil.which("python3") + + if shell_path is None: + self.fail( + "which(python3) returned None.\n" + f"PATH={os.environ.get('PATH')}" + ) + + # We call the interpreter and print its executable because of + # things like pyenv: they install a shim that re-execs python. + # The shim is e.g. /home/user/.pyenv/shims/python3, which then + # runs e.g. /usr/bin/python3 + with tempfile.TemporaryDirectory() as temp_dir: + file_path = os.path.join(temp_dir, "info.py") + with open(file_path, "w") as f: + f.write( + """ +import sys +print(sys.executable) +print(sys._base_executable) +""" + ) + f.flush() + output_lines = ( + subprocess.check_output( + [shell_path, file_path], + text=True, + ) + .strip() + .splitlines() + ) + shell_exe, shell_base_exe = output_lines + + # Call realpath() to help normalize away differences from symlinks. + # Use base executable to ignore a venv the test may be running within. + expected = os.path.realpath(shell_base_exe.strip().lower()) + actual = os.path.realpath(sys._base_executable.lower()) + + msg = f""" +details of executables: +test's runtime: +{sys.executable=} +{sys._base_executable=} +realpath exe : {os.path.realpath(sys.executable)} +realpath base_exe: {os.path.realpath(sys._base_executable)} + +from shell resolution: +which python3: {shell_path=}: +{shell_exe=} +{shell_base_exe=} +realpath exe : {os.path.realpath(shell_exe)} +realpath base_exe: {os.path.realpath(shell_base_exe)} +""".strip() + + # Normalize case: Windows may have case differences + self.assertEqual(expected.lower(), actual.lower(), msg=msg) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/integration/local_toolchains/pbs_archive.bzl b/tests/integration/local_toolchains/pbs_archive.bzl new file mode 100644 index 0000000000..7d817b1b40 --- /dev/null +++ b/tests/integration/local_toolchains/pbs_archive.bzl @@ -0,0 +1,57 @@ +"""A repository rule to download and extract a Python runtime archive.""" + +BUILD_BAZEL = """ +# Generated by pbs_archive.bzl + +package( + default_visibility = ["//visibility:public"], +) + +exports_files(glob(["**"])) +""" + +def _pbs_archive_impl(repository_ctx): + """Implementation of the python_build_standalone_archive rule.""" + os_name = repository_ctx.os.name.lower() + urls = repository_ctx.attr.urls + sha256s = repository_ctx.attr.sha256 + + # os.name for windows contain build and version; simplify it + if "windows" in os_name: + os_name = "windows" + + if os_name not in urls: + fail("Unsupported OS: '{}'. Available OSs are: {}".format( + os_name, + ", ".join(urls.keys()), + )) + + url = urls[os_name] + sha256 = sha256s.get(os_name, "") + + repository_ctx.download_and_extract( + url = url, + sha256 = sha256, + ) + + repository_ctx.file("BUILD.bazel", BUILD_BAZEL) + +pbs_archive = repository_rule( + implementation = _pbs_archive_impl, + attrs = { + "sha256": attr.string_dict( + doc = "A dictionary of SHA256 checksums for the archives, keyed by OS name.", + mandatory = True, + ), + "urls": attr.string_dict( + doc = "A dictionary of URLs to the runtime archives, keyed by OS name (e.g., 'linux', 'windows').", + mandatory = True, + ), + }, + doc = """ +Downloads and extracts a Python runtime archive for the current OS. + +This rule selects a URL from the `urls` attribute based on the host OS, +downloads the archive, and extracts it. +""", +) diff --git a/tests/integration/local_toolchains/repo_runtime_test.py b/tests/integration/local_toolchains/repo_runtime_test.py new file mode 100644 index 0000000000..0eb2ef4248 --- /dev/null +++ b/tests/integration/local_toolchains/repo_runtime_test.py @@ -0,0 +1,16 @@ +import os.path +import sys +import unittest + + +class RepoToolchainTest(unittest.TestCase): + maxDiff = None + + def test_python_from_repo_used(self): + actual = os.path.realpath(sys._base_executable.lower()) + # Normalize case: Windows may have case differences + self.assertIn("pbs_runtime", actual.lower()) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/integration/local_toolchains/test.py b/tests/integration/local_toolchains/test.py deleted file mode 100644 index d85a4c386b..0000000000 --- a/tests/integration/local_toolchains/test.py +++ /dev/null @@ -1,27 +0,0 @@ -import shutil -import subprocess -import sys -import unittest - - -class LocalToolchainTest(unittest.TestCase): - maxDiff = None - - def test_python_from_path_used(self): - shell_path = shutil.which("python3") - - # We call the interpreter and print its executable because of - # things like pyenv: they install a shim that re-execs python. - # The shim is e.g. /home/user/.pyenv/shims/python3, which then - # runs e.g. /usr/bin/python3 - expected = subprocess.check_output( - [shell_path, "-c", "import sys; print(sys.executable)"], - text=True, - ) - expected = expected.strip().lower() - # Normalize case: Windows may have case differences - self.assertEqual(expected.lower(), sys.executable.lower()) - - -if __name__ == "__main__": - unittest.main() diff --git a/tests/integration/pip_parse_isolated/.bazelrc b/tests/integration/pip_parse_isolated/.bazelrc new file mode 100644 index 0000000000..227ce5a4cd --- /dev/null +++ b/tests/integration/pip_parse_isolated/.bazelrc @@ -0,0 +1,8 @@ +# Bazel configuration flags + +build --enable_runfiles + +common --experimental_isolated_extension_usages + +# https://docs.bazel.build/versions/main/best-practices.html#using-the-bazelrc-file +try-import %workspace%/user.bazelrc diff --git a/tests/integration/pip_parse_isolated/BUILD.bazel b/tests/integration/pip_parse_isolated/BUILD.bazel new file mode 100644 index 0000000000..2f825107f1 --- /dev/null +++ b/tests/integration/pip_parse_isolated/BUILD.bazel @@ -0,0 +1,7 @@ +load("@rules_python//python:py_test.bzl", "py_test") + +py_test( + name = "test_isolated", + srcs = ["test_isolated.py"], + deps = ["@pypi//six"], +) diff --git a/tests/integration/pip_parse_isolated/MODULE.bazel b/tests/integration/pip_parse_isolated/MODULE.bazel new file mode 100644 index 0000000000..c25e9dc024 --- /dev/null +++ b/tests/integration/pip_parse_isolated/MODULE.bazel @@ -0,0 +1,19 @@ +module(name = "pip_parse_isolated") + +bazel_dep(name = "rules_python") +local_path_override( + module_name = "rules_python", + path = "../../..", +) + +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.toolchain(python_version = "3.13") + +# This test module verifies that dependencies can be used with `isolate = True`. +pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip", isolate = True) +pip.parse( + hub_name = "pypi_isolated", + python_version = "3.13", + requirements_lock = "//:requirements_lock.txt", +) +use_repo(pip, pypi = "pypi_isolated") diff --git a/tests/integration/pip_parse_isolated/WORKSPACE b/tests/integration/pip_parse_isolated/WORKSPACE new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/integration/pip_parse_isolated/requirements_lock.txt b/tests/integration/pip_parse_isolated/requirements_lock.txt new file mode 100644 index 0000000000..b1445a37ae --- /dev/null +++ b/tests/integration/pip_parse_isolated/requirements_lock.txt @@ -0,0 +1,2 @@ +six==1.17.0 \ + --hash=sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274 diff --git a/tests/integration/pip_parse_isolated/test_isolated.py b/tests/integration/pip_parse_isolated/test_isolated.py new file mode 100644 index 0000000000..5620801955 --- /dev/null +++ b/tests/integration/pip_parse_isolated/test_isolated.py @@ -0,0 +1,17 @@ +""" +Verify that a dependency added using an isolated extension can be imported. +See MODULE.bazel. +""" + +import unittest + +import six + + +class TestIsolated(unittest.TestCase): + def test_import(self): + self.assertTrue(hasattr(six, "PY3")) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/integration/runner.py b/tests/integration/runner.py index 9414a865c0..c187623b3c 100644 --- a/tests/integration/runner.py +++ b/tests/integration/runner.py @@ -23,12 +23,15 @@ _logger = logging.getLogger(__name__) + class ExecuteError(Exception): def __init__(self, result): self.result = result + def __str__(self): return self.result.describe() + class ExecuteResult: def __init__( self, @@ -77,13 +80,16 @@ def setUp(self): # Put the global tmp not under the test tmp to better match how a real # execution has entirely different directories for these. self.tmp_dir = outer_test_tmpdir / "bit_tmp" + self.test_tmp_dir.mkdir(parents=True, exist_ok=True) + self.tmp_dir.mkdir(parents=True, exist_ok=True) self.bazel_env = { "PATH": os.environ["PATH"], "TEST_TMPDIR": str(self.test_tmp_dir), "TMP": str(self.tmp_dir), + "TEMP": str(self.tmp_dir), # For some reason, this is necessary for Bazel 6.4 to work. # If not present, it can't find some bash helpers in @bazel_tools - "RUNFILES_DIR": os.environ["TEST_SRCDIR"] + "RUNFILES_DIR": os.environ["TEST_SRCDIR"], } def run_bazel(self, *args: str, check: bool = True) -> ExecuteResult: diff --git a/tests/integration/runtime_manifests/.bazelrc b/tests/integration/runtime_manifests/.bazelrc new file mode 100644 index 0000000000..d4d45a5ea7 --- /dev/null +++ b/tests/integration/runtime_manifests/.bazelrc @@ -0,0 +1,4 @@ +# Copy of fast-tests config +common:fast-tests --build_tests_only=true +common:fast-tests --build_tag_filters=-large,-enormous,-integration-test +common:fast-tests --test_tag_filters=-large,-enormous,-integration-test diff --git a/tests/integration/runtime_manifests/BUILD.bazel b/tests/integration/runtime_manifests/BUILD.bazel new file mode 100644 index 0000000000..4746fd419b --- /dev/null +++ b/tests/integration/runtime_manifests/BUILD.bazel @@ -0,0 +1,7 @@ +load("@rules_python//python:py_test.bzl", "py_test") + +py_test( + name = "basic_test", + srcs = ["basic_test.py"], + python_version = "3.11", +) diff --git a/tests/integration/runtime_manifests/MODULE.bazel b/tests/integration/runtime_manifests/MODULE.bazel new file mode 100644 index 0000000000..6891b07818 --- /dev/null +++ b/tests/integration/runtime_manifests/MODULE.bazel @@ -0,0 +1,19 @@ +module(name = "runtime_manifests") + +bazel_dep(name = "rules_python", version = "0.0.0") +local_path_override( + module_name = "rules_python", + path = "../../..", +) + +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.override( + add_runtime_manifest_urls = [ + "https://github.com/astral-sh/python-build-standalone/releases/download/20260414/SHA256SUMS", + ], + register_all_versions = True, + runtime_manifest_sha = "ce18fdfd47c66830a40ea9b9e314a14b1636bbfd684501bc5ca1fc6d55a7933f", +) +python.toolchain( + python_version = "3.11.15", +) diff --git a/tests/integration/runtime_manifests/WORKSPACE b/tests/integration/runtime_manifests/WORKSPACE new file mode 100644 index 0000000000..8277ec8090 --- /dev/null +++ b/tests/integration/runtime_manifests/WORKSPACE @@ -0,0 +1 @@ +# Workspace boundary file required by rules_bazel_integration_test diff --git a/tests/integration/runtime_manifests/basic_test.py b/tests/integration/runtime_manifests/basic_test.py new file mode 100644 index 0000000000..35f0e93b6e --- /dev/null +++ b/tests/integration/runtime_manifests/basic_test.py @@ -0,0 +1,27 @@ +import datetime +import platform +import sys +import unittest + + +class BasicTest(unittest.TestCase): + def test_basic(self): + print("Hello World from Python {}!".format(sys.version)) + print("Interpreter executable path: {}".format(sys.executable)) + + # Verify that the hermetic interpreter inside Bazel's output/sandbox tree is used + self.assertIn(".cache/bazel", sys.executable) + + # Verify that the exact custom version (3.11.15) parsed from the manifest is used + self.assertEqual(sys.version_info[:3], (3, 11, 15)) + + # Verify that the exact build version (20260414) parsed from the manifest is used + buildno, builddate = platform.python_build() + date_str = " ".join(builddate.split()[:3]) + dt = datetime.datetime.strptime(date_str, "%b %d %Y") + formatted_date = dt.strftime("%Y%m%d") + self.assertEqual(formatted_date, "20260414") + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/integration/toolchain_target_settings/.bazelrc b/tests/integration/toolchain_target_settings/.bazelrc new file mode 100644 index 0000000000..e1df951d06 --- /dev/null +++ b/tests/integration/toolchain_target_settings/.bazelrc @@ -0,0 +1,3 @@ +common --lockfile_mode=off +test --test_output=errors +build --enable_runfiles diff --git a/tests/integration/toolchain_target_settings/BUILD.bazel b/tests/integration/toolchain_target_settings/BUILD.bazel new file mode 100644 index 0000000000..0e28e461b3 --- /dev/null +++ b/tests/integration/toolchain_target_settings/BUILD.bazel @@ -0,0 +1,56 @@ +load("@bazel_skylib//rules:common_settings.bzl", "string_flag") +load("@rules_python//python:py_test.bzl", "py_test") + +# A flag to select which "family" of toolchains to use. +string_flag( + name = "family", + build_setting_default = "prebuilt", + values = [ + "prebuilt", + "custom", + ], +) + +# Matches when the "prebuilt" family is selected. +# This is referenced in MODULE.bazel's python.override(add_target_settings=...). +config_setting( + name = "is_prebuilt", + flag_values = { + ":family": "prebuilt", + }, +) + +# Matches when the "custom" family is selected. +# No toolchains use this setting, so selecting it should produce an error. +config_setting( + name = "is_custom", + flag_values = { + ":family": "custom", + }, +) + +# This target selects the "prebuilt" family via config_settings transition. +# Since python.override(add_target_settings = ["@@//:is_prebuilt"]) gates +# the default toolchains, this should succeed: the flag matches, the config_setting +# is satisfied, and the default 3.13 toolchain resolves. +py_test( + name = "prebuilt_test", + srcs = ["main.py"], + config_settings = { + "//:family": "prebuilt", + }, + main = "main.py", +) + +# This target selects the "custom" family via config_settings transition. +# No toolchains have target_settings = [":is_custom"], so toolchain resolution +# should fail -- the default toolchains are gated behind ":is_prebuilt" and +# won't match. +py_test( + name = "custom_no_toolchain_test", + srcs = ["main.py"], + config_settings = { + "//:family": "custom", + }, + main = "main.py", +) diff --git a/tests/integration/toolchain_target_settings/MODULE.bazel b/tests/integration/toolchain_target_settings/MODULE.bazel new file mode 100644 index 0000000000..ec3a56749e --- /dev/null +++ b/tests/integration/toolchain_target_settings/MODULE.bazel @@ -0,0 +1,24 @@ +module(name = "module_under_test") + +bazel_dep(name = "rules_python", version = "0.0.0") +bazel_dep(name = "bazel_skylib", version = "1.7.1") + +local_path_override( + module_name = "rules_python", + path = "../../..", +) + +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.toolchain(python_version = "3.13") + +# Gate ALL default-registered toolchains behind the "prebuilt" config setting. +# This prevents them from being a silent fallback when a different toolchain +# family is requested. +python.override( + add_target_settings = ["@@//:is_prebuilt"], +) + +# Register //:family as a transition setting so py_binary/py_test +# config_settings can set it. +config = use_extension("@rules_python//python/extensions:config.bzl", "config") +config.add_transition_setting(setting = "//:family") diff --git a/tests/integration/toolchain_target_settings/REPO.bazel b/tests/integration/toolchain_target_settings/REPO.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/integration/toolchain_target_settings/WORKSPACE b/tests/integration/toolchain_target_settings/WORKSPACE new file mode 100644 index 0000000000..d7be0c96c7 --- /dev/null +++ b/tests/integration/toolchain_target_settings/WORKSPACE @@ -0,0 +1 @@ +# Intentionally blank; bzlmod is used. diff --git a/tests/integration/toolchain_target_settings/WORKSPACE.bzlmod b/tests/integration/toolchain_target_settings/WORKSPACE.bzlmod new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/integration/toolchain_target_settings/main.py b/tests/integration/toolchain_target_settings/main.py new file mode 100644 index 0000000000..52ec605a55 --- /dev/null +++ b/tests/integration/toolchain_target_settings/main.py @@ -0,0 +1,3 @@ +import sys + +print(f"Python {sys.version}") diff --git a/tests/integration/toolchain_target_settings_test.py b/tests/integration/toolchain_target_settings_test.py new file mode 100644 index 0000000000..a0c9d42fc3 --- /dev/null +++ b/tests/integration/toolchain_target_settings_test.py @@ -0,0 +1,39 @@ +"""Integration test for python.override(add_target_settings=...). + +Verifies that when all default toolchains are gated behind a config_setting, +requesting a different (unregistered) toolchain family produces a toolchain +resolution error instead of silently falling back to the default toolchains. +""" + +import unittest + +from tests.integration import runner + + +class AddTargetSettingsTest(runner.TestCase): + def test_prebuilt_family_resolves(self): + """Building with the 'prebuilt' family should succeed. + + The default toolchains have target_settings = [":is_prebuilt"], + and the transition sets //:family=prebuilt, so the config_setting + matches and toolchain resolution finds the default 3.13 toolchain. + """ + self.run_bazel("test", "//:prebuilt_test") + + def test_custom_family_without_toolchain_fails(self): + """Building with the 'custom' family should fail. + + No toolchains have target_settings = [":is_custom"], and the default + toolchains are gated behind ":is_prebuilt" (via add_target_settings), + so toolchain resolution should fail with no matching toolchain. + """ + result = self.run_bazel("build", "//:custom_no_toolchain_test", check=False) + self.assertNotEqual(result.exit_code, 0, "Expected build to fail") + self.assert_result_matches( + result, + r"No matching toolchains found for types", + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/integration/unified_pypi/.bazelrc b/tests/integration/unified_pypi/.bazelrc new file mode 100644 index 0000000000..b3a24e8605 --- /dev/null +++ b/tests/integration/unified_pypi/.bazelrc @@ -0,0 +1 @@ +common --experimental_enable_bzlmod diff --git a/tests/integration/unified_pypi/BUILD.bazel b/tests/integration/unified_pypi/BUILD.bazel new file mode 100644 index 0000000000..c0b9905fa6 --- /dev/null +++ b/tests/integration/unified_pypi/BUILD.bazel @@ -0,0 +1,61 @@ +load("@rules_python//python:py_binary.bzl", "py_binary") +load("@rules_python//python:py_test.bzl", "py_test") + +package(default_visibility = ["//visibility:public"]) + +py_test( + name = "test_default", + srcs = ["test_default.py"], + deps = ["@pypi//colorama"], +) + +py_test( + name = "test_cli", + srcs = ["test_cli.py"], + deps = ["@pypi//colorama"], +) + +py_test( + name = "test_a", + srcs = ["test_a.py"], + config_settings = { + "@rules_python//python/config_settings:venv": "pypi_a", + }, + deps = [ + "@pypi//colorama", + "@pypi//colorama:my_colorama", + ], +) + +# Sibling extra alias failure target (my_colorama is missing in pypi_b): +py_binary( + name = "bin_extra_b", + srcs = ["bin_extra_b.py"], + config_settings = { + "@rules_python//python/config_settings:venv": "pypi_b", + }, + deps = ["@pypi//colorama:my_colorama"], +) + +# Disjoint package failure target (six is missing in pypi_a): +py_binary( + name = "bin_six_a", + srcs = ["bin_six_a.py"], + config_settings = { + "@rules_python//python/config_settings:venv": "pypi_a", + }, + deps = ["@pypi//six"], +) + +py_binary( + name = "bin_declared_only", + srcs = ["bin_declared_only.py"], + deps = ["@pypi//declared_only_pkg"], +) + +py_binary( + name = "bin_declared_only_alias", + srcs = ["bin_declared_only.py"], + main = "bin_declared_only.py", + deps = ["@pypi//declared_only_pkg:declared-only-alias"], +) diff --git a/tests/integration/unified_pypi/MODULE.bazel b/tests/integration/unified_pypi/MODULE.bazel new file mode 100644 index 0000000000..6a4b87e015 --- /dev/null +++ b/tests/integration/unified_pypi/MODULE.bazel @@ -0,0 +1,52 @@ +module(name = "unified_pypi") + +bazel_dep(name = "rules_python", version = "0.0.0") +local_path_override( + module_name = "rules_python", + path = "../../..", +) + +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.toolchain(python_version = "3.11") + +pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") +pip.whl_mods( + additive_build_content = """\ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "my_colorama", + deps = [":pkg"], +) +""", + hub_name = "whl_mods_hub", + whl_name = "colorama", +) +use_repo(pip, "whl_mods_hub") + +# pypi_a has colorama and an extra alias +pip.parse( + extra_hub_aliases = {"colorama": ["my_colorama"]}, + hub_name = "pypi_a", + python_version = "3.11", + requirements_lock = "//:requirements_a.txt", + whl_modifications = { + "@whl_mods_hub//:colorama.json": "colorama", + }, +) +use_repo(pip, "pypi_a") + +# pypi_b has colorama and six, and acts as designated fallback +pip.parse( + hub_name = "pypi_b", + python_version = "3.11", + requirements_lock = "//:requirements_b.txt", +) +use_repo(pip, "pypi_b") + +pip.default(default_hub = "pypi_b") +pip.dep( + name = "declared-only-pkg", + extra_targets = ["declared-only-alias"], +) +use_repo(pip, "pypi") diff --git a/tests/integration/unified_pypi/WORKSPACE b/tests/integration/unified_pypi/WORKSPACE new file mode 100644 index 0000000000..0a08afe832 --- /dev/null +++ b/tests/integration/unified_pypi/WORKSPACE @@ -0,0 +1 @@ +# Minimal WORKSPACE file diff --git a/tests/integration/unified_pypi/WORKSPACE.bzlmod b/tests/integration/unified_pypi/WORKSPACE.bzlmod new file mode 100644 index 0000000000..7bd1c969b9 --- /dev/null +++ b/tests/integration/unified_pypi/WORKSPACE.bzlmod @@ -0,0 +1 @@ +# Minimal WORKSPACE.bzlmod diff --git a/tests/integration/unified_pypi/bin_declared_only.py b/tests/integration/unified_pypi/bin_declared_only.py new file mode 100644 index 0000000000..0b03607e77 --- /dev/null +++ b/tests/integration/unified_pypi/bin_declared_only.py @@ -0,0 +1,2 @@ +# Dummy file for integration test +print("declared_only") diff --git a/tests/integration/unified_pypi/bin_extra_b.py b/tests/integration/unified_pypi/bin_extra_b.py new file mode 100644 index 0000000000..f900d16fd2 --- /dev/null +++ b/tests/integration/unified_pypi/bin_extra_b.py @@ -0,0 +1 @@ +print("Should not be executed") diff --git a/tests/integration/unified_pypi/bin_six_a.py b/tests/integration/unified_pypi/bin_six_a.py new file mode 100644 index 0000000000..f900d16fd2 --- /dev/null +++ b/tests/integration/unified_pypi/bin_six_a.py @@ -0,0 +1 @@ +print("Should not be executed") diff --git a/tests/integration/unified_pypi/requirements_a.txt b/tests/integration/unified_pypi/requirements_a.txt new file mode 100644 index 0000000000..788f12f818 --- /dev/null +++ b/tests/integration/unified_pypi/requirements_a.txt @@ -0,0 +1,3 @@ +colorama==0.4.6 \ + --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \ + --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6 diff --git a/tests/integration/unified_pypi/requirements_b.txt b/tests/integration/unified_pypi/requirements_b.txt new file mode 100644 index 0000000000..c69f3631b2 --- /dev/null +++ b/tests/integration/unified_pypi/requirements_b.txt @@ -0,0 +1,6 @@ +colorama==0.4.5 \ + --hash=sha256:854bf444933e37f5824ae7bfc1e98d5bce2ebe4160d46b5edf346a89358e99da \ + --hash=sha256:e6c6b4334fc50988a639d9b98ae42f5c90ec94cb1495b4fe76c5f72cf7f79435 +six==1.17.0 \ + --hash=sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274 \ + --hash=sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81 diff --git a/tests/integration/unified_pypi/test_a.py b/tests/integration/unified_pypi/test_a.py new file mode 100644 index 0000000000..a9127ba0c7 --- /dev/null +++ b/tests/integration/unified_pypi/test_a.py @@ -0,0 +1,3 @@ +import colorama + +assert colorama.__version__ == "0.4.6" diff --git a/tests/integration/unified_pypi/test_cli.py b/tests/integration/unified_pypi/test_cli.py new file mode 100644 index 0000000000..a9127ba0c7 --- /dev/null +++ b/tests/integration/unified_pypi/test_cli.py @@ -0,0 +1,3 @@ +import colorama + +assert colorama.__version__ == "0.4.6" diff --git a/tests/integration/unified_pypi/test_default.py b/tests/integration/unified_pypi/test_default.py new file mode 100644 index 0000000000..559df59961 --- /dev/null +++ b/tests/integration/unified_pypi/test_default.py @@ -0,0 +1,3 @@ +import colorama + +assert colorama.__version__ == "0.4.5" diff --git a/tests/integration/unified_pypi_test.py b/tests/integration/unified_pypi_test.py new file mode 100644 index 0000000000..a19a9fcfbe --- /dev/null +++ b/tests/integration/unified_pypi_test.py @@ -0,0 +1,100 @@ +"""Integration test for Unified PyPI Hub dynamic dependency resolution.""" + +import contextlib +import unittest + +from tests.integration import runner + + +class UnifiedPypiTest(runner.TestCase): + def test_default_fallback_hub(self): + self.run_bazel("test", "//:test_default") + + def test_transitioned_hub(self): + self.run_bazel("test", "//:test_a") + + def test_cli_override(self): + self.run_bazel( + "run", + "--@rules_python//python/config_settings:venv=pypi_a", + "//:test_cli", + ) + + def test_disjoint_package_cquery_succeeds_but_build_fails(self): + self.run_bazel("cquery", "//:bin_six_a") + result = self.run_bazel("build", "//:bin_six_a", check=False) + self.assertNotEqual( + result.exit_code, + 0, + "Expected build to fail during execution phase", + ) + self.assert_result_matches( + result, + 'ERROR: PyPI package "six" is not available when building under PyPI hub "pypi_a"\\. Try adding it to the requirements of this hub', + ) + + def test_sibling_extra_alias_cquery_succeeds_but_build_fails(self): + self.run_bazel("cquery", "//:bin_extra_b") + result = self.run_bazel("build", "//:bin_extra_b", check=False) + self.assertNotEqual( + result.exit_code, + 0, + "Expected build to fail during execution phase", + ) + self.assert_result_matches( + result, + 'ERROR: PyPI package "colorama:my_colorama" is not available when building under PyPI hub "pypi_b"\\. Try adding it to the requirements of this hub', + ) + + @contextlib.contextmanager + def _temp_modify_file(self, path, new_content): + original_content = path.read_text() + path.write_text(new_content) + try: + yield + finally: + path.write_text(original_content) + + def test_invalid_default_hub_fails_evaluation(self): + module_bazel = self.repo_root / "MODULE.bazel" + invalid_content = module_bazel.read_text().replace( + 'pip.default(default_hub = "pypi_b")', + 'pip.default(default_hub = "invalid_hub")', + ) + with self._temp_modify_file(module_bazel, invalid_content): + # Run bazel cquery and expect it to fail during loading/extension phase + result = self.run_bazel("cquery", "//:test_default", check=False) + self.assertNotEqual( + result.exit_code, + 0, + "Expected extension evaluation to fail due to invalid default_hub", + ) + self.assert_result_matches( + result, + "default_hub 'invalid_hub' is not a defined PyPI hub", + ) + + def test_unimplemented_declared_dep_fails_build(self): + # Even though cquery succeeds: + self.run_bazel("cquery", "//:bin_declared_only") + + # Build must fail because the package is not implemented by any concrete hub + result = self.run_bazel("build", "//:bin_declared_only", check=False) + self.assertNotEqual(result.exit_code, 0) + self.assert_result_matches( + result, + 'ERROR: PyPI package "declared_only_pkg" is not available when building under PyPI hub "pypi_b"\\. Try adding it to the requirements of this hub', + ) + + def test_unimplemented_declared_dep_alias_fails_build(self): + # Build must fail for alias too + result = self.run_bazel("build", "//:bin_declared_only_alias", check=False) + self.assertNotEqual(result.exit_code, 0) + self.assert_result_matches( + result, + 'ERROR: PyPI package "declared_only_pkg:declared-only-alias" is not available when building under PyPI hub "pypi_b"\\. Try adding it to the requirements of this hub', + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/integration/uv_lock/.bazelrc b/tests/integration/uv_lock/.bazelrc new file mode 100644 index 0000000000..511f8a2413 --- /dev/null +++ b/tests/integration/uv_lock/.bazelrc @@ -0,0 +1,5 @@ +build --enable_runfiles +common --experimental_isolated_extension_usages +common --action_env=UV_EXTRA_INDEX_URL + +try-import %workspace%/user.bazelrc diff --git a/tests/integration/uv_lock/.bazelversion b/tests/integration/uv_lock/.bazelversion new file mode 100644 index 0000000000..47da986f86 --- /dev/null +++ b/tests/integration/uv_lock/.bazelversion @@ -0,0 +1 @@ +9.1.0 diff --git a/tests/integration/uv_lock/BUILD.bazel b/tests/integration/uv_lock/BUILD.bazel new file mode 100644 index 0000000000..da6482ba08 --- /dev/null +++ b/tests/integration/uv_lock/BUILD.bazel @@ -0,0 +1,26 @@ +load("@bazel_skylib//rules:diff_test.bzl", "diff_test") +load("@rules_python//python/uv:lock.bzl", "lock") +load(":uv_runner.bzl", "uv_runner") + +lock( + name = "requirements", + srcs = ["requirements.in"], + out = "requirements.txt", + tags = ["no-remote-exec"], +) + +uv_runner( + name = "uv", + is_windows = select({ + "@platforms//os:windows": True, + "//conditions:default": False, + }), + tags = ["manual"], +) + +diff_test( + name = "requirements_diff_test", + timeout = "short", + file1 = ":requirements", + file2 = ":requirements.txt", +) diff --git a/tests/integration/uv_lock/MODULE.bazel b/tests/integration/uv_lock/MODULE.bazel new file mode 100644 index 0000000000..0e0978ed36 --- /dev/null +++ b/tests/integration/uv_lock/MODULE.bazel @@ -0,0 +1,18 @@ +module(name = "uv_lock") + +bazel_dep(name = "bazel_skylib", version = "1.7.1") +bazel_dep(name = "platforms", version = "0.0.10") +bazel_dep(name = "rules_python") +local_path_override( + module_name = "rules_python", + path = "../../..", +) + +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.toolchain(python_version = "3.13") + +uv = use_extension("@rules_python//python/uv:uv.bzl", "uv") +uv.configure(version = "0.11.2") +use_repo(uv, "uv") + +register_toolchains("@uv//:all") diff --git a/tests/integration/uv_lock/README.md b/tests/integration/uv_lock/README.md new file mode 100644 index 0000000000..cce142852c --- /dev/null +++ b/tests/integration/uv_lock/README.md @@ -0,0 +1,64 @@ +# uv_lock integration test workspace + +This directory is a self-contained Bazel workspace used by the +`//tests/integration:uv_lock_test` integration test. + +It demonstrates how to use the `lock()` macro from `@rules_python//python/uv` +to pin requirements with `uv pip compile`. + +## Targets + +| Target | Description | +|--------|-------------| +| `//:requirements` | Build action that produces the locked requirements file | +| `//:requirements.update` | Update the in-source `requirements.txt` via `bazel run` | +| `//:requirements.run` | Run `uv pip compile` with extra command-line args | +| `//:requirements_diff_test` | Diff test comparing the lock output to the in-source file | +| `//:uv` | The `uv` binary from the registered toolchain | + +## Workflow for debugging + +If you want to debug and play around, you can start the server and then run the uv lock command +manually. + +### Start the local PyPI server + +In a separate terminal, start the pypiserver that serves the `my-local-pkg` test wheel: + +```shell +bazel run //tests/integration:uv_lock_pypi_server [-- --no-auth] +``` + +The server prints the URL to use (with and without authentication) and the +SHA256 of the wheel. Pass `--no-auth` to allow anonymous access. + +### Lock the requirements + +With the server running, lock the requirements from this directory: + +```shell +cd tests/integration/uv_lock +bazel run //:requirements.update \ + --action_env=UV_EXTRA_INDEX_URL="" \ + --action_env=UV_CREDENTIALS_DIR= +``` + +The `` and `` values are printed by the pypi-server. + +### Verify the lock output matches the in-source file + +```shell +bazel test //:requirements_diff_test +``` + +## bazel-in-bazel Testing + +When iterating on changes to `lock.bzl`, the integration test can be run +directly from rules_python: + +```shell +bazel test //tests/integration:uv_lock_test_bazel_self \ + --config=fast-tests \ + --test_output=streamed \ + --test_filter= +``` diff --git a/tests/integration/uv_lock/WORKSPACE b/tests/integration/uv_lock/WORKSPACE new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/integration/uv_lock/requirements.in b/tests/integration/uv_lock/requirements.in new file mode 100644 index 0000000000..fba55a7329 --- /dev/null +++ b/tests/integration/uv_lock/requirements.in @@ -0,0 +1 @@ +my-local-pkg==1.0.0 diff --git a/tests/integration/uv_lock/requirements.txt b/tests/integration/uv_lock/requirements.txt new file mode 100644 index 0000000000..52f5c757f5 --- /dev/null +++ b/tests/integration/uv_lock/requirements.txt @@ -0,0 +1,5 @@ +# This file was autogenerated by uv via the following command: +# bazel run //:requirements.update +my-local-pkg==1.0.0 \ + --hash=sha256:be24d5183a182e8da4465ae1b7e60324864d1a72866ec9aefa6aaf80a4529eb1 + # via -r requirements.in diff --git a/tests/integration/uv_lock/uv_runner.bzl b/tests/integration/uv_lock/uv_runner.bzl new file mode 100644 index 0000000000..3faa3c6fc0 --- /dev/null +++ b/tests/integration/uv_lock/uv_runner.bzl @@ -0,0 +1,30 @@ +"""A rule exposing the uv binary from the registered toolchain as an executable target. + +This allows running ``bazel run //:uv -- `` from the test workspace. +""" + +def _uv_runner_impl(ctx): + toolchain_info = ctx.toolchains["@rules_python//python/uv:uv_toolchain_type"] + original_uv_executable = toolchain_info.uv_toolchain_info.uv[DefaultInfo].files_to_run.executable + + ext = "" + if ctx.attr.is_windows: + ext = ".exe" + + uv_exe = ctx.actions.declare_file("uv" + ext) + ctx.actions.symlink(output = uv_exe, target_file = original_uv_executable) + + return DefaultInfo( + files = depset([uv_exe]), + executable = uv_exe, + runfiles = toolchain_info.default_info.default_runfiles, + ) + +uv_runner = rule( + implementation = _uv_runner_impl, + executable = True, + attrs = { + "is_windows": attr.bool(mandatory = True), + }, + toolchains = ["@rules_python//python/uv:uv_toolchain_type"], +) diff --git a/tests/integration/uv_lock_pypi_server.py b/tests/integration/uv_lock_pypi_server.py new file mode 100644 index 0000000000..0d940e7569 --- /dev/null +++ b/tests/integration/uv_lock_pypi_server.py @@ -0,0 +1,148 @@ +import argparse +import hashlib +import io +import os +import sys +import uuid +import zipfile +from wsgiref.simple_server import make_server + +from pypiserver import app_from_config, setup_routes_from_config +from pypiserver.config import Config + + +def _create_wheel_bytes(name, version): + pkg_name_normalized = name.replace("-", "_") + wheel_name = "{}-{}-py3-none-any.whl".format(pkg_name_normalized, version) + dist_info = "{}-{}.dist-info".format(pkg_name_normalized, version) + + metadata = ( + "Metadata-Version: 2.1\n" + "Name: {name}\n" + "Version: {version}\n" + "Summary: A test package\n" + ).format(name=pkg_name_normalized, version=version) + + wheel_file = ( + "Wheel-Version: 1.0\n" + "Generator: test\n" + "Root-Is-Purelib: true\n" + "Tag: py3-none-any\n" + ) + + record_entries = [ + "{}/__init__.py,".format(pkg_name_normalized), + "{}/METADATA,".format(dist_info), + "{}/WHEEL,".format(dist_info), + "{}/RECORD,".format(dist_info), + ] + + buf = io.BytesIO() + with zipfile.ZipFile(buf, "w", zipfile.ZIP_DEFLATED) as zf: + zf.writestr("{}/__init__.py".format(pkg_name_normalized), "# empty\n") + zf.writestr("{}/METADATA".format(dist_info), metadata) + zf.writestr("{}/WHEEL".format(dist_info), wheel_file) + zf.writestr("{}/RECORD".format(dist_info), "\n".join(record_entries)) + + wheel_data = buf.getvalue() + sha256 = hashlib.sha256(wheel_data).hexdigest() + return wheel_data, sha256, wheel_name + + +def main(): + parser = argparse.ArgumentParser( + description="Standalone pypiserver for uv_lock integration tests" + ) + parser.add_argument( + "--packages-dir", + type=str, + default=None, + help="Directory for the test wheels (default: $TEST_TMPDIR/pypi-server-packages)", + ) + parser.add_argument( + "--no-auth", + action="store_true", + default=False, + help="Disable authentication (allows anonymous access)", + ) + parser.add_argument( + "--port", + type=int, + default=0, + help="Port to listen on (0 = find free port)", + ) + parser.add_argument( + "--host", + default="localhost", + help="Host to bind to", + ) + args = parser.parse_args() + + if args.packages_dir is None: + sandbox_root = ( + os.environ.get("TEST_TMPDIR") or os.environ.get("TMPDIR") or "/tmp" + ) + args.packages_dir = os.path.join(sandbox_root, "pypi-server-packages") + packages_dir = args.packages_dir + os.makedirs(packages_dir, exist_ok=True) + + wheel_data, sha256, wheel_name = _create_wheel_bytes("my-local-pkg", "1.0.0") + wheel_path = os.path.join(packages_dir, wheel_name) + with open(wheel_path, "wb") as f: + f.write(wheel_data) + + print("Wheel: {}".format(wheel_path), flush=True) + print("SHA256: {}".format(sha256), flush=True) + + password = uuid.uuid4().hex + username = "testuser" + + if args.no_auth: + authenticate = [] + else: + authenticate = ["download", "list", "update"] + + config = Config.default_with_overrides( + roots=[packages_dir], + port=args.port, + host=args.host, + authenticate=authenticate, + password_file=None, + auther=lambda u, p: u == username and p == password, + disable_fallback=True, + fallback_url="", + server_method="wsgiref", + verbosity=0, + log_stream=None, + ) + app = app_from_config(config) + app = setup_routes_from_config(app, config) + + server = make_server(args.host, args.port, app) + port = server.server_address[1] + + base_url = "http://{}:{}".format(args.host, port) + auth_url = "http://{}:{}@{}:{}".format(username, password, args.host, port) + + print("\npypiserver listening on:\n", flush=True) + print(" URL (no auth): {}".format(base_url), flush=True) + print(" URL (auth): {}".format(auth_url), flush=True) + print("\nRequired dependency in requirements.in:", flush=True) + print(" my-local-pkg==1.0.0", flush=True) + print("\nTo use from the uv_lock test workspace, run:", flush=True) + print(" cd tests/integration/uv_lock", flush=True) + print(" bazel run //:requirements.update \\", flush=True) + print(' --action_env=UV_EXTRA_INDEX_URL="{}" \\'.format(auth_url), flush=True) + print(" --action_env=UV_CREDENTIALS_DIR=", flush=True) + print("\nPress Ctrl+C to stop the server.\n", flush=True) + + try: + server.serve_forever() + except KeyboardInterrupt: + print("\nShutting down...", flush=True) + server.shutdown() + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tests/integration/uv_lock_test.py b/tests/integration/uv_lock_test.py new file mode 100644 index 0000000000..8efd3cb84f --- /dev/null +++ b/tests/integration/uv_lock_test.py @@ -0,0 +1,288 @@ +import base64 +import hashlib +import os +import re +import threading +import time +import unittest +import uuid +from pathlib import Path +from urllib.error import URLError +from urllib.request import Request, urlopen +from wsgiref.simple_server import make_server + +from pypiserver import app_from_config, setup_routes_from_config +from pypiserver.config import Config + +from tests.integration import runner +from tests.integration.uv_lock_pypi_server import _create_wheel_bytes + + +def _make_server_on_free_port(app): + server = make_server("localhost", 0, app) + port = server.server_address[1] + return server, port + + +class UvLockIntegrationTest(runner.TestCase): + def setUp(self): + super().setUp() + + self.username = "testuser" + self.password = uuid.uuid4().hex + + self.dir = Path(os.environ["TEST_TMPDIR"]) + self.docroot = self.dir / "simple" + self.docroot.mkdir(exist_ok=True) + + self.wheel_data, self.wheel_sha256, wheel_name = _create_wheel_bytes( + "my-local-pkg", + "1.0.0", + ) + + packages_dir = self.docroot / "packages" + packages_dir.mkdir(exist_ok=True) + self.wheel_path = packages_dir / wheel_name + self.wheel_path.write_bytes(self.wheel_data) + + config = Config.default_with_overrides( + roots=[packages_dir], + port=0, + host="localhost", + authenticate=["download", "list", "update"], + password_file=None, + auther=lambda u, p: u == self.username and p == self.password, + disable_fallback=True, + fallback_url="", + server_method="wsgiref", + verbosity=0, + log_stream=None, + ) + app = app_from_config(config) + app = setup_routes_from_config(app, config) + + self._server, self.port = _make_server_on_free_port(app) + self.server_url = "http://localhost:{port}".format(port=self.port) + self.auth_url = "http://{user}:{passwd}@localhost:{port}".format( + user=self.username, + passwd=self.password, + port=self.port, + ) + + self._thread = threading.Thread(target=self._server.serve_forever) + self._thread.daemon = True + self._thread.start() + + interval = 0.1 + wait_seconds = 40 + for _ in range(int(wait_seconds / interval)): + try: + req = Request(self.server_url) + with urlopen(req, timeout=1) as response: + if response.status in (200, 401): + break + except (URLError, OSError): + pass + time.sleep(interval) + else: + raise RuntimeError( + "Could not start the server, waited for {}s".format(wait_seconds) + ) + + # Set a default value for UV_EXTRA_INDEX_URL in the bazel env so that + # the workspace .bazelrc `--action_env=UV_EXTRA_INDEX_URL` doesn't + # fail on Windows when the variable is unset in the client env. + self.bazel_env.setdefault("UV_EXTRA_INDEX_URL", "") + + # Use a sandbox-local credential store so credentials don't leak + # to the host system. + self.creds_dir = self.repo_root / ".uv-creds" + self.creds_dir.mkdir(parents=True, exist_ok=True) + self.bazel_env["UV_CREDENTIALS_DIR"] = str(self.creds_dir) + + # Log in to uv's credential store so `uv auth helper` can later + # serve the credentials to Bazel or uv itself. + self.run_bazel( + "run", + "//:uv", + "--", + "auth", + "login", + f"--username={self.username}", + f"--password={self.password}", + self.server_url, + ) + + def tearDown(self): + # Clear credentials from uv's credential store to ensure we are not + # logged into the service after the test. + self.run_bazel( + "run", + "//:uv", + "--", + "auth", + "logout", + self.server_url, + check=False, + ) + self._server.shutdown() + + def _assert_server_requires_auth(self): + req = Request(self.server_url + "/my-local-pkg/") + try: + urlopen(req, timeout=5) + self.fail("Expected 401 without auth") + except URLError: + pass + + def _auth_header(self): + return "Basic " + base64.b64encode( + "{user}:{passwd}".format( + user=self.username, + passwd=self.password, + ).encode("utf-8") + ).decode("utf-8") + + def _assert_simple_api_sha256(self): + auth_header = self._auth_header() + req = Request(self.server_url + "/simple/my-local-pkg/") + req.add_header("Authorization", auth_header) + resp = urlopen(req, timeout=5) + html = resp.read().decode("utf-8") + + match = re.search(r"#sha256=([a-f0-9]+)", html) + self.assertIsNotNone(match, "No sha256 found in simple API: {}".format(html)) + pypiserver_sha256 = match.group(1) + disk_sha256 = hashlib.sha256(self.wheel_path.read_bytes()).hexdigest() + self.assertEqual( + pypiserver_sha256, + disk_sha256, + "pypiserver hash {} != disk hash {}".format(pypiserver_sha256, disk_sha256), + ) + + def _creds_auth_args(self): + return [ + "--strategy=PyRequirementsLockUv=local", + "--action_env={key}={value}".format( + key="UV_CREDENTIALS_DIR", + value=str(self.creds_dir), + ), + "--action_env={key}={value}".format( + key="UV_EXTRA_INDEX_URL", + value=self.server_url, + ), + ] + + def _assert_lock_file(self, result): + self.assertEqual( + result.exit_code, + 0, + "Lock update failed:\n{}".format(result.describe()), + ) + lock_file = self.repo_root / "requirements.txt" + self.assertTrue(lock_file.exists(), "Lock file was not created") + contents = lock_file.read_text() + self.assertIn("my-local-pkg", contents) + self.assertIn("--hash=sha256:", contents) + + def test_lock_update_with_custom_index(self): + self._assert_server_requires_auth() + self._assert_simple_api_sha256() + + result = self.run_bazel( + "run", + "--action_env={key}={value}".format( + key="UV_EXTRA_INDEX_URL", + value=self.auth_url, + ), + "//:requirements.update", + ) + self._assert_lock_file(result) + + def test_update_with_credential_helper(self): + """Use a credential helper for authentication.""" + self._assert_server_requires_auth() + result = self.run_bazel( + "run", + *self._creds_auth_args(), + "//:requirements.update", + ) + self._assert_lock_file(result) + + def test_update_with_uv_auth_helper(self): + """Use the uv auth helper for authentication.""" + self._assert_server_requires_auth() + result = self.run_bazel( + "run", + *self._creds_auth_args(), + "//:requirements.update", + ) + self._assert_lock_file(result) + + def test_diff_test_with_requirements(self): + """Verify that ``diff_test`` can verify the generated lock file.""" + self._assert_server_requires_auth() + + # First generate the lock file + result = self.run_bazel( + "run", + *self._creds_auth_args(), + "//:requirements.update", + ) + self._assert_lock_file(result) + + # Copy the generated lock file to the expected location. The inner + # Bazel workspace is writable because it is a temporary copy created + # by the integration test framework. + generated = self.repo_root / "requirements.txt" + expected = self.repo_root / "requirements_expected.txt" + expected.write_text(generated.read_text()) + + # Run the diff_test: it builds the lock action, then compares the + # output to our expected file. + result = self.run_bazel( + "test", + *self._creds_auth_args(), + "//:requirements_diff_test", + ) + self.assertEqual( + result.exit_code, + 0, + "diff_test failed:\n{}".format(result.describe()), + ) + + def test_no_existing_requirements(self): + """Verify that ``bazel run`` and ``diff_test`` work when + ``requirements.txt`` does not yet exist.""" + self._assert_server_requires_auth() + + # Remove the existing lock file to simulate a fresh checkout + existing = self.repo_root / "requirements.txt" + existing.unlink() + self.assertFalse(existing.exists()) + + # Run ``requirements.update`` to generate the lock from scratch. The + # underlying lock rule will have no ``existing_output`` to copy, but + # ``uv pip compile`` should still produce the output. + result = self.run_bazel( + "run", + *self._creds_auth_args(), + "//:requirements.update", + ) + self._assert_lock_file(result) + + # diff_test should pass now that ``requirements.txt`` exists again + result = self.run_bazel( + "test", + *self._creds_auth_args(), + "//:requirements_diff_test", + ) + self.assertEqual( + result.exit_code, + 0, + "diff_test failed:\n{}".format(result.describe()), + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/integration/validate_test_main/BUILD.bazel b/tests/integration/validate_test_main/BUILD.bazel new file mode 100644 index 0000000000..cf931d3303 --- /dev/null +++ b/tests/integration/validate_test_main/BUILD.bazel @@ -0,0 +1,21 @@ +load("@rules_python//python:py_test.bzl", "py_test") + +# A test whose main module only defines a test case but never runs it. With +# --validate_test_main=enabled this should fail to build. +py_test( + name = "inert_test", + srcs = ["inert_test.py"], +) + +# A test whose main module invokes a runner. This should always build. +py_test( + name = "good_test", + srcs = ["good_test.py"], +) + +# A test whose main module defines nothing and only imports. This is allowed +# even with validation enabled, since it isn't the "defined but never run" case. +py_test( + name = "import_only_test", + srcs = ["import_only_test.py"], +) diff --git a/tests/integration/validate_test_main/MODULE.bazel b/tests/integration/validate_test_main/MODULE.bazel new file mode 100644 index 0000000000..c154ac169e --- /dev/null +++ b/tests/integration/validate_test_main/MODULE.bazel @@ -0,0 +1,10 @@ +module(name = "module_under_test") + +bazel_dep(name = "rules_python", version = "0.0.0") +local_path_override( + module_name = "rules_python", + path = "../../..", +) + +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.toolchain(python_version = "3.11") diff --git a/tests/integration/validate_test_main/WORKSPACE b/tests/integration/validate_test_main/WORKSPACE new file mode 100644 index 0000000000..de908549c0 --- /dev/null +++ b/tests/integration/validate_test_main/WORKSPACE @@ -0,0 +1,13 @@ +local_repository( + name = "rules_python", + path = "../../..", +) + +load("@rules_python//python:repositories.bzl", "py_repositories", "python_register_toolchains") + +py_repositories() + +python_register_toolchains( + name = "python_3_11", + python_version = "3.11", +) diff --git a/tests/integration/validate_test_main/WORKSPACE.bzlmod b/tests/integration/validate_test_main/WORKSPACE.bzlmod new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/integration/validate_test_main/good_test.py b/tests/integration/validate_test_main/good_test.py new file mode 100644 index 0000000000..46107c96ec --- /dev/null +++ b/tests/integration/validate_test_main/good_test.py @@ -0,0 +1,10 @@ +import unittest + + +class GoodTest(unittest.TestCase): + def test_something(self): + self.assertTrue(True) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/integration/validate_test_main/import_only_test.py b/tests/integration/validate_test_main/import_only_test.py new file mode 100644 index 0000000000..1bd46dead7 --- /dev/null +++ b/tests/integration/validate_test_main/import_only_test.py @@ -0,0 +1,3 @@ +# This module defines no classes or functions; it only imports. The validation +# action allows it even when enabled. +import os # noqa: F401 diff --git a/tests/integration/validate_test_main/inert_test.py b/tests/integration/validate_test_main/inert_test.py new file mode 100644 index 0000000000..8decf676d4 --- /dev/null +++ b/tests/integration/validate_test_main/inert_test.py @@ -0,0 +1,7 @@ +import unittest + + +class InertTest(unittest.TestCase): + def test_nothing_runs(self): + # This test case is never executed because nothing invokes a runner. + self.assertTrue(True) diff --git a/tests/integration/validate_test_main_test.py b/tests/integration/validate_test_main_test.py new file mode 100644 index 0000000000..4d108411d5 --- /dev/null +++ b/tests/integration/validate_test_main_test.py @@ -0,0 +1,40 @@ +import unittest + +from tests.integration import runner + +_FLAG = "--@rules_python//python/config_settings:validate_test_main" + + +class ValidateTestMainTest(runner.TestCase): + def test_inert_test_fails_when_enabled(self): + """An inert test main should fail the build when validation is on.""" + result = self.run_bazel( + "build", + f"{_FLAG}=enabled", + "//:inert_test", + check=False, + ) + self.assertNotEqual(result.exit_code, 0, "Expected build to fail") + self.assert_result_matches(result, r"will not run any tests") + + def test_good_test_builds_when_enabled(self): + """A main that invokes a runner should build when validation is on.""" + self.run_bazel("build", f"{_FLAG}=enabled", "//:good_test") + + def test_import_only_test_builds_when_enabled(self): + """A main that only imports (defines nothing) is allowed when on.""" + self.run_bazel("build", f"{_FLAG}=enabled", "//:import_only_test") + + def test_inert_test_builds_when_disabled(self): + """Validation is off by default, so even an inert test builds.""" + self.run_bazel("build", f"{_FLAG}=disabled", "//:inert_test") + + def test_inert_test_builds_by_default(self): + """The default (auto) resolves to disabled, so an inert test builds.""" + self.run_bazel("build", "//:inert_test") + + +if __name__ == "__main__": + # Enabling this makes the runner log subprocesses as the test goes along. + # logging.basicConfig(level = "INFO") + unittest.main() diff --git a/tests/integration/whl_library/.bazelrc b/tests/integration/whl_library/.bazelrc new file mode 100644 index 0000000000..eee47a8f19 --- /dev/null +++ b/tests/integration/whl_library/.bazelrc @@ -0,0 +1,2 @@ +common --incompatible_default_to_explicit_init_py +test --test_output=errors diff --git a/tests/integration/whl_library/BUILD.bazel b/tests/integration/whl_library/BUILD.bazel new file mode 100644 index 0000000000..0c26fee4cf --- /dev/null +++ b/tests/integration/whl_library/BUILD.bazel @@ -0,0 +1,65 @@ +load("@rules_python//python:py_test.bzl", "py_test") + +[ + alias( + name = "{}_pkg".format(pkg), + actual = "@rules_python//python:none", + ) + for pkg in [ + "charset_normalizer", + "certifi", + "idna", + "urllib3", + ] +] + +[ + filegroup( + name = "{}_whl".format(pkg), + srcs = [], + visibility = ["//visibility:public"], + ) + for pkg in [ + "charset_normalizer", + "certifi", + "idna", + "urllib3", + ] +] + +# Extract all deps +genquery( + name = "whl_target_deps", + expression = "deps(@whl_archive//:pkg)", + scope = ["@whl_archive//:pkg"], +) + +# Extract all deps +genquery( + name = "whl_deps_target_deps", + expression = "deps(@whl_deps_library//:pkg)", + scope = ["@whl_deps_library//:pkg"], +) + +py_test( + name = "test_contents", + srcs = ["test_contents.py"], + data = [ + ":whl_deps_target_deps", + ":whl_target_deps", + "@pip_archive//:srcs", + "@pip_sdist_archive//:srcs", + "@whl_archive//:srcs", + "@whl_archive//:whl", + "@whl_deps_library//:whl", + ], + env = { + "SDIST_SRC_FILES": "$(locations @pip_sdist_archive//:srcs)", + "SRC_FILES": "$(locations @pip_archive//:srcs)", + "WHL_DEPS": "$(location :whl_target_deps)", + "WHL_DEPS_LOCATION": "$(location @whl_deps_library//:whl)", + "WHL_FILES": "$(locations @whl_archive//:srcs)", + "WHL_LOCATION": "$(location @whl_archive//:whl)", + "WHL_TARGET_DEPS": "$(location :whl_deps_target_deps)", + }, +) diff --git a/tests/integration/whl_library/MODULE.bazel b/tests/integration/whl_library/MODULE.bazel new file mode 100644 index 0000000000..39a579f79e --- /dev/null +++ b/tests/integration/whl_library/MODULE.bazel @@ -0,0 +1,57 @@ +module(name = "integration_test") + +bazel_dep(name = "package_metadata", version = "0.0.7") +bazel_dep(name = "rules_python", version = "0.0.0") +local_path_override( + module_name = "rules_python", + path = "../../..", +) + +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.toolchain( + python_version = "3.14", +) +use_repo(python, pbs_host = "python_3_14_host") + +pip_archive = use_repo_rule("@rules_python//python/private/pypi:whl_library.bzl", "pip_archive") + +pip_archive( + name = "pip_sdist_archive", + filename = "requests-2.34.2.tar.gz", + python_interpreter_target = "@pbs_host//:python", + requirement = "requests", + # https://pypi.org/project/requests/#requests-2.34.2.tar.gz + sha256 = "f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed", + urls = [ + "https://files.pythonhosted.org/packages/ac/c3/e2a2b89f2d3e2179abd6d00ebd70bff6273f37fb3e0cc209f48b39d00cbf/requests-2.34.2.tar.gz", + ], +) + +pip_archive( + name = "pip_archive", + python_interpreter_target = "@pbs_host//:python", + requirement = "requests", +) + +whl_archive = use_repo_rule("@rules_python//python/private/pypi:whl_library.bzl", "whl_archive") + +whl_archive( + name = "whl_archive", + dep_template = "@integration_test//:{name}_{target}", + filename = "requests-2.34.2-py3-none-any.whl", + # https://pypi.org/project/requests/#requests-2.34.2-py3-none-any.whl + requirement = "requests", + sha256 = "2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0", + urls = [ + "https://files.pythonhosted.org/packages/a0/f4/c67b0b3f1b9245e8d266f0f112c500d50e5b4e83cb6f3b71b6528104182a/requests-2.34.2-py3-none-any.whl", + ], +) + +whl_deps_library = use_repo_rule("@rules_python//python/private/pypi:whl_library.bzl", "whl_deps_library") + +whl_deps_library( + name = "whl_deps_library", + dep_template = "@integration_test//:{name}_{target}", + metadata_file = "@whl_archive//:metadata.json", + requirement = "requests", +) diff --git a/tests/integration/whl_library/WORKSPACE b/tests/integration/whl_library/WORKSPACE new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/integration/whl_library/test_contents.py b/tests/integration/whl_library/test_contents.py new file mode 100644 index 0000000000..ab352a5d98 --- /dev/null +++ b/tests/integration/whl_library/test_contents.py @@ -0,0 +1,179 @@ +import os +import re +import unittest +from pathlib import Path + + +class TestContents(unittest.TestCase): + maxDiff = None + + @staticmethod + def _get_files(env_var: str) -> list[str]: + return [ + f.partition("site-packages/")[-1] for f in os.environ[env_var].split(" ") + ] + + def test_sdist_srcs(self): + self.assertEqual( + self._get_files("SDIST_SRC_FILES"), + [ + "requests/__init__.py", + "requests/__version__.py", + "requests/_internal_utils.py", + "requests/_types.py", + "requests/adapters.py", + "requests/api.py", + "requests/auth.py", + "requests/certs.py", + "requests/compat.py", + "requests/cookies.py", + "requests/exceptions.py", + "requests/help.py", + "requests/hooks.py", + "requests/models.py", + "requests/packages.py", + "requests/sessions.py", + "requests/status_codes.py", + "requests/structures.py", + "requests/utils.py", + ], + ) + + def test_srcs(self): + self.assertEqual( + self._get_files("SRC_FILES"), + [ + "requests/__init__.py", + "requests/__version__.py", + "requests/_internal_utils.py", + "requests/_types.py", + "requests/adapters.py", + "requests/api.py", + "requests/auth.py", + "requests/certs.py", + "requests/compat.py", + "requests/cookies.py", + "requests/exceptions.py", + "requests/help.py", + "requests/hooks.py", + "requests/models.py", + "requests/packages.py", + "requests/sessions.py", + "requests/status_codes.py", + "requests/structures.py", + "requests/utils.py", + ], + ) + + def test_whl_srcs(self): + self.assertEqual( + self._get_files("WHL_FILES"), + [ + "requests/__init__.py", + "requests/__version__.py", + "requests/_internal_utils.py", + "requests/_types.py", + "requests/adapters.py", + "requests/api.py", + "requests/auth.py", + "requests/certs.py", + "requests/compat.py", + "requests/cookies.py", + "requests/exceptions.py", + "requests/help.py", + "requests/hooks.py", + "requests/models.py", + "requests/packages.py", + "requests/sessions.py", + "requests/status_codes.py", + "requests/structures.py", + "requests/utils.py", + ], + ) + + def test_whl_location(self): + self.assertTrue( + os.environ["WHL_LOCATION"].endswith("requests-2.34.2-py3-none-any.whl"), + msg=os.environ["WHL_LOCATION"], + ) + self.assertTrue( + os.environ["WHL_DEPS_LOCATION"].endswith( + "requests-2.34.2-py3-none-any.whl" + ), + msg=os.environ["WHL_DEPS_LOCATION"], + ) + + @staticmethod + def _read_file(env_var: str) -> list[str]: + return set(Path(os.environ[env_var]).read_text().splitlines()) + + @staticmethod + def _normalize_label(label: str) -> str: + if not label.startswith("@@"): + return label + repo, _, rest = label.partition("//") + parts = [p for p in re.split(r"[~+]", repo[2:]) if p] + if parts: + return f"@{parts[-1]}//{rest}" + return label + + def test_whl_deps_ar_the_same(self): + for var, main_dep in { + "WHL_DEPS": "@whl_archive//:pkg", + "WHL_TARGET_DEPS": "@whl_deps_library//:pkg", + }.items(): + self.assertEqual( + { + self._normalize_label(x) + for x in self._read_file(var) + if not x.endswith("toolchain_type") + }, + { + main_dep, + "//:certifi_pkg", + "//:charset_normalizer_pkg", + "//:idna_pkg", + "//:urllib3_pkg", + "@whl_archive//:data", + "@whl_archive//:package_metadata", + "@whl_archive//:site-packages/requests-2.34.2.dist-info/INSTALLER", + "@whl_archive//:site-packages/requests-2.34.2.dist-info/METADATA", + "@whl_archive//:site-packages/requests-2.34.2.dist-info/RECORD", + "@whl_archive//:site-packages/requests-2.34.2.dist-info/WHEEL", + "@whl_archive//:site-packages/requests-2.34.2.dist-info/licenses/LICENSE", + "@whl_archive//:site-packages/requests-2.34.2.dist-info/licenses/NOTICE", + "@whl_archive//:site-packages/requests-2.34.2.dist-info/top_level.txt", + "@whl_archive//:site-packages/requests/__init__.py", + "@whl_archive//:site-packages/requests/__version__.py", + "@whl_archive//:site-packages/requests/_internal_utils.py", + "@whl_archive//:site-packages/requests/_types.py", + "@whl_archive//:site-packages/requests/adapters.py", + "@whl_archive//:site-packages/requests/api.py", + "@whl_archive//:site-packages/requests/auth.py", + "@whl_archive//:site-packages/requests/certs.py", + "@whl_archive//:site-packages/requests/compat.py", + "@whl_archive//:site-packages/requests/cookies.py", + "@whl_archive//:site-packages/requests/exceptions.py", + "@whl_archive//:site-packages/requests/help.py", + "@whl_archive//:site-packages/requests/hooks.py", + "@whl_archive//:site-packages/requests/models.py", + "@whl_archive//:site-packages/requests/packages.py", + "@whl_archive//:site-packages/requests/py.typed", + "@whl_archive//:site-packages/requests/sessions.py", + "@whl_archive//:site-packages/requests/status_codes.py", + "@whl_archive//:site-packages/requests/structures.py", + "@whl_archive//:site-packages/requests/utils.py", + "@whl_archive//:srcs", + "@rules_python//python:none", + "@rules_python//python/config_settings:_is_venvs_site_packages_yes", + "@rules_python//python/config_settings:add_srcs_to_runfiles", + "@rules_python//python/config_settings:precompile", + "@rules_python//python/config_settings:precompile_source_retention", + "@rules_python//python/config_settings:venvs_site_packages", + "@rules_python//python/private:sentinel", + }, + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/interpreter/BUILD.bazel b/tests/interpreter/BUILD.bazel new file mode 100644 index 0000000000..5d89ede28a --- /dev/null +++ b/tests/interpreter/BUILD.bazel @@ -0,0 +1,52 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +load(":interpreter_tests.bzl", "PYTHON_VERSIONS_TO_TEST", "py_reconfig_interpreter_tests") + +# For this test the interpreter is sourced from the current configuration. That +# means both the interpreter and the test itself are expected to run under the +# same Python version. +py_reconfig_interpreter_tests( + name = "interpreter_version_test", + srcs = ["interpreter_test.py"], + data = [ + "//python/bin:python", + ], + env = { + "PYTHON_BIN": "$(rootpath //python/bin:python)", + }, + main = "interpreter_test.py", + python_versions = PYTHON_VERSIONS_TO_TEST, +) + +# For this test the interpreter is sourced from a binary pinned at a specific +# Python version. That means the interpreter and the test itself can run +# different Python versions. +py_reconfig_interpreter_tests( + name = "python_src_test", + srcs = ["interpreter_test.py"], + data = [ + "//python/bin:python", + ], + env = { + # Since we're grabbing the interpreter from a binary with a fixed + # version, we expect to always see that version. It doesn't matter what + # Python version the test itself is running with. + "EXPECTED_INTERPRETER_VERSION": "3.11", + "PYTHON_BIN": "$(rootpath //python/bin:python)", + }, + main = "interpreter_test.py", + python_src = "//tools/publish:twine", + python_versions = PYTHON_VERSIONS_TO_TEST, +) diff --git a/tests/interpreter/interpreter_test.py b/tests/interpreter/interpreter_test.py new file mode 100644 index 0000000000..0971fa2eba --- /dev/null +++ b/tests/interpreter/interpreter_test.py @@ -0,0 +1,80 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import os +import subprocess +import sys +import unittest + + +class InterpreterTest(unittest.TestCase): + def setUp(self): + super().setUp() + self.interpreter = os.environ["PYTHON_BIN"] + + v = sys.version_info + self.version = f"{v.major}.{v.minor}" + + def test_self_version(self): + """Performs a sanity check on the Python version used for this test.""" + expected_version = os.environ["EXPECTED_SELF_VERSION"] + self.assertEqual(expected_version, self.version) + + def test_interpreter_version(self): + """Validates that we can successfully execute arbitrary code from the CLI.""" + expected_version = os.environ.get("EXPECTED_INTERPRETER_VERSION", self.version) + + try: + result = subprocess.check_output( + [self.interpreter], + text=True, + stderr=subprocess.STDOUT, + input="\r".join( + [ + "import sys", + "v = sys.version_info", + "print(f'version: {v.major}.{v.minor}')", + ] + ), + ).strip() + except subprocess.CalledProcessError as error: + print("OUTPUT:", error.stdout) + raise + + self.assertEqual(result, f"version: {expected_version}") + + def test_json_tool(self): + """Validates that we can successfully invoke a module from the CLI.""" + # Pass unformatted JSON to the json.tool module. + try: + result = subprocess.check_output( + [ + self.interpreter, + "-m", + "json.tool", + ], + text=True, + stderr=subprocess.STDOUT, + input='{"json":"obj"}', + ).strip() + except subprocess.CalledProcessError as error: + print("OUTPUT:", error.stdout) + raise + + # Validate that we get formatted JSON back. + self.assertEqual(result, '{\n "json": "obj"\n}') + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/interpreter/interpreter_tests.bzl b/tests/interpreter/interpreter_tests.bzl new file mode 100644 index 0000000000..3c5882afa0 --- /dev/null +++ b/tests/interpreter/interpreter_tests.bzl @@ -0,0 +1,54 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""This file contains helpers for testing the interpreter rule.""" + +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") + +# The versions of Python that we want to run the interpreter tests against. +PYTHON_VERSIONS_TO_TEST = ( + "3.10", + "3.11", + "3.12", +) + +def py_reconfig_interpreter_tests(name, python_versions, env = {}, **kwargs): + """Runs the specified test against each of the specified Python versions. + + One test gets generated for each Python version. The following environment + variable gets set for the test: + + EXPECTED_SELF_VERSION: Contains the Python version that the test itself + is running under. + + Args: + name: Name of the test. + python_versions: A list of Python versions to test. + env: The environment to set on the test. + **kwargs: Passed to the underlying py_reconfig_test targets. + """ + for python_version in python_versions: + py_reconfig_test( + name = "{}_{}".format(name, python_version), + env = env | { + "EXPECTED_SELF_VERSION": python_version, + }, + python_version = python_version, + **kwargs + ) + + native.test_suite( + name = name, + tests = [":{}_{}".format(name, python_version) for python_version in python_versions], + ) diff --git a/tests/load_from_macro/BUILD.bazel b/tests/load_from_macro/BUILD.bazel index 00d7bf90ca..ecb5de51a7 100644 --- a/tests/load_from_macro/BUILD.bazel +++ b/tests/load_from_macro/BUILD.bazel @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -load("//python:defs.bzl", "py_library") +load("//python:py_library.bzl", "py_library") load(":tags.bzl", "TAGS") licenses(["notice"]) diff --git a/tests/modules/another_module/BUILD.bazel b/tests/modules/another_module/BUILD.bazel new file mode 100644 index 0000000000..3b56b6ee83 --- /dev/null +++ b/tests/modules/another_module/BUILD.bazel @@ -0,0 +1,5 @@ +filegroup( + name = "data", + srcs = ["another_module_data.txt"], + visibility = ["//visibility:public"], +) diff --git a/tests/modules/another_module/MODULE.bazel b/tests/modules/another_module/MODULE.bazel new file mode 100644 index 0000000000..8ed5a5543b --- /dev/null +++ b/tests/modules/another_module/MODULE.bazel @@ -0,0 +1 @@ +module(name = "another_module") diff --git a/tests/modules/another_module/another_module_data.txt b/tests/modules/another_module/another_module_data.txt new file mode 100644 index 0000000000..f742ebab60 --- /dev/null +++ b/tests/modules/another_module/another_module_data.txt @@ -0,0 +1 @@ +print("token") diff --git a/tests/modules/other/BUILD.bazel b/tests/modules/other/BUILD.bazel new file mode 100644 index 0000000000..665049b9f5 --- /dev/null +++ b/tests/modules/other/BUILD.bazel @@ -0,0 +1,30 @@ +load("@rules_python//python:py_binary.bzl", "py_binary") +load("@rules_python//tests/support:py_reconfig.bzl", "py_reconfig_binary") + +package( + default_visibility = ["//visibility:public"], +) + +py_reconfig_binary( + name = "external_main", + srcs = [":external_main.py"], + # We're testing a system_python specific code path, + # so force using that bootstrap + bootstrap_impl = "system_python", + main = "external_main.py", +) + +py_binary( + name = "venv_bin", + srcs = ["venv_bin.py"], + config_settings = { + "@rules_python//python/config_settings:bootstrap_impl": "script", + "@rules_python//python/config_settings:venvs_site_packages": "yes", + }, + deps = [ + # Add two packages that install into the same directory. This is + # to test that namespace packages install correctly and are importable. + "//nspkg_delta", + "//nspkg_gamma", + ], +) diff --git a/tests/modules/other/MODULE.bazel b/tests/modules/other/MODULE.bazel new file mode 100644 index 0000000000..11a633d56b --- /dev/null +++ b/tests/modules/other/MODULE.bazel @@ -0,0 +1,5 @@ +module(name = "other") + +bazel_dep(name = "rules_python", version = "0") +bazel_dep(name = "bazel_skylib", version = "1.7.1") +bazel_dep(name = "another_module", version = "0") diff --git a/tests/modules/other/external_main.py b/tests/modules/other/external_main.py new file mode 100644 index 0000000000..f742ebab60 --- /dev/null +++ b/tests/modules/other/external_main.py @@ -0,0 +1 @@ +print("token") diff --git a/tests/modules/other/nspkg_delta/BUILD.bazel b/tests/modules/other/nspkg_delta/BUILD.bazel new file mode 100644 index 0000000000..ca142d2c10 --- /dev/null +++ b/tests/modules/other/nspkg_delta/BUILD.bazel @@ -0,0 +1,10 @@ +load("@rules_python//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "nspkg_delta", + srcs = glob(["site-packages/**/*.py"]), + experimental_venvs_site_packages = "@rules_python//python/config_settings:venvs_site_packages", + imports = ["site-packages"], +) diff --git a/tests/modules/other/nspkg_delta/site-packages/nspkg/subnspkg/delta/__init__.py b/tests/modules/other/nspkg_delta/site-packages/nspkg/subnspkg/delta/__init__.py new file mode 100644 index 0000000000..bb7b160deb --- /dev/null +++ b/tests/modules/other/nspkg_delta/site-packages/nspkg/subnspkg/delta/__init__.py @@ -0,0 +1 @@ +# Intentionally empty diff --git a/tests/modules/other/nspkg_gamma/BUILD.bazel b/tests/modules/other/nspkg_gamma/BUILD.bazel new file mode 100644 index 0000000000..0fe099eb0a --- /dev/null +++ b/tests/modules/other/nspkg_gamma/BUILD.bazel @@ -0,0 +1,10 @@ +load("@rules_python//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "nspkg_gamma", + srcs = glob(["site-packages/**/*.py"]), + experimental_venvs_site_packages = "@rules_python//python/config_settings:venvs_site_packages", + imports = ["site-packages"], +) diff --git a/tests/modules/other/nspkg_gamma/site-packages/nspkg/subnspkg/gamma/__init__.py b/tests/modules/other/nspkg_gamma/site-packages/nspkg/subnspkg/gamma/__init__.py new file mode 100644 index 0000000000..bb7b160deb --- /dev/null +++ b/tests/modules/other/nspkg_gamma/site-packages/nspkg/subnspkg/gamma/__init__.py @@ -0,0 +1 @@ +# Intentionally empty diff --git a/tests/modules/other/nspkg_single/BUILD.bazel b/tests/modules/other/nspkg_single/BUILD.bazel new file mode 100644 index 0000000000..07e269b878 --- /dev/null +++ b/tests/modules/other/nspkg_single/BUILD.bazel @@ -0,0 +1,10 @@ +load("@rules_python//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "nspkg_single", + srcs = glob(["site-packages/**/*.py"]), + experimental_venvs_site_packages = "@rules_python//python/config_settings:venvs_site_packages", + imports = ["site-packages"], +) diff --git a/tests/modules/other/nspkg_single/site-packages/__init__.py b/tests/modules/other/nspkg_single/site-packages/__init__.py new file mode 100644 index 0000000000..bb26c87599 --- /dev/null +++ b/tests/modules/other/nspkg_single/site-packages/__init__.py @@ -0,0 +1 @@ +# empty, will not be added to the site-packages dir diff --git a/tests/modules/other/nspkg_single/site-packages/single_file.py b/tests/modules/other/nspkg_single/site-packages/single_file.py new file mode 100644 index 0000000000..f6d7dfd640 --- /dev/null +++ b/tests/modules/other/nspkg_single/site-packages/single_file.py @@ -0,0 +1,5 @@ +__all__ = [ + "SOMETHING", +] + +SOMETHING = "nothing" diff --git a/tests/modules/other/simple_v1/BUILD.bazel b/tests/modules/other/simple_v1/BUILD.bazel new file mode 100644 index 0000000000..0fa2f14a88 --- /dev/null +++ b/tests/modules/other/simple_v1/BUILD.bazel @@ -0,0 +1,14 @@ +load("@rules_python//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "simple_v1", + srcs = glob(["site-packages/**/*.py"]), + data = glob( + ["**/*"], + exclude = ["site-packages/**/*.py"], + ), + experimental_venvs_site_packages = "@rules_python//python/config_settings:venvs_site_packages", + imports = ["site-packages"], +) diff --git a/tests/modules/other/simple_v1/site-packages/simple-1.0.0.dist-info/METADATA b/tests/modules/other/simple_v1/site-packages/simple-1.0.0.dist-info/METADATA new file mode 100644 index 0000000000..ee76ec48a4 --- /dev/null +++ b/tests/modules/other/simple_v1/site-packages/simple-1.0.0.dist-info/METADATA @@ -0,0 +1 @@ +inside is v1 diff --git a/tests/modules/other/simple_v1/site-packages/simple/__init__.py b/tests/modules/other/simple_v1/site-packages/simple/__init__.py new file mode 100644 index 0000000000..5becc17c04 --- /dev/null +++ b/tests/modules/other/simple_v1/site-packages/simple/__init__.py @@ -0,0 +1 @@ +__version__ = "1.0.0" diff --git a/tests/modules/other/simple_v1/site-packages/simple_v1_extras/data.txt b/tests/modules/other/simple_v1/site-packages/simple_v1_extras/data.txt new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/modules/other/simple_v2/BUILD.bazel b/tests/modules/other/simple_v2/BUILD.bazel new file mode 100644 index 0000000000..5a7e066aec --- /dev/null +++ b/tests/modules/other/simple_v2/BUILD.bazel @@ -0,0 +1,15 @@ +load("@rules_python//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "simple_v2", + srcs = glob(["site-packages/**/*.py"]), + data = glob( + ["**/*"], + exclude = ["site-packages/**/*.py"], + ), + experimental_venvs_site_packages = "@rules_python//python/config_settings:venvs_site_packages", + imports = ["site-packages"], + pyi_srcs = glob(["**/*.pyi"]), +) diff --git a/tests/modules/other/simple_v2/site-packages/simple-2.0.0.dist-info/METADATA b/tests/modules/other/simple_v2/site-packages/simple-2.0.0.dist-info/METADATA new file mode 100644 index 0000000000..ee76ec48a4 --- /dev/null +++ b/tests/modules/other/simple_v2/site-packages/simple-2.0.0.dist-info/METADATA @@ -0,0 +1 @@ +inside is v1 diff --git a/tests/modules/other/simple_v2/site-packages/simple-2.0.0.dist-info/licenses/LICENSE b/tests/modules/other/simple_v2/site-packages/simple-2.0.0.dist-info/licenses/LICENSE new file mode 100644 index 0000000000..0cb5e79499 --- /dev/null +++ b/tests/modules/other/simple_v2/site-packages/simple-2.0.0.dist-info/licenses/LICENSE @@ -0,0 +1 @@ +Some License diff --git a/tests/modules/other/simple_v2/site-packages/simple.libs/data.so b/tests/modules/other/simple_v2/site-packages/simple.libs/data.so new file mode 100644 index 0000000000..f023e3b9ae --- /dev/null +++ b/tests/modules/other/simple_v2/site-packages/simple.libs/data.so @@ -0,0 +1,2 @@ +# This is usually created by auditwheel when processing linux wheels and including +# dependencies. diff --git a/tests/modules/other/simple_v2/site-packages/simple/__init__.py b/tests/modules/other/simple_v2/site-packages/simple/__init__.py new file mode 100644 index 0000000000..8c0d5d5bb2 --- /dev/null +++ b/tests/modules/other/simple_v2/site-packages/simple/__init__.py @@ -0,0 +1 @@ +__version__ = "2.0.0" diff --git a/tests/modules/other/simple_v2/site-packages/simple/__init__.pyi b/tests/modules/other/simple_v2/site-packages/simple/__init__.pyi new file mode 100644 index 0000000000..bb7b160deb --- /dev/null +++ b/tests/modules/other/simple_v2/site-packages/simple/__init__.pyi @@ -0,0 +1 @@ +# Intentionally empty diff --git a/tests/modules/other/venv_bin.py b/tests/modules/other/venv_bin.py new file mode 100644 index 0000000000..81de1a9ab9 --- /dev/null +++ b/tests/modules/other/venv_bin.py @@ -0,0 +1,17 @@ +import nspkg + +print(nspkg) + +import nspkg.subnspkg # noqa: E402 + +print(nspkg.subnspkg) + +import nspkg.subnspkg.delta # noqa: E402 + +print(nspkg.subnspkg.delta) + +import nspkg.subnspkg.gamma # noqa: E402 + +print(nspkg.subnspkg.gamma) + +print("@other//:venv_bin ran successfully.") diff --git a/tests/modules/other/with_external_data/BUILD.bazel b/tests/modules/other/with_external_data/BUILD.bazel new file mode 100644 index 0000000000..338f77947d --- /dev/null +++ b/tests/modules/other/with_external_data/BUILD.bazel @@ -0,0 +1,23 @@ +load("@bazel_skylib//rules:copy_file.bzl", "copy_file") +load("@rules_python//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +# The users may include data through other repos via annotations and copy_file +# just add this edge case. +# +# NOTE: if the data is not copied to `site-packages/` then it will not +# appear. +copy_file( + name = "external_data", + src = "@another_module//:data", + out = "site-packages/external_data/another_module_data.txt", +) + +py_library( + name = "with_external_data", + srcs = ["site-packages/with_external_data.py"], + data = [":external_data"], + experimental_venvs_site_packages = "@rules_python//python/config_settings:venvs_site_packages", + imports = ["site-packages"], +) diff --git a/tests/modules/other/with_external_data/site-packages/with_external_data.py b/tests/modules/other/with_external_data/site-packages/with_external_data.py new file mode 100644 index 0000000000..ccd9dcef9e --- /dev/null +++ b/tests/modules/other/with_external_data/site-packages/with_external_data.py @@ -0,0 +1 @@ +# Intentionally blank diff --git a/tests/modules/rules_pyrefly_stub/WORKSPACE b/tests/modules/rules_pyrefly_stub/WORKSPACE new file mode 100644 index 0000000000..48a0b3083d --- /dev/null +++ b/tests/modules/rules_pyrefly_stub/WORKSPACE @@ -0,0 +1 @@ +workspace(name = "rules_pyrefly") diff --git a/tests/modules/rules_pyrefly_stub/pyrefly/BUILD.bazel b/tests/modules/rules_pyrefly_stub/pyrefly/BUILD.bazel new file mode 100644 index 0000000000..2f14f71b3b --- /dev/null +++ b/tests/modules/rules_pyrefly_stub/pyrefly/BUILD.bazel @@ -0,0 +1,3 @@ +package(default_visibility = ["//visibility:public"]) + +exports_files(["pyrefly.bzl"]) diff --git a/tests/modules/rules_pyrefly_stub/pyrefly/pyrefly.bzl b/tests/modules/rules_pyrefly_stub/pyrefly/pyrefly.bzl new file mode 100644 index 0000000000..fc1a44ed31 --- /dev/null +++ b/tests/modules/rules_pyrefly_stub/pyrefly/pyrefly.bzl @@ -0,0 +1,21 @@ +"""Stub implementation of rules_pyrefly for WORKSPACE mode.""" + +# buildifier: disable=unused-variable +def _noop_aspect_impl(_target, _ctx): + return [] + +_noop_aspect = aspect( + implementation = _noop_aspect_impl, + doc = "No-op Pyrefly aspect stub for WORKSPACE mode.", +) + +def pyrefly(**_kwargs): + """Stub pyrefly aspect constructor. + + Args: + **_kwargs: Ignored keyword arguments. + + Returns: + A no-op aspect. + """ + return _noop_aspect diff --git a/tests/multi_pypi/BUILD.bazel b/tests/multi_pypi/BUILD.bazel new file mode 100644 index 0000000000..a119ebe116 --- /dev/null +++ b/tests/multi_pypi/BUILD.bazel @@ -0,0 +1,29 @@ +load("@bazel_skylib//rules:common_settings.bzl", "string_flag") +load("//python:defs.bzl", "py_library") + +string_flag( + name = "external_deps_name", + build_setting_default = "", + visibility = ["//visibility:public"], +) + +py_library( + name = "common", + srcs = [], + visibility = ["//visibility:public"], + deps = select({ + ":is_external_alpha": ["@pypi_alpha//more_itertools"], + ":is_external_beta": ["@pypi_beta//more_itertools"], + "//conditions:default": [], + }), +) + +config_setting( + name = "is_external_alpha", + flag_values = {"//tests/multi_pypi:external_deps_name": "alpha"}, +) + +config_setting( + name = "is_external_beta", + flag_values = {"//tests/multi_pypi:external_deps_name": "beta"}, +) diff --git a/tests/multi_pypi/alpha/BUILD.bazel b/tests/multi_pypi/alpha/BUILD.bazel new file mode 100644 index 0000000000..7b56e0a547 --- /dev/null +++ b/tests/multi_pypi/alpha/BUILD.bazel @@ -0,0 +1,7 @@ +load("//python/uv:lock.bzl", "lock") + +lock( + name = "requirements", + srcs = ["pyproject.toml"], + out = "requirements.txt", +) diff --git a/tests/multi_pypi/alpha/pyproject.toml b/tests/multi_pypi/alpha/pyproject.toml new file mode 100644 index 0000000000..8f99cd08fc --- /dev/null +++ b/tests/multi_pypi/alpha/pyproject.toml @@ -0,0 +1,6 @@ +[project] +name = "multi-pypi-test-alpha" +version = "0.1.0" +dependencies = [ + "more-itertools==9.1.0" +] diff --git a/tests/multi_pypi/alpha/requirements.txt b/tests/multi_pypi/alpha/requirements.txt new file mode 100644 index 0000000000..febb6b72ae --- /dev/null +++ b/tests/multi_pypi/alpha/requirements.txt @@ -0,0 +1,6 @@ +# This file was autogenerated by uv via the following command: +# bazel run //tests/multi_pypi/alpha:requirements.update +more-itertools==9.1.0 \ + --hash=sha256:cabaa341ad0389ea83c17a94566a53ae4c9d07349861ecb14dc6d0345cf9ac5d \ + --hash=sha256:d2bc7f02446e86a68911e58ded76d6561eea00cddfb2a91e7019bbb586c799f3 + # via multi-pypi-test-alpha (tests/multi_pypi/alpha/pyproject.toml) diff --git a/tests/multi_pypi/beta/BUILD.bazel b/tests/multi_pypi/beta/BUILD.bazel new file mode 100644 index 0000000000..7b56e0a547 --- /dev/null +++ b/tests/multi_pypi/beta/BUILD.bazel @@ -0,0 +1,7 @@ +load("//python/uv:lock.bzl", "lock") + +lock( + name = "requirements", + srcs = ["pyproject.toml"], + out = "requirements.txt", +) diff --git a/tests/multi_pypi/beta/pyproject.toml b/tests/multi_pypi/beta/pyproject.toml new file mode 100644 index 0000000000..02a510ffa2 --- /dev/null +++ b/tests/multi_pypi/beta/pyproject.toml @@ -0,0 +1,6 @@ +[project] +name = "multi-pypi-test-beta" +version = "0.1.0" +dependencies = [ + "more-itertools==9.0.0" +] diff --git a/tests/multi_pypi/beta/requirements.txt b/tests/multi_pypi/beta/requirements.txt new file mode 100644 index 0000000000..de05f6dc9a --- /dev/null +++ b/tests/multi_pypi/beta/requirements.txt @@ -0,0 +1,6 @@ +# This file was autogenerated by uv via the following command: +# bazel run //tests/multi_pypi/beta:requirements.update +more-itertools==9.0.0 \ + --hash=sha256:250e83d7e81d0c87ca6bd942e6aeab8cc9daa6096d12c5308f3f92fa5e5c1f41 \ + --hash=sha256:5a6257e40878ef0520b1803990e3e22303a41b5714006c32a3fd8304b26ea1ab + # via multi-pypi-test-beta (tests/multi_pypi/beta/pyproject.toml) diff --git a/tests/multi_pypi/pypi_alpha/BUILD.bazel b/tests/multi_pypi/pypi_alpha/BUILD.bazel new file mode 100644 index 0000000000..47e3b2fa88 --- /dev/null +++ b/tests/multi_pypi/pypi_alpha/BUILD.bazel @@ -0,0 +1,11 @@ +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") + +py_reconfig_test( + name = "pypi_alpha_test", + srcs = ["pypi_alpha_test.py"], + config_settings = { + "//tests/multi_pypi:external_deps_name": "alpha", + }, + main = "pypi_alpha_test.py", + deps = ["//tests/multi_pypi:common"], +) diff --git a/tests/multi_pypi/pypi_alpha/pypi_alpha_test.py b/tests/multi_pypi/pypi_alpha/pypi_alpha_test.py new file mode 100644 index 0000000000..0521327563 --- /dev/null +++ b/tests/multi_pypi/pypi_alpha/pypi_alpha_test.py @@ -0,0 +1,8 @@ +import sys + +from more_itertools import __version__ + +if __name__ == "__main__": + expected_version = "9.1.0" + if __version__ != expected_version: + sys.exit(f"Expected version {expected_version}, got {__version__}") diff --git a/tests/multi_pypi/pypi_beta/BUILD.bazel b/tests/multi_pypi/pypi_beta/BUILD.bazel new file mode 100644 index 0000000000..077d87bdf0 --- /dev/null +++ b/tests/multi_pypi/pypi_beta/BUILD.bazel @@ -0,0 +1,11 @@ +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") + +py_reconfig_test( + name = "pypi_beta_test", + srcs = ["pypi_beta_test.py"], + config_settings = { + "//tests/multi_pypi:external_deps_name": "beta", + }, + main = "pypi_beta_test.py", + deps = ["//tests/multi_pypi:common"], +) diff --git a/tests/multi_pypi/pypi_beta/pypi_beta_test.py b/tests/multi_pypi/pypi_beta/pypi_beta_test.py new file mode 100644 index 0000000000..8c34de0735 --- /dev/null +++ b/tests/multi_pypi/pypi_beta/pypi_beta_test.py @@ -0,0 +1,8 @@ +import sys + +from more_itertools import __version__ + +if __name__ == "__main__": + expected_version = "9.0.0" + if __version__ != expected_version: + sys.exit(f"Expected version {expected_version}, got {__version__}") diff --git a/tests/news/BUILD.bazel b/tests/news/BUILD.bazel new file mode 100644 index 0000000000..31abff1a8a --- /dev/null +++ b/tests/news/BUILD.bazel @@ -0,0 +1,8 @@ +load("//python:py_test.bzl", "py_test") + +py_test( + name = "news_test", + srcs = ["news_test.py"], + data = ["//news:news_files"], + deps = ["//python/runfiles"], +) diff --git a/tests/news/news_test.py b/tests/news/news_test.py new file mode 100644 index 0000000000..a8ed7a2849 --- /dev/null +++ b/tests/news/news_test.py @@ -0,0 +1,68 @@ +import pathlib +import unittest + +from python.runfiles import runfiles + + +def _get_news_dir(): + rf = runfiles.Create() + path = rf.Rlocation("rules_python/news") + if path: + return pathlib.Path(path) + return None + + +class NewsTest(unittest.TestCase): + def test_all_news_files_are_valid(self): + news_dir = _get_news_dir() + self.assertIsNotNone(news_dir, "Could not locate news directory in runfiles") + self.assertTrue(news_dir.exists(), "News directory does not exist in runfiles") + + allowed_categories = {"added", "changed", "fixed", "removed"} + + for p in news_dir.iterdir(): + if not p.is_file(): + continue + # Ignore BUILD files and .gitkeep if they are in the directory + if p.name in ("BUILD", "BUILD.bazel", ".gitkeep"): + continue + + filename = p.name + + # Collapse extension and filename check into a single assertRegex + self.assertRegex( + filename, + r"^[^.]+\.[^.]+\.md$", + f"News filename {filename} must follow ..md pattern", + ) + + parts = filename.split(".") + category = parts[1].lower() + + # Category must be valid + self.assertIn( + category, + allowed_categories, + f"News file {filename} has invalid category '{category}'. " + f"Must be one of {allowed_categories}", + ) + + # Must be readable as UTF-8 + try: + content = p.read_text(encoding="utf-8").strip() + except (IOError, UnicodeDecodeError) as e: + self.fail(f"Failed to read news file {filename} as UTF-8: {e}") + + # Content must not be empty + self.assertTrue(len(content) > 0, f"News file {filename} must not be empty") + + # Content must NOT start with bullet points (* or -) + self.assertFalse( + content.startswith("* ") or content.startswith("- "), + f"News file {filename} must not start with bullet points (* or -). " + "The release tool adds them automatically.", + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/no_unsafe_paths/BUILD.bazel b/tests/no_unsafe_paths/BUILD.bazel new file mode 100644 index 0000000000..c9a681daa9 --- /dev/null +++ b/tests/no_unsafe_paths/BUILD.bazel @@ -0,0 +1,33 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") +load("//tests/support:support.bzl", "SUPPORTS_BOOTSTRAP_SCRIPT") + +py_reconfig_test( + name = "no_unsafe_paths_3.10_test", + srcs = ["test.py"], + bootstrap_impl = "script", + main = "test.py", + python_version = "3.10", + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, +) + +py_reconfig_test( + name = "no_unsafe_paths_3.11_test", + srcs = ["test.py"], + bootstrap_impl = "script", + main = "test.py", + python_version = "3.11", + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, +) diff --git a/tests/no_unsafe_paths/test.py b/tests/no_unsafe_paths/test.py new file mode 100644 index 0000000000..4727a02995 --- /dev/null +++ b/tests/no_unsafe_paths/test.py @@ -0,0 +1,44 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import os +import sys +import unittest + + +class NoUnsafePathsTest(unittest.TestCase): + def test_no_unsafe_paths_in_search_path(self): + # Based on sys.path documentation, the first item added is the zip + # archive + # (see: https://docs.python.org/3/library/sys_path_init.html) + # + # We can use this as a marker to verify that during bootstrapping, + # (1) no unexpected paths were prepended, and (2) no paths were + # accidentally dropped. + # + major, minor, *_ = sys.version_info + archive = f"python{major}{minor}.zip" + + # < Python 3.11 behaviour + if (major, minor) < (3, 11): + # Because of https://github.com/bazel-contrib/rules_python/blob/0.39.0/python/private/stage2_bootstrap_template.py#L415-L436 + self.assertEqual(os.path.dirname(sys.argv[0]), sys.path[0]) + self.assertEqual(os.path.basename(sys.path[1]), archive) + # >= Python 3.11 behaviour + else: + self.assertEqual(os.path.basename(sys.path[0]), archive) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/packaging/BUILD.bazel b/tests/packaging/BUILD.bazel new file mode 100644 index 0000000000..d88a593006 --- /dev/null +++ b/tests/packaging/BUILD.bazel @@ -0,0 +1,44 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +load("@bazel_skylib//rules:build_test.bzl", "build_test") +load("@rules_pkg//pkg:tar.bzl", "pkg_tar") +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") +load("//tests/support:support.bzl", "SUPPORTS_BOOTSTRAP_SCRIPT") + +build_test( + name = "bzl_libraries_build_test", + targets = [ + # keep sorted + ":bin_tar", + ], +) + +py_reconfig_test( + name = "bin", + srcs = ["bin.py"], + bootstrap_impl = "script", + main = "bin.py", + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, + # Needed until https://github.com/bazelbuild/rules_pkg/issues/929 is fixed + # See: https://github.com/bazel-contrib/rules_python/issues/2489 + venvs_use_declare_symlink = "no", +) + +pkg_tar( + name = "bin_tar", + testonly = True, + srcs = [":bin"], + include_runfiles = True, +) diff --git a/tests/packaging/bin.py b/tests/packaging/bin.py new file mode 100644 index 0000000000..2f9a147db1 --- /dev/null +++ b/tests/packaging/bin.py @@ -0,0 +1 @@ +print("Hello") diff --git a/tests/py_runtime/py_runtime_tests.bzl b/tests/py_runtime/py_runtime_tests.bzl index d5a6076153..47cd97f142 100644 --- a/tests/py_runtime/py_runtime_tests.bzl +++ b/tests/py_runtime/py_runtime_tests.bzl @@ -13,22 +13,56 @@ # limitations under the License. """Starlark tests for py_runtime rule.""" -load("@rules_python_internal//:rules_python_config.bzl", "config") load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:truth.bzl", "matching") load("@rules_testing//lib:util.bzl", rt_util = "util") load("//python:py_runtime.bzl", "py_runtime") load("//python:py_runtime_info.bzl", "PyRuntimeInfo") -load("//tests/base_rules:util.bzl", br_util = "util") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility +load("//python/private:py_runtime_rule.bzl", "coverage_tool_missing_message") # buildifier: disable=bzl-visibility load("//tests/support:py_runtime_info_subject.bzl", "py_runtime_info_subject") -load("//tests/support:support.bzl", "PYTHON_VERSION") _tests = [] +_basic_tests = [] -_SKIP_TEST = { - "target_compatible_with": ["@platforms//:incompatible"], -} +def _test_coverage_warning_when_enabled_and_no_tool(env): + # The only case that warrants a warning: coverage is being collected and the + # runtime that was selected cannot produce any. + msg = coverage_tool_missing_message( + coverage_enabled = True, + coverage_tool = None, + label = Label("//fake:runtime"), + ) + env.expect.that_bool(msg == None).equals(False) + env.expect.that_str(msg).contains("has no coverage_tool") + env.expect.that_str(msg).contains("//fake:runtime") + +_basic_tests.append(_test_coverage_warning_when_enabled_and_no_tool) + +def _test_no_coverage_warning_when_coverage_disabled(env): + # Without `bazel coverage` the missing tool has no observable effect, so + # saying anything would be noise on every ordinary build. + msg = coverage_tool_missing_message( + coverage_enabled = False, + coverage_tool = None, + label = Label("//fake:runtime"), + ) + env.expect.that_bool(msg == None).equals(True) + +_basic_tests.append(_test_no_coverage_warning_when_coverage_disabled) + +def _test_no_coverage_warning_when_tool_present(env): + # A configured coverage tool is the working case, whether it came from the + # bundled wheel set or from py_runtime.coverage_tool directly. + msg = coverage_tool_missing_message( + coverage_enabled = True, + coverage_tool = "some-coverage-tool-file", + label = Label("//fake:runtime"), + ) + env.expect.that_bool(msg == None).equals(True) + +_basic_tests.append(_test_no_coverage_warning_when_tool_present) def _simple_binary_impl(ctx): executable = ctx.actions.declare_file(ctx.label.name) @@ -48,28 +82,33 @@ _simple_binary = rule( executable = True, ) -def _test_bootstrap_template(name): - # The bootstrap_template arg isn't present in older Bazel versions, so - # we have to conditionally pass the arg and mark the test incompatible. - if config.enable_pystar: - py_runtime_kwargs = {"bootstrap_template": "bootstrap.txt"} - attr_values = {} - else: - py_runtime_kwargs = {} - attr_values = _SKIP_TEST +def _source_file_wrapper_impl(ctx): + return [DefaultInfo( + files = depset([ctx.file.src]), + runfiles = ctx.runfiles(files = ctx.files.data), + )] + +_source_file_wrapper = rule( + implementation = _source_file_wrapper_impl, + attrs = { + "data": attr.label_list(allow_files = True), + "src": attr.label(allow_single_file = True, mandatory = True), + }, +) +def _test_bootstrap_template(name): rt_util.helper_target( py_runtime, name = name + "_subject", interpreter_path = "/py", python_version = "PY3", - **py_runtime_kwargs + bootstrap_template = "bootstrap.txt", ) analysis_test( name = name, target = name + "_subject", impl = _test_bootstrap_template_impl, - attr_values = attr_values, + attr_values = {}, ) def _test_bootstrap_template_impl(env, target): @@ -81,29 +120,19 @@ def _test_bootstrap_template_impl(env, target): _tests.append(_test_bootstrap_template) def _test_cannot_have_both_inbuild_and_system_interpreter(name): - if br_util.is_bazel_6_or_higher(): - py_runtime_kwargs = { - "interpreter": "fake_interpreter", - "interpreter_path": "/some/path", - } - attr_values = {} - else: - py_runtime_kwargs = { - "interpreter_path": "/some/path", - } - attr_values = _SKIP_TEST rt_util.helper_target( py_runtime, name = name + "_subject", python_version = "PY3", - **py_runtime_kwargs + interpreter = "fake_interpreter", + interpreter_path = "/some/path", ) analysis_test( name = name, target = name + "_subject", impl = _test_cannot_have_both_inbuild_and_system_interpreter_impl, expect_failure = True, - attr_values = attr_values, + attr_values = {}, ) def _test_cannot_have_both_inbuild_and_system_interpreter_impl(env, target): @@ -114,25 +143,19 @@ def _test_cannot_have_both_inbuild_and_system_interpreter_impl(env, target): _tests.append(_test_cannot_have_both_inbuild_and_system_interpreter) def _test_cannot_specify_files_for_system_interpreter(name): - if br_util.is_bazel_6_or_higher(): - py_runtime_kwargs = {"files": ["foo.txt"]} - attr_values = {} - else: - py_runtime_kwargs = {} - attr_values = _SKIP_TEST rt_util.helper_target( py_runtime, name = name + "_subject", interpreter_path = "/foo", python_version = "PY3", - **py_runtime_kwargs + files = ["foo.txt"], ) analysis_test( name = name, target = name + "_subject", impl = _test_cannot_specify_files_for_system_interpreter_impl, expect_failure = True, - attr_values = attr_values, + attr_values = {}, ) def _test_cannot_specify_files_for_system_interpreter_impl(env, target): @@ -143,21 +166,12 @@ def _test_cannot_specify_files_for_system_interpreter_impl(env, target): _tests.append(_test_cannot_specify_files_for_system_interpreter) def _test_coverage_tool_executable(name): - if br_util.is_bazel_6_or_higher(): - py_runtime_kwargs = { - "coverage_tool": name + "_coverage_tool", - } - attr_values = {} - else: - py_runtime_kwargs = {} - attr_values = _SKIP_TEST - rt_util.helper_target( py_runtime, name = name + "_subject", python_version = "PY3", interpreter_path = "/bogus", - **py_runtime_kwargs + coverage_tool = name + "_coverage_tool", ) rt_util.helper_target( _simple_binary, @@ -168,7 +182,7 @@ def _test_coverage_tool_executable(name): name = name, target = name + "_subject", impl = _test_coverage_tool_executable_impl, - attr_values = attr_values, + attr_values = {}, ) def _test_coverage_tool_executable_impl(env, target): @@ -183,14 +197,10 @@ def _test_coverage_tool_executable_impl(env, target): _tests.append(_test_coverage_tool_executable) def _test_coverage_tool_plain_files(name): - if br_util.is_bazel_6_or_higher(): - py_runtime_kwargs = { - "coverage_tool": name + "_coverage_tool", - } - attr_values = {} - else: - py_runtime_kwargs = {} - attr_values = _SKIP_TEST + py_runtime_kwargs = { + "coverage_tool": name + "_coverage_tool", + } + attr_values = {} rt_util.helper_target( py_runtime, name = name + "_subject", @@ -239,11 +249,54 @@ def _test_in_build_interpreter(name): def _test_in_build_interpreter_impl(env, target): info = env.expect.that_target(target).provider(PyRuntimeInfo, factory = py_runtime_info_subject) info.python_version().equals("PY3") - info.files().contains_predicate(matching.file_basename_equals("file1.txt")) + info.files().contains_exactly([ + "{package}/fake_interpreter", + "{package}/file1.txt", + ]) info.interpreter().path().contains("fake_interpreter") + env.expect.that_bool(info.actual.interpreter_files_to_run == None).equals(True) _tests.append(_test_in_build_interpreter) +def _test_non_executable_source_file_interpreter_keeps_file_only_behavior(name): + rt_util.helper_target( + _source_file_wrapper, + name = name + "_wrapped_interpreter", + src = "fake_interpreter", + data = ["runfile.txt"], + ) + + rt_util.helper_target( + py_runtime, + name = name + "_subject", + interpreter = name + "_wrapped_interpreter", + python_version = "PY3", + files = ["file1.txt"], + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_non_executable_source_file_interpreter_keeps_file_only_behavior_impl, + ) + +def _test_non_executable_source_file_interpreter_keeps_file_only_behavior_impl(env, target): + target = env.expect.that_target(target) + py_runtime_info = target.provider( + PyRuntimeInfo, + factory = py_runtime_info_subject, + ) + py_runtime_info.interpreter().short_path_equals("{package}/fake_interpreter") + env.expect.that_bool(py_runtime_info.actual.interpreter_files_to_run == None).equals(True) + py_runtime_info.files().contains_exactly([ + "{package}/file1.txt", + ]) + + target.default_outputs().contains_exactly([ + "{package}/file1.txt", + ]) + +_tests.append(_test_non_executable_source_file_interpreter_keeps_file_only_behavior) + def _test_interpreter_binary_with_multiple_outputs(name): rt_util.helper_target( _simple_binary, @@ -271,6 +324,9 @@ def _test_interpreter_binary_with_multiple_outputs_impl(env, target): factory = py_runtime_info_subject, ) py_runtime_info.interpreter().short_path_equals("{package}/{test_name}_built_interpreter") + py_runtime_info.interpreter_files_to_run().executable().short_path_equals( + "{package}/{test_name}_built_interpreter", + ) py_runtime_info.files().contains_exactly([ "{package}/extra_default_output.txt", "{package}/runfile.txt", @@ -316,6 +372,9 @@ def _test_interpreter_binary_with_single_output_and_runfiles_impl(env, target): factory = py_runtime_info_subject, ) py_runtime_info.interpreter().short_path_equals("{package}/{test_name}_built_interpreter") + py_runtime_info.interpreter_files_to_run().executable().short_path_equals( + "{package}/{test_name}_built_interpreter", + ) py_runtime_info.files().contains_exactly([ "{package}/runfile.txt", "{package}/{test_name}_built_interpreter", @@ -334,14 +393,8 @@ def _test_interpreter_binary_with_single_output_and_runfiles_impl(env, target): _tests.append(_test_interpreter_binary_with_single_output_and_runfiles) def _test_must_have_either_inbuild_or_system_interpreter(name): - if br_util.is_bazel_6_or_higher(): - py_runtime_kwargs = {} - attr_values = {} - else: - py_runtime_kwargs = { - "interpreter_path": "/some/path", - } - attr_values = _SKIP_TEST + py_runtime_kwargs = {} + attr_values = {} rt_util.helper_target( py_runtime, name = name + "_subject", @@ -377,22 +430,18 @@ def _test_system_interpreter(name): ) def _test_system_interpreter_impl(env, target): - env.expect.that_target(target).provider( + info = env.expect.that_target(target).provider( PyRuntimeInfo, factory = py_runtime_info_subject, - ).interpreter_path().equals("/system/python") + ) + info.interpreter_path().equals("/system/python") + env.expect.that_bool(info.actual.interpreter_files_to_run == None).equals(True) _tests.append(_test_system_interpreter) def _test_system_interpreter_must_be_absolute(name): - # Bazel 5.4 will entirely crash when an invalid interpreter_path - # is given. - if br_util.is_bazel_6_or_higher(): - py_runtime_kwargs = {"interpreter_path": "relative/path"} - attr_values = {} - else: - py_runtime_kwargs = {"interpreter_path": "/junk/value/for/bazel5.4"} - attr_values = _SKIP_TEST + py_runtime_kwargs = {"interpreter_path": "relative/path"} + attr_values = {} rt_util.helper_target( py_runtime, name = name + "_subject", @@ -415,28 +464,19 @@ def _test_system_interpreter_must_be_absolute_impl(env, target): _tests.append(_test_system_interpreter_must_be_absolute) def _interpreter_version_info_test(name, interpreter_version_info, impl, expect_failure = True): - if config.enable_pystar: - py_runtime_kwargs = { - "interpreter_version_info": interpreter_version_info, - } - attr_values = {} - else: - py_runtime_kwargs = {} - attr_values = _SKIP_TEST - rt_util.helper_target( py_runtime, name = name + "_subject", python_version = "PY3", interpreter_path = "/py", - **py_runtime_kwargs + interpreter_version_info = interpreter_version_info, ) analysis_test( name = name, target = name + "_subject", impl = impl, expect_failure = expect_failure, - attr_values = attr_values, + attr_values = {}, ) def _test_interpreter_version_info_must_define_major_and_minor_only_major(name): @@ -530,9 +570,6 @@ def _test_interpreter_version_info_parses_values_to_struct_impl(env, target): _tests.append(_test_interpreter_version_info_parses_values_to_struct) def _test_version_info_from_flag(name): - if not config.enable_pystar: - rt_util.skip_test(name) - return py_runtime( name = name + "_subject", interpreter_version_info = None, @@ -543,7 +580,7 @@ def _test_version_info_from_flag(name): target = name + "_subject", impl = _test_version_info_from_flag_impl, config_settings = { - PYTHON_VERSION: "3.12", + labels.PYTHON_VERSION: "3.12", }, ) @@ -560,5 +597,6 @@ _tests.append(_test_version_info_from_flag) def py_runtime_test_suite(name): test_suite( name = name, + basic_tests = _basic_tests, tests = _tests, ) diff --git a/tests/py_runtime_info/py_runtime_info_tests.bzl b/tests/py_runtime_info/py_runtime_info_tests.bzl index 9acf541683..6b9ecac605 100644 --- a/tests/py_runtime_info/py_runtime_info_tests.bzl +++ b/tests/py_runtime_info/py_runtime_info_tests.bzl @@ -15,19 +15,16 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("@rules_testing//lib:truth.bzl", "matching") load("//python:py_runtime_info.bzl", "PyRuntimeInfo") -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility +load("//tests/support:py_runtime_info_subject.bzl", "py_runtime_info_subject") def _create_py_runtime_info_without_interpreter_version_info_impl(ctx): - kwargs = {} - if IS_BAZEL_7_OR_HIGHER: - kwargs["bootstrap_template"] = ctx.attr.bootstrap_template - return [PyRuntimeInfo( interpreter = ctx.file.interpreter, files = depset(ctx.files.files), python_version = "PY3", - **kwargs + bootstrap_template = ctx.attr.bootstrap_template, )] _create_py_runtime_info_without_interpreter_version_info = rule( @@ -40,6 +37,57 @@ _create_py_runtime_info_without_interpreter_version_info = rule( }, ) +def _simple_binary_impl(ctx): + executable = ctx.actions.declare_file(ctx.label.name) + ctx.actions.write(executable, "", is_executable = True) + return [DefaultInfo( + executable = executable, + files = depset([executable]), + )] + +_simple_binary = rule( + implementation = _simple_binary_impl, + executable = True, +) + +def _file_target_impl(ctx): + output = ctx.actions.declare_file(ctx.label.name + ".txt") + ctx.actions.write(output, "") + return [DefaultInfo(files = depset([output]))] + +_file_target = rule( + implementation = _file_target_impl, +) + +def _create_py_runtime_info_with_interpreter_files_to_run_impl(ctx): + files_to_run = ctx.attr.files_to_run[DefaultInfo].files_to_run + kwargs = dict( + bootstrap_template = ctx.file.bootstrap_template, + interpreter_files_to_run = files_to_run, + python_version = "PY3", + ) + if ctx.attr.use_interpreter_path: + kwargs["interpreter_path"] = "/python" + else: + kwargs["files"] = depset() + kwargs["interpreter"] = ctx.executable.interpreter + + return [PyRuntimeInfo(**kwargs)] + +_create_py_runtime_info_with_interpreter_files_to_run = rule( + implementation = _create_py_runtime_info_with_interpreter_files_to_run_impl, + attrs = { + "bootstrap_template": attr.label(allow_single_file = True, default = "bootstrap.txt"), + "files_to_run": attr.label(mandatory = True), + "interpreter": attr.label( + cfg = "target", + executable = True, + mandatory = True, + ), + "use_interpreter_path": attr.bool(), + }, +) + _tests = [] def _test_can_create_py_runtime_info_without_interpreter_version_info(name): @@ -58,6 +106,112 @@ def _test_can_create_py_runtime_info_without_interpreter_version_info_impl(env, _tests.append(_test_can_create_py_runtime_info_without_interpreter_version_info) +def _test_interpreter_files_to_run_with_interpreter(name): + _simple_binary( + name = name + "_interpreter", + ) + _create_py_runtime_info_with_interpreter_files_to_run( + name = name + "_subject", + files_to_run = name + "_interpreter", + interpreter = name + "_interpreter", + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_interpreter_files_to_run_with_interpreter_impl, + ) + +def _test_interpreter_files_to_run_with_interpreter_impl(env, target): + info = env.expect.that_target(target).provider( + PyRuntimeInfo, + factory = py_runtime_info_subject, + ) + info.interpreter().short_path_equals("{package}/{test_name}_interpreter") + info.interpreter_files_to_run().executable().short_path_equals( + "{package}/{test_name}_interpreter", + ) + +_tests.append(_test_interpreter_files_to_run_with_interpreter) + +def _test_interpreter_files_to_run_disallows_interpreter_path(name): + _simple_binary( + name = name + "_interpreter", + ) + _create_py_runtime_info_with_interpreter_files_to_run( + name = name + "_subject", + files_to_run = name + "_interpreter", + interpreter = name + "_interpreter", + tags = ["manual"], + use_interpreter_path = True, + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_interpreter_files_to_run_disallows_interpreter_path_impl, + expect_failure = True, + ) + +def _test_interpreter_files_to_run_disallows_interpreter_path_impl(env, target): + env.expect.that_target(target).failures().contains_predicate( + matching.str_matches("*interpreter_files_to_run*interpreter_path*"), + ) + +_tests.append(_test_interpreter_files_to_run_disallows_interpreter_path) + +def _test_interpreter_files_to_run_requires_executable(name): + _simple_binary( + name = name + "_interpreter", + ) + _file_target( + name = name + "_files_to_run", + ) + _create_py_runtime_info_with_interpreter_files_to_run( + name = name + "_subject", + files_to_run = name + "_files_to_run", + interpreter = name + "_interpreter", + tags = ["manual"], + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_interpreter_files_to_run_requires_executable_impl, + expect_failure = True, + ) + +def _test_interpreter_files_to_run_requires_executable_impl(env, target): + env.expect.that_target(target).failures().contains_predicate( + matching.str_matches("*interpreter_files_to_run*executable*"), + ) + +_tests.append(_test_interpreter_files_to_run_requires_executable) + +def _test_interpreter_files_to_run_requires_matching_interpreter(name): + _simple_binary( + name = name + "_interpreter", + ) + _simple_binary( + name = name + "_other_interpreter", + ) + _create_py_runtime_info_with_interpreter_files_to_run( + name = name + "_subject", + files_to_run = name + "_other_interpreter", + interpreter = name + "_interpreter", + tags = ["manual"], + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_interpreter_files_to_run_requires_matching_interpreter_impl, + expect_failure = True, + ) + +def _test_interpreter_files_to_run_requires_matching_interpreter_impl(env, target): + env.expect.that_target(target).failures().contains_predicate( + matching.str_matches("*interpreter_files_to_run.executable*interpreter*"), + ) + +_tests.append(_test_interpreter_files_to_run_requires_matching_interpreter) + def py_runtime_info_test_suite(name): test_suite( name = name, diff --git a/tests/py_wheel/py_wheel_tests.bzl b/tests/py_wheel/py_wheel_tests.bzl index 091e01c37d..75fef3a622 100644 --- a/tests/py_wheel/py_wheel_tests.bzl +++ b/tests/py_wheel/py_wheel_tests.bzl @@ -14,10 +14,10 @@ """Test for py_wheel.""" load("@rules_testing//lib:analysis_test.bzl", "analysis_test", "test_suite") -load("@rules_testing//lib:truth.bzl", "matching") +load("@rules_testing//lib:truth.bzl", "matching", "subjects") load("@rules_testing//lib:util.bzl", rt_util = "util") load("//python:packaging.bzl", "py_wheel") -load("//python/private:py_wheel_normalize_pep440.bzl", "normalize_pep440") # buildifier: disable=bzl-visibility +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility _basic_tests = [] _tests = [] @@ -168,105 +168,43 @@ def _test_content_type_from_description_impl(env, target): _tests.append(_test_content_type_from_description) -def _test_pep440_normalization(env): - prefixes = ["v", " v", " \t\r\nv"] - epochs = { - "": ["", "0!", "00!"], - "1!": ["1!", "001!"], - "200!": ["200!", "00200!"], - } - releases = { - "0.1": ["0.1", "0.01"], - "2023.7.19": ["2023.7.19", "2023.07.19"], - } - pres = { - "": [""], - "a0": ["a", ".a", "-ALPHA0", "_alpha0", ".a0"], - "a4": ["alpha4", ".a04"], - "b0": ["b", ".b", "-BETA0", "_beta0", ".b0"], - "b5": ["beta05", ".b5"], - "rc0": ["C", "_c0", "RC", "_rc0", "-preview_0"], - } - explicit_posts = { - "": [""], - ".post0": [], - ".post1": [".post1", "-r1", "_rev1"], - } - implicit_posts = [[".post1", "-1"], [".post2", "-2"]] - devs = { - "": [""], - ".dev0": ["dev", "-DEV", "_Dev-0"], - ".dev9": ["DEV9", ".dev09", ".dev9"], - ".dev{BUILD_TIMESTAMP}": [ - "-DEV{BUILD_TIMESTAMP}", - "_dev_{BUILD_TIMESTAMP}", - ], - } - locals = { - "": [""], - "+ubuntu.7": ["+Ubuntu_7", "+ubuntu-007"], - "+ubuntu.r007": ["+Ubuntu_R007"], - } - epochs = [ - [normalized_epoch, input_epoch] - for normalized_epoch, input_epochs in epochs.items() - for input_epoch in input_epochs - ] - releases = [ - [normalized_release, input_release] - for normalized_release, input_releases in releases.items() - for input_release in input_releases - ] - pres = [ - [normalized_pre, input_pre] - for normalized_pre, input_pres in pres.items() - for input_pre in input_pres - ] - explicit_posts = [ - [normalized_post, input_post] - for normalized_post, input_posts in explicit_posts.items() - for input_post in input_posts - ] - pres_and_posts = [ - [normalized_pre + normalized_post, input_pre + input_post] - for normalized_pre, input_pre in pres - for normalized_post, input_post in explicit_posts - ] + [ - [normalized_pre + normalized_post, input_pre + input_post] - for normalized_pre, input_pre in pres - for normalized_post, input_post in implicit_posts - if input_pre == "" or input_pre[-1].isdigit() - ] - devs = [ - [normalized_dev, input_dev] - for normalized_dev, input_devs in devs.items() - for input_dev in input_devs - ] - locals = [ - [normalized_local, input_local] - for normalized_local, input_locals in locals.items() - for input_local in input_locals - ] - postfixes = ["", " ", " \t\r\n"] - i = 0 - for nepoch, iepoch in epochs: - for nrelease, irelease in releases: - for nprepost, iprepost in pres_and_posts: - for ndev, idev in devs: - for nlocal, ilocal in locals: - prefix = prefixes[i % len(prefixes)] - postfix = postfixes[(i // len(prefixes)) % len(postfixes)] - env.expect.that_str( - normalize_pep440( - prefix + iepoch + irelease + iprepost + - idev + ilocal + postfix, - ), - ).equals( - nepoch + nrelease + nprepost + ndev + nlocal, - ) - i += 1 +def _test_config_settings(name): + rt_util.helper_target( + native.config_setting, + name = "is_py_39", + flag_values = { + labels.PYTHON_VERSION_MAJOR_MINOR: "3.9", + }, + ) + rt_util.helper_target( + py_wheel, + name = name + "_subject", + distribution = "mydist_" + name, + version = select({ + ":is_py_39": "3.9", + "//conditions:default": "not-3.9", + }), + config_settings = { + labels.PYTHON_VERSION: "3.9", + }, + ) + analysis_test( + name = name, + impl = _test_config_settings_impl, + target = name + "_subject", + config_settings = { + # Ensure a different value than the target under test. + labels.PYTHON_VERSION: "3.11", + }, + ) + +def _test_config_settings_impl(env, target): + env.expect.that_target(target).attr( + "version", + factory = subjects.str, + ).equals("3.9") -_basic_tests.append(_test_pep440_normalization) +_tests.append(_test_config_settings) def py_wheel_test_suite(name): test_suite( diff --git a/tests/py_zipapp/BUILD.bazel b/tests/py_zipapp/BUILD.bazel new file mode 100644 index 0000000000..a68448d964 --- /dev/null +++ b/tests/py_zipapp/BUILD.bazel @@ -0,0 +1,116 @@ +load("@rules_shell//shell:sh_test.bzl", "sh_test") +load("//python:py_binary.bzl", "py_binary") +load("//python:py_library.bzl", "py_library") +load("//python:py_test.bzl", "py_test") +load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility +load("//python/zipapp:py_zipapp_binary.bzl", "py_zipapp_binary") + +py_binary( + name = "venv_bin", + srcs = ["main.py"], + config_settings = { + "//python/config_settings:venvs_site_packages": "yes", + } | select({ + "@platforms//os:windows": {}, + "//conditions:default": { + "//python/config_settings:bootstrap_impl": "script", + }, + }), + main = "main.py", + deps = [":bin_deps"], +) + +py_zipapp_binary( + name = "venv_zipapp", + binary = ":venv_bin", +) + +py_test( + name = "venv_zipapp_test", + srcs = ["venv_zipapp_test.py"], + data = [":venv_zipapp"], + env = { + "BZLMOD_ENABLED": str(int(BZLMOD_ENABLED)), + "TEST_ZIPAPP": "$(location :venv_zipapp)", + }, +) + +# Create the app with a supported level of compression +py_zipapp_binary( + name = "venv_zipapp_compressed", + binary = ":venv_bin", + compression = "4", +) + +py_test( + name = "venv_zipapp_compressed_test", + srcs = ["venv_zipapp_test.py"], + data = [":venv_zipapp_compressed"], + env = { + "BZLMOD_ENABLED": str(int(BZLMOD_ENABLED)), + "COMPRESSED": "1", + "TEST_ZIPAPP": "$(location :venv_zipapp_compressed)", + }, + main = "venv_zipapp_test.py", +) + +py_binary( + name = "system_python_bin", + srcs = ["main.py"], + config_settings = { + "//python/config_settings:bootstrap_impl": "system_python", + "//python/config_settings:venvs_site_packages": "no", + }, + main = "main.py", + deps = [":bin_deps"], +) + +py_zipapp_binary( + name = "system_python_zipapp", + binary = ":system_python_bin", +) + +py_test( + name = "system_python_zipapp_test", + srcs = ["system_python_zipapp_test.py"], + data = [":system_python_zipapp"], + env = { + "TEST_ZIPAPP": "$(location :system_python_zipapp)", + }, +) + +sh_test( + name = "system_python_zipapp_external_bootstrap_test", + srcs = ["system_python_zipapp_external_bootstrap_test.sh"], + data = [ + ":system_python_zipapp", + "//python:current_py_toolchain", + ], + env = { + "PYTHON": "$(PYTHON3_ROOTPATH)", + "ZIPAPP": "$(location :system_python_zipapp)", + }, + toolchains = ["//python:current_py_toolchain"], +) + +py_library( + name = "bin_deps", + deps = [ + ":pkgdep", + ":some_dep", + ], +) + +py_library( + name = "some_dep", + srcs = ["some_dep.py"], + experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", + imports = ["."], +) + +py_library( + name = "pkgdep", + srcs = glob(["site-packages/**"]), + experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", + imports = ["site-packages"], +) diff --git a/tests/py_zipapp/main.py b/tests/py_zipapp/main.py new file mode 100644 index 0000000000..5770170d2c --- /dev/null +++ b/tests/py_zipapp/main.py @@ -0,0 +1,24 @@ +"A trivial zipapp that prints a message" + + +def main(): + print("Hello from zipapp") + try: + import some_dep + + print(f"dep: {some_dep}") + + import pkgdep.pkgmod + + print(f"dep: {pkgdep.pkgmod}") + except ImportError as e: + import sys + + e.add_note( + "Failed to import a dependency.\n" + "sys.path:\n" + "\n".join(sys.path) + ) + raise + + +if __name__ == "__main__": + main() diff --git a/tests/py_zipapp/site-packages/pkgdep/__init__.py b/tests/py_zipapp/site-packages/pkgdep/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/py_zipapp/site-packages/pkgdep/pkgmod.py b/tests/py_zipapp/site-packages/pkgdep/pkgmod.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/py_zipapp/some_dep.py b/tests/py_zipapp/some_dep.py new file mode 100644 index 0000000000..b64ecfb84a --- /dev/null +++ b/tests/py_zipapp/some_dep.py @@ -0,0 +1 @@ +"""empty module""" diff --git a/tests/py_zipapp/system_python_zipapp_external_bootstrap_test.sh b/tests/py_zipapp/system_python_zipapp_external_bootstrap_test.sh new file mode 100755 index 0000000000..e7396007d9 --- /dev/null +++ b/tests/py_zipapp/system_python_zipapp_external_bootstrap_test.sh @@ -0,0 +1,54 @@ +#!/usr/bin/env bash + +set -xeuo pipefail + +# This test expects ZIPAPP env var to point to the zipapp file. +if [[ -z "${ZIPAPP:-}" ]]; then + echo "ZIPAPP env var not set" + exit 1 +fi + +# On Windows, the executable file is an exe, and the .zip is a sibling +# output. +ZIPAPP="${ZIPAPP/.exe/.zip}" + +export RULES_PYTHON_BOOTSTRAP_VERBOSE=1 + +# We're testing the invocation of `__main__.py`, so we have to +# manually pass the zipapp to python. +echo "=====================================================================" +echo "Running zipapp using an automatic temp directory..." +echo "=====================================================================" +"$PYTHON" "$ZIPAPP" + +echo +echo + +echo "=====================================================================" +echo "Running zipapp with extract root set..." +echo "=====================================================================" +export RULES_PYTHON_EXTRACT_ROOT="${TEST_TMPDIR:-/tmp}/extract_root_test" +"$PYTHON" "$ZIPAPP" + +# Verify that the directory was created +if [[ ! -d "$RULES_PYTHON_EXTRACT_ROOT" ]]; then + echo "Error: Extract root directory $RULES_PYTHON_EXTRACT_ROOT was not created!" + exit 1 +fi + +# On windows, the path is shortened to just the basename to avoid long path errors. +# Other platforms use the full path. +# Note: [ -d ... ] expands globs, while [[ -d ... ]] does not. +if [ -d "$RULES_PYTHON_EXTRACT_ROOT/_main/tests/py_zipapp/system_python_zipapp"/*/runfiles ]; then + echo "Found runfiles at $RULES_PYTHON_EXTRACT_ROOT/_main/tests/py_zipapp/system_python_zipapp/*/runfiles" +elif [ -d "$RULES_PYTHON_EXTRACT_ROOT/system_python_zipapp"/*/runfiles ]; then + echo "Found runfiles at $RULES_PYTHON_EXTRACT_ROOT/system_python_zipapp/*/runfiles" +else + echo "Error: Could not find 'runfiles' directory" + exit 1 +fi + +echo "=====================================================================" +echo "Running zipapp with extract root set a second time..." +echo "=====================================================================" +"$PYTHON" "$ZIPAPP" diff --git a/tests/py_zipapp/system_python_zipapp_test.py b/tests/py_zipapp/system_python_zipapp_test.py new file mode 100644 index 0000000000..7c3e2deeaf --- /dev/null +++ b/tests/py_zipapp/system_python_zipapp_test.py @@ -0,0 +1,32 @@ +import os +import subprocess +import unittest + + +class SystemPythonZipAppTest(unittest.TestCase): + def test_zipapp_runnable(self): + zipapp_path = os.environ["TEST_ZIPAPP"] + + self.assertTrue(os.path.exists(zipapp_path)) + self.assertTrue(os.path.isfile(zipapp_path)) + + try: + output = ( + subprocess.check_output([zipapp_path], stderr=subprocess.STDOUT) + .decode("utf-8") + .strip() + ) + except subprocess.CalledProcessError as e: + self.fail( + "exit code: {}\n" + " command: {}\n" + "===== stdout/stderr start ==={}===== stdout/stderr end ====".format( + e.returncode, e.cmd, e.output.decode("utf-8") + ) + ) + self.assertIn("Hello from zipapp", output) + self.assertIn("dep:", output) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/py_zipapp/venv_zipapp_test.py b/tests/py_zipapp/venv_zipapp_test.py new file mode 100644 index 0000000000..bd26d533a3 --- /dev/null +++ b/tests/py_zipapp/venv_zipapp_test.py @@ -0,0 +1,109 @@ +import contextlib +import os +import subprocess +import unittest +import zipfile + + +class PyZipAppTest(unittest.TestCase): + def test_zipapp_runnable(self): + zipapp_path = os.environ["TEST_ZIPAPP"] + + try: + output = ( + subprocess.check_output([zipapp_path], stderr=subprocess.STDOUT) + .decode("utf-8") + .strip() + ) + except subprocess.CalledProcessError as e: + self.fail( + ( + "exec failed: {}\n" + + "exit code: {}\n" + + "=== stdout/stderr start ===\n" + "{}\n" + "=== stdout/stderr end ===" + ).format(zipapp_path, e.returncode, e.output.decode("utf-8")) + ) + self.assertIn("Hello from zipapp", output) + self.assertIn("dep:", output) + + def assertHasPathMatchingSuffix(self, namelist, suffix, msg=None): + if not any(name.endswith(suffix) for name in namelist): + self.fail( + (msg or f"No path in zipapp matching suffix '{suffix}'") + + "\nAvailable paths:\n" + + "\n".join(namelist) + ) + + def assertZipEntryIsSymlink(self, zip_file, path, msg=None): + try: + info = zip_file.getinfo(path) + except KeyError: + self.fail(msg or f"Path '{path}' not found in zipfile") + + # S_IFLNK is 0o120000. + # ZipInfo.external_attr is 32 bits: the high 16 bits are Unix attributes. + is_symlink = (info.external_attr >> 16) & 0o170000 == 0o120000 + if not is_symlink: + self.fail(msg or f"Path '{path}' is not a symlink") + + def _is_bzlmod_enabled(self): + return os.environ["BZLMOD_ENABLED"] == "1" + + @contextlib.contextmanager + def _open_zipapp(self, path): + zf = None + try: + try: + zf = zipfile.ZipFile(path, "r") + except zipfile.BadZipFile: + # On windows, the main output is the launcher .exe file, and the + # zip file is a sibling file. + path = path.replace(".exe", ".zip") + zf = zipfile.ZipFile(path, "r") + if zf: + yield zf + finally: + if zf: + zf.close() + + def test_zipapp_structure(self): + zipapp_path = os.environ["TEST_ZIPAPP"] + + with self._open_zipapp(zipapp_path) as zf: + info = zf.infolist()[0] + if os.getenv("COMPRESSED", "0") == "1": + self.assertEqual(info.compress_type, zipfile.ZIP_DEFLATED) + else: + self.assertEqual(info.compress_type, zipfile.ZIP_STORED) + + namelist = zf.namelist() + + if self._is_bzlmod_enabled(): + self.assertIn("runfiles/_repo_mapping", namelist) + + # On Windows, pyvenv.cfg and bin/python3 are generated at runtime. + if os.name != "nt": + self.assertHasPathMatchingSuffix(namelist, "/pyvenv.cfg") + + # The venv directory name depends on the target name, so find it + # by looking for pyvenv.cfg. + venv_config = next( + (name for name in namelist if name.endswith("/pyvenv.cfg")), None + ) + self.assertIsNotNone(venv_config) + + venv_root = os.path.dirname(venv_config) + + # Verify bin/python3 exists and is a symlink + python_bin = f"{venv_root}/bin/python3" + self.assertZipEntryIsSymlink(zf, python_bin) + + # Verify _bazel_site_init.py exists in site-packages + self.assertHasPathMatchingSuffix( + namelist, "/site-packages/_bazel_site_init.py" + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/pycross/0001-Add-new-file-for-testing-patch-support.patch b/tests/pycross/0001-Add-new-file-for-testing-patch-support.patch deleted file mode 100644 index fcbc3096ef..0000000000 --- a/tests/pycross/0001-Add-new-file-for-testing-patch-support.patch +++ /dev/null @@ -1,17 +0,0 @@ -From b2ebe6fe67ff48edaf2ae937d24b1f0b67c16f81 Mon Sep 17 00:00:00 2001 -From: Philipp Schrader -Date: Thu, 28 Sep 2023 09:02:44 -0700 -Subject: [PATCH] Add new file for testing patch support - ---- - site-packages/numpy/file_added_via_patch.txt | 1 + - 1 file changed, 1 insertion(+) - create mode 100644 site-packages/numpy/file_added_via_patch.txt - -diff --git a/site-packages/numpy/file_added_via_patch.txt b/site-packages/numpy/file_added_via_patch.txt -new file mode 100644 -index 0000000..9d947a4 ---- /dev/null -+++ b/site-packages/numpy/file_added_via_patch.txt -@@ -0,0 +1 @@ -+Hello from a patch! diff --git a/tests/pycross/BUILD.bazel b/tests/pycross/BUILD.bazel deleted file mode 100644 index 52d1d18480..0000000000 --- a/tests/pycross/BUILD.bazel +++ /dev/null @@ -1,64 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -load("//python:defs.bzl", "py_test") -load("//third_party/rules_pycross/pycross/private:wheel_library.bzl", "py_wheel_library") # buildifier: disable=bzl-visibility - -py_wheel_library( - name = "extracted_wheel_for_testing", - wheel = "@wheel_for_testing//file", -) - -py_test( - name = "py_wheel_library_test", - srcs = [ - "py_wheel_library_test.py", - ], - data = [ - ":extracted_wheel_for_testing", - ], - deps = [ - "//python/runfiles", - ], -) - -py_wheel_library( - name = "patched_extracted_wheel_for_testing", - patch_args = [ - "-p1", - ], - patch_tool = "patch", - patches = [ - "0001-Add-new-file-for-testing-patch-support.patch", - ], - target_compatible_with = select({ - # We don't have `patch` available on the Windows CI machines. - "@platforms//os:windows": ["@platforms//:incompatible"], - "//conditions:default": [], - }), - wheel = "@wheel_for_testing//file", -) - -py_test( - name = "patched_py_wheel_library_test", - srcs = [ - "patched_py_wheel_library_test.py", - ], - data = [ - ":patched_extracted_wheel_for_testing", - ], - deps = [ - "//python/runfiles", - ], -) diff --git a/tests/pycross/patched_py_wheel_library_test.py b/tests/pycross/patched_py_wheel_library_test.py deleted file mode 100644 index e1b404a0ef..0000000000 --- a/tests/pycross/patched_py_wheel_library_test.py +++ /dev/null @@ -1,40 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -import unittest -from pathlib import Path - -from python.runfiles import runfiles - -RUNFILES = runfiles.Create() - - -class TestPyWheelLibrary(unittest.TestCase): - def setUp(self): - self.extraction_dir = Path( - RUNFILES.Rlocation( - "rules_python/tests/pycross/patched_extracted_wheel_for_testing" - ) - ) - self.assertTrue(self.extraction_dir.exists(), self.extraction_dir) - self.assertTrue(self.extraction_dir.is_dir(), self.extraction_dir) - - def test_patched_file_contents(self): - """Validate that the patch got applied correctly.""" - file = self.extraction_dir / "site-packages/numpy/file_added_via_patch.txt" - self.assertEqual(file.read_text(), "Hello from a patch!\n") - - -if __name__ == "__main__": - unittest.main() diff --git a/tests/pycross/py_wheel_library_test.py b/tests/pycross/py_wheel_library_test.py deleted file mode 100644 index 25d896a1ae..0000000000 --- a/tests/pycross/py_wheel_library_test.py +++ /dev/null @@ -1,46 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -import unittest -from pathlib import Path - -from python.runfiles import runfiles - -RUNFILES = runfiles.Create() - - -class TestPyWheelLibrary(unittest.TestCase): - def setUp(self): - self.extraction_dir = Path( - RUNFILES.Rlocation("rules_python/tests/pycross/extracted_wheel_for_testing") - ) - self.assertTrue(self.extraction_dir.exists(), self.extraction_dir) - self.assertTrue(self.extraction_dir.is_dir(), self.extraction_dir) - - def test_file_presence(self): - """Validate that the basic file layout looks good.""" - for path in ( - "bin/f2py", - "site-packages/numpy.libs/libgfortran-daac5196.so.5.0.0", - "site-packages/numpy/dtypes.py", - "site-packages/numpy/core/_umath_tests.cpython-311-aarch64-linux-gnu.so", - ): - print(self.extraction_dir / path) - self.assertTrue( - (self.extraction_dir / path).exists(), f"{path} does not exist" - ) - - -if __name__ == "__main__": - unittest.main() diff --git a/tests/pypi/argparse/BUILD.bazel b/tests/pypi/argparse/BUILD.bazel new file mode 100644 index 0000000000..b04da685e6 --- /dev/null +++ b/tests/pypi/argparse/BUILD.bazel @@ -0,0 +1,3 @@ +load(":argparse_tests.bzl", "argparse_test_suite") + +argparse_test_suite(name = "argparse_tests") diff --git a/tests/pypi/argparse/argparse_tests.bzl b/tests/pypi/argparse/argparse_tests.bzl new file mode 100644 index 0000000000..f8fd1c9481 --- /dev/null +++ b/tests/pypi/argparse/argparse_tests.bzl @@ -0,0 +1,48 @@ +"" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private/pypi:argparse.bzl", "argparse") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_index_url(env): + env.expect.that_str(argparse.index_url([], "default")).equals("default") + env.expect.that_str(argparse.index_url([], None)).equals(None) + + env.expect.that_str(argparse.index_url(["-i", "https://example.com/simple"], "default")).equals("https://example.com/simple") + env.expect.that_str(argparse.index_url(["--index-url", "https://example.com/simple"], "default")).equals("https://example.com/simple") + env.expect.that_str(argparse.index_url(["--index-url=https://example.com/simple"], "default")).equals("https://example.com/simple") + + env.expect.that_str(argparse.index_url(["--extra-index-url", "https://extra.com", "-i", "https://index.com"], "default")).equals("https://index.com") + +_tests.append(_test_index_url) + +def _test_extra_index_url(env): + env.expect.that_collection(argparse.extra_index_url([], ["default"])).contains_exactly(["default"]) + env.expect.that_collection(argparse.extra_index_url([], None)).contains_exactly([]) + + env.expect.that_collection(argparse.extra_index_url(["--extra-index-url", "https://extra.com/simple"], [])).contains_exactly(["https://extra.com/simple"]) + env.expect.that_collection(argparse.extra_index_url(["--extra-index-url=https://extra.com/simple"], [])).contains_exactly(["https://extra.com/simple"]) + + env.expect.that_collection(argparse.extra_index_url(["--extra-index-url", "https://first.com", "--extra-index-url", "https://second.com"], [])).contains_exactly(["https://first.com", "https://second.com"]) + +_tests.append(_test_extra_index_url) + +def _test_platform(env): + env.expect.that_collection(argparse.platform([], ["default"])).contains_exactly(["default"]) + env.expect.that_collection(argparse.platform([], None)).contains_exactly([]) + + env.expect.that_collection(argparse.platform(["--platform", "manylinux_2_17_x86_64"], [])).contains_exactly(["manylinux_2_17_x86_64"]) + env.expect.that_collection(argparse.platform(["--platform=manylinux_2_17_x86_64"], [])).contains_exactly(["manylinux_2_17_x86_64"]) + + env.expect.that_collection(argparse.platform(["--platform", "macosx_10_9_x86_64", "--platform", "linux_x86_64"], [])).contains_exactly(["macosx_10_9_x86_64", "linux_x86_64"]) + +_tests.append(_test_platform) + +def argparse_test_suite(name): + """Create the test suite. + + Args: + name: the name of the test suite + """ + test_suite(name = name, basic_tests = _tests) diff --git a/tests/pypi/config_settings/config_settings_tests.bzl b/tests/pypi/config_settings/config_settings_tests.bzl index a77fa5b66b..c7e51b78b4 100644 --- a/tests/pypi/config_settings/config_settings_tests.bzl +++ b/tests/pypi/config_settings/config_settings_tests.bzl @@ -31,13 +31,7 @@ _subject = rule( ) _flag = struct( - platform = lambda x: ("//command_line_option:platforms", str(Label("//tests/support:" + x))), - pip_whl = lambda x: (str(Label("//python/config_settings:pip_whl")), str(x)), - pip_whl_glibc_version = lambda x: (str(Label("//python/config_settings:pip_whl_glibc_version")), str(x)), - pip_whl_muslc_version = lambda x: (str(Label("//python/config_settings:pip_whl_muslc_version")), str(x)), - pip_whl_osx_version = lambda x: (str(Label("//python/config_settings:pip_whl_osx_version")), str(x)), - pip_whl_osx_arch = lambda x: (str(Label("//python/config_settings:pip_whl_osx_arch")), str(x)), - py_linux_libc = lambda x: (str(Label("//python/config_settings:py_linux_libc")), str(x)), + platform = lambda x: ("//command_line_option:platforms", str(Label("//tests/support/platforms:" + x))), python_version = lambda x: (str(Label("//python/config_settings:python_version")), str(x)), ) @@ -71,455 +65,30 @@ def _match(env, target, want): _tests = [] -# Tests when we only have an `sdist` present. +# Legacy pip config setting tests -def _test_sdist_default(name): +def _test_legacy_default(name): _analysis_test( name = name, dist = { - "is_cp3.7_sdist": "sdist", + "is_cp37": "legacy", }, - want = "sdist", + want = "legacy", ) -_tests.append(_test_sdist_default) +_tests.append(_test_legacy_default) -def _test_sdist_no_whl(name): +def _test_legacy_with_constraint_values(name): _analysis_test( name = name, dist = { - "is_cp3.7_sdist": "sdist", + "is_cp37": "legacy", + "is_cp37_linux_aarch64": "legacy_platform_override", }, - config_settings = [ - _flag.platform("linux_aarch64"), - _flag.pip_whl("no"), - ], - want = "sdist", + want = "legacy_platform_override", ) -_tests.append(_test_sdist_no_whl) - -def _test_sdist_no_sdist(name): - _analysis_test( - name = name, - dist = { - "is_cp3.7_sdist": "sdist", - }, - config_settings = [ - _flag.platform("linux_aarch64"), - _flag.pip_whl("only"), - ], - # We will use `no_match_error` in the real case to indicate that `sdist` is not - # allowed to be used. - want = "no_match", - ) - -_tests.append(_test_sdist_no_sdist) - -def _test_basic_whl_default(name): - _analysis_test( - name = name, - dist = { - "is_cp3.7_py_none_any": "whl", - "is_cp3.7_sdist": "sdist", - }, - want = "whl", - ) - -_tests.append(_test_basic_whl_default) - -def _test_basic_whl_nowhl(name): - _analysis_test( - name = name, - dist = { - "is_cp3.7_py_none_any": "whl", - "is_cp3.7_sdist": "sdist", - }, - config_settings = [ - _flag.platform("linux_aarch64"), - _flag.pip_whl("no"), - ], - want = "sdist", - ) - -_tests.append(_test_basic_whl_nowhl) - -def _test_basic_whl_nosdist(name): - _analysis_test( - name = name, - dist = { - "is_cp3.7_py_none_any": "whl", - "is_cp3.7_sdist": "sdist", - }, - config_settings = [ - _flag.platform("linux_aarch64"), - _flag.pip_whl("only"), - ], - want = "whl", - ) - -_tests.append(_test_basic_whl_nosdist) - -def _test_whl_default(name): - _analysis_test( - name = name, - dist = { - "is_cp3.7_py3_none_any": "whl", - "is_cp3.7_py_none_any": "basic_whl", - }, - want = "whl", - ) - -_tests.append(_test_whl_default) - -def _test_whl_nowhl(name): - _analysis_test( - name = name, - dist = { - "is_cp3.7_py3_none_any": "whl", - "is_cp3.7_py_none_any": "basic_whl", - }, - config_settings = [ - _flag.platform("linux_aarch64"), - _flag.pip_whl("no"), - ], - want = "no_match", - ) - -_tests.append(_test_whl_nowhl) - -def _test_whl_nosdist(name): - _analysis_test( - name = name, - dist = { - "is_cp3.7_py3_none_any": "whl", - }, - config_settings = [ - _flag.platform("linux_aarch64"), - _flag.pip_whl("only"), - ], - want = "whl", - ) - -_tests.append(_test_whl_nosdist) - -def _test_abi_whl_is_prefered(name): - _analysis_test( - name = name, - dist = { - "is_cp3.7_py3_abi3_any": "abi_whl", - "is_cp3.7_py3_none_any": "whl", - }, - want = "abi_whl", - ) - -_tests.append(_test_abi_whl_is_prefered) - -def _test_whl_with_constraints_is_prefered(name): - _analysis_test( - name = name, - dist = { - "is_cp3.7_py3_none_any": "default_whl", - "is_cp3.7_py3_none_any_linux_aarch64": "whl", - "is_cp3.7_py3_none_any_linux_x86_64": "amd64_whl", - }, - want = "whl", - ) - -_tests.append(_test_whl_with_constraints_is_prefered) - -def _test_cp_whl_is_prefered_over_py3(name): - _analysis_test( - name = name, - dist = { - "is_cp3.7_cp3x_none_any": "cp", - "is_cp3.7_py3_abi3_any": "py3_abi3", - "is_cp3.7_py3_none_any": "py3", - }, - want = "cp", - ) - -_tests.append(_test_cp_whl_is_prefered_over_py3) - -def _test_cp_abi_whl_is_prefered_over_py3(name): - _analysis_test( - name = name, - dist = { - "is_cp3.7_cp3x_abi3_any": "cp", - "is_cp3.7_py3_abi3_any": "py3", - }, - want = "cp", - ) - -_tests.append(_test_cp_abi_whl_is_prefered_over_py3) - -def _test_cp_version_is_selected_when_python_version_is_specified(name): - _analysis_test( - name = name, - dist = { - "is_cp3.10_cp3x_none_any": "cp310", - "is_cp3.8_cp3x_none_any": "cp38", - "is_cp3.9_cp3x_none_any": "cp39", - }, - want = "cp310", - config_settings = [ - _flag.python_version("3.10.9"), - _flag.platform("linux_aarch64"), - ], - ) - -_tests.append(_test_cp_version_is_selected_when_python_version_is_specified) - -def _test_py_none_any_versioned(name): - _analysis_test( - name = name, - dist = { - "is_cp3.10_py_none_any": "whl", - "is_cp3.9_py_none_any": "too-low", - }, - want = "whl", - config_settings = [ - _flag.python_version("3.10.9"), - _flag.platform("linux_aarch64"), - ], - ) - -_tests.append(_test_py_none_any_versioned) - -def _test_cp_cp_whl(name): - _analysis_test( - name = name, - dist = { - "is_cp3.10_cp3x_cp_linux_aarch64": "whl", - }, - want = "whl", - config_settings = [ - _flag.python_version("3.10.9"), - _flag.platform("linux_aarch64"), - ], - ) - -_tests.append(_test_cp_cp_whl) - -def _test_cp_version_sdist_is_selected(name): - _analysis_test( - name = name, - dist = { - "is_cp3.10_sdist": "sdist", - }, - want = "sdist", - config_settings = [ - _flag.python_version("3.10.9"), - _flag.platform("linux_aarch64"), - ], - ) - -_tests.append(_test_cp_version_sdist_is_selected) - -def _test_platform_whl_is_prefered_over_any_whl_with_constraints(name): - _analysis_test( - name = name, - dist = { - "is_cp3.7_py3_abi3_any": "better_default_whl", - "is_cp3.7_py3_abi3_any_linux_aarch64": "better_default_any_whl", - "is_cp3.7_py3_none_any": "default_whl", - "is_cp3.7_py3_none_any_linux_aarch64": "whl", - "is_cp3.7_py3_none_linux_aarch64": "platform_whl", - }, - want = "platform_whl", - ) - -_tests.append(_test_platform_whl_is_prefered_over_any_whl_with_constraints) - -def _test_abi3_platform_whl_preference(name): - _analysis_test( - name = name, - dist = { - "is_cp3.7_py3_abi3_linux_aarch64": "abi3_platform", - "is_cp3.7_py3_none_linux_aarch64": "platform", - }, - want = "abi3_platform", - ) - -_tests.append(_test_abi3_platform_whl_preference) - -def _test_glibc(name): - _analysis_test( - name = name, - dist = { - "is_cp3.7_cp3x_cp_manylinux_aarch64": "glibc", - "is_cp3.7_py3_abi3_linux_aarch64": "abi3_platform", - }, - want = "glibc", - ) - -_tests.append(_test_glibc) - -def _test_glibc_versioned(name): - _analysis_test( - name = name, - dist = { - "is_cp3.7_cp3x_cp_manylinux_2_14_aarch64": "glibc", - "is_cp3.7_cp3x_cp_manylinux_2_17_aarch64": "glibc", - "is_cp3.7_py3_abi3_linux_aarch64": "abi3_platform", - }, - want = "glibc", - config_settings = [ - _flag.py_linux_libc("glibc"), - _flag.pip_whl_glibc_version("2.17"), - _flag.platform("linux_aarch64"), - ], - ) - -_tests.append(_test_glibc_versioned) - -def _test_glibc_compatible_exists(name): - _analysis_test( - name = name, - dist = { - # Code using the conditions will need to construct selects, which - # do the version matching correctly. - "is_cp3.7_cp3x_cp_manylinux_2_14_aarch64": "2_14_whl_via_2_14_branch", - "is_cp3.7_cp3x_cp_manylinux_2_17_aarch64": "2_14_whl_via_2_17_branch", - }, - want = "2_14_whl_via_2_17_branch", - config_settings = [ - _flag.py_linux_libc("glibc"), - _flag.pip_whl_glibc_version("2.17"), - _flag.platform("linux_aarch64"), - ], - ) - -_tests.append(_test_glibc_compatible_exists) - -def _test_musl(name): - _analysis_test( - name = name, - dist = { - "is_cp3.7_cp3x_cp_musllinux_aarch64": "musl", - }, - want = "musl", - config_settings = [ - _flag.py_linux_libc("musl"), - _flag.platform("linux_aarch64"), - ], - ) - -_tests.append(_test_musl) - -def _test_windows(name): - _analysis_test( - name = name, - dist = { - "is_cp3.7_cp3x_cp_windows_x86_64": "whl", - }, - want = "whl", - config_settings = [ - _flag.platform("windows_x86_64"), - ], - ) - -_tests.append(_test_windows) - -def _test_osx(name): - _analysis_test( - name = name, - dist = { - # We prefer arch specific whls over universal - "is_cp3.7_cp3x_cp_osx_x86_64": "whl", - "is_cp3.7_cp3x_cp_osx_x86_64_universal2": "universal_whl", - }, - want = "whl", - config_settings = [ - _flag.platform("mac_x86_64"), - ], - ) - -_tests.append(_test_osx) - -def _test_osx_universal_default(name): - _analysis_test( - name = name, - dist = { - # We default to universal if only that exists - "is_cp3.7_cp3x_cp_osx_x86_64_universal2": "whl", - }, - want = "whl", - config_settings = [ - _flag.platform("mac_x86_64"), - ], - ) - -_tests.append(_test_osx_universal_default) - -def _test_osx_universal_only(name): - _analysis_test( - name = name, - dist = { - # If we prefer universal, then we use that - "is_cp3.7_cp3x_cp_osx_x86_64": "whl", - "is_cp3.7_cp3x_cp_osx_x86_64_universal2": "universal", - }, - want = "universal", - config_settings = [ - _flag.pip_whl_osx_arch("universal"), - _flag.platform("mac_x86_64"), - ], - ) - -_tests.append(_test_osx_universal_only) - -def _test_osx_os_version(name): - _analysis_test( - name = name, - dist = { - # Similarly to the libc version, the user of the config settings will have to - # construct the select so that the version selection is correct. - "is_cp3.7_cp3x_cp_osx_10_9_x86_64": "whl", - }, - want = "whl", - config_settings = [ - _flag.pip_whl_osx_version("10.9"), - _flag.platform("mac_x86_64"), - ], - ) - -_tests.append(_test_osx_os_version) - -def _test_all(name): - _analysis_test( - name = name, - dist = { - "is_cp3.7_" + f: f - for f in [ - "{py}_{abi}_{plat}".format(py = valid_py, abi = valid_abi, plat = valid_plat) - # we have py2.py3, py3, cp3x - for valid_py in ["py", "py3", "cp3x"] - # cp abi usually comes with a version and we only need one - # config setting variant for all of them because the python - # version will discriminate between different versions. - for valid_abi in ["none", "abi3", "cp"] - for valid_plat in [ - "any", - "manylinux_2_17_x86_64", - "manylinux_2_17_aarch64", - "osx_x86_64", - "windows_x86_64", - ] - if not ( - valid_abi == "abi3" and valid_py == "py" or - valid_abi == "cp" and valid_py != "cp3x" - ) - ] - }, - want = "cp3x_cp_manylinux_2_17_x86_64", - config_settings = [ - _flag.pip_whl_glibc_version("2.17"), - _flag.platform("linux_x86_64"), - ], - ) - -_tests.append(_test_all) +_tests.append(_test_legacy_with_constraint_values) def config_settings_test_suite(name): # buildifier: disable=function-docstring test_suite( @@ -530,16 +99,34 @@ def config_settings_test_suite(name): # buildifier: disable=function-docstring config_settings( name = "dummy", python_versions = ["3.7", "3.8", "3.9", "3.10"], - glibc_versions = [(2, 14), (2, 17)], - muslc_versions = [(1, 1)], - osx_versions = [(10, 9), (11, 0)], - target_platforms = [ - "windows_x86_64", - "windows_aarch64", - "linux_x86_64", - "linux_ppc", - "linux_aarch64", - "osx_x86_64", - "osx_aarch64", - ], + platform_config_settings = { + "linux_aarch64": [ + "@platforms//cpu:aarch64", + "@platforms//os:linux", + ], + "linux_ppc": [ + "@platforms//cpu:ppc", + "@platforms//os:linux", + ], + "linux_x86_64": [ + "@platforms//cpu:x86_64", + "@platforms//os:linux", + ], + "osx_aarch64": [ + "@platforms//cpu:aarch64", + "@platforms//os:osx", + ], + "osx_x86_64": [ + "@platforms//cpu:x86_64", + "@platforms//os:osx", + ], + "windows_aarch64": [ + "@platforms//cpu:aarch64", + "@platforms//os:windows", + ], + "windows_x86_64": [ + "@platforms//cpu:x86_64", + "@platforms//os:windows", + ], + }, ) diff --git a/tests/pypi/env_marker_setting/BUILD.bazel b/tests/pypi/env_marker_setting/BUILD.bazel new file mode 100644 index 0000000000..9605e650ce --- /dev/null +++ b/tests/pypi/env_marker_setting/BUILD.bazel @@ -0,0 +1,5 @@ +load(":env_marker_setting_tests.bzl", "env_marker_setting_test_suite") + +env_marker_setting_test_suite( + name = "env_marker_setting_tests", +) diff --git a/tests/pypi/env_marker_setting/env_marker_setting_tests.bzl b/tests/pypi/env_marker_setting/env_marker_setting_tests.bzl new file mode 100644 index 0000000000..c5b3f72d8c --- /dev/null +++ b/tests/pypi/env_marker_setting/env_marker_setting_tests.bzl @@ -0,0 +1,104 @@ +"""env_marker_setting tests.""" + +load("@rules_testing//lib:analysis_test.bzl", "analysis_test") +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("@rules_testing//lib:util.bzl", "TestingAspectInfo") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility +load("//python/private/pypi:env_marker_info.bzl", "EnvMarkerInfo") # buildifier: disable=bzl-visibility +load("//python/private/pypi:env_marker_setting.bzl", "env_marker_setting") # buildifier: disable=bzl-visibility + +def _custom_env_markers_impl(ctx): + _ = ctx # @unused + return [EnvMarkerInfo(env = { + "os_name": "testos", + })] + +_custom_env_markers = rule( + implementation = _custom_env_markers_impl, +) + +_tests = [] + +def _test_custom_env_markers(name): + def _impl(env, target): + env.expect.where( + expression = target[TestingAspectInfo].attrs.expression, + ).that_str( + target[config_common.FeatureFlagInfo].value, + ).equals("TRUE") + + env_marker_setting( + name = name + "_subject", + expression = "os_name == 'testos'", + ) + _custom_env_markers(name = name + "_env") + analysis_test( + name = name, + impl = _impl, + target = name + "_subject", + config_settings = { + labels.PIP_ENV_MARKER_CONFIG: str(Label(name + "_env")), + }, + ) + +_tests.append(_test_custom_env_markers) + +def _test_expr(name): + def impl(env, target): + env.expect.where( + expression = target[TestingAspectInfo].attrs.expression, + ).that_str( + target[config_common.FeatureFlagInfo].value, + ).equals( + env.ctx.attr.expected, + ) + + cases = { + "python_full_version_lt_negative": { + "config_settings": { + labels.PYTHON_VERSION: "3.12.0", + }, + "expected": "FALSE", + "expression": "python_full_version < '3.8'", + }, + "python_version_gte": { + "config_settings": { + labels.PYTHON_VERSION: "3.12.0", + }, + "expected": "TRUE", + "expression": "python_version >= '3.12.0'", + }, + } + + tests = [] + for case_name, case in cases.items(): + test_name = name + "_" + case_name + tests.append(test_name) + env_marker_setting( + name = test_name + "_subject", + expression = case["expression"], + ) + analysis_test( + name = test_name, + impl = impl, + target = test_name + "_subject", + config_settings = case["config_settings"], + attr_values = { + "expected": case["expected"], + }, + attrs = { + "expected": attr.string(), + }, + ) + native.test_suite( + name = name, + tests = tests, + ) + +_tests.append(_test_expr) + +def env_marker_setting_test_suite(name): + test_suite( + name = name, + tests = _tests, + ) diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index b9427795ec..97849e882d 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -16,42 +16,97 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:truth.bzl", "subjects") -load("//python/private/pypi:extension.bzl", "parse_modules") # buildifier: disable=bzl-visibility +load("//python/private/pypi:extension.bzl", "build_config", "default_platforms", "parse_modules") # buildifier: disable=bzl-visibility +load("//python/private/pypi:platform.bzl", _plat = "platform") # buildifier: disable=bzl-visibility load("//python/private/pypi:whl_config_setting.bzl", "whl_config_setting") # buildifier: disable=bzl-visibility +load("//tests/support/mocks:mocks.bzl", "mocks") +load(":pip_parse.bzl", _parse = "pip_parse") _tests = [] -def _mock_mctx(*modules, environ = {}, read = None): +def _pypi_mock_mctx(*modules, os_name = "unittest", arch_name = "exotic", environ = {}, read = None, mock_files = {}): + _ = read # @unused + return mocks.mctx( + modules = list(modules), + os_name = os_name, + arch_name = arch_name, + environ = environ, + mock_files = { + "requirements.txt": """\ +simple==0.0.1 \ + --hash=sha256:deadbeef \ + --hash=sha256:deadbaaf""", + } | mock_files, + ) + +def _default( + *, + arch_name = None, + auth_patterns = None, + config_settings = None, + default_hub = "", + env = None, + index_url = None, + marker = None, + netrc = None, + os_name = None, + platform = None, + pyproject_toml = None, + whl_platform_tags = None, + whl_abi_tags = None): return struct( - os = struct( - environ = environ, - name = "unittest", - arch = "exotic", - ), - read = read or (lambda _: "simple==0.0.1 --hash=sha256:deadbeef --hash=sha256:deadbaaf"), - modules = [ - struct( - name = modules[0].name, - tags = modules[0].tags, - is_root = modules[0].is_root, - ), - ] + [ - struct( - name = mod.name, - tags = mod.tags, - is_root = False, - ) - for mod in modules[1:] + arch_name = arch_name, + auth_patterns = auth_patterns or {}, + config_settings = config_settings, + default_hub = default_hub, + env = env or {}, + index_url = index_url or "", + marker = marker or "", + netrc = netrc, + os_name = os_name, + platform = platform, + pyproject_toml = pyproject_toml, + whl_abi_tags = whl_abi_tags or [], + whl_platform_tags = whl_platform_tags or [], + ) + +# The default value for the default platforms tags use in `_mod`. +_default_tags_default = [ + _default( + platform = "{}_{}{}".format(os, cpu, freethreaded), + os_name = os, + arch_name = cpu, + config_settings = [ + "@platforms//os:{}".format(os), + "@platforms//cpu:{}".format(cpu), ], + whl_abi_tags = ["cp{major}{minor}t"] if freethreaded else ["abi3", "cp{major}{minor}"], + whl_platform_tags = whl_platform_tags, + ) + for (os, cpu, freethreaded), whl_platform_tags in { + ("linux", "x86_64", ""): ["linux_x86_64", "manylinux_*_x86_64"], + ("linux", "x86_64", "_freethreaded"): ["linux_x86_64", "manylinux_*_x86_64"], + ("linux", "aarch64", ""): ["linux_aarch64", "manylinux_*_aarch64"], + ("osx", "aarch64", ""): ["macosx_*_arm64"], + ("windows", "aarch64", ""): ["win_arm64"], + }.items() +] + +def _dep(*, name, extra_targets = []): + return struct( + name = name, + extra_targets = extra_targets, ) -def _mod(*, name, parse = [], override = [], whl_mods = [], is_root = True): +def _mod(*, name, default = _default_tags_default, parse = [], override = [], whl_mods = [], dep = [], is_root = True): return struct( name = name, tags = struct( parse = parse, override = override, whl_mods = whl_mods, + default = default, + dep = dep, ), is_root = is_root, ) @@ -60,7 +115,8 @@ def _parse_modules(env, **kwargs): return env.expect.that_struct( parse_modules(**kwargs), attrs = dict( - is_reproducible = subjects.bool, + declared_deps = subjects.dict, + default_hub = subjects.str, exposed_packages = subjects.dict, hub_group_map = subjects.dict, hub_whl_map = subjects.dict, @@ -69,92 +125,43 @@ def _parse_modules(env, **kwargs): ), ) -def _parse( - *, - hub_name, - python_version, - _evaluate_markers_srcs = [], - auth_patterns = {}, - download_only = False, - enable_implicit_namespace_pkgs = False, - environment = {}, - envsubst = {}, - experimental_index_url = "", - experimental_requirement_cycles = {}, - experimental_target_platforms = [], - extra_hub_aliases = {}, - extra_pip_args = [], - isolated = True, - netrc = None, - parse_all_requirements_files = True, - pip_data_exclude = None, - python_interpreter = None, - python_interpreter_target = None, - quiet = True, - requirements_by_platform = {}, - requirements_darwin = None, - requirements_linux = None, - requirements_lock = None, - requirements_windows = None, - timeout = 600, - whl_modifications = {}, - **kwargs): - return struct( - _evaluate_markers_srcs = _evaluate_markers_srcs, - auth_patterns = auth_patterns, - download_only = download_only, - enable_implicit_namespace_pkgs = enable_implicit_namespace_pkgs, - environment = environment, - envsubst = envsubst, - experimental_index_url = experimental_index_url, - experimental_requirement_cycles = experimental_requirement_cycles, - experimental_target_platforms = experimental_target_platforms, - extra_hub_aliases = extra_hub_aliases, - extra_pip_args = extra_pip_args, - hub_name = hub_name, - isolated = isolated, - netrc = netrc, - parse_all_requirements_files = parse_all_requirements_files, - pip_data_exclude = pip_data_exclude, - python_interpreter = python_interpreter, - python_interpreter_target = python_interpreter_target, - python_version = python_version, - quiet = quiet, - requirements_by_platform = requirements_by_platform, - requirements_darwin = requirements_darwin, - requirements_linux = requirements_linux, - requirements_lock = requirements_lock, - requirements_windows = requirements_windows, - timeout = timeout, - whl_modifications = whl_modifications, - # The following are covered by other unit tests - experimental_extra_index_urls = [], - parallel_download = False, - experimental_index_url_overrides = {}, - **kwargs +def _build_config(env, **kwargs): + return env.expect.that_struct( + build_config( + enable_pipstar_extract = True, + **kwargs + ), + attrs = dict( + auth_patterns = subjects.dict, + netrc = subjects.str, + platforms = subjects.dict, + ), ) def _test_simple(env): pypi = _parse_modules( env, - module_ctx = _mock_mctx( + module_ctx = _pypi_mock_mctx( _mod( name = "rules_python", parse = [ _parse( hub_name = "pypi", python_version = "3.15", + simpleapi_skip = ["simple"], requirements_lock = "requirements.txt", ), ], ), + os_name = "linux", + arch_name = "x86_64", ), available_interpreters = { "python_3_15_host": "unit_test_interpreter_target", }, + minor_mapping = {"3.15": "3.15.19"}, ) - pypi.is_reproducible().equals(True) pypi.exposed_packages().contains_exactly({"pypi": ["simple"]}) pypi.hub_group_map().contains_exactly({"pypi": {}}) pypi.hub_whl_map().contains_exactly({"pypi": { @@ -168,9 +175,9 @@ def _test_simple(env): }}) pypi.whl_libraries().contains_exactly({ "pypi_315_simple": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", "requirement": "simple==0.0.1 --hash=sha256:deadbeef --hash=sha256:deadbaaf", }, }) @@ -178,401 +185,400 @@ def _test_simple(env): _tests.append(_test_simple) -def _test_simple_multiple_requirements(env): +def _test_pip_parse_pyproject_toml(env): + # pip.parse() reads the version from pyproject.toml's requires-python when + # python_version is not set explicitly. pypi = _parse_modules( env, - module_ctx = _mock_mctx( + module_ctx = _pypi_mock_mctx( _mod( name = "rules_python", parse = [ _parse( hub_name = "pypi", - python_version = "3.15", - requirements_darwin = "darwin.txt", - requirements_windows = "win.txt", + pyproject_toml = "pyproject.toml", + simpleapi_skip = ["simple"], + requirements_lock = "requirements.txt", ), ], ), - read = lambda x: { - "darwin.txt": "simple==0.0.2 --hash=sha256:deadb00f", - "win.txt": "simple==0.0.1 --hash=sha256:deadbeef", - }[x], + os_name = "linux", + arch_name = "x86_64", + mock_files = { + "pyproject.toml": "[project]\nrequires-python = \"==3.15.19\"\n", + }, ), available_interpreters = { - "python_3_15_host": "unit_test_interpreter_target", + "python_3_15_19_host": "unit_test_interpreter_target", }, + minor_mapping = {"3.15": "3.15.19"}, ) - pypi.is_reproducible().equals(True) + # Resolves identically to passing python_version = "3.15.19" explicitly: + # the full version drives interpreter selection, hub naming uses major.minor. pypi.exposed_packages().contains_exactly({"pypi": ["simple"]}) - pypi.hub_group_map().contains_exactly({"pypi": {}}) pypi.hub_whl_map().contains_exactly({"pypi": { "simple": { - "pypi_315_simple_osx_aarch64_osx_x86_64": [ - whl_config_setting( - target_platforms = [ - "cp315_osx_aarch64", - "cp315_osx_x86_64", - ], - version = "3.15", - ), - ], - "pypi_315_simple_windows_x86_64": [ + "pypi_315_simple": [ whl_config_setting( - target_platforms = [ - "cp315_windows_x86_64", - ], version = "3.15", ), ], }, }}) pypi.whl_libraries().contains_exactly({ - "pypi_315_simple_osx_aarch64_osx_x86_64": { - "dep_template": "@pypi//{name}:{target}", - "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", - "requirement": "simple==0.0.2 --hash=sha256:deadb00f", - }, - "pypi_315_simple_windows_x86_64": { + "pypi_315_simple": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", - "requirement": "simple==0.0.1 --hash=sha256:deadbeef", + "requirement": "simple==0.0.1 --hash=sha256:deadbeef --hash=sha256:deadbaaf", }, }) - pypi.whl_mods().contains_exactly({}) -_tests.append(_test_simple_multiple_requirements) +_tests.append(_test_pip_parse_pyproject_toml) + +def _test_simple_isolated(env): + """Simulate `isolate = True` with parse_modules. -def _test_simple_with_markers(env): + No pip.default tags, but requirements parsing still produces the expected + hub output. + """ pypi = _parse_modules( env, - module_ctx = _mock_mctx( + module_ctx = _pypi_mock_mctx( _mod( - name = "rules_python", + name = "my_module", + default = [], # no platform tags parse = [ _parse( hub_name = "pypi", python_version = "3.15", - requirements_lock = "universal.txt", + simpleapi_skip = ["simple"], + requirements_lock = "requirements.txt", ), ], ), - read = lambda x: { - "universal.txt": """\ -torch==2.4.1+cpu ; platform_machine == 'x86_64' -torch==2.4.1 ; platform_machine != 'x86_64' -""", - }[x], + os_name = "linux", + arch_name = "x86_64", ), available_interpreters = { "python_3_15_host": "unit_test_interpreter_target", }, - evaluate_markers = lambda _, requirements, **__: { - key: [ - platform - for platform in platforms - if ("x86_64" in platform and "platform_machine ==" in key) or ("x86_64" not in platform and "platform_machine !=" in key) - ] - for key, platforms in requirements.items() - }, + minor_mapping = {"3.15": "3.15.19"}, ) - pypi.is_reproducible().equals(True) - pypi.exposed_packages().contains_exactly({"pypi": ["torch"]}) + pypi.exposed_packages().contains_exactly({"pypi": ["simple"]}) pypi.hub_group_map().contains_exactly({"pypi": {}}) pypi.hub_whl_map().contains_exactly({"pypi": { - "torch": { - "pypi_315_torch_linux_aarch64_linux_arm_linux_ppc_linux_s390x_osx_aarch64": [ - whl_config_setting( - target_platforms = [ - "cp315_linux_aarch64", - "cp315_linux_arm", - "cp315_linux_ppc", - "cp315_linux_s390x", - "cp315_osx_aarch64", - ], - version = "3.15", - ), - ], - "pypi_315_torch_linux_x86_64_osx_x86_64_windows_x86_64": [ + "simple": { + "pypi_315_simple": [ whl_config_setting( - target_platforms = [ - "cp315_linux_x86_64", - "cp315_osx_x86_64", - "cp315_windows_x86_64", - ], version = "3.15", ), ], }, }}) pypi.whl_libraries().contains_exactly({ - "pypi_315_torch_linux_aarch64_linux_arm_linux_ppc_linux_s390x_osx_aarch64": { - "dep_template": "@pypi//{name}:{target}", - "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", - "requirement": "torch==2.4.1 ; platform_machine != 'x86_64'", - }, - "pypi_315_torch_linux_x86_64_osx_x86_64_windows_x86_64": { + "pypi_315_simple": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", - "requirement": "torch==2.4.1+cpu ; platform_machine == 'x86_64'", + "requirement": "simple==0.0.1 --hash=sha256:deadbeef --hash=sha256:deadbaaf", }, }) pypi.whl_mods().contains_exactly({}) -_tests.append(_test_simple_with_markers) +_tests.append(_test_simple_isolated) -def _test_download_only_multiple(env): +def _test_build_pipstar_platform(env): + config = _build_config( + env, + module_ctx = _pypi_mock_mctx( + _mod( + name = "rules_python", + default = [ + _default( + platform = "myplat", + os_name = "linux", + arch_name = "x86_64", + config_settings = [ + "@platforms//os:linux", + "@platforms//cpu:x86_64", + ], + ), + _default(), + _default( + platform = "myplat2", + os_name = "linux", + arch_name = "x86_64", + config_settings = [ + "@platforms//os:linux", + "@platforms//cpu:x86_64", + ], + ), + _default(platform = "myplat2"), + _default( + netrc = "my_netrc", + auth_patterns = {"foo": "bar"}, + ), + ], + ), + ), + ) + config.auth_patterns().contains_exactly({"foo": "bar"}) + config.netrc().equals("my_netrc") + config.platforms().contains_exactly({ + "myplat": struct( + name = "myplat", + os_name = "linux", + arch_name = "x86_64", + config_settings = [ + "@platforms//os:linux", + "@platforms//cpu:x86_64", + ], + env = {"implementation_name": "cpython"}, + marker = "", + whl_abi_tags = ["none", "abi3", "cp{major}{minor}"], + whl_platform_tags = ["any"], + ), + } | { + name: _plat(**values) + for name, values in default_platforms().items() + }) + +_tests.append(_test_build_pipstar_platform) + +def _test_multiple_default_tags(env): + """Test that multiple pip.default tags do not trigger duplicate default hub failures. + + Only when multiple tags explicitly define default_hub should it fail. + """ pypi = _parse_modules( env, - module_ctx = _mock_mctx( + module_ctx = _pypi_mock_mctx( _mod( name = "rules_python", + default = _default_tags_default + [ + _default(platform = "extra_custom_platform"), + ], parse = [ _parse( hub_name = "pypi", python_version = "3.15", - download_only = True, - requirements_by_platform = { - "requirements.linux_x86_64.txt": "linux_x86_64", - "requirements.osx_aarch64.txt": "osx_aarch64", - }, + simpleapi_skip = ["simple"], + requirements_lock = "requirements.txt", ), ], ), - read = lambda x: { - "requirements.linux_x86_64.txt": """\ ---platform=manylinux_2_17_x86_64 ---python-version=315 ---implementation=cp ---abi=cp315 - -simple==0.0.1 --hash=sha256:deadbeef -extra==0.0.1 --hash=sha256:deadb00f -""", - "requirements.osx_aarch64.txt": """\ ---platform=macosx_10_9_arm64 ---python-version=315 ---implementation=cp ---abi=cp315 - -simple==0.0.3 --hash=sha256:deadbaaf -""", - }[x], + os_name = "linux", + arch_name = "x86_64", ), available_interpreters = { "python_3_15_host": "unit_test_interpreter_target", }, + minor_mapping = {"3.15": "3.15.19"}, ) - - pypi.is_reproducible().equals(True) pypi.exposed_packages().contains_exactly({"pypi": ["simple"]}) - pypi.hub_group_map().contains_exactly({"pypi": {}}) - pypi.hub_whl_map().contains_exactly({"pypi": { - "extra": { - "pypi_315_extra": [ - whl_config_setting(version = "3.15"), - ], - }, - "simple": { - "pypi_315_simple_linux_x86_64": [ - whl_config_setting( - target_platforms = ["cp315_linux_x86_64"], - version = "3.15", - ), - ], - "pypi_315_simple_osx_aarch64": [ - whl_config_setting( - target_platforms = ["cp315_osx_aarch64"], - version = "3.15", - ), - ], - }, - }}) - pypi.whl_libraries().contains_exactly({ - "pypi_315_extra": { - "dep_template": "@pypi//{name}:{target}", - "download_only": True, - "experimental_target_platforms": ["cp315_linux_x86_64"], - "extra_pip_args": ["--platform=manylinux_2_17_x86_64", "--python-version=315", "--implementation=cp", "--abi=cp315"], - "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", - "requirement": "extra==0.0.1 --hash=sha256:deadb00f", - }, - "pypi_315_simple_linux_x86_64": { - "dep_template": "@pypi//{name}:{target}", - "download_only": True, - "experimental_target_platforms": ["cp315_linux_x86_64"], - "extra_pip_args": ["--platform=manylinux_2_17_x86_64", "--python-version=315", "--implementation=cp", "--abi=cp315"], - "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", - "requirement": "simple==0.0.1 --hash=sha256:deadbeef", - }, - "pypi_315_simple_osx_aarch64": { - "dep_template": "@pypi//{name}:{target}", - "download_only": True, - "experimental_target_platforms": ["cp315_osx_aarch64"], - "extra_pip_args": ["--platform=macosx_10_9_arm64", "--python-version=315", "--implementation=cp", "--abi=cp315"], - "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", - "requirement": "simple==0.0.3 --hash=sha256:deadbaaf", - }, - }) - pypi.whl_mods().contains_exactly({}) -_tests.append(_test_download_only_multiple) - -def _test_simple_get_index(env): - got_simpleapi_download_args = [] - got_simpleapi_download_kwargs = {} - - def mocksimpleapi_download(*args, **kwargs): - got_simpleapi_download_args.extend(args) - got_simpleapi_download_kwargs.update(kwargs) - return { - "simple": struct( - whls = { - "deadb00f": struct( - yanked = False, - filename = "simple-0.0.1-py3-none-any.whl", - sha256 = "deadb00f", - url = "example2.org", - ), - }, - sdists = { - "deadbeef": struct( - yanked = False, - filename = "simple-0.0.1.tar.gz", - sha256 = "deadbeef", - url = "example.org", +_tests.append(_test_multiple_default_tags) + +def _test_name_collision_no_env(env): + """Test that a hub named 'pypi' is NOT renamed when the env var is not set.""" + pypi = _parse_modules( + env, + module_ctx = _pypi_mock_mctx( + _mod( + name = "rules_python", + parse = [ + _parse( + hub_name = "pypi", + python_version = "3.15", + simpleapi_skip = ["simple"], + requirements_lock = "requirements.txt", ), - }, + ], ), - } + os_name = "linux", + arch_name = "x86_64", + environ = {}, # Env var NOT set + ), + available_interpreters = { + "python_3_15_host": "unit_test_interpreter_target", + }, + minor_mapping = {"3.15": "3.15.19"}, + ) + + # The hub name remains 'pypi' + pypi.exposed_packages().contains_exactly({"pypi": ["simple"]}) + pypi.default_hub().equals(None) + +_tests.append(_test_name_collision_no_env) +def _test_name_collision_with_env(env): + """Test that a hub named 'pypi' is silently renamed to module_name_pypi and routed as default_hub when the env var is set.""" pypi = _parse_modules( env, - module_ctx = _mock_mctx( + module_ctx = _pypi_mock_mctx( _mod( name = "rules_python", parse = [ _parse( hub_name = "pypi", python_version = "3.15", + simpleapi_skip = ["simple"], requirements_lock = "requirements.txt", - experimental_index_url = "pypi.org", - extra_pip_args = [ - "--extra-args-for-sdist-building", - ], ), ], ), - read = lambda x: { - "requirements.txt": """ -simple==0.0.1 --hash=sha256:deadbeef --hash=sha256:deadb00f -some_pkg==0.0.1 -""", - }[x], + os_name = "linux", + arch_name = "x86_64", + environ = {"RULES_PYTHON_PYPI_HUB_RESERVED": "1"}, ), available_interpreters = { "python_3_15_host": "unit_test_interpreter_target", }, - simpleapi_download = mocksimpleapi_download, + minor_mapping = {"3.15": "3.15.19"}, ) - pypi.is_reproducible().equals(False) - pypi.exposed_packages().contains_exactly({"pypi": ["simple", "some_pkg"]}) - pypi.hub_group_map().contains_exactly({"pypi": {}}) - pypi.hub_whl_map().contains_exactly({ - "pypi": { - "simple": { - "pypi_315_simple_py3_none_any_deadb00f": [ - whl_config_setting( - filename = "simple-0.0.1-py3-none-any.whl", - version = "3.15", + # The hub name is renamed to 'rules_python_pypi' + pypi.exposed_packages().contains_exactly({"rules_python_pypi": ["simple"]}) + + # It is used as the default_hub + pypi.default_hub().equals("rules_python_pypi") + +_tests.append(_test_name_collision_with_env) + +def _test_default_hub_precedence(env): + """Test that pip.default(default_hub = ...) has precedence over the fallback renamed default hub.""" + pypi = _parse_modules( + env, + module_ctx = _pypi_mock_mctx( + _mod( + name = "rules_python", + default = _default_tags_default + [ + _default( + platform = "extra_custom_platform", + default_hub = "other_pypi", ), ], - "pypi_315_simple_sdist_deadbeef": [ - whl_config_setting( - filename = "simple-0.0.1.tar.gz", - version = "3.15", + parse = [ + _parse( + hub_name = "pypi", + python_version = "3.15", + simpleapi_skip = ["simple"], + requirements_lock = "requirements.txt", + ), + _parse( + hub_name = "other_pypi", + python_version = "3.15", + simpleapi_skip = ["simple"], + requirements_lock = "requirements.txt", ), ], - }, - "some_pkg": { - "pypi_315_some_pkg": [whl_config_setting(version = "3.15")], - }, + ), + os_name = "linux", + arch_name = "x86_64", + environ = {"RULES_PYTHON_PYPI_HUB_RESERVED": "1"}, + ), + available_interpreters = { + "python_3_15_host": "unit_test_interpreter_target", }, + minor_mapping = {"3.15": "3.15.19"}, + ) + + # The hub named 'pypi' is renamed to 'rules_python_pypi' + pypi.exposed_packages().contains_exactly({ + "other_pypi": ["simple"], + "rules_python_pypi": ["simple"], }) - pypi.whl_libraries().contains_exactly({ - "pypi_315_simple_py3_none_any_deadb00f": { - "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": [ - "cp315_linux_aarch64", - "cp315_linux_arm", - "cp315_linux_ppc", - "cp315_linux_s390x", - "cp315_linux_x86_64", - "cp315_osx_aarch64", - "cp315_osx_x86_64", - "cp315_windows_x86_64", - ], - "filename": "simple-0.0.1-py3-none-any.whl", - "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", - "requirement": "simple==0.0.1", - "sha256": "deadb00f", - "urls": ["example2.org"], + + # But the default_hub remains 'other_pypi' because pip.default has higher precedence! + pypi.default_hub().equals("other_pypi") + +_tests.append(_test_default_hub_precedence) + +def _test_extension_dep(env): + pypi = _parse_modules( + env, + module_ctx = _pypi_mock_mctx( + _mod( + name = "my_module", + dep = [ + _dep( + name = "declared-pkg", + extra_targets = ["declared-alias"], + ), + ], + ), + os_name = "linux", + arch_name = "x86_64", + ), + available_interpreters = {}, + minor_mapping = {}, + ) + + pypi.declared_deps().contains_exactly({"declared_pkg": {"declared-alias": None}}) + pypi.exposed_packages().contains_exactly({}) + pypi.hub_group_map().contains_exactly({}) + pypi.hub_whl_map().contains_exactly({}) + pypi.whl_libraries().contains_exactly({}) + pypi.whl_mods().contains_exactly({}) + +_tests.append(_test_extension_dep) + +def _test_extension_dep_coexists_with_concrete_hub(env): + pypi = _parse_modules( + env, + module_ctx = _pypi_mock_mctx( + _mod( + name = "my_module", + parse = [ + _parse( + hub_name = "pypi_a", + python_version = "3.15", + simpleapi_skip = ["simple"], + requirements_lock = "requirements.txt", + ), + ], + dep = [ + _dep( + name = "simple", + extra_targets = ["extra-target"], + ), + ], + ), + os_name = "linux", + arch_name = "x86_64", + ), + available_interpreters = { + "python_3_15_host": "unit_test_interpreter_target", }, - "pypi_315_simple_sdist_deadbeef": { - "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": [ - "cp315_linux_aarch64", - "cp315_linux_arm", - "cp315_linux_ppc", - "cp315_linux_s390x", - "cp315_linux_x86_64", - "cp315_osx_aarch64", - "cp315_osx_x86_64", - "cp315_windows_x86_64", + minor_mapping = {"3.15": "3.15.19"}, + ) + + pypi.declared_deps().contains_exactly({"simple": {"extra-target": None}}) + pypi.exposed_packages().contains_exactly({"pypi_a": ["simple"]}) + pypi.hub_group_map().contains_exactly({"pypi_a": {}}) + pypi.hub_whl_map().contains_exactly({"pypi_a": { + "simple": { + "pypi_a_315_simple": [ + whl_config_setting( + version = "3.15", + ), ], - "extra_pip_args": ["--extra-args-for-sdist-building"], - "filename": "simple-0.0.1.tar.gz", - "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", - "requirement": "simple==0.0.1", - "sha256": "deadbeef", - "urls": ["example.org"], }, - # We are falling back to regular `pip` - "pypi_315_some_pkg": { - "dep_template": "@pypi//{name}:{target}", - "extra_pip_args": ["--extra-args-for-sdist-building"], + }}) + pypi.whl_libraries().contains_exactly({ + "pypi_a_315_simple": { + "config_load": "@pypi_a//:config.bzl", + "dep_template": "@pypi_a//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", - "requirement": "some_pkg==0.0.1", + "requirement": "simple==0.0.1 --hash=sha256:deadbeef --hash=sha256:deadbaaf", }, }) pypi.whl_mods().contains_exactly({}) - env.expect.that_dict(got_simpleapi_download_kwargs).contains_exactly({ - "attr": struct( - auth_patterns = {}, - envsubst = {}, - extra_index_urls = [], - index_url = "pypi.org", - index_url_overrides = {}, - netrc = None, - sources = ["simple", "some_pkg"], - ), - "cache": {}, - "parallel_download": False, - }) -_tests.append(_test_simple_get_index) +_tests.append(_test_extension_dep_coexists_with_concrete_hub) def extension_test_suite(name): """Create the test suite. diff --git a/tests/pypi/extension/pip_parse.bzl b/tests/pypi/extension/pip_parse.bzl new file mode 100644 index 0000000000..7b5bdfdfd6 --- /dev/null +++ b/tests/pypi/extension/pip_parse.bzl @@ -0,0 +1,73 @@ +"""A simple test helper""" + +def pip_parse( + *, + hub_name, + python_version = None, + add_libdir_to_library_search_path = False, + auth_patterns = {}, + download_only = False, + enable_implicit_namespace_pkgs = False, + environment = {}, + envsubst = {}, + experimental_extra_index_urls = [], + experimental_index_url = "", + experimental_requirement_cycles = {}, + extra_hub_aliases = {}, + extra_pip_args = [], + isolated = True, + netrc = None, + parse_all_requirements_files = True, + pip_data_exclude = None, + pyproject_toml = None, + python_interpreter = None, + python_interpreter_target = None, + quiet = True, + requirements_by_platform = {}, + requirements_darwin = None, + requirements_linux = None, + requirements_lock = None, + requirements_windows = None, + target_platforms = [], + simpleapi_skip = [], + timeout = 600, + uv_lock = None, + whl_modifications = {}, + **kwargs): + """A simple helper for testing to simulate the PyPI extension parse tag class""" + return struct( + auth_patterns = auth_patterns, + add_libdir_to_library_search_path = add_libdir_to_library_search_path, + download_only = download_only, + enable_implicit_namespace_pkgs = enable_implicit_namespace_pkgs, + environment = environment, + envsubst = envsubst, + experimental_extra_index_urls = experimental_extra_index_urls, + experimental_index_url = experimental_index_url, + experimental_requirement_cycles = experimental_requirement_cycles, + target_platforms = target_platforms, + extra_hub_aliases = extra_hub_aliases, + extra_pip_args = extra_pip_args, + hub_name = hub_name, + isolated = isolated, + netrc = netrc, + parse_all_requirements_files = parse_all_requirements_files, + pip_data_exclude = pip_data_exclude, + pyproject_toml = pyproject_toml, + python_interpreter = python_interpreter, + python_interpreter_target = python_interpreter_target, + python_version = python_version, + quiet = quiet, + requirements_by_platform = requirements_by_platform, + requirements_darwin = requirements_darwin, + requirements_linux = requirements_linux, + requirements_lock = requirements_lock, + requirements_windows = requirements_windows, + timeout = timeout, + uv_lock = uv_lock, + whl_modifications = whl_modifications, + parallel_download = False, + experimental_index_url_overrides = {}, + simpleapi_skip = simpleapi_skip, + **kwargs + ) diff --git a/tests/pypi/generate_group_library_build_bazel/generate_group_library_build_bazel_tests.bzl b/tests/pypi/generate_group_library_build_bazel/generate_group_library_build_bazel_tests.bzl index a46aa413a3..a91f861a36 100644 --- a/tests/pypi/generate_group_library_build_bazel/generate_group_library_build_bazel_tests.bzl +++ b/tests/pypi/generate_group_library_build_bazel/generate_group_library_build_bazel_tests.bzl @@ -21,7 +21,7 @@ _tests = [] def _test_simple(env): want = """\ -load("@rules_python//python:defs.bzl", "py_library") +load("@rules_python//python:py_library.bzl", "py_library") ## Group vbap @@ -62,7 +62,7 @@ _tests.append(_test_simple) def _test_in_hub(env): want = """\ -load("@rules_python//python:defs.bzl", "py_library") +load("@rules_python//python:py_library.bzl", "py_library") ## Group vbap diff --git a/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl b/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl index b0d8f6d17e..bb03d9a589 100644 --- a/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl +++ b/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl @@ -19,8 +19,10 @@ load("//python/private/pypi:generate_whl_library_build_bazel.bzl", "generate_whl _tests = [] -def _test_all(env): +def _test_all_workspace(env): want = """\ +load("@package_metadata//rules:package_metadata.bzl", "package_metadata") +load("@pypi//:config.bzl", "packages") load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets") package(default_visibility = ["//visibility:public"]) @@ -38,51 +40,115 @@ whl_library_targets( "data_exclude_all", ], dep_template = "@pypi//{name}:{target}", - dependencies = [ + group_deps = [ + "foo", + "fox", + "qux", + ], + group_name = "qux", + include = packages, + metadata_name = "foo", + name = "foo.whl", + requires_dist = [ "foo", "bar-baz", "qux", ], - dependencies_by_platform = { - "linux_x86_64": [ - "box", - "box-amd64", - ], - "windows_x86_64": ["fox"], - "@platforms//os:linux": ["box"], + srcs_exclude = ["srcs_exclude_all"], + tags = [ + "pypi_name=foo", + "pypi_version=0", + ], +) + +package_metadata( + name = "package_metadata", + purl = "foo", + visibility = ["//:__subpackages__"], +) + +# SOMETHING SPECIAL AT THE END +""" + actual = generate_whl_library_build_bazel( + metadata_version = "0", + metadata_name = "foo", + dep_template = "@pypi//{name}:{target}", + name = "foo.whl", + requires_dist = ["foo", "bar-baz", "qux"], + data_exclude = ["exclude_via_attr"], + annotation = struct( + copy_files = {"file_src": "file_dest"}, + copy_executables = {"exec_src": "exec_dest"}, + data = ["extra_target"], + data_exclude_glob = ["data_exclude_all"], + srcs_exclude_glob = ["srcs_exclude_all"], + additive_build_content = """# SOMETHING SPECIAL AT THE END""", + ), + config_load = "@pypi//:config.bzl", + group_name = "qux", + group_deps = ["foo", "fox", "qux"], + purl = "foo", + ) + env.expect.that_str(actual.replace("@@", "@")).equals(want) + +_tests.append(_test_all_workspace) + +def _test_all(env): + want = """\ +load("@package_metadata//rules:package_metadata.bzl", "package_metadata") +load("@pypi//:config.bzl", "packages") +load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets") + +package(default_visibility = ["//visibility:public"]) + +whl_library_targets( + copy_executables = { + "exec_src": "exec_dest", }, - entry_points = { - "foo": "bar.py", + copy_files = { + "file_src": "file_dest", }, + data = ["extra_target"], + data_exclude = [ + "exclude_via_attr", + "data_exclude_all", + ], + dep_template = "@pypi//{name}:{target}", group_deps = [ "foo", "fox", "qux", ], group_name = "qux", + include = packages, + metadata_name = "foo", name = "foo.whl", + requires_dist = [ + "foo", + "bar-baz", + "qux", + ], srcs_exclude = ["srcs_exclude_all"], tags = [ - "tag2", - "tag1", + "pypi_name=foo", + "pypi_version=0", ], ) +package_metadata( + name = "package_metadata", + purl = "foo", + visibility = ["//:__subpackages__"], +) + # SOMETHING SPECIAL AT THE END """ actual = generate_whl_library_build_bazel( + metadata_version = "0", + metadata_name = "foo", dep_template = "@pypi//{name}:{target}", name = "foo.whl", - dependencies = ["foo", "bar-baz", "qux"], - dependencies_by_platform = { - "linux_x86_64": ["box", "box-amd64"], - "windows_x86_64": ["fox"], - "@platforms//os:linux": ["box"], # buildifier: disable=unsorted-dict-items to check that we sort inside the test - }, - tags = ["tag2", "tag1"], - entry_points = { - "foo": "bar.py", - }, + requires_dist = ["foo", "bar-baz", "qux"], data_exclude = ["exclude_via_attr"], annotation = struct( copy_files = {"file_src": "file_dest"}, @@ -92,13 +158,89 @@ whl_library_targets( srcs_exclude_glob = ["srcs_exclude_all"], additive_build_content = """# SOMETHING SPECIAL AT THE END""", ), + config_load = "@pypi//:config.bzl", group_name = "qux", + purl = "foo", group_deps = ["foo", "fox", "qux"], ) env.expect.that_str(actual.replace("@@", "@")).equals(want) _tests.append(_test_all) +def _test_all_with_loads(env): + want = """\ +load("@package_metadata//rules:package_metadata.bzl", "package_metadata") +load("@pypi//:config.bzl", "packages") +load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets") + +package(default_visibility = ["//visibility:public"]) + +whl_library_targets( + copy_executables = { + "exec_src": "exec_dest", + }, + copy_files = { + "file_src": "file_dest", + }, + data = ["extra_target"], + data_exclude = [ + "exclude_via_attr", + "data_exclude_all", + ], + dep_template = "@pypi//{name}:{target}", + group_deps = [ + "foo", + "fox", + "qux", + ], + group_name = "qux", + include = packages, + metadata_name = "foo", + name = "foo.whl", + requires_dist = [ + "foo", + "bar-baz", + "qux", + ], + srcs_exclude = ["srcs_exclude_all"], + tags = [ + "pypi_name=foo", + "pypi_version=0", + ], +) + +package_metadata( + name = "package_metadata", + purl = "foo", + visibility = ["//:__subpackages__"], +) + +# SOMETHING SPECIAL AT THE END +""" + actual = generate_whl_library_build_bazel( + metadata_version = "0", + metadata_name = "foo", + dep_template = "@pypi//{name}:{target}", + name = "foo.whl", + requires_dist = ["foo", "bar-baz", "qux"], + data_exclude = ["exclude_via_attr"], + annotation = struct( + copy_files = {"file_src": "file_dest"}, + copy_executables = {"exec_src": "exec_dest"}, + data = ["extra_target"], + data_exclude_glob = ["data_exclude_all"], + srcs_exclude_glob = ["srcs_exclude_all"], + additive_build_content = """# SOMETHING SPECIAL AT THE END""", + ), + group_name = "qux", + config_load = "@pypi//:config.bzl", + group_deps = ["foo", "fox", "qux"], + purl = "foo", + ) + env.expect.that_str(actual.replace("@@", "@")).equals(want) + +_tests.append(_test_all_with_loads) + def generate_whl_library_build_bazel_test_suite(name): """Create the test suite. diff --git a/tests/pypi/hash/BUILD.bazel b/tests/pypi/hash/BUILD.bazel new file mode 100644 index 0000000000..1e052af141 --- /dev/null +++ b/tests/pypi/hash/BUILD.bazel @@ -0,0 +1,3 @@ +load(":hash_tests.bzl", "hash_test_suite") + +hash_test_suite(name = "hash_tests") diff --git a/tests/pypi/hash/hash_tests.bzl b/tests/pypi/hash/hash_tests.bzl new file mode 100644 index 0000000000..6866a79d07 --- /dev/null +++ b/tests/pypi/hash/hash_tests.bzl @@ -0,0 +1,102 @@ +# Copyright 2026 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private/pypi:hash.bzl", "hash") # buildifier: disable=bzl-visibility + +_tests = [] + +# The digests of b"rules_python" with the SRI values calculated with: +# python3 -c 'import base64, hashlib; h = hashlib.sha512(b"rules_python").digest(); print(base64.b64encode(h))' +_SHA256 = "a90bb21ddf552d508565aa2f0d78ac13297e6c407e39eb578eeac09e62c5da3a" +_SHA256_SRI = "sha256-qQuyHd9VLVCFZaovDXisEyl+bEB+OetXjurAnmLF2jo=" +_SHA384 = "7407ad30f83623cabf9139bd80edb332516880321ab2a20fa0f281767675bfc10b53555eb5042aa58de4b9345f5f70b6" +_SHA384_SRI = "sha384-dAetMPg2I8q/kTm9gO2zMlFogDIasqIPoPKBdnZ1v8ELU1VetQQqpY3kuTRfX3C2" +_SHA512 = "127af31fba52ce5ac96a4bc0f1b31c31c12367feace2b7dbb81066df479883add9dd6166495b126b76ac8c624d254b6a4ba3fbb628e8dcf0a34e1b0b065f619f" +_SHA512_SRI = "sha512-EnrzH7pSzlrJakvA8bMcMcEjZ/6s4rfbuBBm30eYg63Z3WFmSVsSa3asjGJNJUtqS6P7tijo3PCjThsLBl9hnw==" + +def _test_digest(env): + env.expect.that_str(hash.digest("sha256", _SHA256)).equals("sha256:" + _SHA256) + + # The algorithm name is normalized to lower case. + env.expect.that_str(hash.digest("SHA256", _SHA256)).equals("sha256:" + _SHA256) + + # Unknown algorithms and empty digests yield nothing. + env.expect.that_str(hash.digest("egg", "foo")).equals("") + env.expect.that_str(hash.digest("sha256", "")).equals("") + env.expect.that_str(hash.digest("", "")).equals("") + +_tests.append(_test_digest) + +def _test_hex_to_sri(env): + env.expect.that_str(hash.hex_to_sri("sha256", _SHA256)).equals(_SHA256_SRI) + env.expect.that_str(hash.hex_to_sri("sha384", _SHA384)).equals(_SHA384_SRI) + env.expect.that_str(hash.hex_to_sri("sha512", _SHA512)).equals(_SHA512_SRI) + + # Upper case digests are accepted. + env.expect.that_str(hash.hex_to_sri("sha512", _SHA512.upper())).equals(_SHA512_SRI) + + # Algorithms that SRI does not support yield nothing. + env.expect.that_str(hash.hex_to_sri("md5", "0" * 32)).equals("") + env.expect.that_str(hash.hex_to_sri("blake2b", "0" * 128)).equals("") + + # Invalid digests yield nothing. + env.expect.that_str(hash.hex_to_sri("sha256", "")).equals("") + env.expect.that_str(hash.hex_to_sri("sha256", "abc")).equals("") + env.expect.that_str(hash.hex_to_sri("sha256", "not-hex!")).equals("") + +_tests.append(_test_hex_to_sri) + +def _test_integrity(env): + env.expect.that_str(hash.integrity("")).equals("") + env.expect.that_str(hash.integrity("md5:" + "0" * 32)).equals("") + env.expect.that_str(hash.integrity("sha256:" + _SHA256)).equals(_SHA256_SRI) + env.expect.that_str(hash.integrity("sha384:" + _SHA384)).equals(_SHA384_SRI) + env.expect.that_str(hash.integrity("sha512:" + _SHA512)).equals(_SHA512_SRI) + +_tests.append(_test_integrity) + +def _test_preferred_digest(env): + env.expect.that_str(hash.preferred_digest([])).equals("") + env.expect.that_str(hash.preferred_digest([""])).equals("") + + # sha256 stays preferred so that the repo names remain stable. + env.expect.that_str(hash.preferred_digest([ + "sha256:" + _SHA256, + "sha512:" + _SHA512, + ])).equals("sha256:" + _SHA256) + + # Otherwise the strongest SRI supported algorithm wins. + env.expect.that_str(hash.preferred_digest([ + "sha384:" + _SHA384, + "sha512:" + _SHA512, + ])).equals("sha512:" + _SHA512) + + # And anything else is picked alphabetically for determinism. + env.expect.that_str(hash.preferred_digest([ + "md5:deadb00f", + "blake2b:deadbeef", + ])).equals("blake2b:deadbeef") + +_tests.append(_test_preferred_digest) + +def hash_test_suite(name): + """Create the test suite. + + Args: + name: the name of the test suite + """ + test_suite(name = name, basic_tests = _tests) diff --git a/tests/pypi/hub_builder/BUILD.bazel b/tests/pypi/hub_builder/BUILD.bazel new file mode 100644 index 0000000000..eb52fff01c --- /dev/null +++ b/tests/pypi/hub_builder/BUILD.bazel @@ -0,0 +1,3 @@ +load(":hub_builder_tests.bzl", "hub_builder_test_suite") + +hub_builder_test_suite(name = "hub_builder_tests") diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl new file mode 100644 index 0000000000..f421c7e351 --- /dev/null +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -0,0 +1,1617 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("@rules_testing//lib:truth.bzl", "subjects") +load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "REPO_VERBOSITY_ENV_VAR", "repo_utils") # buildifier: disable=bzl-visibility +load("//python/private/pypi:hub_builder.bzl", _hub_builder = "hub_builder") # buildifier: disable=bzl-visibility +load("//python/private/pypi:parse_simpleapi_html.bzl", "parse_simpleapi_html") # buildifier: disable=bzl-visibility +load("//python/private/pypi:platform.bzl", _plat = "platform") # buildifier: disable=bzl-visibility +load("//python/private/pypi:simpleapi_download.bzl", "simpleapi_download") # buildifier: disable=bzl-visibility +load("//python/private/pypi:whl_config_setting.bzl", "whl_config_setting") # buildifier: disable=bzl-visibility +load("//tests/pypi/extension:pip_parse.bzl", _parse = "pip_parse") +load("//tests/support/mocks:mocks.bzl", "mocks") + +_tests = [] + +def _mock_mctx(os_name = "unittest", arch_name = "exotic", environ = {}, mock_files = None): + return mocks.mctx( + os_name = os_name, + arch_name = arch_name, + environ = environ, + mock_files = mock_files or { + "requirements.txt": """\ +simple==0.0.1 \ + --hash=sha256:deadbeef \ + --hash=sha256:deadbaaf""", + }, + ) + +def hub_builder( + env, + enable_pipstar_extract = True, + debug = False, + config = None, + minor_mapping = {}, + whl_overrides = {}, + simpleapi_download_fn = None, + log_printer = None, + available_interpreters = {}): + builder = _hub_builder( + name = "pypi", + module_name = "unit_test", + config = config or struct( + # no need to evaluate the markers with the interpreter + enable_pipstar_extract = enable_pipstar_extract, + index_url = "https://pypi.org/simple", + platforms = { + "{}_{}{}".format(os, cpu, freethreaded): _plat( + name = "{}_{}{}".format(os, cpu, freethreaded), + os_name = os, + arch_name = cpu, + config_settings = [ + "@platforms//os:{}".format(os), + "@platforms//cpu:{}".format(cpu), + ], + whl_abi_tags = ["cp{major}{minor}t"] if freethreaded else ["abi3", "cp{major}{minor}"], + whl_platform_tags = whl_platform_tags, + ) + for (os, cpu, freethreaded), whl_platform_tags in { + ("linux", "x86_64", ""): ["linux_x86_64", "manylinux_*_x86_64"], + ("linux", "x86_64", "_freethreaded"): ["linux_x86_64", "manylinux_*_x86_64"], + ("linux", "aarch64", ""): ["linux_aarch64", "manylinux_*_aarch64"], + ("osx", "aarch64", ""): ["macosx_*_arm64"], + ("windows", "aarch64", ""): ["win_arm64"], + }.items() + }, + netrc = None, + auth_patterns = None, + toml_decode = json.decode, + ), + whl_overrides = whl_overrides, + minor_mapping = minor_mapping or {"3.15": "3.15.19"}, + available_interpreters = available_interpreters or { + "python_3_15_host": "unit_test_interpreter_target", + }, + simpleapi_download_fn = simpleapi_download_fn or (lambda *a, **k: {}), + logger = repo_utils.logger( + struct( + getenv = { + REPO_DEBUG_ENV_VAR: "1", + REPO_VERBOSITY_ENV_VAR: "TRACE" if debug else "FAIL", + }.get, + ), + "unit-test", + printer = log_printer, + ), + simpleapi_cache = {}, + ) + self = struct( + build = lambda: env.expect.that_struct( + builder.build(), + attrs = dict( + exposed_packages = subjects.collection, + group_map = subjects.dict, + whl_map = subjects.dict, + whl_libraries = subjects.dict, + extra_aliases = subjects.dict, + ), + ), + pip_parse = builder.pip_parse, + ) + return self + +def _test_simple(env): + builder = hub_builder(env) + builder.pip_parse( + _mock_mctx( + os_name = "osx", + arch_name = "aarch64", + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_lock = "requirements.txt", + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["simple"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "simple": { + "pypi_315_simple": [ + whl_config_setting( + version = "3.15", + ), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_simple": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "simple==0.0.1 --hash=sha256:deadbeef --hash=sha256:deadbaaf", + }, + }) + pypi.extra_aliases().contains_exactly({}) + +_tests.append(_test_simple) + +def _test_uv_lock_only(env): + builder = hub_builder(env) + builder.pip_parse( + _mock_mctx( + os_name = "osx", + arch_name = "aarch64", + mock_files = { + "uv.lock": """{"package":[{"name":"simple","source":{"registry":"https://pypi.org/simple"},"version":"0.0.1","wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/simple-0.0.1-py3-none-any.whl"}]}]}""", + }, + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_lock = None, + uv_lock = "uv.lock", + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["simple"]) + pypi.group_map().contains_exactly({}) + +_tests.append(_test_uv_lock_only) + +def _test_simple_multiple_requirements(env): + sub_tests = { + ("osx", "aarch64"): "simple==0.0.2 --hash=sha256:deadb00f", + ("windows", "aarch64"): "simple==0.0.1 --hash=sha256:deadbeef", + } + for (host_os, host_arch), want_requirement in sub_tests.items(): + builder = hub_builder(env) + builder.pip_parse( + mocks.mctx( + mock_files = { + "darwin.txt": "simple==0.0.2 --hash=sha256:deadb00f", + "win.txt": "simple==0.0.1 --hash=sha256:deadbeef", + }, + os_name = host_os, + arch_name = host_arch, + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_darwin = "darwin.txt", + requirements_windows = "win.txt", + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["simple"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "simple": { + "pypi_315_simple": [ + whl_config_setting(version = "3.15"), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_simple": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": want_requirement, + }, + }) + pypi.extra_aliases().contains_exactly({}) + +_tests.append(_test_simple_multiple_requirements) + +def _test_simple_extras_vs_no_extras(env): + sub_tests = { + ("osx", "aarch64"): "simple[foo]==0.0.1 --hash=sha256:deadbeef", + ("windows", "aarch64"): "simple==0.0.1 --hash=sha256:deadbeef", + } + for (host_os, host_arch), want_requirement in sub_tests.items(): + builder = hub_builder(env) + builder.pip_parse( + mocks.mctx( + mock_files = { + "darwin.txt": "simple[foo]==0.0.1 --hash=sha256:deadbeef", + "win.txt": "simple==0.0.1 --hash=sha256:deadbeef", + }, + os_name = host_os, + arch_name = host_arch, + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_darwin = "darwin.txt", + requirements_windows = "win.txt", + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["simple"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "simple": { + "pypi_315_simple": [ + whl_config_setting(version = "3.15"), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_simple": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": want_requirement, + }, + }) + pypi.extra_aliases().contains_exactly({}) + +_tests.append(_test_simple_extras_vs_no_extras) + +def _test_simple_extras_vs_no_extras_simpleapi(env): + def mockread_simpleapi(*_, parse_index, **__): + if parse_index: + content = """\ + simple-0.0.1-py3-none-any.whl
    +""" + return struct( + output = parse_simpleapi_html( + content = content, + parse_index = parse_index, + ), + success = True, + ) + + builder = hub_builder( + env, + simpleapi_download_fn = lambda *args, **kwargs: simpleapi_download( + read_simpleapi = mockread_simpleapi, + *args, + **kwargs + ), + ) + builder.pip_parse( + mocks.mctx( + mock_files = { + "darwin.txt": "simple[foo]==0.0.1 --hash=sha256:deadbeef", + "win.txt": "simple==0.0.1 --hash=sha256:deadbeef", + }, + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_darwin = "darwin.txt", + requirements_windows = "win.txt", + experimental_index_url = "https://example.com", + target_platforms = ["osx_aarch64", "windows_aarch64"], + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["simple"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "simple": { + "pypi_315_simple_py3_none_any_deadbeef_osx_aarch64": [ + whl_config_setting( + target_platforms = [ + "cp315_osx_aarch64", + ], + version = "3.15", + ), + ], + "pypi_315_simple_py3_none_any_deadbeef_windows_aarch64": [ + whl_config_setting( + target_platforms = [ + "cp315_windows_aarch64", + ], + version = "3.15", + ), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_simple_py3_none_any_deadbeef_osx_aarch64": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "filename": "simple-0.0.1-py3-none-any.whl", + "index_url": "https://example.com/simple/", + "integrity": "sha256-3q2+7w==", + "requirement": "simple[foo]==0.0.1", + "urls": ["/simple-0.0.1-py3-none-any.whl"], + }, + "pypi_315_simple_py3_none_any_deadbeef_windows_aarch64": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "filename": "simple-0.0.1-py3-none-any.whl", + "index_url": "https://example.com/simple/", + "integrity": "sha256-3q2+7w==", + "requirement": "simple==0.0.1", + "urls": ["/simple-0.0.1-py3-none-any.whl"], + }, + }) + pypi.extra_aliases().contains_exactly({}) + +_tests.append(_test_simple_extras_vs_no_extras_simpleapi) + +def _test_simple_sha512_simpleapi(env): + """Non-sha256 pins match the index digests and are downloaded with `integrity`.""" + + def mockread_simpleapi(*_, parse_index, **__): + if parse_index: + content = """\ + simple-0.0.1-py3-none-any.whl
    +""" + return struct( + output = parse_simpleapi_html( + content = content, + parse_index = parse_index, + ), + success = True, + ) + + builder = hub_builder( + env, + simpleapi_download_fn = lambda *args, **kwargs: simpleapi_download( + read_simpleapi = mockread_simpleapi, + *args, + **kwargs + ), + ) + builder.pip_parse( + mocks.mctx( + mock_files = { + "win.txt": "simple==0.0.1 --hash=sha512:deadbeef", + }, + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_windows = "win.txt", + experimental_index_url = "https://example.com", + target_platforms = ["windows_aarch64"], + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["simple"]) + pypi.whl_map().contains_exactly({ + "simple": { + "pypi_315_simple_py3_none_any_deadbeef": [ + whl_config_setting( + target_platforms = [ + "cp315_windows_aarch64", + ], + version = "3.15", + ), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_simple_py3_none_any_deadbeef": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "filename": "simple-0.0.1-py3-none-any.whl", + "index_url": "https://example.com/simple/", + "integrity": "sha512-3q2+7w==", + "requirement": "simple==0.0.1", + "urls": ["/simple-0.0.1-py3-none-any.whl"], + }, + }) + +_tests.append(_test_simple_sha512_simpleapi) + +def _test_simple_multiple_python_versions(env): + builder = hub_builder( + env, + available_interpreters = { + "python_3_15_host": "unit_test_interpreter_target", + "python_3_16_host": "unit_test_interpreter_target", + }, + minor_mapping = { + "3.15": "3.15.19", + "3.16": "3.16.9", + }, + ) + builder.pip_parse( + mocks.mctx( + mock_files = { + "requirements_3_15.txt": """ +simple==0.0.1 --hash=sha256:deadbeef +old-package==0.0.1 --hash=sha256:deadbaaf +""", + }, + os_name = "linux", + arch_name = "amd64", + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_lock = "requirements_3_15.txt", + ), + ) + builder.pip_parse( + mocks.mctx( + mock_files = { + "requirements_3_16.txt": """ +simple==0.0.2 --hash=sha256:deadb00f +new-package==0.0.1 --hash=sha256:deadb00f2 +""", + }, + os_name = "linux", + arch_name = "amd64", + ), + _parse( + hub_name = "pypi", + python_version = "3.16", + requirements_lock = "requirements_3_16.txt", + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["simple"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "new_package": { + "pypi_316_new_package": [ + whl_config_setting(version = "3.16"), + ], + }, + "old_package": { + "pypi_315_old_package": [ + whl_config_setting(version = "3.15"), + ], + }, + "simple": { + "pypi_315_simple": [ + whl_config_setting(version = "3.15"), + ], + "pypi_316_simple": [ + whl_config_setting(version = "3.16"), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_old_package": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "old-package==0.0.1 --hash=sha256:deadbaaf", + }, + "pypi_315_simple": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "simple==0.0.1 --hash=sha256:deadbeef", + }, + "pypi_316_new_package": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "new-package==0.0.1 --hash=sha256:deadb00f2", + }, + "pypi_316_simple": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "simple==0.0.2 --hash=sha256:deadb00f", + }, + }) + pypi.extra_aliases().contains_exactly({}) + +_tests.append(_test_simple_multiple_python_versions) + +def _test_simple_with_markers(env): + sub_tests = { + ("osx", "aarch64"): "torch==2.4.1 --hash=sha256:deadbeef", + ("linux", "x86_64"): "torch==2.4.1+cpu", + } + for (host_os, host_arch), want_requirement in sub_tests.items(): + builder = hub_builder(env) + builder.pip_parse( + mocks.mctx( + mock_files = { + "universal.txt": """\ + torch==2.4.1+cpu ; platform_machine == 'x86_64' + torch==2.4.1 ; platform_machine != 'x86_64' \ + --hash=sha256:deadbeef + """, + }, + os_name = host_os, + arch_name = host_arch, + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_lock = "universal.txt", + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["torch"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "torch": { + "pypi_315_torch": [ + whl_config_setting( + version = "3.15", + ), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_torch": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": want_requirement, + }, + }) + pypi.extra_aliases().contains_exactly({}) + +_tests.append(_test_simple_with_markers) + +def _test_torch_experimental_index_url(env): + def mockread_simpleapi(*_, parse_index, **__): + if parse_index: + content = """\ + torch +""" + else: + content = """\ + torch-2.4.1+cpu-cp310-cp310-linux_x86_64.whl
    + torch-2.4.1+cpu-cp310-cp310-win_amd64.whl
    + torch-2.4.1+cpu-cp311-cp311-linux_x86_64.whl
    + torch-2.4.1+cpu-cp311-cp311-win_amd64.whl
    + torch-2.4.1+cpu-cp312-cp312-linux_x86_64.whl
    + torch-2.4.1+cpu-cp312-cp312-win_amd64.whl
    + torch-2.4.1+cpu-cp38-cp38-linux_x86_64.whl
    + torch-2.4.1+cpu-cp38-cp38-win_amd64.whl
    + torch-2.4.1+cpu-cp39-cp39-linux_x86_64.whl
    + torch-2.4.1+cpu-cp39-cp39-win_amd64.whl
    + torch-2.4.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
    + torch-2.4.1-cp310-none-macosx_11_0_arm64.whl
    + torch-2.4.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
    + torch-2.4.1-cp311-none-macosx_11_0_arm64.whl
    + torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
    + torch-2.4.1-cp312-none-macosx_11_0_arm64.whl
    + torch-2.4.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
    + torch-2.4.1-cp38-none-macosx_11_0_arm64.whl
    + torch-2.4.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
    + torch-2.4.1-cp39-none-macosx_11_0_arm64.whl
    +""" + + return struct( + output = parse_simpleapi_html( + content = content, + parse_index = parse_index, + ), + success = True, + ) + + builder = hub_builder( + env, + config = struct( + netrc = None, + enable_pipstar_extract = True, + index_url = "https://pypi.org/simple", + auth_patterns = {}, + platforms = { + "{}_{}".format(os, cpu): _plat( + name = "{}_{}".format(os, cpu), + os_name = os, + arch_name = cpu, + config_settings = [ + "@platforms//os:{}".format(os), + "@platforms//cpu:{}".format(cpu), + ], + whl_platform_tags = whl_platform_tags, + ) + for (os, cpu), whl_platform_tags in { + ("linux", "x86_64"): ["linux_x86_64", "manylinux_*_x86_64"], + ("linux", "aarch64"): ["linux_aarch64", "manylinux_*_aarch64"], + # this should be ignored as well because there is no sdist and no whls + # for intel Macs + ("osx", "x86_64"): ["macosx_*_x86_64"], + ("osx", "aarch64"): ["macosx_*_arm64"], + ("windows", "x86_64"): ["win_amd64"], + ("windows", "aarch64"): ["win_arm64"], # this should be ignored + }.items() + }, + ), + available_interpreters = { + "python_3_12_host": "unit_test_interpreter_target", + }, + minor_mapping = {"3.12": "3.12.19"}, + simpleapi_download_fn = lambda *args, **kwargs: simpleapi_download( + read_simpleapi = mockread_simpleapi, + *args, + **kwargs + ), + ) + builder.pip_parse( + mocks.mctx( + mock_files = { + "universal.txt": """\ +torch==2.4.1 ; platform_machine != 'x86_64' \ + --hash=sha256:1495132f30f722af1a091950088baea383fe39903db06b20e6936fd99402803e \ + --hash=sha256:30be2844d0c939161a11073bfbaf645f1c7cb43f62f46cc6e4df1c119fb2a798 \ + --hash=sha256:36109432b10bd7163c9b30ce896f3c2cca1b86b9765f956a1594f0ff43091e2a \ + --hash=sha256:56ad2a760b7a7882725a1eebf5657abbb3b5144eb26bcb47b52059357463c548 \ + --hash=sha256:5fc1d4d7ed265ef853579caf272686d1ed87cebdcd04f2a498f800ffc53dab71 \ + --hash=sha256:72b484d5b6cec1a735bf3fa5a1c4883d01748698c5e9cfdbeb4ffab7c7987e0d \ + --hash=sha256:a38de2803ee6050309aac032676536c3d3b6a9804248537e38e098d0e14817ec \ + --hash=sha256:d36a8ef100f5bff3e9c3cea934b9e0d7ea277cb8210c7152d34a9a6c5830eadd \ + --hash=sha256:ddddbd8b066e743934a4200b3d54267a46db02106876d21cf31f7da7a96f98ea \ + --hash=sha256:fa27b048d32198cda6e9cff0bf768e8683d98743903b7e5d2b1f5098ded1d343 + # via -r requirements.in +torch==2.4.1+cpu ; platform_machine == 'x86_64' \ + --hash=sha256:0c0a7cc4f7c74ff024d5a5e21230a01289b65346b27a626f6c815d94b4b8c955 \ + --hash=sha256:1dd062d296fb78aa7cfab8690bf03704995a821b5ef69cfc807af5c0831b4202 \ + --hash=sha256:2b03e20f37557d211d14e3fb3f71709325336402db132a1e0dd8b47392185baf \ + --hash=sha256:330e780f478707478f797fdc82c2a96e9b8c5f60b6f1f57bb6ad1dd5b1e7e97e \ + --hash=sha256:3a570e5c553415cdbddfe679207327b3a3806b21c6adea14fba77684d1619e97 \ + --hash=sha256:3c99506980a2fb4b634008ccb758f42dd82f93ae2830c1e41f64536e310bf562 \ + --hash=sha256:76a6fe7b10491b650c630bc9ae328df40f79a948296b41d3b087b29a8a63cbad \ + --hash=sha256:833490a28ac156762ed6adaa7c695879564fa2fd0dc51bcf3fdb2c7b47dc55e6 \ + --hash=sha256:8800deef0026011d502c0c256cc4b67d002347f63c3a38cd8e45f1f445c61364 \ + --hash=sha256:c4f2c3c026e876d4dad7629170ec14fff48c076d6c2ae0e354ab3fdc09024f00 + # via -r requirements.in +""", + }, + ), + _parse( + hub_name = "pypi", + python_version = "3.12", + experimental_index_url = "https://torch.index", + requirements_lock = "universal.txt", + target_platforms = [ + "linux_x86_64", + "linux_aarch64", + # this should be ignored as well because there is no sdist and no whls + # for intel Macs + "osx_x86_64", + "osx_aarch64", + "windows_x86_64", + "windows_aarch64", + ], + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["torch"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "torch": { + "pypi_312_torch_cp312_cp312_linux_x86_64_8800deef_linux_x86_64": [ + whl_config_setting( + target_platforms = ["cp312_linux_x86_64"], + version = "3.12", + ), + ], + "pypi_312_torch_cp312_cp312_manylinux_2_17_aarch64_36109432_linux_aarch64": [ + whl_config_setting( + target_platforms = ["cp312_linux_aarch64"], + version = "3.12", + ), + ], + "pypi_312_torch_cp312_cp312_win_amd64_3a570e5c_windows_x86_64": [ + whl_config_setting( + target_platforms = ["cp312_windows_x86_64"], + version = "3.12", + ), + ], + "pypi_312_torch_cp312_none_macosx_11_0_arm64_72b484d5_osx_aarch64": [ + whl_config_setting( + target_platforms = ["cp312_osx_aarch64"], + version = "3.12", + ), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_312_torch_cp312_cp312_linux_x86_64_8800deef_linux_x86_64": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "filename": "torch-2.4.1+cpu-cp312-cp312-linux_x86_64.whl", + "index_url": "https://torch.index/torch/", + "integrity": "sha256-iADe7wAmAR1QLAwlbMS2fQAjR/Y8OjjNjkXx9EXGE2Q=", + "requirement": "torch==2.4.1+cpu", + "urls": ["/whl/cpu/torch-2.4.1%2Bcpu-cp312-cp312-linux_x86_64.whl"], + }, + "pypi_312_torch_cp312_cp312_manylinux_2_17_aarch64_36109432_linux_aarch64": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "filename": "torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", + "index_url": "https://torch.index/torch/", + "integrity": "sha256-NhCUMrEL1xY8mzDOiW88LMobhrl2X5VqFZTw/0MJHio=", + "requirement": "torch==2.4.1", + "urls": ["/whl/cpu/torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl"], + }, + "pypi_312_torch_cp312_cp312_win_amd64_3a570e5c_windows_x86_64": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "filename": "torch-2.4.1+cpu-cp312-cp312-win_amd64.whl", + "index_url": "https://torch.index/torch/", + "integrity": "sha256-OlcOXFU0Fc293+Z5IHMns6OAayHGreoU+6d2hNFhnpc=", + "requirement": "torch==2.4.1+cpu", + "urls": ["/whl/cpu/torch-2.4.1%2Bcpu-cp312-cp312-win_amd64.whl"], + }, + "pypi_312_torch_cp312_none_macosx_11_0_arm64_72b484d5_osx_aarch64": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "filename": "torch-2.4.1-cp312-none-macosx_11_0_arm64.whl", + "index_url": "https://torch.index/torch/", + "integrity": "sha256-crSE1bbOwac1vz+locSIPQF0hpjF6c/b60/6t8eYfg0=", + "requirement": "torch==2.4.1", + "urls": ["/whl/cpu/torch-2.4.1-cp312-none-macosx_11_0_arm64.whl"], + }, + }) + pypi.extra_aliases().contains_exactly({}) + +_tests.append(_test_torch_experimental_index_url) + +def _test_index_url_precedence(env): + for test in [ + struct( + requirements_txt = "simple==0.0.1 --hash=sha256:deadb00f", + experimental_index_url = "https://experimental.example.com/simple", + experimental_extra_index_urls = [], + expect_index_url = "https://experimental.example.com/simple", + expect_extra_index_urls = [], + expect_url = "experimental.example.com/simple/", + ), + struct( + requirements_txt = """\ +--index-url=https://file.example.com/simple +simple==0.0.1 --hash=sha256:deadb00f +""", + experimental_index_url = "https://experimental.example.com/simple", + experimental_extra_index_urls = [], + expect_index_url = "https://file.example.com/simple", + expect_extra_index_urls = [], + expect_url = "file.example.com/simple/", + ), + struct( + requirements_txt = "simple==0.0.1 --hash=sha256:deadb00f", + experimental_index_url = "", + experimental_extra_index_urls = [], + expect_index_url = "https://pypi.org/simple", + expect_extra_index_urls = [], + expect_url = "pypi.org/simple/", + ), + struct( + requirements_txt = """\ +--extra-index-url=https://extra1.example.com/simple +--extra-index-url=https://extra2.example.com/simple +simple==0.0.1 --hash=sha256:deadb00f +""", + experimental_index_url = "", + experimental_extra_index_urls = [ + "https://ignored.example.com/simple", + ], + expect_index_url = "https://pypi.org/simple", + expect_extra_index_urls = [ + "https://extra1.example.com/simple", + "https://extra2.example.com/simple", + ], + expect_url = "pypi.org/simple/", + ), + # Regression: an unsubstituted ``$VAR`` template with the env var + # unset must expand to "" and fall back to the config default, + # rather than activating the experimental index-url path. + struct( + requirements_txt = "simple==0.0.1 --hash=sha256:deadb00f", + experimental_index_url = "$RULES_PYTHON_PIP_INDEX_URL", + experimental_extra_index_urls = [], + envsubst = ["RULES_PYTHON_PIP_INDEX_URL"], + environ = {}, + expect_index_url = "https://pypi.org/simple", + expect_extra_index_urls = [], + expect_url = "pypi.org/simple/", + ), + # When the env var is set, the resolved value drives the + # experimental index-url path. + struct( + requirements_txt = "simple==0.0.1 --hash=sha256:deadb00f", + experimental_index_url = "${RULES_PYTHON_PIP_INDEX_URL:-}", + experimental_extra_index_urls = [], + envsubst = ["RULES_PYTHON_PIP_INDEX_URL"], + environ = {"RULES_PYTHON_PIP_INDEX_URL": "https://from-env.example.com/simple"}, + expect_index_url = "https://from-env.example.com/simple", + expect_extra_index_urls = [], + expect_url = "from-env.example.com/simple/", + ), + ]: + got_kwargs = {} + + def mock_simpleapi_download(*_, **kwargs): + got_kwargs.update(kwargs) + return { + "simple": struct( + whls = { + "sha256:deadb00f": struct( + yanked = None, + filename = "simple-0.0.1-py3-none-any.whl", + digest = "sha256:deadb00f", + url = test.expect_url, + ), + }, + sdists = {}, + hashes_by_version = {}, + index_url = test.expect_index_url, + ), + } + + builder = hub_builder( + env, + simpleapi_download_fn = mock_simpleapi_download, + ) + builder.pip_parse( + _mock_mctx( + environ = getattr(test, "environ", {}), + mock_files = { + "requirements.txt": test.requirements_txt, + }, + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + experimental_index_url = test.experimental_index_url, + experimental_extra_index_urls = test.experimental_extra_index_urls, + envsubst = getattr(test, "envsubst", []), + requirements_lock = "requirements.txt", + target_platforms = [ + "linux_x86_64", + "osx_aarch64", + ], + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["simple"]) + pypi.whl_map().contains_exactly({ + "simple": { + "pypi_315_simple_py3_none_any_deadb00f": [ + whl_config_setting( + target_platforms = ("cp315_linux_x86_64", "cp315_osx_aarch64"), + version = "3.15", + ), + ], + }, + }) + want_whl_library = { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "filename": "simple-0.0.1-py3-none-any.whl", + "index_url": test.expect_index_url, + "integrity": "sha256-3q2wDw==", + "requirement": "simple==0.0.1", + "urls": [test.expect_url], + } + if getattr(test, "envsubst", []): + want_whl_library["envsubst"] = test.envsubst + pypi.whl_libraries().contains_exactly({ + "pypi_315_simple_py3_none_any_deadb00f": want_whl_library, + }) + pypi.extra_aliases().contains_exactly({}) + + env.expect.that_dict(got_kwargs).contains_exactly({ + "attr": struct( + auth_patterns = {}, + envsubst = getattr(test, "envsubst", []), + extra_index_urls = test.expect_extra_index_urls, + index_url = test.expect_index_url, + index_url_overrides = {}, + netrc = None, + sources = { + "simple": ["0.0.1"], + }, + ), + "cache": {}, + "parallel_download": False, + }) + +_tests.append(_test_index_url_precedence) + +def _test_download_only_multiple(env): + builder = hub_builder(env) + builder.pip_parse( + mocks.mctx( + mock_files = { + "requirements.linux_x86_64.txt": """\ +--platform=manylinux_2_17_x86_64 +--python-version=315 +--implementation=cp +--abi=cp315 + +simple==0.0.1 \ + --hash=sha256:deadbeef +extra==0.0.1 \ + --hash=sha256:deadb00f +""", + "requirements.osx_aarch64.txt": """\ +--platform=macosx_10_9_arm64 +--python-version=315 +--implementation=cp +--abi=cp315 + +simple==0.0.3 \ + --hash=sha256:deadbaaf +""", + }, + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + download_only = True, + requirements_by_platform = { + "requirements.linux_x86_64.txt": "linux_x86_64", + "requirements.osx_aarch64.txt": "osx_aarch64", + }, + target_platforms = [ + "linux_x86_64", + "osx_aarch64", + ], + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["simple"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "extra": { + "pypi_315_extra": [ + whl_config_setting(version = "3.15"), + ], + }, + "simple": { + "pypi_315_simple_linux_x86_64": [ + whl_config_setting( + target_platforms = ["cp315_linux_x86_64"], + version = "3.15", + ), + ], + "pypi_315_simple_osx_aarch64": [ + whl_config_setting( + target_platforms = ["cp315_osx_aarch64"], + version = "3.15", + ), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_extra": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "download_only": True, + "extra_pip_args": ["--platform=manylinux_2_17_x86_64", "--python-version=315", "--implementation=cp", "--abi=cp315"], + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "extra==0.0.1 --hash=sha256:deadb00f", + }, + "pypi_315_simple_linux_x86_64": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "download_only": True, + "extra_pip_args": ["--platform=manylinux_2_17_x86_64", "--python-version=315", "--implementation=cp", "--abi=cp315"], + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "simple==0.0.1 --hash=sha256:deadbeef", + }, + "pypi_315_simple_osx_aarch64": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "download_only": True, + "extra_pip_args": ["--platform=macosx_10_9_arm64", "--python-version=315", "--implementation=cp", "--abi=cp315"], + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "simple==0.0.3 --hash=sha256:deadbaaf", + }, + }) + pypi.extra_aliases().contains_exactly({}) + +_tests.append(_test_download_only_multiple) + +def _test_simple_get_index(env): + got_simpleapi_download_args = [] + got_simpleapi_download_kwargs = {} + + def mocksimpleapi_download(*args, **kwargs): + got_simpleapi_download_args.extend(args) + got_simpleapi_download_kwargs.update(kwargs) + return { + "plat_pkg": struct( + whls = { + "sha256:deadb44f": struct( + yanked = None, + filename = "plat-pkg-0.0.4-py3-none-linux_x86_64.whl", + digest = "sha256:deadb44f", + url = "example2.org/index/plat_pkg/", + ), + }, + sdists = {}, + hashes_by_version = { + "0.0.4": ["sha256:deadb44f"], + }, + index_url = "https://pypi.org/simple", + ), + "simple": struct( + whls = { + "sha256:deadb00f": struct( + yanked = None, + filename = "simple-0.0.1-py3-none-any.whl", + digest = "sha256:deadb00f", + url = "example2.org", + ), + }, + sdists = { + "sha256:deadbeef": struct( + yanked = None, + filename = "simple-0.0.1.tar.gz", + digest = "sha256:deadbeef", + url = "example.org", + ), + }, + index_url = "https://pypi.org/simple", + ), + "some_other_pkg": struct( + whls = { + "sha256:deadb33f": struct( + yanked = None, + filename = "some-other-pkg-0.0.1-py3-none-any.whl", + digest = "sha256:deadb33f", + url = "example2.org/index/some_other_pkg/", + ), + }, + sdists = {}, + hashes_by_version = { + "0.0.1": ["sha256:deadb33f"], + "0.0.3": ["sha256:deadbeef"], + }, + index_url = "https://with_index_url", + ), + } + + builder = hub_builder( + env, + simpleapi_download_fn = mocksimpleapi_download, + whl_overrides = { + "direct_without_sha": { + "my_patch": 1, + }, + }, + ) + builder.pip_parse( + mocks.mctx( + mock_files = { + "requirements.txt": """ +simple==0.0.1 \ + --hash=sha256:deadbeef \ + --hash=sha256:deadb00f +some_pkg==0.0.1 @ example-direct.org/some_pkg-0.0.1-py3-none-any.whl \ + --hash=sha256:deadbaaf +direct_without_sha==0.0.1 @ example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl +some_other_pkg==0.0.1 +plat_pkg==0.0.4 +pip_fallback==0.0.1 +direct_sdist_without_sha @ some-archive/any-name.tar.gz +git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef +""", + }, + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_lock = "requirements.txt", + extra_pip_args = [ + "--extra-args-for-sdist-building", + ], + target_platforms = [ + "linux_aarch64", + "linux_x86_64", + "linux_x86_64_freethreaded", + "osx_aarch64", + "windows_aarch64", + ], + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly([ + "direct_sdist_without_sha", + "direct_without_sha", + "git_dep", + "pip_fallback", + "plat_pkg", + "simple", + "some_other_pkg", + "some_pkg", + ]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "direct_sdist_without_sha": { + "pypi_315_any_name": [ + whl_config_setting( + target_platforms = ( + "cp315_linux_aarch64", + "cp315_linux_x86_64", + "cp315_linux_x86_64_freethreaded", + "cp315_osx_aarch64", + "cp315_windows_aarch64", + ), + version = "3.15", + ), + ], + }, + "direct_without_sha": { + "pypi_315_direct_without_sha_0_0_1_py3_none_any": [ + whl_config_setting( + target_platforms = ( + "cp315_linux_aarch64", + "cp315_linux_x86_64", + "cp315_linux_x86_64_freethreaded", + "cp315_osx_aarch64", + "cp315_windows_aarch64", + ), + version = "3.15", + ), + ], + }, + "git_dep": { + "pypi_315_git_dep": [ + whl_config_setting( + version = "3.15", + ), + ], + }, + "pip_fallback": { + "pypi_315_pip_fallback": [ + whl_config_setting( + version = "3.15", + ), + ], + }, + "plat_pkg": { + "pypi_315_plat_py3_none_linux_x86_64_deadb44f": [ + whl_config_setting( + target_platforms = [ + "cp315_linux_x86_64", + "cp315_linux_x86_64_freethreaded", + ], + version = "3.15", + ), + ], + }, + "simple": { + "pypi_315_simple_py3_none_any_deadb00f": [ + whl_config_setting( + target_platforms = ( + "cp315_linux_aarch64", + "cp315_linux_x86_64", + "cp315_linux_x86_64_freethreaded", + "cp315_osx_aarch64", + "cp315_windows_aarch64", + ), + version = "3.15", + ), + ], + }, + "some_other_pkg": { + "pypi_315_some_py3_none_any_deadb33f": [ + whl_config_setting( + target_platforms = ( + "cp315_linux_aarch64", + "cp315_linux_x86_64", + "cp315_linux_x86_64_freethreaded", + "cp315_osx_aarch64", + "cp315_windows_aarch64", + ), + version = "3.15", + ), + ], + }, + "some_pkg": { + "pypi_315_some_pkg_py3_none_any_deadbaaf": [ + whl_config_setting( + target_platforms = ( + "cp315_linux_aarch64", + "cp315_linux_x86_64", + "cp315_linux_x86_64_freethreaded", + "cp315_osx_aarch64", + "cp315_windows_aarch64", + ), + version = "3.15", + ), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_any_name": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "extra_pip_args": ["--extra-args-for-sdist-building"], + "filename": "any-name.tar.gz", + "integrity": "", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "direct_sdist_without_sha @ some-archive/any-name.tar.gz", + "urls": ["some-archive/any-name.tar.gz"], + }, + "pypi_315_direct_without_sha_0_0_1_py3_none_any": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "filename": "direct_without_sha-0.0.1-py3-none-any.whl", + "integrity": "", + "requirement": "direct_without_sha==0.0.1", + "urls": ["example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl"], + "whl_patches": {"my_patch": "1"}, + }, + "pypi_315_git_dep": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "extra_pip_args": ["--extra-args-for-sdist-building"], + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef", + }, + "pypi_315_pip_fallback": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "extra_pip_args": ["--extra-args-for-sdist-building"], + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "pip_fallback==0.0.1", + }, + "pypi_315_plat_py3_none_linux_x86_64_deadb44f": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "filename": "plat-pkg-0.0.4-py3-none-linux_x86_64.whl", + "index_url": "https://pypi.org/simple", + "integrity": "sha256-3q20Tw==", + "requirement": "plat_pkg==0.0.4", + "urls": ["example2.org/index/plat_pkg/"], + }, + "pypi_315_simple_py3_none_any_deadb00f": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "filename": "simple-0.0.1-py3-none-any.whl", + "index_url": "https://pypi.org/simple", + "integrity": "sha256-3q2wDw==", + "requirement": "simple==0.0.1", + "urls": ["example2.org"], + }, + "pypi_315_some_pkg_py3_none_any_deadbaaf": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "filename": "some_pkg-0.0.1-py3-none-any.whl", + "integrity": "sha256-3q26rw==", + "requirement": "some_pkg==0.0.1", + "urls": ["example-direct.org/some_pkg-0.0.1-py3-none-any.whl"], + }, + "pypi_315_some_py3_none_any_deadb33f": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "filename": "some-other-pkg-0.0.1-py3-none-any.whl", + "index_url": "https://with_index_url", + "integrity": "sha256-3q2zPw==", + "requirement": "some_other_pkg==0.0.1", + "urls": ["example2.org/index/some_other_pkg/"], + }, + }) + pypi.extra_aliases().contains_exactly({}) + env.expect.that_dict(got_simpleapi_download_kwargs).contains_exactly( + { + "attr": struct( + auth_patterns = {}, + envsubst = {}, + extra_index_urls = [], + index_url = "https://pypi.org/simple", + index_url_overrides = {}, + netrc = None, + sources = { + "pip_fallback": ["0.0.1"], + "plat_pkg": ["0.0.4"], + "simple": ["0.0.1"], + "some_other_pkg": ["0.0.1"], + }, + ), + "cache": {}, + "parallel_download": False, + }, + ) + +_tests.append(_test_simple_get_index) + +def _test_optimum_sys_platform_extra(env): + sub_tests = { + ("osx", "aarch64"): "optimum[onnxruntime]==1.17.1", + ("linux", "aarch64"): "optimum[onnxruntime-gpu]==1.17.1", + } + for (host_os, host_arch), want_requirement in sub_tests.items(): + builder = hub_builder( + env, + ) + builder.pip_parse( + mocks.mctx( + mock_files = { + "universal.txt": """\ +optimum[onnxruntime]==1.17.1 ; sys_platform == 'darwin' +optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' +""", + }, + os_name = host_os, + arch_name = host_arch, + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_lock = "universal.txt", + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["optimum"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "optimum": { + "pypi_315_optimum": [ + whl_config_setting(version = "3.15"), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_optimum": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": want_requirement, + }, + }) + pypi.extra_aliases().contains_exactly({}) + +_tests.append(_test_optimum_sys_platform_extra) + +def _test_pipstar_platforms(env): + builder = hub_builder( + env, + config = struct( + enable_pipstar_extract = True, + index_url = "https://pypi.org/simple", + netrc = None, + auth_patterns = {}, + platforms = { + "my{}{}".format(os, cpu): _plat( + name = "my{}{}".format(os, cpu), + os_name = os, + arch_name = cpu, + marker = "python_version ~= \"3.13\"", + config_settings = [ + "@platforms//os:{}".format(os), + "@platforms//cpu:{}".format(cpu), + ], + ) + for os, cpu in [ + ("linux", "x86_64"), + ("osx", "aarch64"), + ] + }, + ), + ) + builder.pip_parse( + mocks.mctx( + mock_files = { + "universal.txt": """\ +optimum[onnxruntime]==1.17.1 ; sys_platform == 'darwin' +optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' +""", + }, + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_lock = "universal.txt", + target_platforms = ["mylinuxx86_64", "myosxaarch64"], + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["optimum"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "optimum": { + "pypi_315_optimum_mylinuxx86_64": [ + whl_config_setting( + version = "3.15", + target_platforms = [ + "cp315_mylinuxx86_64", + ], + ), + ], + "pypi_315_optimum_myosxaarch64": [ + whl_config_setting( + version = "3.15", + target_platforms = [ + "cp315_myosxaarch64", + ], + ), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_optimum_mylinuxx86_64": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "optimum[onnxruntime-gpu]==1.17.1", + }, + "pypi_315_optimum_myosxaarch64": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "optimum[onnxruntime]==1.17.1", + }, + }) + pypi.extra_aliases().contains_exactly({}) + +_tests.append(_test_pipstar_platforms) + +def _test_pipstar_platforms_limit(env): + builder = hub_builder( + env, + config = struct( + enable_pipstar_extract = True, + index_url = "https://pypi.org/simple", + netrc = None, + auth_patterns = {}, + platforms = { + "my{}{}".format(os, cpu): _plat( + name = "my{}{}".format(os, cpu), + os_name = os, + arch_name = cpu, + marker = "python_version ~= \"3.13\"", + config_settings = [ + "@platforms//os:{}".format(os), + "@platforms//cpu:{}".format(cpu), + ], + ) + for os, cpu in [ + ("linux", "x86_64"), + ("osx", "aarch64"), + ] + }, + ), + ) + builder.pip_parse( + mocks.mctx( + os_name = "linux", + arch_name = "amd64", + mock_files = { + "universal.txt": """\ +optimum[onnxruntime]==1.17.1 ; sys_platform == 'darwin' +optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' +""", + }, + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_lock = "universal.txt", + target_platforms = ["my{os}{arch}"], + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["optimum"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "optimum": { + "pypi_315_optimum": [ + whl_config_setting(version = "3.15"), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_optimum": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "optimum[onnxruntime-gpu]==1.17.1", + }, + }) + pypi.extra_aliases().contains_exactly({}) + +_tests.append(_test_pipstar_platforms_limit) + +def _test_err_duplicate_repos(env): + logs = {} + log_printer = lambda key, message: logs.setdefault(key.strip(), []).append(message) + builder = hub_builder( + env, + available_interpreters = { + "python_3_15_1_host": "unit_test_interpreter_target_1", + "python_3_15_2_host": "unit_test_interpreter_target_2", + }, + log_printer = log_printer, + ) + builder.pip_parse( + _mock_mctx( + os_name = "osx", + arch_name = "aarch64", + mock_files = { + "requirements.txt": "foo==0.0.1", + }, + ), + _parse( + hub_name = "pypi", + python_version = "3.15.1", + requirements_lock = "requirements.txt", + ), + ) + builder.pip_parse( + _mock_mctx( + os_name = "osx", + arch_name = "aarch64", + mock_files = { + "requirements.txt": "foo==0.0.1", + }, + ), + _parse( + hub_name = "pypi", + python_version = "3.15.2", + requirements_lock = "requirements.txt", + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly([]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({}) + pypi.whl_libraries().contains_exactly({}) + pypi.extra_aliases().contains_exactly({}) + env.expect.that_dict(logs).keys().contains_exactly(["rules_python:unit-test FAIL:"]) + env.expect.that_collection(logs["rules_python:unit-test FAIL:"]).contains_exactly([ + """\ +Attempting to create a duplicate library pypi_315_foo for foo with different arguments. Already existing declaration has: + common: { + "dep_template": "@pypi//{name}:{target}", + "config_load": "@pypi//:config.bzl", + "requirement": "foo==0.0.1", + } + different: { + "python_interpreter_target": ("unit_test_interpreter_target_1", "unit_test_interpreter_target_2"), + }\ +""", + ]).in_order() + +_tests.append(_test_err_duplicate_repos) + +def hub_builder_test_suite(name): + """Create the test suite. + + Args: + name: the name of the test suite + """ + test_suite(name = name, basic_tests = _tests) diff --git a/tests/pypi/index_sources/index_sources_tests.bzl b/tests/pypi/index_sources/index_sources_tests.bzl index 0a767078ba..3a605b9eb9 100644 --- a/tests/pypi/index_sources/index_sources_tests.bzl +++ b/tests/pypi/index_sources/index_sources_tests.bzl @@ -20,37 +20,182 @@ load("//python/private/pypi:index_sources.bzl", "index_sources") # buildifier: _tests = [] def _test_no_simple_api_sources(env): - inputs = [ - "foo==0.0.1", - "foo==0.0.1 @ https://someurl.org", - "foo==0.0.1 @ https://someurl.org --hash=sha256:deadbeef", - "foo==0.0.1 @ https://someurl.org; python_version < 2.7 --hash=sha256:deadbeef", - ] - for input in inputs: + inputs = { + "foo @ git+https://github.com/org/foo.git@deadbeef": struct( + requirement = "foo @ git+https://github.com/org/foo.git@deadbeef", + requirement_line = "foo @ git+https://github.com/org/foo.git@deadbeef", + marker = "", + url = "git+https://github.com/org/foo.git@deadbeef", + hashes = [], + version = "", + filename = "", + ), + "foo==0.0.1": struct( + requirement = "foo==0.0.1", + requirement_line = "foo==0.0.1", + marker = "", + url = "", + version = "0.0.1", + filename = "", + ), + "foo==0.0.1 @ https://someurl.org": struct( + requirement = "foo==0.0.1 @ https://someurl.org", + requirement_line = "foo==0.0.1 @ https://someurl.org", + marker = "", + url = "https://someurl.org", + version = "0.0.1", + filename = "", + ), + "foo==0.0.1 @ https://someurl.org/package.whl": struct( + requirement = "foo==0.0.1", + requirement_line = "foo==0.0.1 @ https://someurl.org/package.whl", + marker = "", + url = "https://someurl.org/package.whl", + version = "0.0.1", + filename = "package.whl", + ), + "foo==0.0.1 @ https://someurl.org/package.whl --hash=sha256:deadbeef": struct( + requirement = "foo==0.0.1", + requirement_line = "foo==0.0.1 @ https://someurl.org/package.whl --hash=sha256:deadbeef", + marker = "", + url = "https://someurl.org/package.whl", + hashes = ["sha256:deadbeef"], + version = "0.0.1", + filename = "package.whl", + ), + "foo==0.0.1 @ https://someurl.org/package.whl; python_version < \"2.7\"\\ --hash=sha256:deadbeef": struct( + requirement = "foo==0.0.1", + requirement_line = "foo==0.0.1 @ https://someurl.org/package.whl --hash=sha256:deadbeef", + marker = "python_version < \"2.7\"", + url = "https://someurl.org/package.whl", + hashes = ["sha256:deadbeef"], + version = "0.0.1", + filename = "package.whl", + ), + "foo[extra] @ https://example.org/foo-1.0.tar.gz --hash=sha256:deadbe0f": struct( + # NOTE @aignas 2025-08-03: we need to ensure that sdists continue working + # when we are using pip to install them even if the experimental_index_url + # code path is used. + requirement = "foo[extra] @ https://example.org/foo-1.0.tar.gz --hash=sha256:deadbe0f", + requirement_line = "foo[extra] @ https://example.org/foo-1.0.tar.gz --hash=sha256:deadbe0f", + marker = "", + url = "https://example.org/foo-1.0.tar.gz", + hashes = ["sha256:deadbe0f"], + version = "", + filename = "foo-1.0.tar.gz", + ), + "torch @ https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp311-cp311-linux_x86_64.whl#sha256=deadbeef": struct( + requirement = "torch", + requirement_line = "torch @ https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp311-cp311-linux_x86_64.whl#sha256=deadbeef", + marker = "", + url = "https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp311-cp311-linux_x86_64.whl", + hashes = ["sha256:deadbeef"], + version = "", + filename = "torch-2.6.0+cpu-cp311-cp311-linux_x86_64.whl", + ), + } + for input, want in inputs.items(): got = index_sources(input) - env.expect.that_collection(got.shas).contains_exactly([]) - env.expect.that_str(got.version).equals("0.0.1") + env.expect.that_collection(got.hashes).contains_exactly(want.hashes if hasattr(want, "hashes") else []) + env.expect.that_str(got.version).equals(want.version) + env.expect.that_str(got.requirement).equals(want.requirement) + env.expect.that_str(got.requirement_line).equals(got.requirement_line) + env.expect.that_str(got.marker).equals(want.marker) + env.expect.that_str(got.url).equals(want.url) + env.expect.that_str(got.filename).equals(want.filename) _tests.append(_test_no_simple_api_sources) def _test_simple_api_sources(env): tests = { - "foo==0.0.2 --hash=sha256:deafbeef --hash=sha256:deadbeef": [ - "deadbeef", - "deafbeef", - ], - "foo[extra]==0.0.2; (python_version < 2.7 or something_else == \"@\") --hash=sha256:deafbeef --hash=sha256:deadbeef": [ - "deadbeef", - "deafbeef", - ], + "foo==0.0.2 --hash=sha256:deafbeef --hash=sha256:deadbeef": struct( + hashes = [ + "sha256:deadbeef", + "sha256:deafbeef", + ], + marker = "", + requirement = "foo==0.0.2", + requirement_line = "foo==0.0.2 --hash=sha256:deafbeef --hash=sha256:deadbeef", + url = "", + ), + "foo[extra]==0.0.2; (python_version < 2.7 or extra == \"@\") --hash=sha256:deafbeef --hash=sha256:deadbeef": struct( + hashes = [ + "sha256:deadbeef", + "sha256:deafbeef", + ], + marker = "(python_version < 2.7 or extra == \"@\")", + requirement = "foo[extra]==0.0.2", + requirement_line = "foo[extra]==0.0.2 --hash=sha256:deafbeef --hash=sha256:deadbeef", + url = "", + ), } - for input, want_shas in tests.items(): + for input, want in tests.items(): got = index_sources(input) - env.expect.that_collection(got.shas).contains_exactly(want_shas) + env.expect.that_collection(got.hashes).contains_exactly(want.hashes) env.expect.that_str(got.version).equals("0.0.2") + env.expect.that_str(got.requirement).equals(want.requirement) + env.expect.that_str(got.requirement_line).equals(want.requirement_line) + env.expect.that_str(got.marker).equals(want.marker) + env.expect.that_str(got.url).equals(want.url) _tests.append(_test_simple_api_sources) +def _test_non_sha256_hashes(env): + tests = { + # A `#sha512=` URL fragment, as any hash algorithm is allowed by PEP 503. + "foo @ https://example.org/foo-0.0.1-py3-none-any.whl#sha512=deadbeef": struct( + hashes = ["sha512:deadbeef"], + marker = "", + requirement = "foo", + requirement_line = "foo @ https://example.org/foo-0.0.1-py3-none-any.whl#sha512=deadbeef", + url = "https://example.org/foo-0.0.1-py3-none-any.whl", + version = "", + filename = "foo-0.0.1-py3-none-any.whl", + ), + # Unknown algorithms are dropped. + "foo==0.0.2 --hash=egg:deadbeef": struct( + hashes = [], + marker = "", + requirement = "foo==0.0.2", + requirement_line = "foo==0.0.2", + url = "", + version = "0.0.2", + filename = "", + ), + "foo==0.0.2 --hash=sha512:deadbeef": struct( + hashes = ["sha512:deadbeef"], + marker = "", + requirement = "foo==0.0.2", + requirement_line = "foo==0.0.2 --hash=sha512:deadbeef", + url = "", + version = "0.0.2", + filename = "", + ), + "foo==0.0.2 --hash=sha512:deafbeef --hash=sha256:deadbeef": struct( + hashes = [ + "sha256:deadbeef", + "sha512:deafbeef", + ], + marker = "", + requirement = "foo==0.0.2", + requirement_line = "foo==0.0.2 --hash=sha512:deafbeef --hash=sha256:deadbeef", + url = "", + version = "0.0.2", + filename = "", + ), + } + for input, want in tests.items(): + got = index_sources(input) + env.expect.that_collection(got.hashes).contains_exactly(want.hashes) + env.expect.that_str(got.version).equals(want.version) + env.expect.that_str(got.requirement).equals(want.requirement) + env.expect.that_str(got.requirement_line).equals(want.requirement_line) + env.expect.that_str(got.marker).equals(want.marker) + env.expect.that_str(got.url).equals(want.url) + env.expect.that_str(got.filename).equals(want.filename) + +_tests.append(_test_non_sha256_hashes) + def index_sources_test_suite(name): """Create the test suite. diff --git a/tests/pypi/integration/BUILD.bazel b/tests/pypi/integration/BUILD.bazel index 9ea8dcebe4..316abe8271 100644 --- a/tests/pypi/integration/BUILD.bazel +++ b/tests/pypi/integration/BUILD.bazel @@ -1,20 +1,7 @@ load("@bazel_skylib//rules:build_test.bzl", "build_test") load("@rules_python_publish_deps//:requirements.bzl", "all_requirements") -load(":transitions.bzl", "transition_rule") build_test( name = "all_requirements_build_test", targets = all_requirements, ) - -# Rule that transitions dependencies to be built from sdist -transition_rule( - name = "all_requirements_from_sdist", - testonly = True, - deps = all_requirements, -) - -build_test( - name = "all_requirements_from_sdist_build_test", - targets = ["all_requirements_from_sdist"], -) diff --git a/tests/pypi/namespace_pkgs/BUILD.bazel b/tests/pypi/namespace_pkgs/BUILD.bazel new file mode 100644 index 0000000000..57f7962524 --- /dev/null +++ b/tests/pypi/namespace_pkgs/BUILD.bazel @@ -0,0 +1,5 @@ +load(":namespace_pkgs_tests.bzl", "namespace_pkgs_test_suite") + +namespace_pkgs_test_suite( + name = "namespace_pkgs_tests", +) diff --git a/tests/pypi/namespace_pkgs/namespace_pkgs_tests.bzl b/tests/pypi/namespace_pkgs/namespace_pkgs_tests.bzl new file mode 100644 index 0000000000..9c382d070c --- /dev/null +++ b/tests/pypi/namespace_pkgs/namespace_pkgs_tests.bzl @@ -0,0 +1,206 @@ +"" + +load("@rules_testing//lib:analysis_test.bzl", "test_suite") +load("//python/private/pypi:namespace_pkgs.bzl", "create_inits", "get_files") # buildifier: disable=bzl-visibility + +_tests = [] + +def test_in_current_dir(env): + srcs = [ + "foo/bar/biz.py", + "foo/bee/boo.py", + "foo/buu/__init__.py", + "foo/buu/bii.py", + ] + got = get_files(srcs = srcs) + expected = [ + "foo", + "foo/bar", + "foo/bee", + ] + env.expect.that_collection(got).contains_exactly(expected) + +_tests.append(test_in_current_dir) + +def test_find_correct_namespace_packages(env): + srcs = [ + "nested/root/foo/bar/biz.py", + "nested/root/foo/bee/boo.py", + "nested/root/foo/buu/__init__.py", + "nested/root/foo/buu/bii.py", + ] + + got = get_files(srcs = srcs, root = "nested/root") + expected = [ + "nested/root/foo", + "nested/root/foo/bar", + "nested/root/foo/bee", + ] + env.expect.that_collection(got).contains_exactly(expected) + +_tests.append(test_find_correct_namespace_packages) + +def test_ignores_empty_directories(_): + # because globs do not add directories, this test is not needed + pass + +_tests.append(test_ignores_empty_directories) + +def test_empty_case(env): + srcs = [ + "foo/__init__.py", + "foo/bar/__init__.py", + "foo/bar/biz.py", + ] + + got = get_files(srcs = srcs) + expected = [] + env.expect.that_collection(got).contains_exactly(expected) + +_tests.append(test_empty_case) + +def test_ignores_non_module_files_in_directories(env): + srcs = [ + "foo/__init__.pyi", + "foo/py.typed", + ] + + got = get_files(srcs = srcs) + expected = [] + env.expect.that_collection(got).contains_exactly(expected) + +_tests.append(test_ignores_non_module_files_in_directories) + +def test_parent_child_relationship_of_namespace_pkgs(env): + srcs = [ + "foo/bar/biff/my_module.py", + "foo/bar/biff/another_module.py", + ] + + got = get_files(srcs = srcs) + expected = [ + "foo", + "foo/bar", + "foo/bar/biff", + ] + env.expect.that_collection(got).contains_exactly(expected) + +_tests.append(test_parent_child_relationship_of_namespace_pkgs) + +def test_parent_child_relationship_of_namespace_and_standard_pkgs(env): + srcs = [ + "foo/bar/biff/__init__.py", + "foo/bar/biff/another_module.py", + ] + + got = get_files(srcs = srcs) + expected = [ + "foo", + "foo/bar", + ] + env.expect.that_collection(got).contains_exactly(expected) + +_tests.append(test_parent_child_relationship_of_namespace_and_standard_pkgs) + +def test_parent_child_relationship_of_namespace_and_nested_standard_pkgs(env): + srcs = [ + "foo/bar/__init__.py", + "foo/bar/biff/another_module.py", + "foo/bar/biff/__init__.py", + "foo/bar/boof/big_module.py", + "foo/bar/boof/__init__.py", + "fim/in_a_ns_pkg.py", + ] + + got = get_files(srcs = srcs) + expected = [ + "foo", + "fim", + ] + env.expect.that_collection(got).contains_exactly(expected) + +_tests.append(test_parent_child_relationship_of_namespace_and_nested_standard_pkgs) + +def test_recognized_all_nonstandard_module_types(env): + srcs = [ + "ayy/my_module.pyc", + "bee/ccc/dee/eee.so", + "eff/jee/aych.pyd", + ] + + expected = [ + "ayy", + "bee", + "bee/ccc", + "bee/ccc/dee", + "eff", + "eff/jee", + ] + got = get_files(srcs = srcs) + env.expect.that_collection(got).contains_exactly(expected) + +_tests.append(test_recognized_all_nonstandard_module_types) + +def test_skips_ignored_directories(env): + srcs = [ + "root/foo/boo/my_module.py", + "root/foo/bar/another_module.py", + ] + + expected = [ + "root/foo", + "root/foo/bar", + ] + got = get_files( + srcs = srcs, + ignored_dirnames = ["root/foo/boo"], + root = "root", + ) + env.expect.that_collection(got).contains_exactly(expected) + +_tests.append(test_skips_ignored_directories) + +def _test_create_inits(env): + srcs = [ + "nested/root/foo/bar/biz.py", + "nested/root/foo/bee/boo.py", + "nested/root/foo/buu/__init__.py", + "nested/root/foo/buu/bii.py", + ] + copy_file_calls = [] + template = Label("//python/private/pypi:namespace_pkg_tmpl.py") + + got = create_inits( + srcs = srcs, + root = "nested/root", + copy_file = lambda **kwargs: copy_file_calls.append(kwargs), + ) + env.expect.that_collection(got).contains_exactly([ + call["out"] + for call in copy_file_calls + ]) + env.expect.that_collection(copy_file_calls).contains_exactly([ + { + "name": "_cp_0_namespace", + "out": "nested/root/foo/__init__.py", + "src": template, + }, + { + "name": "_cp_1_namespace", + "out": "nested/root/foo/bar/__init__.py", + "src": template, + }, + { + "name": "_cp_2_namespace", + "out": "nested/root/foo/bee/__init__.py", + "src": template, + }, + ]) + +_tests.append(_test_create_inits) + +def namespace_pkgs_test_suite(name): + test_suite( + name = name, + basic_tests = _tests, + ) diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index dfa1fef5c3..54f906929a 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -15,24 +15,38 @@ "" load("@rules_testing//lib:test_suite.bzl", "test_suite") -load("//python/private/pypi:parse_requirements.bzl", "parse_requirements", "select_requirement") # buildifier: disable=bzl-visibility +load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "REPO_VERBOSITY_ENV_VAR", "repo_utils") # buildifier: disable=bzl-visibility +load("//python/private/pypi:parse_requirements.bzl", "select_requirement", _parse_requirements = "parse_requirements") # buildifier: disable=bzl-visibility +load("//python/private/pypi:pep508_env.bzl", pep508_env = "env") # buildifier: disable=bzl-visibility +load("//tests/support/mocks:mocks.bzl", "mocks") def _mock_ctx(): testdata = { - "requirements_different_package_version": """\ -foo==0.0.1+local --hash=sha256:deadbeef -foo==0.0.1 --hash=sha256:deadb00f -""", "requirements_direct": """\ -foo[extra] @ https://some-url +foo[extra] @ https://some-url/package.whl +""", + "requirements_direct_sdist": """ +foo @ https://github.com/org/foo/downloads/foo-1.1.tar.gz """, "requirements_extra_args": """\ --index-url=example.org -foo[extra]==0.0.1 --hash=sha256:deadbeef +foo[extra]==0.0.1 \ + --hash=sha256:deadbeef +""", + "requirements_foo": """\ +foo==0.0.1 \ + --hash=sha256:deadb00f +""", + "requirements_foo_local": """\ +foo==0.0.1+local \ + --hash=sha256:deadbeef +""", + "requirements_git": """ +foo @ git+https://github.com/org/foo.git@deadbeef """, "requirements_linux": """\ -foo==0.0.3 --hash=sha256:deadbaaf +foo==0.0.3 --hash=sha256:deadbaaf --hash=sha256:5d15t """, # download_only = True "requirements_linux_download_only": """\ @@ -53,11 +67,23 @@ foo==0.0.1 --hash=sha256:deadbeef foo[extra]==0.0.1 --hash=sha256:deadbeef """, "requirements_marker": """\ -foo[extra]==0.0.1 ;marker --hash=sha256:deadbeef +foo[extra]==0.0.1 ; os_name == 'nt' --hash=sha256:deadbeef bar==0.0.1 --hash=sha256:deadbeef +""", + "requirements_multi_version": """\ +foo==0.0.1; python_full_version < '3.10.0' \ + --hash=sha256:deadbeef +foo==0.0.2; python_full_version >= '3.10.0' \ + --hash=sha256:deadb11f +boo==0.0.4; python_full_version < '3.10.0' \ + --hash=sha256:deadbaaf +""", + "requirements_optional_hash": """ +bar==0.0.4 @ https://example.org/bar-0.0.4.whl +foo==0.0.5 @ https://example.org/foo-0.0.5.whl --hash=sha256:deadbeef """, "requirements_osx": """\ -foo==0.0.3 --hash=sha256:deadbaaf +foo==0.0.3 --hash=sha256:deadbaaf --hash=sha256:deadb11f --hash=sha256:5d15t """, "requirements_osx_download_only": """\ --platform=macosx_10_9_arm64 @@ -66,252 +92,393 @@ foo==0.0.3 --hash=sha256:deadbaaf --abi=cp39 foo==0.0.3 --hash=sha256:deadbaaf +""", + "requirements_sha512": """\ +foo==0.0.1 --hash=sha512:deadbeef """, "requirements_windows": """\ foo[extra]==0.0.2 --hash=sha256:deadbeef bar==0.0.1 --hash=sha256:deadb00f """, + "uv_lock_empty": """{"package":[]}""", + "uv_lock_foo": """{"package":[{"dependencies":[{"extra":"extra","name":"bar"}],"name":"foo","source":{"registry":"https://pypi.org/simple"},"version":"0.0.1","wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]}]}""", + "uv_lock_foo_bar": """{"package":[{"name":"bar","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"sdist":{"hash":"sha256:deadb00f","url":"https://files.pythonhosted.org/packages/bar-0.0.1.tar.gz"}},{"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]}]}""", + "uv_lock_foo_dep_extra": """{"package":[{"name":"bar","version":"0.0.2","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/bar-0.0.2-py3-none-any.whl"}]},{"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"dependencies":[{"name":"bar","extra":["extra1"]}],"wheels":[{"hash":"sha256:baadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]}]}""", + "uv_lock_foo_multi_versions": """{"package":[{"name":"foo","source":{"registry":"https://pypi.org/simple"},"version":"0.0.1","wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]},{"name":"foo","source":{"registry":"https://pypi.org/simple"},"version":"0.0.2","wheels":[{"hash":"sha256:deadb11f","url":"https://files.pythonhosted.org/packages/foo-0.0.2-py3-none-any.whl"}]}]}""", + "uv_lock_foo_multi_wheel_dedup": """{"package":[{"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:aaa","url":"https://files.pythonhosted.org/packages/foo-0.0.1-cp39-cp39-manylinux_2_17_x86_64.whl"},{"hash":"sha256:bbb","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]}]}""", + "uv_lock_foo_only": """{"package":[{"name":"foo","source":{"registry":"https://pypi.org/simple"},"version":"0.0.2"}]}""", + "uv_lock_foo_optional_deps": """{"package":[{"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"optional-dependencies":{"extra1":[],"extra2":[]},"wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]}]}""", + "uv_lock_foo_requires_dist_extras": """{"package":[{"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]},{"name":"root-pkg","source":{"virtual":"."},"version":"0.0.0","dependencies":[{"name":"foo"}],"metadata":{"requires-dist":[{"name":"foo","extras":["all"]}]}}]}""", + "uv_lock_foo_resolution_markers_dedup": """{"package":[{"name":"foo","source":{"registry":"https://pypi.org/simple"},"version":"0.0.1","resolution-markers":["sys_platform == 'linux'"],"wheels":[{"hash":"sha256:aaa","url":"https://files.pythonhosted.org/packages/foo-0.0.1-cp39-cp39-manylinux_2_17_x86_64.whl"},{"hash":"sha256:bbb","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]},{"name":"foo","source":{"registry":"https://pypi.org/simple"},"version":"0.0.2","resolution-markers":["sys_platform == 'darwin'"],"wheels":[{"hash":"sha256:ccc","url":"https://files.pythonhosted.org/packages/foo-0.0.2-cp39-cp39-macosx_11_0_arm64.whl"},{"hash":"sha256:ddd","url":"https://files.pythonhosted.org/packages/foo-0.0.2-py3-none-any.whl"}]}]}""", + "uv_lock_foo_sdist": """{"package":[{"name":"foo","sdist":{"hash":"sha256:feedcafe","url":"https://files.pythonhosted.org/packages/foo-0.0.1.tar.gz"},"source":{"registry":"https://pypi.org/simple"},"version":"0.0.1","wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]}]}""", + "uv_lock_foo_sha512": """{"package":[{"name":"foo","source":{"registry":"https://pypi.org/simple"},"version":"0.0.1","wheels":[{"hash":"sha512:deadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]}]}""", + "uv_lock_foo_virtual": """{"package":[{"name":"foo","source":{"registry":"https://pypi.org/simple"},"version":"0.0.1","wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]},{"name":"virtual-pkg","source":{"virtual":true},"version":"0.0.0"}]}""", + "uv_lock_foo_with_extras": """{"package":[{"name":"foo","provides-extras":["extra"],"source":{"registry":"https://pypi.org/simple"},"version":"0.0.1","wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]}]}""", + "uv_lock_git_vcs": """{"package":[{"name":"foo","source":{"git":"https://github.com/org/foo.git"},"version":"0.1.0"}]}""", + "uv_lock_rules_python_pkg": """{"package":[{"name":"rules_python","source":{"registry":"https://pypi.org/simple"},"version":"0.0.1","wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/rules_python-0.0.1-py3-none-any.whl"}]}]}""", } - return struct( - os = struct( - name = "linux", - arch = "x86_64", - ), - read = lambda x: testdata[x], + return mocks.mctx( + os_name = "linux", + arch_name = "x86_64", + mock_files = testdata, ) _tests = [] +def _make_platforms(platform_names): + """Create minimal platform structs for testing, matching py3-none-any wheels.""" + platforms = {} + for name in platform_names: + platforms[name] = struct( + env = pep508_env(python_version = "3.11.0", os = "linux", arch = "x86_64"), + whl_abi_tags = ["none"], + whl_platform_tags = ["any"], + ) + return platforms + +def parse_requirements(debug = False, **kwargs): + """Get requirements by calling the original parse_requirements. + + Args: + debug: If True, set verbosity to TRACE. + **kwargs: forwarded to the underlying function. + + Returns: + The result of the underlying parse_requirements call. + """ + kwargs.setdefault("toml_decode", json.decode) + + # Provide default platforms when not specified. + if "platforms" not in kwargs: + if "requirements_by_platform" in kwargs: + platform_names = {} + for _plats in kwargs["requirements_by_platform"].values(): + for _p in _plats: + platform_names[_p] = None + platform_names = sorted(platform_names) + kwargs["platforms"] = _make_platforms(platform_names) + elif "uv_lock" in kwargs: + kwargs["platforms"] = _make_platforms(["linux_x86_64"]) + + return _parse_requirements( + ctx = _mock_ctx(), + logger = repo_utils.logger(struct( + getenv = { + REPO_DEBUG_ENV_VAR: "1", + REPO_VERBOSITY_ENV_VAR: "TRACE" if debug else "INFO", + }.get, + ), "unit-test"), + **kwargs + ) + def _test_simple(env): + """Test basic parsing of a single ``requirements_lock`` file.""" got = parse_requirements( - ctx = _mock_ctx(), requirements_by_platform = { "requirements_lock": ["linux_x86_64", "windows_x86_64"], }, ) - env.expect.that_dict(got).contains_exactly({ - "foo": [ - struct( - distribution = "foo", - extra_pip_args = [], - requirement_line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", - srcs = struct( - requirement = "foo[extra]==0.0.1", - shas = ["deadbeef"], - version = "0.0.1", - ), - target_platforms = [ - "linux_x86_64", - "windows_x86_64", - ], - whls = [], - sdist = None, - is_exposed = True, - ), - ], - }) - env.expect.that_str( - select_requirement( - got["foo"], - platform = "linux_x86_64", - ).srcs.version, - ).equals("0.0.1") + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", + target_platforms = [ + "linux_x86_64", + "windows_x86_64", + ], + url = "", + filename = "", + digest = "", + yanked = None, + ), + ], + ), + ]) _tests.append(_test_simple) +def _test_direct_urls_integration(env): + """Check that we are using the filename from index_sources.""" + got = parse_requirements( + requirements_by_platform = { + "requirements_direct": ["linux_x86_64"], + "requirements_direct_sdist": ["osx_x86_64"], + }, + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = True, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + filename = "foo-1.1.tar.gz", + requirement_line = "foo @ https://github.com/org/foo/downloads/foo-1.1.tar.gz", + digest = "", + target_platforms = ["osx_x86_64"], + url = "https://github.com/org/foo/downloads/foo-1.1.tar.gz", + yanked = None, + ), + struct( + distribution = "foo", + extra_pip_args = [], + filename = "package.whl", + requirement_line = "foo[extra]", + digest = "", + target_platforms = ["linux_x86_64"], + url = "https://some-url/package.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_direct_urls_integration) + +def _test_direct_urls_no_extract(env): + """Check that URL requirements are not dropped when extract_url_srcs=False.""" + got = parse_requirements( + requirements_by_platform = { + "requirements_direct": ["linux_x86_64"], + "requirements_direct_sdist": ["osx_x86_64"], + }, + extract_url_srcs = False, + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = True, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + filename = "", + requirement_line = "foo @ https://github.com/org/foo/downloads/foo-1.1.tar.gz", + digest = "", + target_platforms = ["osx_x86_64"], + url = "", + yanked = None, + ), + struct( + distribution = "foo", + extra_pip_args = [], + filename = "", + requirement_line = "foo[extra] @ https://some-url/package.whl", + digest = "", + target_platforms = ["linux_x86_64"], + url = "", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_direct_urls_no_extract) + def _test_extra_pip_args(env): + """Test that ``extra_pip_args`` are merged with per-requirement-file args.""" got = parse_requirements( - ctx = _mock_ctx(), requirements_by_platform = { "requirements_extra_args": ["linux_x86_64"], }, extra_pip_args = ["--trusted-host=example.org"], ) - env.expect.that_dict(got).contains_exactly({ - "foo": [ - struct( - distribution = "foo", - extra_pip_args = ["--index-url=example.org", "--trusted-host=example.org"], - requirement_line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", - srcs = struct( - requirement = "foo[extra]==0.0.1", - shas = ["deadbeef"], - version = "0.0.1", - ), - target_platforms = [ - "linux_x86_64", - ], - whls = [], - sdist = None, - is_exposed = True, - ), - ], - }) - env.expect.that_str( - select_requirement( - got["foo"], - platform = "linux_x86_64", - ).srcs.version, - ).equals("0.0.1") + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = ["--index-url=example.org", "--trusted-host=example.org"], + requirement_line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", + target_platforms = [ + "linux_x86_64", + ], + url = "", + filename = "", + digest = "", + yanked = None, + ), + ], + ), + ]) _tests.append(_test_extra_pip_args) def _test_dupe_requirements(env): + """Test that duplicate requirement entries are deduplicated.""" got = parse_requirements( - ctx = _mock_ctx(), requirements_by_platform = { "requirements_lock_dupe": ["linux_x86_64"], }, ) - env.expect.that_dict(got).contains_exactly({ - "foo": [ - struct( - distribution = "foo", - extra_pip_args = [], - requirement_line = "foo[extra,extra_2]==0.0.1 --hash=sha256:deadbeef", - srcs = struct( - requirement = "foo[extra,extra_2]==0.0.1", - shas = ["deadbeef"], - version = "0.0.1", + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo[extra,extra_2]==0.0.1 --hash=sha256:deadbeef", + target_platforms = ["linux_x86_64"], + url = "", + filename = "", + digest = "", + yanked = None, ), - target_platforms = ["linux_x86_64"], - whls = [], - sdist = None, - is_exposed = True, - ), - ], - }) + ], + ), + ]) _tests.append(_test_dupe_requirements) def _test_multi_os(env): + """Test per-OS requirements parsing with ``select_requirement``.""" got = parse_requirements( - ctx = _mock_ctx(), requirements_by_platform = { "requirements_linux": ["linux_x86_64"], "requirements_windows": ["windows_x86_64"], }, ) - env.expect.that_dict(got).contains_exactly({ - "bar": [ - struct( - distribution = "bar", - extra_pip_args = [], - requirement_line = "bar==0.0.1 --hash=sha256:deadb00f", - srcs = struct( - requirement = "bar==0.0.1", - shas = ["deadb00f"], - version = "0.0.1", - ), - target_platforms = ["windows_x86_64"], - whls = [], - sdist = None, - is_exposed = False, - ), - ], - "foo": [ - struct( - distribution = "foo", - extra_pip_args = [], - requirement_line = "foo==0.0.3 --hash=sha256:deadbaaf", - srcs = struct( - requirement = "foo==0.0.3", - shas = ["deadbaaf"], - version = "0.0.3", + env.expect.that_collection(got).contains_exactly([ + struct( + name = "bar", + index_url = "", + is_exposed = False, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "bar", + extra_pip_args = [], + requirement_line = "bar==0.0.1 --hash=sha256:deadb00f", + target_platforms = ["windows_x86_64"], + url = "", + filename = "", + digest = "", + yanked = None, ), - target_platforms = ["linux_x86_64"], - whls = [], - sdist = None, - is_exposed = True, - ), - struct( - distribution = "foo", - extra_pip_args = [], - requirement_line = "foo[extra]==0.0.2 --hash=sha256:deadbeef", - srcs = struct( - requirement = "foo[extra]==0.0.2", - shas = ["deadbeef"], - version = "0.0.2", - ), - target_platforms = ["windows_x86_64"], - whls = [], - sdist = None, - is_exposed = True, - ), - ], - }) + ], + ), + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = True, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.3 --hash=sha256:deadbaaf --hash=sha256:5d15t", + target_platforms = ["linux_x86_64"], + url = "", + filename = "", + digest = "", + yanked = None, + ), + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo[extra]==0.0.2 --hash=sha256:deadbeef", + target_platforms = ["windows_x86_64"], + url = "", + filename = "", + digest = "", + yanked = None, + ), + ], + ), + ]) env.expect.that_str( select_requirement( - got["foo"], + got[1].srcs, platform = "windows_x86_64", - ).srcs.version, - ).equals("0.0.2") + ).requirement_line, + ).equals("foo[extra]==0.0.2 --hash=sha256:deadbeef") _tests.append(_test_multi_os) def _test_multi_os_legacy(env): + """Test download-only per-OS requirements parsing.""" got = parse_requirements( - ctx = _mock_ctx(), requirements_by_platform = { "requirements_linux_download_only": ["cp39_linux_x86_64"], "requirements_osx_download_only": ["cp39_osx_aarch64"], }, ) - env.expect.that_dict(got).contains_exactly({ - "bar": [ - struct( - distribution = "bar", - extra_pip_args = ["--platform=manylinux_2_17_x86_64", "--python-version=39", "--implementation=cp", "--abi=cp39"], - is_exposed = False, - requirement_line = "bar==0.0.1 --hash=sha256:deadb00f", - sdist = None, - srcs = struct( - requirement = "bar==0.0.1", - shas = ["deadb00f"], - version = "0.0.1", - ), - target_platforms = ["cp39_linux_x86_64"], - whls = [], - ), - ], - "foo": [ - struct( - distribution = "foo", - extra_pip_args = ["--platform=manylinux_2_17_x86_64", "--python-version=39", "--implementation=cp", "--abi=cp39"], - is_exposed = True, - requirement_line = "foo==0.0.1 --hash=sha256:deadbeef", - sdist = None, - srcs = struct( - requirement = "foo==0.0.1", - shas = ["deadbeef"], - version = "0.0.1", - ), - target_platforms = ["cp39_linux_x86_64"], - whls = [], - ), - struct( - distribution = "foo", - extra_pip_args = ["--platform=macosx_10_9_arm64", "--python-version=39", "--implementation=cp", "--abi=cp39"], - is_exposed = True, - requirement_line = "foo==0.0.3 --hash=sha256:deadbaaf", - sdist = None, - srcs = struct( - requirement = "foo==0.0.3", - shas = ["deadbaaf"], - version = "0.0.3", - ), - target_platforms = ["cp39_osx_aarch64"], - whls = [], - ), - ], - }) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "bar", + index_url = "", + is_exposed = False, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "bar", + extra_pip_args = ["--platform=manylinux_2_17_x86_64", "--python-version=39", "--implementation=cp", "--abi=cp39"], + requirement_line = "bar==0.0.1 --hash=sha256:deadb00f", + target_platforms = ["cp39_linux_x86_64"], + url = "", + filename = "", + digest = "", + yanked = None, + ), + ], + ), + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = True, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = ["--platform=manylinux_2_17_x86_64", "--python-version=39", "--implementation=cp", "--abi=cp39"], + requirement_line = "foo==0.0.1 --hash=sha256:deadbeef", + target_platforms = ["cp39_linux_x86_64"], + url = "", + filename = "", + digest = "", + yanked = None, + ), + struct( + distribution = "foo", + extra_pip_args = ["--platform=macosx_10_9_arm64", "--python-version=39", "--implementation=cp", "--abi=cp39"], + requirement_line = "foo==0.0.3 --hash=sha256:deadbaaf", + target_platforms = ["cp39_osx_aarch64"], + url = "", + filename = "", + digest = "", + yanked = None, + ), + ], + ), + ]) _tests.append(_test_multi_os_legacy) def _test_select_requirement_none_platform(env): + """Test that ``select_requirement`` returns the first src when platform is ``None``.""" got = select_requirement( [ struct( @@ -326,107 +493,1228 @@ def _test_select_requirement_none_platform(env): _tests.append(_test_select_requirement_none_platform) def _test_env_marker_resolution(env): - def _mock_eval_markers(_, input): - ret = { - "foo[extra]==0.0.1 ;marker --hash=sha256:deadbeef": ["cp311_windows_x86_64"], - } - - env.expect.that_collection(input.keys()).contains_exactly(ret.keys()) - env.expect.that_collection(input.values()[0]).contains_exactly(["cp311_linux_super_exotic", "cp311_windows_x86_64"]) - return ret + """Test environment marker resolution with platform env information.""" got = parse_requirements( - ctx = _mock_ctx(), requirements_by_platform = { "requirements_marker": ["cp311_linux_super_exotic", "cp311_windows_x86_64"], }, - evaluate_markers = _mock_eval_markers, - ) - env.expect.that_dict(got).contains_exactly({ - "bar": [ - struct( - distribution = "bar", - extra_pip_args = [], - is_exposed = True, - requirement_line = "bar==0.0.1 --hash=sha256:deadbeef", - sdist = None, - srcs = struct( - requirement = "bar==0.0.1", - shas = ["deadbeef"], - version = "0.0.1", - ), - target_platforms = ["cp311_linux_super_exotic", "cp311_windows_x86_64"], - whls = [], + platforms = { + "cp311_linux_super_exotic": struct( + env = pep508_env(os = "linux", arch = "x86_64", python_version = "3.11.0"), + whl_abi_tags = [], + whl_platform_tags = [], ), - ], - "foo": [ - struct( - distribution = "foo", - extra_pip_args = [], - # This is not exposed because we also have `linux_super_exotic` in the platform list - is_exposed = False, - requirement_line = "foo[extra]==0.0.1 ;marker --hash=sha256:deadbeef", - sdist = None, - srcs = struct( - requirement = "foo[extra]==0.0.1 ;marker", - shas = ["deadbeef"], - version = "0.0.1", - ), - target_platforms = ["cp311_windows_x86_64"], - whls = [], + "cp311_windows_x86_64": struct( + env = pep508_env(os = "windows", arch = "x86_64", python_version = "3.11.0"), + whl_abi_tags = [], + whl_platform_tags = [], ), - ], - }) - env.expect.that_str( - select_requirement( - got["foo"], - platform = "windows_x86_64", - ).srcs.version, - ).equals("0.0.1") + }, + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "bar", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "bar", + extra_pip_args = [], + requirement_line = "bar==0.0.1 --hash=sha256:deadbeef", + target_platforms = ["cp311_linux_super_exotic", "cp311_windows_x86_64"], + url = "", + filename = "", + digest = "", + yanked = None, + ), + ], + ), + struct( + name = "foo", + index_url = "", + is_exposed = False, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", + target_platforms = ["cp311_windows_x86_64"], + url = "", + filename = "", + digest = "", + yanked = None, + ), + ], + ), + ]) _tests.append(_test_env_marker_resolution) def _test_different_package_version(env): + """Test that different package versions across platforms are handled.""" got = parse_requirements( - ctx = _mock_ctx(), requirements_by_platform = { - "requirements_different_package_version": ["linux_x86_64"], + "requirements_foo": ["linux_aarch64"], + "requirements_foo_local": ["linux_x86_64"], }, ) - env.expect.that_dict(got).contains_exactly({ - "foo": [ - struct( - distribution = "foo", - extra_pip_args = [], - requirement_line = "foo==0.0.1 --hash=sha256:deadb00f", - srcs = struct( - requirement = "foo==0.0.1", - shas = ["deadb00f"], - version = "0.0.1", + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = True, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1 --hash=sha256:deadb00f", + target_platforms = ["linux_aarch64"], + url = "", + filename = "", + digest = "", + yanked = None, ), - target_platforms = ["linux_x86_64"], - whls = [], - sdist = None, - is_exposed = True, + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1+local --hash=sha256:deadbeef", + target_platforms = ["linux_x86_64"], + url = "", + filename = "", + digest = "", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_different_package_version) + +def _test_different_package_extras(env): + """Test that different extras across platforms are handled.""" + got = parse_requirements( + requirements_by_platform = { + "requirements_foo": ["linux_aarch64"], + "requirements_lock": ["linux_x86_64"], + }, + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = True, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1 --hash=sha256:deadb00f", + target_platforms = ["linux_aarch64"], + url = "", + filename = "", + digest = "", + yanked = None, + ), + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", + target_platforms = ["linux_x86_64"], + url = "", + filename = "", + digest = "", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_different_package_extras) + +def _test_optional_hash(env): + """Test parsing of requirements with optional hashes and URLs.""" + got = parse_requirements( + requirements_by_platform = { + "requirements_optional_hash": ["linux_x86_64"], + }, + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "bar", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "bar", + extra_pip_args = [], + requirement_line = "bar==0.0.4", + target_platforms = ["linux_x86_64"], + url = "https://example.org/bar-0.0.4.whl", + filename = "bar-0.0.4.whl", + digest = "", + yanked = None, + ), + ], + ), + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.5", + target_platforms = ["linux_x86_64"], + url = "https://example.org/foo-0.0.5.whl", + filename = "foo-0.0.5.whl", + digest = "sha256:deadbeef", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_optional_hash) + +def _test_git_sources(env): + """Test parsing of git-sourced requirements.""" + got = parse_requirements( + requirements_by_platform = { + "requirements_git": ["linux_x86_64"], + }, + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo @ git+https://github.com/org/foo.git@deadbeef", + target_platforms = ["linux_x86_64"], + url = "", + filename = "", + digest = "", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_git_sources) + +def _test_overlapping_shas_with_index_results(env): + """Test that index results with overlapping shas are matched to the correct platform.""" + got = parse_requirements( + requirements_by_platform = { + "requirements_linux": ["cp39_linux_x86_64"], + "requirements_osx": ["cp39_osx_x86_64"], + }, + platforms = { + "cp39_linux_x86_64": struct( + env = pep508_env( + python_version = "3.9.0", + os = "linux", + arch = "x86_64", + ), + whl_abi_tags = ["none"], + whl_platform_tags = ["any"], ), - struct( - distribution = "foo", - extra_pip_args = [], - requirement_line = "foo==0.0.1+local --hash=sha256:deadbeef", - srcs = struct( - requirement = "foo==0.0.1+local", - shas = ["deadbeef"], - version = "0.0.1+local", + "cp39_osx_x86_64": struct( + env = pep508_env( + python_version = "3.9.0", + os = "osx", + arch = "x86_64", ), - target_platforms = ["linux_x86_64"], - whls = [], - sdist = None, - is_exposed = True, + whl_abi_tags = ["none"], + whl_platform_tags = ["macosx_*_x86_64"], ), - ], - }) + }, + get_index_urls = lambda _, __, **kwargs: { + "foo": struct( + index_url = "https://example.com", + sdists = { + "sha256:5d15t": struct( + url = "sdist", + digest = "sha256:5d15t", + filename = "foo-0.0.1.tar.gz", + yanked = None, + ), + }, + whls = { + "sha256:deadb11f": struct( + url = "super2", + digest = "sha256:deadb11f", + filename = "foo-0.0.1-py3-none-macosx_14_0_x86_64.whl", + yanked = None, + ), + "sha256:deadbaaf": struct( + url = "super2", + digest = "sha256:deadbaaf", + filename = "foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + }, + ), + }, + ) -_tests.append(_test_different_package_version) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "https://example.com", + is_exposed = True, + is_multiple_versions = True, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + filename = "foo-0.0.1-py3-none-any.whl", + requirement_line = "foo==0.0.3", + digest = "sha256:deadbaaf", + target_platforms = ["cp39_linux_x86_64"], + url = "super2", + yanked = None, + ), + struct( + distribution = "foo", + extra_pip_args = [], + filename = "foo-0.0.1-py3-none-macosx_14_0_x86_64.whl", + requirement_line = "foo==0.0.3", + digest = "sha256:deadb11f", + target_platforms = ["cp39_osx_x86_64"], + url = "super2", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_overlapping_shas_with_index_results) + +def _test_non_sha256_hash_matching(env): + """Test that non-sha256 pins are matched against the index digests.""" + got = parse_requirements( + requirements_by_platform = { + "requirements_sha512": ["cp311_linux_x86_64"], + }, + platforms = { + "cp311_linux_x86_64": struct( + env = pep508_env( + python_version = "3.11.0", + os = "linux", + arch = "x86_64", + ), + whl_abi_tags = ["none"], + whl_platform_tags = ["any"], + ), + }, + get_index_urls = lambda _, __, **kwargs: { + "foo": struct( + index_url = "https://example.com", + sdists = {}, + whls = { + "sha512:deadbeef": struct( + url = "https://example.com/foo-0.0.1-py3-none-any.whl", + digest = "sha512:deadbeef", + filename = "foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + }, + ), + }, + ) + + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "https://example.com", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + filename = "foo-0.0.1-py3-none-any.whl", + requirement_line = "foo==0.0.1", + digest = "sha512:deadbeef", + target_platforms = ["cp311_linux_x86_64"], + url = "https://example.com/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_non_sha256_hash_matching) + +def _test_get_index_urls_different_versions(env): + """Test that different versions from index URLs are matched correctly per platform.""" + got = parse_requirements( + requirements_by_platform = { + "requirements_multi_version": [ + "cp39_linux_x86_64", + "cp310_linux_x86_64", + ], + }, + platforms = { + "cp310_linux_x86_64": struct( + env = pep508_env( + python_version = "3.10.0", + os = "linux", + arch = "x86_64", + ), + whl_abi_tags = ["none"], + whl_platform_tags = ["any"], + ), + "cp39_linux_x86_64": struct( + env = pep508_env( + python_version = "3.9.0", + os = "linux", + arch = "x86_64", + ), + whl_abi_tags = ["none"], + whl_platform_tags = ["any"], + ), + }, + get_index_urls = lambda _, __, **kwargs: { + "foo": struct( + index_url = "", + sdists = {}, + whls = { + "sha256:deadb11f": struct( + url = "super2", + digest = "sha256:deadb11f", + filename = "foo-0.0.2-py3-none-any.whl", + yanked = None, + ), + "sha256:deadbaaf": struct( + url = "super2", + digest = "sha256:deadbaaf", + filename = "foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + }, + ), + }, + ) + + env.expect.that_collection(got).contains_exactly([ + struct( + name = "boo", + index_url = "", + is_exposed = False, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "boo", + extra_pip_args = [], + filename = "", + requirement_line = "boo==0.0.4 --hash=sha256:deadbaaf", + digest = "", + target_platforms = ["cp39_linux_x86_64"], + url = "", + yanked = None, + ), + ], + ), + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = True, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + filename = "", + requirement_line = "foo==0.0.1 --hash=sha256:deadbeef", + digest = "", + target_platforms = ["cp39_linux_x86_64"], + url = "", + yanked = None, + ), + struct( + distribution = "foo", + extra_pip_args = [], + filename = "foo-0.0.2-py3-none-any.whl", + requirement_line = "foo==0.0.2", + digest = "sha256:deadb11f", + target_platforms = ["cp310_linux_x86_64"], + url = "super2", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_get_index_urls_different_versions) + +def _test_get_index_urls_cross_platform(env): + """Verifies that distributions from all requirement files are passed to ``get_index_urls``. + + This ensures the lockfile facts are platform-independent. + """ + calls = [] + + def _get_index_urls(_, distributions, **__): + calls.append({k: list(v) for k, v in distributions.items()}) + return {} + + parse_requirements( + requirements_by_platform = { + "requirements_osx": ["cp39_osx_x86_64"], + # requirements_windows has no matching platforms (simulating + # a macOS build where windows-specific files aren't used). + "requirements_windows": [], + }, + platforms = { + "cp39_osx_x86_64": struct( + env = pep508_env( + python_version = "3.9.0", + os = "osx", + arch = "x86_64", + ), + whl_abi_tags = ["none"], + whl_platform_tags = ["macosx_*_x86_64"], + ), + }, + get_index_urls = _get_index_urls, + ) + + # distributions must include packages from ALL files, even those with + # no matching platforms: + # - foo: 0.0.2 from requirements_windows, 0.0.3 from requirements_osx + # - bar: 0.0.1 from requirements_windows only + env.expect.that_collection(calls).contains_exactly([ + { + "bar": ["0.0.1"], + "foo": ["0.0.2", "0.0.3"], + }, + ]) + +_tests.append(_test_get_index_urls_cross_platform) + +def _test_get_index_urls_single_py_version(env): + """Test index URL matching when only a single Python version is used.""" + got = parse_requirements( + requirements_by_platform = { + "requirements_multi_version": [ + "cp310_linux_x86_64", + ], + }, + platforms = { + "cp310_linux_x86_64": struct( + env = pep508_env( + python_version = "3.10.0", + os = "linux", + arch = "x86_64", + ), + whl_abi_tags = ["none"], + whl_platform_tags = ["any"], + ), + }, + get_index_urls = lambda _, __, **kwargs: { + "foo": struct( + index_url = "", + sdists = {}, + whls = { + "sha256:deadb11f": struct( + url = "super2", + digest = "sha256:deadb11f", + filename = "foo-0.0.2-py3-none-any.whl", + yanked = None, + ), + }, + ), + }, + ) + + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + filename = "foo-0.0.2-py3-none-any.whl", + requirement_line = "foo==0.0.2", + digest = "sha256:deadb11f", + target_platforms = ["cp310_linux_x86_64"], + url = "super2", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_get_index_urls_single_py_version) + +def _test_get_index_urls_all_versions(env): + """Test that all versions from all requirement files are passed to ``get_index_urls``.""" + calls = [] + + def _get_index_urls(_, distributions, **__): + calls.append({k: list(v) for k, v in distributions.items()}) + return {} + + parse_requirements( + requirements_by_platform = { + "requirements_multi_version": ["cp39_linux_x86_64"], + }, + platforms = { + "cp39_linux_x86_64": struct( + env = pep508_env( + python_version = "3.9.0", + os = "linux", + arch = "x86_64", + ), + whl_abi_tags = ["none"], + whl_platform_tags = ["any"], + ), + }, + get_index_urls = _get_index_urls, + ) + + env.expect.that_collection(calls).contains_exactly([ + { + # boo should be also passed even though it is present on one platform. + "boo": ["0.0.4"], + # Both versions 0.0.1 and 0.0.2 should be passed to get_index_urls, even + # though only 0.0.1 matches the cp39_linux_x86_64 platform markers. + "foo": ["0.0.1", "0.0.2"], + }, + ]) + +_tests.append(_test_get_index_urls_all_versions) + +def _test_uv_lock_consistent(env): + """Test that uv_lock with requirements_by_platform uses correct platforms.""" + got = parse_requirements( + requirements_by_platform = { + "requirements_lock": ["linux_x86_64", "windows_x86_64"], + }, + uv_lock = "uv_lock_foo_with_extras", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "https://pypi.org/simple/foo", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo[extra]==0.0.1", + target_platforms = ["linux_x86_64", "windows_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + digest = "sha256:deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_consistent) + +def _test_uv_lock_primary_source(env): + """Test that uv.lock can be used as the sole source without requirements files.""" + got = parse_requirements( + uv_lock = "uv_lock_foo_sdist", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "https://pypi.org/simple/foo", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + digest = "sha256:deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_primary_source) + +def _test_uv_lock_non_sha256_hash(env): + """Test that non-sha256 uv.lock hashes are propagated to the sources.""" + got = parse_requirements( + uv_lock = "uv_lock_foo_sha512", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "https://pypi.org/simple/foo", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + digest = "sha512:deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_non_sha256_hash) + +def _test_uv_lock_primary_source_multiple_versions(env): + """Test that uv.lock with multiple versions of the same package works.""" + got = parse_requirements( + uv_lock = "uv_lock_foo_multi_versions", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "https://pypi.org/simple/foo", + is_exposed = True, + is_multiple_versions = True, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + digest = "sha256:deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.2", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.2-py3-none-any.whl", + digest = "sha256:deadb11f", + url = "https://files.pythonhosted.org/packages/foo-0.0.2-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_primary_source_multiple_versions) + +def _test_uv_lock_primary_source_with_extras(env): + """Test that uv.lock extras are included in requirement lines.""" + got = parse_requirements( + uv_lock = "uv_lock_foo_with_extras", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "https://pypi.org/simple/foo", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo[extra]==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + digest = "sha256:deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_primary_source_with_extras) + +def _test_uv_lock_primary_source_includes_virtual(env): + """Test that virtual packages in uv.lock are included.""" + got = parse_requirements( + uv_lock = "uv_lock_foo_virtual", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "https://pypi.org/simple/foo", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + digest = "sha256:deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + struct( + name = "virtual_pkg", + index_url = "", + is_exposed = False, + is_multiple_versions = False, + srcs = [], + ), + ]) + +_tests.append(_test_uv_lock_primary_source_includes_virtual) + +def _test_uv_lock_cross_consistent(env): + """Test that the uv.lock and requirements work together for cross-platform.""" + got = parse_requirements( + requirements_by_platform = { + "requirements_lock": ["linux_x86_64", "windows_x86_64"], + }, + uv_lock = "uv_lock_foo_with_extras", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "https://pypi.org/simple/foo", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo[extra]==0.0.1", + target_platforms = ["linux_x86_64", "windows_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + digest = "sha256:deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_cross_consistent) + +def _test_uv_lock_vcs_entry(env): + """Test that VCS entries in uv.lock are handled without crashing.""" + got = parse_requirements( + uv_lock = "uv_lock_git_vcs", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.1.0", + target_platforms = ["linux_x86_64"], + filename = "foo.git", + digest = "", + url = "https://github.com/org/foo.git", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_vcs_entry) + +def _test_uv_lock_rules_python_pkg_not_skipped(env): + """Test that 'rules_python' package is not skipped from uv.lock.""" + got = parse_requirements( + uv_lock = "uv_lock_rules_python_pkg", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "rules_python", + index_url = "https://pypi.org/simple/rules-python", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "rules_python", + extra_pip_args = [], + requirement_line = "rules_python==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "rules_python-0.0.1-py3-none-any.whl", + digest = "sha256:deadbeef", + url = "https://files.pythonhosted.org/packages/rules_python-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_rules_python_pkg_not_skipped) + +def _test_uv_lock_no_consistency_check(env): + """Test that uv.lock is used as the primary source when both uv.lock and requirements exist.""" + got = parse_requirements( + requirements_by_platform = { + "requirements_lock": ["linux_x86_64"], + }, + uv_lock = "uv_lock_foo", + ) + + # The result comes from uv.lock (no extras since uv_lock_foo doesn't have provides-extras) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "https://pypi.org/simple/foo", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + digest = "sha256:deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_no_consistency_check) + +def _test_uv_lock_multiple_packages(env): + """Test that multiple packages from uv.lock are all returned.""" + got = parse_requirements( + uv_lock = "uv_lock_foo_bar", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "bar", + index_url = "https://pypi.org/simple/bar", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "bar", + extra_pip_args = [], + requirement_line = "bar==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "bar-0.0.1.tar.gz", + digest = "sha256:deadb00f", + url = "https://files.pythonhosted.org/packages/bar-0.0.1.tar.gz", + yanked = None, + ), + ], + ), + struct( + name = "foo", + index_url = "https://pypi.org/simple/foo", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + digest = "sha256:deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_multiple_packages) + +def _test_uv_lock_with_extra_pip_args(env): + """Test that extra_pip_args are passed through with uv.lock.""" + got = parse_requirements( + uv_lock = "uv_lock_foo", + extra_pip_args = ["--index-url=example.org"], + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "https://pypi.org/simple/foo", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = ["--index-url=example.org"], + requirement_line = "foo==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + digest = "sha256:deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_with_extra_pip_args) + +def _test_uv_lock_multi_os_with_requirements(env): + """Test that uv.lock works with requirements_by_platform for multi-platform.""" + got = parse_requirements( + requirements_by_platform = { + "requirements_foo": ["linux_aarch64"], + "requirements_lock": ["linux_x86_64", "windows_x86_64"], + }, + uv_lock = "uv_lock_foo", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "https://pypi.org/simple/foo", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1", + target_platforms = ["linux_aarch64", "linux_x86_64", "windows_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + digest = "sha256:deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_multi_os_with_requirements) + +def _test_uv_lock_extras_optional_deps(env): + """Test that extras from optional-dependencies in uv.lock are included.""" + got = parse_requirements( + uv_lock = "uv_lock_foo_optional_deps", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "https://pypi.org/simple/foo", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo[extra1,extra2]==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + digest = "sha256:deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_extras_optional_deps) + +def _test_uv_lock_extras_dep_edge(env): + """Test that dep extra edges in uv.lock add extras to the dependency.""" + got = parse_requirements( + uv_lock = "uv_lock_foo_dep_extra", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "bar", + index_url = "https://pypi.org/simple/bar", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "bar", + extra_pip_args = [], + requirement_line = "bar[extra1]==0.0.2", + target_platforms = ["linux_x86_64"], + filename = "bar-0.0.2-py3-none-any.whl", + digest = "sha256:deadbeef", + url = "https://files.pythonhosted.org/packages/bar-0.0.2-py3-none-any.whl", + yanked = None, + ), + ], + ), + struct( + name = "foo", + index_url = "https://pypi.org/simple/foo", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + digest = "sha256:baadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_extras_dep_edge) + +def _test_uv_lock_wheel_dedup_single_version(env): + """Test that overlapping wheels for a single version are deduplicated to one per platform.""" + got = parse_requirements( + uv_lock = "uv_lock_foo_multi_wheel_dedup", + platforms = { + "cp39_linux_x86_64": struct( + env = pep508_env(python_version = "3.9.0", os = "linux", arch = "x86_64"), + whl_abi_tags = ["none", "abi3", "cp39"], + whl_platform_tags = ["any", "linux_x86_64", "manylinux_*_x86_64"], + ), + }, + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "https://pypi.org/simple/foo", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1", + target_platforms = ["cp39_linux_x86_64"], + filename = "foo-0.0.1-cp39-cp39-manylinux_2_17_x86_64.whl", + digest = "sha256:aaa", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-cp39-cp39-manylinux_2_17_x86_64.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_wheel_dedup_single_version) + +def _test_uv_lock_wheel_dedup_resolution_markers(env): + """Test that resolution-markers filtering and wheel dedup work together. + + Two versions of foo with resolution-markers for different platforms. + Each version has a platform-specific wheel and a generic py3-none-any wheel. + The dedup should pick the platform-specific wheel for each platform and + the resolution-markers should split versions across platforms. + """ + got = parse_requirements( + uv_lock = "uv_lock_foo_resolution_markers_dedup", + platforms = { + "cp39_linux_x86_64": struct( + env = pep508_env(python_version = "3.9.0", os = "linux", arch = "x86_64"), + whl_abi_tags = ["none", "abi3", "cp39"], + whl_platform_tags = ["any", "linux_x86_64", "manylinux_*_x86_64"], + ), + "cp39_osx_aarch64": struct( + env = pep508_env(python_version = "3.9.0", os = "osx", arch = "aarch64"), + whl_abi_tags = ["none", "abi3", "cp39"], + whl_platform_tags = ["any", "macosx_*_arm64"], + ), + }, + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "https://pypi.org/simple/foo", + is_exposed = True, + is_multiple_versions = True, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1", + target_platforms = ["cp39_linux_x86_64"], + filename = "foo-0.0.1-cp39-cp39-manylinux_2_17_x86_64.whl", + digest = "sha256:aaa", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-cp39-cp39-manylinux_2_17_x86_64.whl", + yanked = None, + ), + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.2", + target_platforms = ["cp39_osx_aarch64"], + filename = "foo-0.0.2-cp39-cp39-macosx_11_0_arm64.whl", + digest = "sha256:ccc", + url = "https://files.pythonhosted.org/packages/foo-0.0.2-cp39-cp39-macosx_11_0_arm64.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_wheel_dedup_resolution_markers) + +def _test_uv_lock_requires_dist_extras(env): + """Test that extras from metadata.requires-dist appear in requirement_line.""" + got = parse_requirements( + uv_lock = "uv_lock_foo_requires_dist_extras", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "https://pypi.org/simple/foo", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo[all]==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + digest = "sha256:deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + struct( + name = "root_pkg", + index_url = "", + is_exposed = False, + is_multiple_versions = False, + srcs = [], + ), + ]) + +_tests.append(_test_uv_lock_requires_dist_extras) def parse_requirements_test_suite(name): """Create the test suite. diff --git a/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl b/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl index d3c42a8864..975fcd0ee6 100644 --- a/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl +++ b/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl @@ -42,6 +42,29 @@ def _generate_html(*items): ]), ) +def _test_index(env): + # buildifier: disable=unsorted-dict-items + tests = [ + ( + [ + struct(attrs = ['href="/simple/foo/"'], filename = "foo"), + struct(attrs = ['href="./b-ar/"'], filename = "b-._.-aR"), + ], + { + "b_ar": "./b-ar/", + "foo": "/simple/foo/", + }, + ), + ] + + for (input, want) in tests: + html = _generate_html(*input) + got = parse_simpleapi_html(content = html, parse_index = True) + + env.expect.that_dict(got).contains_exactly(want) + +_tests.append(_test_index) + def _test_sdist(env): # buildifier: disable=unsorted-dict-items tests = [ @@ -52,13 +75,13 @@ def _test_sdist(env): 'data-requires-python=">=3.7"', ], filename = "foo-0.0.1.tar.gz", - url = "ignored", ), struct( filename = "foo-0.0.1.tar.gz", - sha256 = "deadbeefasource", + digest = "sha256:deadbeefasource", url = "https://example.org/full-url/foo-0.0.1.tar.gz", - yanked = False, + yanked = None, + version = "0.0.1", ), ), ( @@ -66,40 +89,79 @@ def _test_sdist(env): attrs = [ 'href="https://example.org/full-url/foo-0.0.1.tar.gz#sha256=deadbeefasource"', 'data-requires-python=">=3.7"', + "data-yanked", ], filename = "foo-0.0.1.tar.gz", - url = "ignored", ), struct( filename = "foo-0.0.1.tar.gz", - sha256 = "deadbeefasource", + digest = "sha256:deadbeefasource", url = "https://example.org/full-url/foo-0.0.1.tar.gz", - yanked = False, + version = "0.0.1", + yanked = "", + ), + ), + ( + struct( + attrs = [ + 'href="https://example.org/full-url/foo-0.0.1.tar.gz#sha256=deadbeefasource"', + 'data-requires-python="<=3.7"', + "data-yanked=\"Something with "quotes" over two lines\"", + ], + filename = "foo-0.0.1.tar.gz", + ), + struct( + filename = "foo-0.0.1.tar.gz", + digest = "sha256:deadbeefasource", + url = "https://example.org/full-url/foo-0.0.1.tar.gz", + version = "0.0.1", + # NOTE @aignas 2026-03-09: we preserve the white space + yanked = "Something \nwith \"quotes\"\nover two lines", + ), + ), + ( + struct( + attrs = [ + 'href="https://example.org/full-url/foo-0.0.1.tar.gz#sha256=deadbeefasource"', + 'data-requires-python=">=3.7"', + 'data-yanked=""', + ], + filename = "foo-0.0.1.tar.gz", + ), + struct( + filename = "foo-0.0.1.tar.gz", + digest = "sha256:deadbeefasource", + url = "https://example.org/full-url/foo-0.0.1.tar.gz", + version = "0.0.1", + yanked = "", ), ), ] for (input, want) in tests: html = _generate_html(input) - got = parse_simpleapi_html(url = input.url, content = html) + got = parse_simpleapi_html(content = html) env.expect.that_collection(got.sdists).has_size(1) env.expect.that_collection(got.whls).has_size(0) + env.expect.that_collection(got.hashes_by_version).has_size(1) if not got: fail("expected at least one element, but did not get anything from:\n{}".format(html)) actual = env.expect.that_struct( - got.sdists[want.sha256], + got.sdists[want.digest], attrs = dict( filename = subjects.str, - sha256 = subjects.str, + digest = subjects.str, url = subjects.str, - yanked = subjects.bool, + yanked = subjects.str, + version = subjects.str, ), ) actual.filename().equals(want.filename) - actual.sha256().equals(want.sha256) + actual.digest().equals(want.digest) actual.url().equals(want.url) actual.yanked().equals(want.yanked) + actual.version().equals(want.version) _tests.append(_test_sdist) @@ -115,35 +177,35 @@ def _test_whls(env): 'data-core-metadata="sha256=deadb00f"', ], filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - url = "ignored", ), struct( filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", metadata_sha256 = "deadb00f", metadata_url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata", - sha256 = "deadbeef", + digest = "sha256:deadbeef", url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - yanked = False, + version = "0.0.2", + yanked = None, ), ), ( struct( attrs = [ 'href="https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl#sha256=deadbeef"', - 'data-requires-python=">=3.7"', + 'data-requires-python=">=3.7"', 'data-dist-info-metadata="sha256=deadb00f"', 'data-core-metadata="sha256=deadb00f"', ], filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - url = "ignored", ), struct( filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", metadata_sha256 = "deadb00f", metadata_url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata", - sha256 = "deadbeef", + digest = "sha256:deadbeef", url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - yanked = False, + version = "0.0.2", + yanked = None, ), ), ( @@ -154,15 +216,15 @@ def _test_whls(env): 'data-core-metadata="sha256=deadb00f"', ], filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - url = "ignored", ), struct( filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", metadata_sha256 = "deadb00f", metadata_url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata", - sha256 = "deadbeef", + digest = "sha256:deadbeef", + version = "0.0.2", url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - yanked = False, + yanked = None, ), ), ( @@ -173,15 +235,15 @@ def _test_whls(env): 'data-dist-info-metadata="sha256=deadb00f"', ], filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - url = "ignored", ), struct( filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", metadata_sha256 = "deadb00f", metadata_url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata", - sha256 = "deadbeef", + digest = "sha256:deadbeef", + version = "0.0.2", url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - yanked = False, + yanked = None, ), ), ( @@ -191,134 +253,111 @@ def _test_whls(env): 'data-requires-python=">=3.7"', ], filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - url = "ignored", ), struct( filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", metadata_sha256 = "", metadata_url = "", - sha256 = "deadbeef", + digest = "sha256:deadbeef", url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - yanked = False, - ), - ), - ( - struct( - attrs = [ - 'href="../../foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl#sha256=deadbeef"', - 'data-requires-python=">=3.7"', - 'data-dist-info-metadata="sha256=deadb00f"', - ], - filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - url = "https://example.org/python-wheels/bar/foo/", - ), - struct( - filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - metadata_sha256 = "deadb00f", - metadata_url = "https://example.org/python-wheels/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata", - sha256 = "deadbeef", - url = "https://example.org/python-wheels/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - yanked = False, - ), - ), - ( - struct( - attrs = [ - 'href="/whl/torch-2.0.0-cp38-cp38-manylinux2014_aarch64.whl#sha256=deadbeef"', - ], - filename = "torch-2.0.0-cp38-cp38-manylinux2014_aarch64.whl", - url = "https://download.pytorch.org/whl/cpu/torch", - ), - struct( - filename = "torch-2.0.0-cp38-cp38-manylinux2014_aarch64.whl", - metadata_sha256 = "", - metadata_url = "", - sha256 = "deadbeef", - url = "https://download.pytorch.org/whl/torch-2.0.0-cp38-cp38-manylinux2014_aarch64.whl", - yanked = False, - ), - ), - ( - struct( - attrs = [ - 'href="/whl/torch-2.0.0-cp38-cp38-manylinux2014_aarch64.whl#sha256=notdeadbeef"', - ], - filename = "torch-2.0.0-cp38-cp38-manylinux2014_aarch64.whl", - url = "http://download.pytorch.org/whl/cpu/torch", - ), - struct( - filename = "torch-2.0.0-cp38-cp38-manylinux2014_aarch64.whl", - metadata_sha256 = "", - metadata_url = "", - sha256 = "notdeadbeef", - url = "http://download.pytorch.org/whl/torch-2.0.0-cp38-cp38-manylinux2014_aarch64.whl", - yanked = False, - ), - ), - ( - struct( - attrs = [ - 'href="1.0.0/mypy_extensions-1.0.0-py3-none-any.whl#sha256=deadbeef"', - ], - filename = "mypy_extensions-1.0.0-py3-none-any.whl", - url = "https://example.org/simple/mypy_extensions", - ), - struct( - filename = "mypy_extensions-1.0.0-py3-none-any.whl", - metadata_sha256 = "", - metadata_url = "", - sha256 = "deadbeef", - url = "https://example.org/simple/mypy_extensions/1.0.0/mypy_extensions-1.0.0-py3-none-any.whl", - yanked = False, - ), - ), - ( - struct( - attrs = [ - 'href="unknown://example.com/mypy_extensions-1.0.0-py3-none-any.whl#sha256=deadbeef"', - ], - filename = "mypy_extensions-1.0.0-py3-none-any.whl", - url = "https://example.org/simple/mypy_extensions", - ), - struct( - filename = "mypy_extensions-1.0.0-py3-none-any.whl", - metadata_sha256 = "", - metadata_url = "", - sha256 = "deadbeef", - url = "https://example.org/simple/mypy_extensions/unknown://example.com/mypy_extensions-1.0.0-py3-none-any.whl", - yanked = False, + version = "0.0.2", + yanked = None, ), ), ] for (input, want) in tests: html = _generate_html(input) - got = parse_simpleapi_html(url = input.url, content = html) + got = parse_simpleapi_html(content = html) env.expect.that_collection(got.sdists).has_size(0) env.expect.that_collection(got.whls).has_size(1) if not got: fail("expected at least one element, but did not get anything from:\n{}".format(html)) actual = env.expect.that_struct( - got.whls[want.sha256], + got.whls[want.digest], attrs = dict( filename = subjects.str, metadata_sha256 = subjects.str, metadata_url = subjects.str, - sha256 = subjects.str, + digest = subjects.str, url = subjects.str, - yanked = subjects.bool, + yanked = subjects.str, + version = subjects.str, ), ) actual.filename().equals(want.filename) actual.metadata_sha256().equals(want.metadata_sha256) actual.metadata_url().equals(want.metadata_url) - actual.sha256().equals(want.sha256) + actual.digest().equals(want.digest) actual.url().equals(want.url) actual.yanked().equals(want.yanked) + actual.version().equals(want.version) _tests.append(_test_whls) +def _test_non_sha256_fragment(env): + """The index may advertise digests calculated with any hash algorithm (PEP 503).""" + + html = _generate_html( + struct( + attrs = [ + 'href="https://example.org/full-url/foo-0.0.1-py3-none-any.whl#sha512=deadbeef"', + ], + filename = "foo-0.0.1-py3-none-any.whl", + ), + struct( + attrs = [ + 'href="https://example.org/full-url/foo-0.0.1.tar.gz#sha512=deadb00f"', + ], + filename = "foo-0.0.1.tar.gz", + ), + struct( + # A non-hash fragment is kept as part of the URL. + attrs = [ + 'href="https://example.org/full-url/foo-0.0.2-py3-none-any.whl#egg=foo"', + ], + filename = "foo-0.0.2-py3-none-any.whl", + ), + ) + got = parse_simpleapi_html(content = html) + + env.expect.that_collection(got.whls).has_size(2) + env.expect.that_collection(got.sdists).has_size(1) + env.expect.that_dict(got.hashes_by_version).contains_exactly({ + "0.0.1": ["sha512:deadbeef", "sha512:deadb00f"], + "0.0.2": [""], + }) + + whl = got.whls["sha512:deadbeef"] + env.expect.that_str(whl.digest).equals("sha512:deadbeef") + env.expect.that_str(whl.url).equals("https://example.org/full-url/foo-0.0.1-py3-none-any.whl") + + sdist = got.sdists["sha512:deadb00f"] + env.expect.that_str(sdist.digest).equals("sha512:deadb00f") + + no_hash_whl = got.whls[""] + env.expect.that_str(no_hash_whl.digest).equals("") + env.expect.that_str(no_hash_whl.url).equals("https://example.org/full-url/foo-0.0.2-py3-none-any.whl#egg=foo") + +_tests.append(_test_non_sha256_fragment) + +def _test_sha256_fragment_digest(env): + html = _generate_html( + struct( + attrs = [ + 'href="https://example.org/full-url/foo-0.0.1-py3-none-any.whl#sha256=deadbeef"', + ], + filename = "foo-0.0.1-py3-none-any.whl", + ), + ) + got = parse_simpleapi_html(content = html) + + whl = got.whls["sha256:deadbeef"] + env.expect.that_str(whl.digest).equals("sha256:deadbeef") + +_tests.append(_test_sha256_fragment_digest) + def parse_simpleapi_html_test_suite(name): """Create the test suite. diff --git a/tests/pypi/patch_whl/BUILD.bazel b/tests/pypi/patch_whl/BUILD.bazel index d6c4f47b36..cfdc3cc12a 100644 --- a/tests/pypi/patch_whl/BUILD.bazel +++ b/tests/pypi/patch_whl/BUILD.bazel @@ -1,3 +1,14 @@ +load("//python:py_test.bzl", "py_test") +load("//tests/support:support.bzl", "SUPPORTS_BZLMOD") load(":patch_whl_tests.bzl", "patch_whl_test_suite") patch_whl_test_suite(name = "patch_whl_tests") + +py_test( + name = "patch_whl_patch_test", + srcs = ["patch_whl_patch_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, + deps = [ + "@patch_whl_pkg//:pkg", + ], +) diff --git a/tests/pypi/patch_whl/patch_whl_patch_test.py b/tests/pypi/patch_whl/patch_whl_patch_test.py new file mode 100644 index 0000000000..56dfe1aa55 --- /dev/null +++ b/tests/pypi/patch_whl/patch_whl_patch_test.py @@ -0,0 +1,17 @@ +"""Test that wheel patching works end-to-end.""" + +import unittest + +import pkg + + +class PatchWhlTest(unittest.TestCase): + def test_patched(self): + self.assertEqual(pkg.PATCHED, True) + + def test_data_unchanged(self): + self.assertEqual(pkg.DATA, "hello") + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/pypi/patch_whl/patch_whl_tests.bzl b/tests/pypi/patch_whl/patch_whl_tests.bzl index f93fe459c9..5010f03dc0 100644 --- a/tests/pypi/patch_whl/patch_whl_tests.bzl +++ b/tests/pypi/patch_whl/patch_whl_tests.bzl @@ -15,7 +15,7 @@ "" load("@rules_testing//lib:test_suite.bzl", "test_suite") -load("//python/private/pypi:patch_whl.bzl", "patched_whl_name") # buildifier: disable=bzl-visibility +load("//python/private/pypi:patch_whl.bzl", "fix_record_content", "patched_whl_name") # buildifier: disable=bzl-visibility _tests = [] @@ -31,6 +31,104 @@ def _test_simple_local_version(env): _tests.append(_test_simple_local_version) +def _test_fix_record_adds_missing(env): + record = """\ +foo/__init__.py,sha256=abc,123 +foo-1.0.dist-info/RECORD,, +""" + all_files = { + "foo-1.0.dist-info/METADATA": True, + "foo-1.0.dist-info/RECORD": True, + "foo/__init__.py": True, + "foo/bar.py": True, + } + result = fix_record_content( + record_content = record, + all_files = all_files, + record_rel = "foo-1.0.dist-info/RECORD", + ) + env.expect.that_str(result).contains("foo/bar.py,sha256=0,0") + env.expect.that_str(result).contains("foo-1.0.dist-info/METADATA,sha256=0,0") + +_tests.append(_test_fix_record_adds_missing) + +def _test_fix_record_no_missing(env): + record = """\ +foo/__init__.py,sha256=abc,123 +foo/bar.py,sha256=def,456 +foo-1.0.dist-info/RECORD,, +""" + all_files = { + "foo-1.0.dist-info/RECORD": True, + "foo/__init__.py": True, + "foo/bar.py": True, + } + result = fix_record_content( + record_content = record, + all_files = all_files, + record_rel = "foo-1.0.dist-info/RECORD", + ) + env.expect.that_bool(result == None).equals(True) + +_tests.append(_test_fix_record_no_missing) + +def _test_fix_record_preserves_quoting(env): + record = '''\ +"foo/__init__.py",sha256=abc,123 +"foo-1.0.dist-info/RECORD",, +''' + all_files = { + "foo-1.0.dist-info/RECORD": True, + "foo/__init__.py": True, + "foo/bar.py": True, + } + result = fix_record_content( + record_content = record, + all_files = all_files, + record_rel = "foo-1.0.dist-info/RECORD", + ) + env.expect.that_str(result).contains('"foo/bar.py",sha256=0,0') + +_tests.append(_test_fix_record_preserves_quoting) + +def _test_fix_record_skips_excluded(env): + record = """\ +foo/__init__.py,sha256=abc,123 +foo-1.0.dist-info/RECORD,, +""" + all_files = { + "foo-1.0.dist-info/INSTALLER": True, + "foo-1.0.dist-info/RECORD": True, + "foo/__init__.py": True, + } + result = fix_record_content( + record_content = record, + all_files = all_files, + record_rel = "foo-1.0.dist-info/RECORD", + ) + env.expect.that_bool(result == None).equals(True) + +_tests.append(_test_fix_record_skips_excluded) + +def _test_fix_record_skips_whl_files(env): + record = """\ +foo/__init__.py,sha256=abc,123 +foo-1.0.dist-info/RECORD,, +""" + all_files = { + "foo-1.0.dist-info/RECORD": True, + "foo-1.0.whl": True, + "foo/__init__.py": True, + } + result = fix_record_content( + record_content = record, + all_files = all_files, + record_rel = "foo-1.0.dist-info/RECORD", + ) + env.expect.that_bool(result == None).equals(True) + +_tests.append(_test_fix_record_skips_whl_files) + def patch_whl_test_suite(name): """Create the test suite. diff --git a/tests/pypi/patch_whl/testdata/BUILD.bazel b/tests/pypi/patch_whl/testdata/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/pypi/patch_whl/testdata/patches/BUILD.bazel b/tests/pypi/patch_whl/testdata/patches/BUILD.bazel new file mode 100644 index 0000000000..85f0d05223 --- /dev/null +++ b/tests/pypi/patch_whl/testdata/patches/BUILD.bazel @@ -0,0 +1 @@ +exports_files(["modify_pkg.patch"]) diff --git a/tests/pypi/patch_whl/testdata/patches/modify_pkg.patch b/tests/pypi/patch_whl/testdata/patches/modify_pkg.patch new file mode 100644 index 0000000000..ba8cca54b5 --- /dev/null +++ b/tests/pypi/patch_whl/testdata/patches/modify_pkg.patch @@ -0,0 +1,6 @@ +--- pkg.py ++++ pkg.py +@@ -1,2 +1,2 @@ +-PATCHED = False ++PATCHED = True + DATA = "hello" diff --git a/tests/pypi/patch_whl/testdata/pkg/BUILD.bazel b/tests/pypi/patch_whl/testdata/pkg/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/pypi/patch_whl/testdata/pkg/pkg-1.0.dist-info/METADATA b/tests/pypi/patch_whl/testdata/pkg/pkg-1.0.dist-info/METADATA new file mode 100644 index 0000000000..a102dd73d4 --- /dev/null +++ b/tests/pypi/patch_whl/testdata/pkg/pkg-1.0.dist-info/METADATA @@ -0,0 +1,2 @@ +Name: pkg +Version: 1.0 diff --git a/tests/pypi/patch_whl/testdata/pkg/pkg-1.0.dist-info/RECORD b/tests/pypi/patch_whl/testdata/pkg/pkg-1.0.dist-info/RECORD new file mode 100644 index 0000000000..dba333a1c1 --- /dev/null +++ b/tests/pypi/patch_whl/testdata/pkg/pkg-1.0.dist-info/RECORD @@ -0,0 +1,4 @@ +pkg.py,sha256=abc,123 +pkg-1.0.dist-info/METADATA,sha256=def,456 +pkg-1.0.dist-info/WHEEL,sha256=ghi,789 +pkg-1.0.dist-info/RECORD,, diff --git a/tests/pypi/patch_whl/testdata/pkg/pkg-1.0.dist-info/WHEEL b/tests/pypi/patch_whl/testdata/pkg/pkg-1.0.dist-info/WHEEL new file mode 100644 index 0000000000..a64521a1cc --- /dev/null +++ b/tests/pypi/patch_whl/testdata/pkg/pkg-1.0.dist-info/WHEEL @@ -0,0 +1 @@ +Wheel-Version: 1.0 diff --git a/tests/pypi/patch_whl/testdata/pkg/pkg.py b/tests/pypi/patch_whl/testdata/pkg/pkg.py new file mode 100644 index 0000000000..0985d5a3dd --- /dev/null +++ b/tests/pypi/patch_whl/testdata/pkg/pkg.py @@ -0,0 +1,2 @@ +PATCHED = False +DATA = "hello" diff --git a/tests/pypi/pep508/BUILD.bazel b/tests/pypi/pep508/BUILD.bazel new file mode 100644 index 0000000000..36fce0fa89 --- /dev/null +++ b/tests/pypi/pep508/BUILD.bazel @@ -0,0 +1,20 @@ +load(":deps_tests.bzl", "deps_test_suite") +load(":env_tests.bzl", "env_test_suite") +load(":evaluate_tests.bzl", "evaluate_test_suite") +load(":requirement_tests.bzl", "requirement_test_suite") + +deps_test_suite( + name = "deps_tests", +) + +env_test_suite( + name = "env_tests", +) + +evaluate_test_suite( + name = "evaluate_tests", +) + +requirement_test_suite( + name = "requirement_tests", +) diff --git a/tests/pypi/pep508/deps_tests.bzl b/tests/pypi/pep508/deps_tests.bzl new file mode 100644 index 0000000000..e88acb8c56 --- /dev/null +++ b/tests/pypi/pep508/deps_tests.bzl @@ -0,0 +1,257 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Tests for construction of Python version matching config settings.""" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private/pypi:pep508_deps.bzl", "deps") # buildifier: disable=bzl-visibility + +_tests = [] + +def test_simple_deps(env): + got = deps( + "foo", + requires_dist = ["bar-Bar"], + ) + env.expect.that_collection(got.deps).contains_exactly(["bar_bar"]) + env.expect.that_dict(got.deps_select).contains_exactly({}) + +_tests.append(test_simple_deps) + +def test_can_add_os_specific_deps(env): + got = deps( + "foo", + requires_dist = [ + "bar", + "an_osx_dep; sys_platform=='darwin'", + "posix_dep; os_name=='posix'", + "win_dep; os_name=='nt'", + ], + ) + + env.expect.that_collection(got.deps).contains_exactly(["bar"]) + env.expect.that_dict(got.deps_select).contains_exactly({ + "an_osx_dep": "sys_platform == \"darwin\"", + "posix_dep": "os_name == \"posix\"", + "win_dep": "os_name == \"nt\"", + }) + +_tests.append(test_can_add_os_specific_deps) + +def test_deps_are_added_to_more_specialized_platforms(env): + got = deps( + "foo", + requires_dist = [ + "m1_dep; sys_platform=='darwin' and platform_machine=='arm64'", + "mac_dep; sys_platform=='darwin'", + ], + ) + + env.expect.that_collection(got.deps).contains_exactly([]) + env.expect.that_dict(got.deps_select).contains_exactly({ + "m1_dep": "sys_platform == \"darwin\" and platform_machine == \"arm64\"", + "mac_dep": "sys_platform == \"darwin\"", + }) + +_tests.append(test_deps_are_added_to_more_specialized_platforms) + +def test_self_is_ignored(env): + got = deps( + "foo", + requires_dist = [ + "bar", + "req_dep; extra == 'requests'", + "foo[requests]; extra == 'ssl'", + "ssl_lib; extra == 'ssl'", + ], + extras = ["ssl"], + ) + + env.expect.that_collection(got.deps).contains_exactly(["bar", "req_dep", "ssl_lib"]) + env.expect.that_dict(got.deps_select).contains_exactly({}) + +_tests.append(test_self_is_ignored) + +def test_self_dependencies_can_come_in_any_order(env): + got = deps( + "foo", + requires_dist = [ + "bar", + "baz; extra == 'feat'", + "foo[feat2]; extra == 'all'", + "foo[feat]; extra == 'feat2'", + "foo[feat3]; extra == 'all'", + "zdep; extra == 'all'", + ], + extras = ["all"], + ) + + env.expect.that_collection(got.deps).contains_exactly(["bar", "baz", "zdep"]) + env.expect.that_dict(got.deps_select).contains_exactly({}) + +_tests.append(test_self_dependencies_can_come_in_any_order) + +def test_self_include_deps_from_previously_visited(env): + got = deps( + "foo", + requires_dist = [ + "bar", + "baz; extra == 'feat'", + "foo[dev]; extra == 'all'", + "foo[feat]; extra == 'feat2'", + "dev_dep; extra == 'dev'", + ], + extras = ["feat2"], + ) + + env.expect.that_collection(got.deps).contains_exactly(["bar", "baz"]) + env.expect.that_dict(got.deps_select).contains_exactly({}) + +_tests.append(test_self_include_deps_from_previously_visited) + +def _test_can_get_deps_based_on_specific_python_version(env): + requires_dist = [ + "bar", + "baz; python_full_version < '3.7.3'", + "posix_dep; os_name=='posix' and python_version >= '3.8'", + ] + + got = deps( + "foo", + requires_dist = requires_dist, + ) + + # since there is a single target platform, the deps_select will be empty + env.expect.that_collection(got.deps).contains_exactly(["bar"]) + env.expect.that_dict(got.deps_select).contains_exactly({ + "baz": "python_full_version < \"3.7.3\"", + "posix_dep": "os_name == \"posix\" and python_version >= \"3.8\"", + }) + +_tests.append(_test_can_get_deps_based_on_specific_python_version) + +def _test_include_only_particular_deps(env): + requires_dist = [ + "bar", + "baz; python_full_version < '3.7.3'", + "posix_dep; os_name=='posix' and python_version >= '3.8'", + ] + + got = deps( + "foo", + requires_dist = requires_dist, + include = ["bar", "posix_dep"], + ) + + # since there is a single target platform, the deps_select will be empty + env.expect.that_collection(got.deps).contains_exactly(["bar"]) + env.expect.that_dict(got.deps_select).contains_exactly({ + "posix_dep": "os_name == \"posix\" and python_version >= \"3.8\"", + }) + +_tests.append(_test_include_only_particular_deps) + +def test_all_markers_are_added(env): + requires_dist = [ + "bar", + "baz (<2,>=1.11) ; python_version < '3.8'", + "baz (<2,>=1.14) ; python_version >= '3.8'", + ] + + got = deps( + "foo", + requires_dist = requires_dist, + ) + + env.expect.that_collection(got.deps).contains_exactly(["bar"]) + env.expect.that_dict(got.deps_select).contains_exactly({ + "baz": "(python_version < \"3.8\") or (python_version >= \"3.8\")", + }) + +_tests.append(test_all_markers_are_added) + +def test_extra_with_conditional_and_unconditional_markers(env): + requires_dist = [ + "bar", + 'baz!=1.56.0; sys_platform == "darwin" and extra == "client"', + 'baz; extra == "client"', + ] + + got = deps( + "foo", + extras = ["client"], + requires_dist = requires_dist, + ) + + env.expect.that_collection(got.deps).contains_exactly(["bar", "baz"]) + env.expect.that_dict(got.deps_select).contains_exactly({}) + +_tests.append(test_extra_with_conditional_and_unconditional_markers) + +def test_span_all_python_versions(env): + requires_dist = [ + "bar>=0.4.0; python_version >= \"3.13.0\"", + "bar>=0.3.0; python_version ~= \"3.12.0\"", + "bar>=0.2.0; python_version ~= \"3.11.0\"", + "bar>=0.1.0; python_version < \"3.11\"", + ] + + got = deps( + "foo", + requires_dist = requires_dist, + ) + + env.expect.that_collection(got.deps).contains_exactly([]) + env.expect.that_dict(got.deps_select).contains_exactly({ + "bar": "(python_version < \"3.11\") or (python_version >= \"3.13.0\") or (python_version ~= \"3.11.0\") or (python_version ~= \"3.12.0\")", + }) + +_tests.append(test_span_all_python_versions) + +def test_extras_with_hyphens_are_normalized(env): + """Test that extras with hyphens in marker expressions are normalized. + + When wheel METADATA uses hyphens in marker expressions + (e.g., extra == "db-backend") but the extras from requirement parsing + are already normalized (e.g., "db_backend"), the deps should still + resolve because marker evaluation normalizes per PEP 685. + + Args: + env: the test environment. + """ + requires_dist = [ + "bar", + 'baz-lib; extra == "db-backend"', + 'qux-async; extra == "async-driver"', + ] + + got = deps( + "foo", + extras = ["db_backend", "async_driver"], + requires_dist = requires_dist, + ) + + env.expect.that_collection(got.deps).contains_exactly([ + "bar", + "baz_lib", + "qux_async", + ]) + env.expect.that_dict(got.deps_select).contains_exactly({}) + +_tests.append(test_extras_with_hyphens_are_normalized) + +def deps_test_suite(name): # buildifier: disable=function-docstring + test_suite( + name = name, + basic_tests = _tests, + ) diff --git a/tests/pypi/pep508/env_tests.bzl b/tests/pypi/pep508/env_tests.bzl new file mode 100644 index 0000000000..cfd94a1b01 --- /dev/null +++ b/tests/pypi/pep508/env_tests.bzl @@ -0,0 +1,69 @@ +"""Tests to check for env construction.""" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private/pypi:pep508_env.bzl", pep508_env = "env") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_env_defaults(env): + got = pep508_env(os = "exotic", arch = "exotic", python_version = "3.1.1") + got.pop("_aliases") + env.expect.that_dict(got).contains_exactly({ + "implementation_name": "cpython", + "implementation_version": "3.1.1", + "os_name": "posix", + "platform_machine": "", + "platform_python_implementation": "CPython", + "platform_release": "", + "platform_system": "", + "platform_version": "0", + "python_full_version": "3.1.1", + "python_version": "3.1", + "sys_platform": "", + }) + +_tests.append(_test_env_defaults) + +def _test_env_freebsd(env): + got = pep508_env(os = "freebsd", arch = "arm64", python_version = "3.1.1") + got.pop("_aliases") + env.expect.that_dict(got).contains_exactly({ + "implementation_name": "cpython", + "implementation_version": "3.1.1", + "os_name": "posix", + "platform_machine": "aarch64", + "platform_python_implementation": "CPython", + "platform_release": "", + "platform_system": "FreeBSD", + "platform_version": "0", + "python_full_version": "3.1.1", + "python_version": "3.1", + "sys_platform": "freebsd", + }) + +_tests.append(_test_env_freebsd) + +def _test_env_macos(env): + got = pep508_env(os = "macos", arch = "arm64", python_version = "3.1.1") + got.pop("_aliases") + env.expect.that_dict(got).contains_exactly({ + "implementation_name": "cpython", + "implementation_version": "3.1.1", + "os_name": "posix", + "platform_machine": "aarch64", + "platform_python_implementation": "CPython", + "platform_release": "", + "platform_system": "Darwin", + "platform_version": "0", + "python_full_version": "3.1.1", + "python_version": "3.1", + "sys_platform": "darwin", + }) + +_tests.append(_test_env_macos) + +def env_test_suite(name): # buildifier: disable=function-docstring + test_suite( + name = name, + basic_tests = _tests, + ) diff --git a/tests/pypi/pep508/evaluate_tests.bzl b/tests/pypi/pep508/evaluate_tests.bzl new file mode 100644 index 0000000000..7c7b5d8173 --- /dev/null +++ b/tests/pypi/pep508/evaluate_tests.bzl @@ -0,0 +1,336 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Tests for construction of Python version matching config settings.""" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private/pypi:pep508_env.bzl", pep508_env = "env") # buildifier: disable=bzl-visibility +load("//python/private/pypi:pep508_evaluate.bzl", "evaluate", "tokenize") # buildifier: disable=bzl-visibility + +_tests = [] + +def _check_evaluate(env, expr, expected, values, strict = True): + env.expect.where( + expression = expr, + values = values, + ).that_bool(evaluate(expr, env = values, strict = strict)).equals(expected) + +def _tokenize_tests(env): + for input, want in { + "": [], + "'osx' == os_name": ['"osx"', "==", "os_name"], + "'x' not in os_name": ['"x"', "not in", "os_name"], + "()": ["(", ")"], + "(os_name == 'osx' and not os_name == 'posix') or os_name == \"win\"": [ + "(", + "os_name", + "==", + '"osx"', + "and", + "not", + "os_name", + "==", + '"posix"', + ")", + "or", + "os_name", + "==", + '"win"', + ], + "os_name\t==\t'osx'": ["os_name", "==", '"osx"'], + "os_name == 'osx'": ["os_name", "==", '"osx"'], + "python_version <= \"1.0\"": ["python_version", "<=", '"1.0"'], + "python_version>='1.0.0'": ["python_version", ">=", '"1.0.0"'], + "python_version~='1.0.0'": ["python_version", "~=", '"1.0.0"'], + }.items(): + got = tokenize(input) + env.expect.that_collection(got).contains_exactly(want).in_order() + +_tests.append(_tokenize_tests) + +def _evaluate_non_version_env_tests(env): + for var_name in [ + "implementation_name", + "os_name", + "platform_machine", + "platform_python_implementation", + "platform_release", + "platform_system", + "sys_platform", + "extra", + ]: + # Given + marker_env = {var_name: "osx"} + + # When + for input, want in { + "'osx' != {}".format(var_name): False, + "'osx' < {}".format(var_name): False, + "'osx' <= {}".format(var_name): True, + "'osx' == {}".format(var_name): True, + "'osx' >= {}".format(var_name): True, + "'w' not in {}".format(var_name): True, + "'x' in {}".format(var_name): True, + "{} != 'osx'".format(var_name): False, + "{} < 'osx'".format(var_name): False, + "{} <= 'osx'".format(var_name): True, + "{} == 'osx'".format(var_name): True, + "{} > 'osx'".format(var_name): False, + "{} >= 'osx'".format(var_name): True, + }.items(): + _check_evaluate(env, input, want, marker_env) + + # Check that the non-strict eval gives us back the input when no + # env is supplied. + _check_evaluate(env, input, input.replace("'", '"'), {}, strict = False) + +_tests.append(_evaluate_non_version_env_tests) + +def _evaluate_version_env_tests(env): + for var_name in [ + "python_version", + "implementation_version", + "platform_version", + "python_full_version", + ]: + # Given + marker_env = {var_name: "3.7.9"} + + # When + for input, want in { + "{} < '3.8'".format(var_name): True, + "{} > '3.7'".format(var_name): True, + "{} >= '3.7.9'".format(var_name): True, + "{} >= '3.7.10'".format(var_name): False, + "{} >= '3.7.8'".format(var_name): True, + "{} <= '3.7.9'".format(var_name): True, + "{} <= '3.7.10'".format(var_name): True, + "{} <= '3.7.8'".format(var_name): False, + "{} == '3.7.9'".format(var_name): True, + "{} == '3.7.*'".format(var_name): True, + "{} != '3.7.9'".format(var_name): False, + "{} ~= '3.7.1'".format(var_name): True, + "{} ~= '3.7.10'".format(var_name): False, + "{} ~= '3.8.0'".format(var_name): False, + "{} === '3.7.9+rc2'".format(var_name): False, + "{} === '3.7.9'".format(var_name): True, + "{} == '3.7.9+rc2'".format(var_name): True, + }.items(): # buildifier: @unsorted-dict-items + _check_evaluate(env, input, want, marker_env) + + # Check that the non-strict eval gives us back the input when no + # env is supplied. + _check_evaluate(env, input, input.replace("'", '"'), {}, strict = False) + +_tests.append(_evaluate_version_env_tests) + +def _evaluate_platform_version_is_special(env): + # Given + marker_env = {"platform_version": "FooBar Linux v1.2.3"} + + # When the platform version is not + input = "platform_version == '0'" + _check_evaluate(env, input, False, marker_env) + + # And when I compare it as string + input = "'FooBar' in platform_version" + _check_evaluate(env, input, True, marker_env) + + # Check that the non-strict eval gives us back the input when no + # env is supplied. + _check_evaluate(env, input, input.replace("'", '"'), {}, strict = False) + +_tests.append(_evaluate_platform_version_is_special) + +def _logical_expression_tests(env): + for input, want in { + # Basic + "": True, + "(())": True, + "()": True, + + # expr + "os_name == 'fo'": False, + "(os_name == 'fo')": False, + "((os_name == 'fo'))": False, + "((os_name == 'foo'))": True, + "not (os_name == 'fo')": True, + + # and + "os_name == 'fo' and os_name == 'foo'": False, + + # and not + "os_name == 'fo' and not os_name == 'foo'": False, + + # or + "os_name == 'oo' or os_name == 'foo'": True, + + # or not + "os_name == 'foo' or not os_name == 'foo'": True, + + # multiple or + "os_name == 'oo' or os_name == 'fo' or os_name == 'foo'": True, + "os_name == 'oo' or os_name == 'foo' or os_name == 'fo'": True, + + # multiple and + "os_name == 'foo' and os_name == 'foo' and os_name == 'fo'": False, + + # x or not y and z != (x or not y), but is instead evaluated as x or (not y and z) + "os_name == 'foo' or not os_name == 'fo' and os_name == 'fo'": True, + + # x or y and z != (x or y) and z, but is instead evaluated as x or (y and z) + "os_name == 'foo' or os_name == 'fo' and os_name == 'fo'": True, + "not (os_name == 'foo' or os_name == 'fo' and os_name == 'fo')": False, + + # x or y and z and w != (x or y and z) and w, but is instead evaluated as x or (y and z and w) + "os_name == 'foo' or os_name == 'fo' and os_name == 'fo' and os_name == 'fo'": True, + + # not not True + "not not os_name == 'foo'": True, + "not not not os_name == 'foo'": False, + }.items(): # buildifier: @unsorted-dict-items + _check_evaluate(env, input, want, {"os_name": "foo"}) + + if not input.strip("()"): + # These cases will just return True, because they will be evaluated + # and the brackets will be processed. + continue + + # Check that the non-strict eval gives us back the input when no env + # is supplied. + _check_evaluate(env, input, input.replace("'", '"'), {}, strict = False) + +_tests.append(_logical_expression_tests) + +def _evaluate_partial_only_extra(env): + # Given + extra = "foo" + + # When + for input, want in { + "os_name == 'osx' and extra == 'bar'": False, + "os_name == 'osx' and extra == 'foo'": "os_name == \"osx\"", + "platform_system == 'aarch64' and os_name == 'osx' and extra == 'foo'": "platform_system == \"aarch64\" and os_name == \"osx\"", + "platform_system == 'aarch64' and extra == 'foo' and os_name == 'osx'": "platform_system == \"aarch64\" and os_name == \"osx\"", + "os_name == 'osx' or extra == 'bar'": "os_name == \"osx\"", + "os_name == 'osx' or extra == 'foo'": "", + "extra == 'bar' or os_name == 'osx'": "os_name == \"osx\"", + "extra == 'foo' or os_name == 'osx'": "", + "os_name == 'win' or extra == 'bar' or os_name == 'osx'": "os_name == \"win\" or os_name == \"osx\"", + "os_name == 'win' or extra == 'foo' or os_name == 'osx'": "", + }.items(): # buildifier: @unsorted-dict-items + got = evaluate( + input, + env = { + "extra": extra, + }, + strict = False, + ) + env.expect.that_bool(got).equals(want) + _check_evaluate(env, input, want, {"extra": extra}, strict = False) + +_tests.append(_evaluate_partial_only_extra) + +def _evaluate_with_aliases(env): + # When + for (os, cpu), tests in { + # buildifier: @unsorted-dict-items + ("osx", "aarch64"): { + "platform_system == 'Darwin' and platform_machine == 'arm64'": True, + "platform_system == 'Darwin' and platform_machine == 'aarch64'": True, + "platform_system == 'Darwin' and platform_machine == 'amd64'": False, + }, + ("osx", "x86_64"): { + "platform_system == 'Darwin' and platform_machine == 'amd64'": True, + "platform_system == 'Darwin' and platform_machine == 'x86_64'": True, + }, + ("osx", "x86_32"): { + "platform_system == 'Darwin' and platform_machine == 'i386'": True, + "platform_system == 'Darwin' and platform_machine == 'i686'": True, + "platform_system == 'Darwin' and platform_machine == 'x86_32'": True, + "platform_system == 'Darwin' and platform_machine == 'x86_64'": False, + }, + ("freebsd", "x86_32"): { + "platform_system == 'FreeBSD' and platform_machine == 'i386'": True, + "platform_system == 'FreeBSD' and platform_machine == 'i686'": True, + "platform_system == 'FreeBSD' and platform_machine == 'x86_32'": True, + "platform_system == 'FreeBSD' and platform_machine == 'x86_64'": False, + "platform_system == 'FreeBSD' and os_name == 'posix'": True, + }, + }.items(): # buildifier: @unsorted-dict-items + for input, want in tests.items(): + _check_evaluate(env, input, want, pep508_env( + os = os, + arch = cpu, + python_version = "3.2", + )) + +_tests.append(_evaluate_with_aliases) + +def _expr_case(expr, want, env): + return struct(expr = expr.strip(), want = want, env = env) + +_MISC_EXPRESSIONS = [ + _expr_case('python_version == "3.*"', True, {"python_version": "3.10.1"}), + _expr_case('python_version != "3.10.*"', False, {"python_version": "3.10.1"}), + _expr_case('python_version != "3.11.*"', True, {"python_version": "3.10.1"}), + _expr_case('python_version != "3.10"', False, {"python_version": "3.10.0"}), + _expr_case('python_version == "3.10"', True, {"python_version": "3.10.0"}), + # Cases for the '>' operator + # Taken from spec: https://peps.python.org/pep-0440/#exclusive-ordered-comparison + _expr_case('python_version > "1.7"', True, {"python_version": "1.7.1"}), + _expr_case('python_version > "1.7"', False, {"python_version": "1.7.0.post0"}), + _expr_case('python_version > "1.7"', True, {"python_version": "1.7.1"}), + _expr_case('python_version > "1.7.post2"', True, {"python_version": "1.7.1"}), + _expr_case('python_version > "1.7.post2"', True, {"python_version": "1.7.post3"}), + _expr_case('python_version > "1.7.post2"', False, {"python_version": "1.7.0"}), + _expr_case('python_version > "1.7.1+local"', False, {"python_version": "1.7.1"}), + _expr_case('python_version > "1.7.1+local"', True, {"python_version": "1.7.2"}), + # Extra cases for the '<' operator + _expr_case('python_version < "1.7.1"', False, {"python_version": "1.7.2"}), + _expr_case('python_version < "1.7.3"', True, {"python_version": "1.7.2"}), + _expr_case('python_version < "1.7.1"', True, {"python_version": "1.7"}), + _expr_case('python_version < "1.7.1"', False, {"python_version": "1.7.1-rc2"}), + _expr_case('python_version < "1.7.1-rc3"', True, {"python_version": "1.7.1-rc2"}), + _expr_case('python_version < "1.7.1-rc1"', False, {"python_version": "1.7.1-rc2"}), + # Extra tests + _expr_case('python_version <= "1.7.1"', True, {"python_version": "1.7.1"}), + _expr_case('python_version <= "1.7.2"', True, {"python_version": "1.7.1"}), + _expr_case('python_version >= "1.7.1"', True, {"python_version": "1.7.1"}), + _expr_case('python_version >= "1.7.0"', True, {"python_version": "1.7.1"}), + # Compatible version tests: + # https://packaging.python.org/en/latest/specifications/version-specifiers/#compatible-release + _expr_case('python_version ~= "2.2"', True, {"python_version": "2.3"}), + _expr_case('python_version ~= "2.2"', False, {"python_version": "2.1"}), + _expr_case('python_version ~= "2.2.0"', True, {"python_version": "2.2"}), + _expr_case('python_version ~= "2.2.post3"', False, {"python_version": "2.2"}), + _expr_case('python_version ~= "2.2.post3"', False, {"python_version": "3.0"}), + _expr_case('python_version ~= "2.2.post3"', True, {"python_version": "2.3"}), + _expr_case('python_version ~= "1!2.2"', False, {"python_version": "2.7"}), + _expr_case('python_version ~= "0!2.2"', True, {"python_version": "2.7"}), + _expr_case('python_version ~= "1!2.2"', True, {"python_version": "1!2.7"}), + _expr_case('python_version ~= "1.2.3"', True, {"python_version": "1.2.4"}), + _expr_case('python_version ~= "1.2.3"', False, {"python_version": "1.3.2"}), +] + +def _misc_expressions(env): + for case in _MISC_EXPRESSIONS: + _check_evaluate(env, case.expr, case.want, case.env) + +_tests.append(_misc_expressions) + +def evaluate_test_suite(name): # buildifier: disable=function-docstring + test_suite( + name = name, + basic_tests = _tests, + ) diff --git a/tests/pypi/pep508/requirement_tests.bzl b/tests/pypi/pep508/requirement_tests.bzl new file mode 100644 index 0000000000..2ce45922da --- /dev/null +++ b/tests/pypi/pep508/requirement_tests.bzl @@ -0,0 +1,49 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Tests for parsing the requirement specifier.""" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private/pypi:pep508_requirement.bzl", "requirement") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_requirement_line_parsing(env): + want = { + " name1[ foo ] ": ("name1", ["foo"], None, ""), + "Name[foo]": ("name", ["foo"], None, ""), + "name [fred,bar] @ http://foo.com ; python_version=='2.7'": ("name", ["fred", "bar"], None, "python_version=='2.7'"), + "name; (os_name=='a' or os_name=='b') and os_name=='c'": ("name", [], None, "(os_name=='a' or os_name=='b') and os_name=='c'"), + "name@http://foo.com": ("name", [], None, ""), + "name[ Foo123 ]": ("name", ["foo123"], None, ""), + "name[extra-one,extra-two.three]==1.0": ("name", ["extra_one", "extra_two_three"], "1.0", ""), + "name[extra]@http://foo.com": ("name", ["extra"], None, ""), + "name[foo]": ("name", ["foo"], None, ""), + "name[quux, strange];python_version<'2.7' and platform_version=='2'": ("name", ["quux", "strange"], None, "python_version<'2.7' and platform_version=='2'"), + "name_foo[bar]": ("name-foo", ["bar"], None, ""), + "name_foo[bar]==0.25": ("name-foo", ["bar"], "0.25", ""), + } + + got = { + i: (parsed.name, parsed.extras, parsed.version, parsed.marker) + for i, parsed in {case: requirement(case) for case in want}.items() + } + env.expect.that_dict(got).contains_exactly(want) + +_tests.append(_test_requirement_line_parsing) + +def requirement_test_suite(name): # buildifier: disable=function-docstring + test_suite( + name = name, + basic_tests = _tests, + ) diff --git a/tests/pypi/pkg_aliases/pkg_aliases_test.bzl b/tests/pypi/pkg_aliases/pkg_aliases_test.bzl index 0fa66d05eb..32be6ba47c 100644 --- a/tests/pypi/pkg_aliases/pkg_aliases_test.bzl +++ b/tests/pypi/pkg_aliases/pkg_aliases_test.bzl @@ -15,6 +15,7 @@ """pkg_aliases tests""" load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private/pypi:config_settings.bzl", "config_settings") # buildifier: disable=bzl-visibility load( "//python/private/pypi:pkg_aliases.bzl", @@ -43,6 +44,7 @@ def _test_legacy_aliases(env): "whl": "@repo//:whl", "data": "@repo//:data", "dist_info": "@repo//:dist_info", + "extracted_whl_files": "@repo//:extracted_whl_files", "my_special": "@repo//:my_special", } @@ -56,12 +58,8 @@ def _test_config_setting_aliases(env): actual_no_match_error = [] def mock_select(value, no_match_error = None): - actual_no_match_error.append(no_match_error) - env.expect.that_str(no_match_error).contains("""\ -configuration settings: - //:my_config_setting - -""") + if no_match_error and no_match_error not in actual_no_match_error: + actual_no_match_error.append(no_match_error) return value pkg_aliases( @@ -71,7 +69,7 @@ configuration settings: }, extra_aliases = ["my_special"], native = struct( - alias = lambda name, actual: got.update({name: actual}), + alias = lambda *, name, actual, visibility = None, tags = None: got.update({name: actual}), ), select = mock_select, ) @@ -80,9 +78,22 @@ configuration settings: want = { "pkg": { "//:my_config_setting": "@bar_baz_repo//:pkg", + "//conditions:default": "_no_matching_repository", }, + # This will be printing the current config values and will make sure we + # have an error. + "_no_matching_repository": {Label("//python/config_settings:is_not_matching_current_config"): labels.NONE}, } env.expect.that_dict(got).contains_at_least(want) + env.expect.that_collection(actual_no_match_error).has_size(1) + env.expect.that_str(actual_no_match_error[0]).contains("""\ +configuration settings: + //:my_config_setting + +""") + env.expect.that_str(actual_no_match_error[0]).contains( + "//python/config_settings:current_config=fail", + ) _tests.append(_test_config_setting_aliases) @@ -92,13 +103,8 @@ def _test_config_setting_aliases_many(env): actual_no_match_error = [] def mock_select(value, no_match_error = None): - actual_no_match_error.append(no_match_error) - env.expect.that_str(no_match_error).contains("""\ -configuration settings: - //:another_config_setting - //:my_config_setting - //:third_config_setting -""") + if no_match_error and no_match_error not in actual_no_match_error: + actual_no_match_error.append(no_match_error) return value pkg_aliases( @@ -112,7 +118,8 @@ configuration settings: }, extra_aliases = ["my_special"], native = struct( - alias = lambda name, actual: got.update({name: actual}), + alias = lambda *, name, actual, visibility = None, tags = None: got.update({name: actual}), + config_setting = lambda **_: None, ), select = mock_select, ) @@ -125,9 +132,17 @@ configuration settings: "//:another_config_setting", ): "@bar_baz_repo//:my_special", "//:third_config_setting": "@foo_repo//:my_special", + "//conditions:default": "_no_matching_repository", }, } env.expect.that_dict(got).contains_at_least(want) + env.expect.that_collection(actual_no_match_error).has_size(1) + env.expect.that_str(actual_no_match_error[0]).contains("""\ +configuration settings: + //:another_config_setting + //:my_config_setting + //:third_config_setting +""") _tests.append(_test_config_setting_aliases_many) @@ -137,55 +152,47 @@ def _test_multiplatform_whl_aliases(env): actual_no_match_error = [] def mock_select(value, no_match_error = None): - actual_no_match_error.append(no_match_error) - env.expect.that_str(no_match_error).contains("""\ -configuration settings: - //:my_config_setting - //_config:is_cp3.9_linux_x86_64 - //_config:is_cp3.9_py3_none_any - //_config:is_cp3.9_py3_none_any_linux_x86_64 - -""") + if no_match_error and no_match_error not in actual_no_match_error: + actual_no_match_error.append(no_match_error) return value pkg_aliases( name = "bar_baz", actual = { whl_config_setting( - filename = "foo-0.0.0-py3-none-any.whl", version = "3.9", - ): "filename_repo", + ): "version_repo", whl_config_setting( - filename = "foo-0.0.0-py3-none-any.whl", version = "3.9", target_platforms = ["cp39_linux_x86_64"], - ): "filename_repo_for_platform", - whl_config_setting( - version = "3.9", - target_platforms = ["cp39_linux_x86_64"], - ): "bzlmod_repo_for_a_particular_platform", + ): "version_platform_repo", "//:my_config_setting": "bzlmod_repo", }, extra_aliases = [], native = struct( - alias = lambda name, actual: got.update({name: actual}), + alias = lambda *, name, actual, visibility = None, tags = None: got.update({name: actual}), ), select = mock_select, - glibc_versions = [], - muslc_versions = [], - osx_versions = [], ) # buildifier: disable=unsorted-dict-items want = { "pkg": { "//:my_config_setting": "@bzlmod_repo//:pkg", - "//_config:is_cp3.9_linux_x86_64": "@bzlmod_repo_for_a_particular_platform//:pkg", - "//_config:is_cp3.9_py3_none_any": "@filename_repo//:pkg", - "//_config:is_cp3.9_py3_none_any_linux_x86_64": "@filename_repo_for_platform//:pkg", + "//_config:is_cp39": "@version_repo//:pkg", + "//_config:is_cp39_linux_x86_64": "@version_platform_repo//:pkg", + "//conditions:default": "_no_matching_repository", }, } env.expect.that_dict(got).contains_at_least(want) + env.expect.that_collection(actual_no_match_error).has_size(1) + env.expect.that_str(actual_no_match_error[0]).contains("""\ +configuration settings: + //:my_config_setting + //_config:is_cp39 + //_config:is_cp39_linux_x86_64 + +""") _tests.append(_test_multiplatform_whl_aliases) @@ -226,6 +233,10 @@ def _test_group_aliases(env): "name": "dist_info", "actual": "@repo//:dist_info", }, + { + "name": "extracted_whl_files", + "actual": "@repo//:extracted_whl_files", + }, { "name": "pkg", "actual": "//_groups:my_group_pkg", @@ -258,7 +269,6 @@ _tests.append(_test_multiplatform_whl_aliases_nofilename) def _test_multiplatform_whl_aliases_nofilename_target_platforms(env): aliases = { whl_config_setting( - config_setting = "//:ignored", version = "3.1", target_platforms = [ "cp31_linux_x86_64", @@ -270,102 +280,19 @@ def _test_multiplatform_whl_aliases_nofilename_target_platforms(env): got = multiplatform_whl_aliases(aliases = aliases) want = { - "//_config:is_cp3.1_linux_aarch64": "foo", - "//_config:is_cp3.1_linux_x86_64": "foo", + "//_config:is_cp31_linux_aarch64": "foo", + "//_config:is_cp31_linux_x86_64": "foo", } env.expect.that_dict(got).contains_exactly(want) _tests.append(_test_multiplatform_whl_aliases_nofilename_target_platforms) -def _test_multiplatform_whl_aliases_filename(env): - aliases = { - whl_config_setting( - filename = "foo-0.0.3-py3-none-any.whl", - version = "3.2", - ): "foo-py3-0.0.3", - whl_config_setting( - filename = "foo-0.0.1-py3-none-any.whl", - version = "3.1", - ): "foo-py3-0.0.1", - whl_config_setting( - filename = "foo-0.0.2-py3-none-any.whl", - version = "3.1", - target_platforms = [ - "cp31_linux_x86_64", - "cp31_linux_aarch64", - ], - ): "foo-0.0.2", - } - got = multiplatform_whl_aliases( - aliases = aliases, - glibc_versions = [], - muslc_versions = [], - osx_versions = [], - ) - want = { - "//_config:is_cp3.1_py3_none_any": "foo-py3-0.0.1", - "//_config:is_cp3.1_py3_none_any_linux_aarch64": "foo-0.0.2", - "//_config:is_cp3.1_py3_none_any_linux_x86_64": "foo-0.0.2", - "//_config:is_cp3.2_py3_none_any": "foo-py3-0.0.3", - } - env.expect.that_dict(got).contains_exactly(want) - -_tests.append(_test_multiplatform_whl_aliases_filename) - -def _test_multiplatform_whl_aliases_filename_versioned(env): - aliases = { - whl_config_setting( - filename = "foo-0.0.1-py3-none-manylinux_2_17_x86_64.whl", - version = "3.1", - ): "glibc-2.17", - whl_config_setting( - filename = "foo-0.0.1-py3-none-manylinux_2_18_x86_64.whl", - version = "3.1", - ): "glibc-2.18", - whl_config_setting( - filename = "foo-0.0.1-py3-none-musllinux_1_1_x86_64.whl", - version = "3.1", - ): "musl-1.1", - } - got = multiplatform_whl_aliases( - aliases = aliases, - glibc_versions = [(2, 17), (2, 18)], - muslc_versions = [(1, 1), (1, 2)], - osx_versions = [], - ) - want = { - # This could just work with: - # select({ - # "//_config:is_gt_eq_2.18": "//_config:is_cp3.1_py3_none_manylinux_x86_64", - # "//conditions:default": "//_config:is_gt_eq_2.18", - # }): "glibc-2.18", - # select({ - # "//_config:is_range_2.17_2.18": "//_config:is_cp3.1_py3_none_manylinux_x86_64", - # "//_config:is_glibc_default": "//_config:is_cp3.1_py3_none_manylinux_x86_64", - # "//conditions:default": "//_config:is_glibc_default", - # }): "glibc-2.17", - # ( - # "//_config:is_gt_musl_1.1": "musl-1.1", - # "//_config:is_musl_default": "musl-1.1", - # ): "musl-1.1", - # - # For this to fully work we need to have the pypi:config_settings.bzl to generate the - # extra targets that use the FeatureFlagInfo and this to generate extra aliases for the - # config settings. - "//_config:is_cp3.1_py3_none_manylinux_2_17_x86_64": "glibc-2.17", - "//_config:is_cp3.1_py3_none_manylinux_2_18_x86_64": "glibc-2.18", - "//_config:is_cp3.1_py3_none_manylinux_x86_64": "glibc-2.17", - "//_config:is_cp3.1_py3_none_musllinux_1_1_x86_64": "musl-1.1", - "//_config:is_cp3.1_py3_none_musllinux_1_2_x86_64": "musl-1.1", - "//_config:is_cp3.1_py3_none_musllinux_x86_64": "musl-1.1", - } - env.expect.that_dict(got).contains_exactly(want) - -_tests.append(_test_multiplatform_whl_aliases_filename_versioned) - def _mock_alias(container): return lambda name, **kwargs: container.append(name) +def _mock_config_setting_group(container): + return lambda name, **kwargs: container.append(name) + def _mock_config_setting(container): def _inner(name, flag_values = None, constraint_values = None, **_): if flag_values or constraint_values: @@ -391,12 +318,21 @@ def _test_config_settings_exist_legacy(env): python_versions = ["3.11"], native = struct( alias = _mock_alias(available_config_settings), - config_setting = _mock_config_setting(available_config_settings), + config_setting = _mock_config_setting([]), + ), + selects = struct( + config_setting_group = _mock_config_setting_group(available_config_settings), ), - target_platforms = [ - "linux_aarch64", - "linux_x86_64", - ], + platform_config_settings = { + "linux_aarch64": [ + "@platforms//cpu:aarch64", + "@platforms//os:linux", + ], + "linux_x86_64": [ + "@platforms//cpu:x86_64", + "@platforms//os:linux", + ], + }, ) got_aliases = multiplatform_whl_aliases( @@ -408,63 +344,6 @@ def _test_config_settings_exist_legacy(env): _tests.append(_test_config_settings_exist_legacy) -def _test_config_settings_exist(env): - for py_tag in ["py2.py3", "py3", "py311", "cp311"]: - if py_tag == "py2.py3": - abis = ["none"] - elif py_tag.startswith("py"): - abis = ["none", "abi3"] - else: - abis = ["none", "abi3", "cp311"] - - for abi_tag in abis: - for platform_tag, kwargs in { - "any": {}, - "macosx_11_0_arm64": { - "osx_versions": [(11, 0)], - "target_platforms": ["osx_aarch64"], - }, - "manylinux_2_17_x86_64": { - "glibc_versions": [(2, 17), (2, 18)], - "target_platforms": ["linux_x86_64"], - }, - "manylinux_2_18_x86_64": { - "glibc_versions": [(2, 17), (2, 18)], - "target_platforms": ["linux_x86_64"], - }, - "musllinux_1_1_aarch64": { - "muslc_versions": [(1, 2), (1, 1), (1, 0)], - "target_platforms": ["linux_aarch64"], - }, - }.items(): - aliases = { - whl_config_setting( - filename = "foo-0.0.1-{}-{}-{}.whl".format(py_tag, abi_tag, platform_tag), - version = "3.11", - ): "repo", - } - available_config_settings = [] - config_settings( - python_versions = ["3.11"], - native = struct( - alias = _mock_alias(available_config_settings), - config_setting = _mock_config_setting(available_config_settings), - ), - **kwargs - ) - - got_aliases = multiplatform_whl_aliases( - aliases = aliases, - glibc_versions = kwargs.get("glibc_versions", []), - muslc_versions = kwargs.get("muslc_versions", []), - osx_versions = kwargs.get("osx_versions", []), - ) - got = [a.partition(":")[-1] for a in got_aliases] - - env.expect.that_collection(available_config_settings).contains_at_least(got) - -_tests.append(_test_config_settings_exist) - def pkg_aliases_test_suite(name): """Create the test suite. diff --git a/tests/pypi/pypi_cache/BUILD.bazel b/tests/pypi/pypi_cache/BUILD.bazel new file mode 100644 index 0000000000..03c20623cd --- /dev/null +++ b/tests/pypi/pypi_cache/BUILD.bazel @@ -0,0 +1,5 @@ +load(":pypi_cache_tests.bzl", "pypi_cache_test_suite") + +pypi_cache_test_suite( + name = "pypi_cache_tests", +) diff --git a/tests/pypi/pypi_cache/pypi_cache_tests.bzl b/tests/pypi/pypi_cache/pypi_cache_tests.bzl new file mode 100644 index 0000000000..21c2f5bd8c --- /dev/null +++ b/tests/pypi/pypi_cache/pypi_cache_tests.bzl @@ -0,0 +1,563 @@ +"" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("@rules_testing//lib:truth.bzl", "subjects") +load("//python/private/pypi:pypi_cache.bzl", "pypi_cache") # buildifier: disable=bzl-visibility +load("//tests/support/mocks:mocks.bzl", "mocks") + +_tests = [] + +def _cache(env, **kwargs): + cache = pypi_cache(**kwargs) + + attrs = { + "hashes_by_version": subjects.dict, + "sdists": subjects.dict, + "whls": subjects.dict, + } + + def _expect(value): + if not value: + return env.expect.that_str(value) + + if type(value) == "dict": + return env.expect.that_dict(value) + + return env.expect.that_struct( + value, + attrs = attrs, + ) + + return struct( + setdefault = lambda *args, **kwargs: _expect( + cache.setdefault(*args, **kwargs), + ), + get = lambda *args, **kwargs: _expect( + cache.get(*args, **kwargs), + ), + get_facts = lambda: env.expect.that_dict(cache.get_facts()), + ) + +def _test_memory_cache_hit(env): + """Verifies that the cache returns stored values for the same real_url.""" + store = {} + + # We pass None for module_ctx to focus solely on memory_cache behavior + cache = _cache(env, mctx = None, store = store) + + # Mocked parsed result from a PyPI-like index + fake_result = struct( + sdists = { + "sha256:sha_1": struct(version = "1.0.0", filename = "pkg-1.0.0.tar.gz"), + }, + whls = { + "sha256:sha_2": struct(version = "1.1.0", filename = "pkg-1.1.0-py3-none-any.whl"), + }, + hashes_by_version = { + "1.0.0": ["sha256:sha_1"], + "1.1.0": ["sha256:sha_2"], + }, + ) + + # Key format: (index_url, real_url, versions) + key = ("https://{PYPI_INDEX_URL}/pkg", "https://pypi.org/simple/pkg", ["1.0.0", "1.1.0"]) + + # When set the cache + cache.setdefault(key, fake_result) + + # And get a value back + got = cache.get(key) + + got.sdists().contains_exactly(fake_result.sdists) + got.whls().contains_exactly(fake_result.whls) + got.hashes_by_version().contains_exactly(fake_result.hashes_by_version) + + # A different key with fewer versions + key = ("https://{PYPI_INDEX_URL}/pkg", "https://pypi.org/simple/pkg", ["1.0.0"]) + + got = cache.get(key) + got.sdists().contains_exactly(fake_result.sdists) + got.whls().contains_exactly({}) + got.hashes_by_version().contains_exactly({"1.0.0": ["sha256:sha_1"]}) + + # A key with no matches + key = ("https://{PYPI_INDEX_URL}/pkg", "https://pypi.org/simple/pkg", ["1.2.0"]) + + cache.get(key).equals(None) + +_tests.append(_test_memory_cache_hit) + +def _test_pypi_cache_writes_to_facts(env): + """Verifies that setting a value in the cache also populates the facts store.""" + mock_ctx = mocks.mctx(facts = {}) + cache = _cache(env, mctx = mock_ctx) + + fake_result = struct( + sdists = { + "sha256:sha_sdist": struct( + version = "1.0.0", + filename = "pkg-1.0.0.tar.gz", + url = "https://pypi.org/files/pkg-1.0.0.tar.gz", + digest = "sha256:sha_sdist", + yanked = "", + ), + }, + whls = { + "sha256:sha_whl": struct( + version = "1.0.0", + filename = "pkg-1.0.0-py3-none-any.whl", + url = "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl", + digest = "sha256:sha_whl", + yanked = "Security issue", + ), + # This won't get stored + "sha256:sha_whl_2": struct( + version = "1.1.0", + filename = "pkg-1.1.0-py3-none-any.whl", + url = "https://pypi.org/files/pkg-1.1.0-py3-none-any.whl", + digest = "sha256:sha_whl_2", + yanked = None, + ), + }, + hashes_by_version = { + "1.0.0": ["sha256:sha_sdist", "sha256:sha_whl"], + "1.1.0": ["sha256:sha_whl_2"], + }, + ) + + key = ("https://{PYPI_INDEX_URL}/pkg/", "https://pypi.org/simple/pkg/", ["1.0.0"]) + + # When we set the cache + cache.setdefault(key, fake_result) + + # Then the key returns us the same items + got = cache.get(key) + got.whls().contains_exactly({ + "sha256:sha_whl": fake_result.whls["sha256:sha_whl"], + }) + got.sdists().contains_exactly(fake_result.sdists) + got.hashes_by_version().contains_exactly({ + "1.0.0": fake_result.hashes_by_version["1.0.0"], + }) + + # Then when we get facts at the end + cache.get_facts().contains_exactly({ + "dist_hashes": { + # We are not using the real index URL, because we may have credentials in here + "https://{PYPI_INDEX_URL}": { + "pkg": { + "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha256:sha_whl", + "https://pypi.org/files/pkg-1.0.0.tar.gz": "sha256:sha_sdist", + }, + }, + }, + "dist_yanked": { + "https://{PYPI_INDEX_URL}": { + "pkg": { + "sha256:sha_sdist": "", + "sha256:sha_whl": "Security issue", + }, + }, + }, + "fact_version": "v2", # Facts version + }) + + # When we get the other items cached in memory, they get written to facts + got = cache.get((key[0], key[1], ["1.1.0"])) + got.whls().contains_exactly({ + "sha256:sha_whl_2": fake_result.whls["sha256:sha_whl_2"], + }) + got.sdists().contains_exactly({}) + got.hashes_by_version().contains_exactly({ + "1.1.0": fake_result.hashes_by_version["1.1.0"], + }) + + # Then when we get facts at the end + cache.get_facts().contains_exactly({ + "dist_hashes": { + # We are not using the real index URL, because we may have credentials in here + "https://{PYPI_INDEX_URL}": { + "pkg": { + "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha256:sha_whl", + "https://pypi.org/files/pkg-1.0.0.tar.gz": "sha256:sha_sdist", + "https://pypi.org/files/pkg-1.1.0-py3-none-any.whl": "sha256:sha_whl_2", + }, + }, + }, + "dist_yanked": { + "https://{PYPI_INDEX_URL}": { + "pkg": { + "sha256:sha_sdist": "", + "sha256:sha_whl": "Security issue", + }, + }, + }, + "fact_version": "v2", # Facts version + }) + +_tests.append(_test_pypi_cache_writes_to_facts) + +def _test_pypi_cache_reads_from_facts(env): + """Verifies that setting a value in the cache also populates the facts store.""" + mock_ctx = mocks.mctx(facts = { + "dist_hashes": { + # We are not using the real index URL, because we may have credentials in here + "https://{PYPI_INDEX_URL}": { + "pkg": { + "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha256:sha_whl", + "https://pypi.org/files/pkg-1.0.0.tar.gz": "sha256:sha_sdist", + }, + }, + }, + "dist_yanked": { + "https://{PYPI_INDEX_URL}": { + "pkg": { + "sha256:sha_sdist": "", + "sha256:sha_whl": "Security issue", + }, + }, + }, + "fact_version": "v2", # Facts version + }) + cache = _cache(env, mctx = mock_ctx) + + key = ("https://{PYPI_INDEX_URL}/pkg/", "https://pypi.org/simple/pkg/", ["1.0.0"]) + + # Then we would get empty facts because we haven't accessed any of the known facts. + # This simulates the dropping of the facts of requirements that are no longer needed. + cache.get_facts().contains_exactly({}) + + # When we get the + got = cache.get(key) + + expected_result = struct( + sdists = { + "sha256:sha_sdist": struct( + digest = "sha256:sha_sdist", + version = "1.0.0", + filename = "pkg-1.0.0.tar.gz", + metadata_url = "", + metadata_sha256 = "", + url = "https://pypi.org/files/pkg-1.0.0.tar.gz", + yanked = "", + ), + }, + whls = { + "sha256:sha_whl": struct( + digest = "sha256:sha_whl", + version = "1.0.0", + filename = "pkg-1.0.0-py3-none-any.whl", + url = "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl", + metadata_url = "", + metadata_sha256 = "", + yanked = "Security issue", + ), + }, + hashes_by_version = { + "1.0.0": ["sha256:sha_sdist", "sha256:sha_whl"], + }, + ) + + got.whls().contains_exactly(expected_result.whls) + got.sdists().contains_exactly(expected_result.sdists) + got.hashes_by_version().contains_exactly(expected_result.hashes_by_version) + + # Then when we store the same facts back again, because we accessed the cached keys. + cache.get_facts().contains_exactly(mock_ctx.facts) + + # When we request more than what we have, we will return nothing + key = ("https://{PYPI_INDEX_URL}/pkg/", "https://pypi.org/simple/pkg/", ["1.0.0", "1.1.0"]) + got = cache.get(key) + got.equals(None) + +_tests.append(_test_pypi_cache_reads_from_facts) + +def _test_pypi_cache_reads_from_facts_drops_unaccessed_dists(env): + """Verifies that dists for unaccessed versions are dropped from computed facts.""" + mock_ctx = mocks.mctx(facts = { + "dist_hashes": { + "https://{PYPI_INDEX_URL}": { + "pkg": { + "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha256:sha_whl_1.0.0", + "https://pypi.org/files/pkg-1.0.0.tar.gz": "sha256:sha_sdist_1.0.0", + "https://pypi.org/files/pkg-1.1.0-py3-none-any.whl": "sha256:sha_whl_1.1.0", + "https://pypi.org/files/pkg-1.1.0.tar.gz": "sha256:sha_sdist_1.1.0", + }, + }, + }, + "fact_version": "v2", + }) + cache = _cache(env, mctx = mock_ctx) + + # Request only version 1.0.0; version 1.1.0 is NOT requested + key = ("https://{PYPI_INDEX_URL}/pkg/", "https://pypi.org/simple/pkg/", ["1.0.0"]) + got = cache.get(key) + + expected = struct( + sdists = { + "sha256:sha_sdist_1.0.0": struct( + digest = "sha256:sha_sdist_1.0.0", + version = "1.0.0", + filename = "pkg-1.0.0.tar.gz", + metadata_url = "", + metadata_sha256 = "", + url = "https://pypi.org/files/pkg-1.0.0.tar.gz", + yanked = None, + ), + }, + whls = { + "sha256:sha_whl_1.0.0": struct( + digest = "sha256:sha_whl_1.0.0", + version = "1.0.0", + filename = "pkg-1.0.0-py3-none-any.whl", + metadata_url = "", + metadata_sha256 = "", + url = "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl", + yanked = None, + ), + }, + hashes_by_version = { + "1.0.0": ["sha256:sha_sdist_1.0.0", "sha256:sha_whl_1.0.0"], + }, + ) + got.whls().contains_exactly(expected.whls) + got.sdists().contains_exactly(expected.sdists) + got.hashes_by_version().contains_exactly(expected.hashes_by_version) + + # get_facts() must only contain version 1.0.0 data; 1.1.0 is dropped + cache.get_facts().contains_exactly({ + "dist_hashes": { + "https://{PYPI_INDEX_URL}": { + "pkg": { + "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha256:sha_whl_1.0.0", + "https://pypi.org/files/pkg-1.0.0.tar.gz": "sha256:sha_sdist_1.0.0", + }, + }, + }, + "fact_version": "v2", + }) + +_tests.append(_test_pypi_cache_reads_from_facts_drops_unaccessed_dists) + +def _test_pypi_cache_facts_non_sha256_round_trip(env): + """Verifies that digests of other hash algorithms survive the facts round trip.""" + mock_ctx = mocks.mctx(facts = {}) + cache = _cache(env, mctx = mock_ctx) + + fake_result = struct( + sdists = {}, + whls = { + "sha512:whl_digest": struct( + version = "1.0.0", + filename = "pkg-1.0.0-py3-none-any.whl", + url = "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl", + digest = "sha512:whl_digest", + yanked = None, + ), + }, + hashes_by_version = { + "1.0.0": ["sha512:whl_digest"], + }, + ) + + key = ("https://{PYPI_INDEX_URL}/pkg/", "https://pypi.org/simple/pkg/", ["1.0.0"]) + cache.setdefault(key, fake_result) + + # The hash algorithm is stored in the facts + cache.get_facts().contains_exactly({ + "dist_hashes": { + "https://{PYPI_INDEX_URL}": { + "pkg": { + "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha512:whl_digest", + }, + }, + }, + "fact_version": "v2", # Facts version + }) + + # And a cache that only has the facts reconstructs the digests + cache = _cache(env, mctx = mocks.mctx(facts = { + "dist_hashes": { + "https://{PYPI_INDEX_URL}": { + "pkg": { + "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha512:whl_digest", + }, + }, + }, + "fact_version": "v2", # Facts version + })) + + got = cache.get(key) + got.whls().contains_exactly({ + "sha512:whl_digest": struct( + digest = "sha512:whl_digest", + version = "1.0.0", + filename = "pkg-1.0.0-py3-none-any.whl", + metadata_url = "", + metadata_sha256 = "", + url = "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl", + yanked = None, + ), + }) + got.hashes_by_version().contains_exactly({"1.0.0": ["sha512:whl_digest"]}) + +_tests.append(_test_pypi_cache_facts_non_sha256_round_trip) + +def _test_memory_cache_index_urls(env): + """Verifies that the cache returns stored values for index_urls.""" + store = {} + cache = _cache(env, mctx = None, store = store) + + fake_result = { + "pkg-a": "https://pypi.org/simple/pkg-a/", + "pkg_b": "https://pypi.org/simple/pkg-b/", + } + + key = ("https://pypi.org/simple/", "https://pypi.org/simple/", {"pkg-a": None, "pkg_b": None}) + + cache.setdefault(key, fake_result) + + got = cache.get(key) + got.contains_exactly(fake_result) + + key = ("https://pypi.org/simple/", "https://pypi.org/simple/", {"pkg-a": None}) + got = cache.get(key) + got.contains_exactly({"pkg-a": "https://pypi.org/simple/pkg-a/"}) + + key = ("https://pypi.org/simple/", "https://pypi.org/simple/", {"pkg-c": None}) + cache.get(key).equals(None) + +_tests.append(_test_memory_cache_index_urls) + +def _test_pypi_cache_writes_index_urls_to_facts(env): + """Verifies that setting index_urls in the cache also populates the facts store.""" + mock_ctx = mocks.mctx(facts = {}) + cache = _cache(env, mctx = mock_ctx) + + fake_result = { + "pkg-a": "https://pypi.org/simple/pkg-a/", + "pkg_b": "https://pypi.org/simple/pkg-b/", + } + + key = ("https://pypi.org/simple/", "https://pypi.org/simple/", {"pkg-a": None}) + + cache.setdefault(key, fake_result) + + cache.get_facts().contains_exactly({ + "fact_version": "v2", + "index_urls": { + "https://pypi.org/simple/": { + "pkg-a": "https://pypi.org/simple/pkg-a/", + }, + }, + }) + + key = ("https://pypi.org/simple/", "https://pypi.org/simple/", {"pkg_b": None}) + cache.setdefault(key, fake_result) + + cache.get_facts().contains_exactly({ + "fact_version": "v2", + "index_urls": { + "https://pypi.org/simple/": { + "pkg-a": "https://pypi.org/simple/pkg-a/", + "pkg_b": "https://pypi.org/simple/pkg-b/", + }, + }, + }) + +_tests.append(_test_pypi_cache_writes_index_urls_to_facts) + +def _test_pypi_cache_reads_index_urls_from_facts(env): + """Verifies that reading index_urls from facts works correctly.""" + mock_ctx = mocks.mctx(facts = { + "fact_version": "v2", + "index_urls": { + "https://pypi.org/simple/": { + "pkg-a": "https://pypi.org/simple/pkg-a/", + "pkg-b": "https://pypi.org/simple/pkg-b/", + }, + }, + }) + cache = _cache(env, mctx = mock_ctx) + + key = ("https://pypi.org/simple/", "https://pypi.org/simple/", {"pkg-a": None}) + got = cache.get(key) + got.contains_exactly({"pkg-a": "https://pypi.org/simple/pkg-a/"}) + + key = ("https://pypi.org/simple/", "https://pypi.org/simple/", {"pkg-a": None, "pkg-b": None}) + got = cache.get(key) + got.contains_exactly({ + "pkg-a": "https://pypi.org/simple/pkg-a/", + "pkg-b": "https://pypi.org/simple/pkg-b/", + }) + + key = ("https://pypi.org/simple/", "https://pypi.org/simple/", {"pkg-c": None}) + cache.get(key).equals(None) + + cache.get_facts().contains_exactly(mock_ctx.facts) + +_tests.append(_test_pypi_cache_reads_index_urls_from_facts) + +def _test_pypi_cache_reads_index_urls_from_facts_incomplete(env): + """Verifies that incomplete index_urls facts returns None (forces fresh download).""" + mock_ctx = mocks.mctx(facts = { + "fact_version": "v2", + "index_urls": { + "https://pypi.org/simple/": { + "pkg-a": "https://pypi.org/simple/pkg-a/", + }, + }, + }) + cache = _cache(env, mctx = mock_ctx) + + # Request pkg-a and pkg-b, but facts only have pkg-a + key = ("https://pypi.org/simple/", "https://pypi.org/simple/", {"pkg-a": None, "pkg-b": None}) + cache.get(key).equals(None) + +_tests.append(_test_pypi_cache_reads_index_urls_from_facts_incomplete) + +def _test_pypi_cache_reads_index_urls_from_facts_drops_unaccessed(env): + """Verifies get_facts() drops unaccessed index_urls entries. + + When known_facts has packages that are not requested, they should not + appear in computed_facts. This ensures stale facts (from packages + removed from all requirements files) get cleaned up from the lockfile. + """ + mock_ctx = mocks.mctx(facts = { + "fact_version": "v2", + "index_urls": { + "https://pypi.org/simple/": { + "pkg-a": "https://pypi.org/simple/pkg-a/", + "pkg-b": "https://pypi.org/simple/pkg-b/", + "pkg-c": "https://pypi.org/simple/pkg-c/", + }, + }, + }) + cache = _cache(env, mctx = mock_ctx) + + # Request only a subset of packages; pkg-c is not requested + key = ("https://pypi.org/simple/", "https://pypi.org/simple/", {"pkg-a": None, "pkg-b": None}) + got = cache.get(key) + got.contains_exactly({ + "pkg-a": "https://pypi.org/simple/pkg-a/", + "pkg-b": "https://pypi.org/simple/pkg-b/", + }) + + # get_facts() must only return the requested (accessed) subset; pkg-c is dropped + cache.get_facts().contains_exactly({ + "fact_version": "v2", + "index_urls": { + "https://pypi.org/simple/": { + "pkg-a": "https://pypi.org/simple/pkg-a/", + "pkg-b": "https://pypi.org/simple/pkg-b/", + }, + }, + }) + +_tests.append(_test_pypi_cache_reads_index_urls_from_facts_drops_unaccessed) + +def pypi_cache_test_suite(name): + test_suite( + name = name, + basic_tests = _tests, + ) diff --git a/tests/pypi/python_tag/BUILD.bazel b/tests/pypi/python_tag/BUILD.bazel new file mode 100644 index 0000000000..d4b37cea16 --- /dev/null +++ b/tests/pypi/python_tag/BUILD.bazel @@ -0,0 +1,3 @@ +load(":python_tag_tests.bzl", "python_tag_test_suite") + +python_tag_test_suite(name = "python_tag_tests") diff --git a/tests/pypi/python_tag/python_tag_tests.bzl b/tests/pypi/python_tag/python_tag_tests.bzl new file mode 100644 index 0000000000..ca86575e5b --- /dev/null +++ b/tests/pypi/python_tag/python_tag_tests.bzl @@ -0,0 +1,34 @@ +"" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private/pypi:python_tag.bzl", "python_tag") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_without_version(env): + for give, expect in { + "cpython": "cp", + "ironpython": "ip", + "jython": "jy", + "pypy": "pp", + "python": "py", + "something_else": "something_else", + }.items(): + got = python_tag(give) + env.expect.that_str(got).equals(expect) + +_tests.append(_test_without_version) + +def _test_with_version(env): + got = python_tag("cpython", "3.1.15") + env.expect.that_str(got).equals("cp31") + +_tests.append(_test_with_version) + +def python_tag_test_suite(name): + """Create the test suite. + + Args: + name: the name of the test suite + """ + test_suite(name = name, basic_tests = _tests) diff --git a/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl b/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl index ca1651aa1d..9114f59279 100644 --- a/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl +++ b/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl @@ -15,10 +15,6 @@ """render_pkg_aliases tests""" load("@rules_testing//lib:test_suite.bzl", "test_suite") -load( - "//python/private/pypi:pkg_aliases.bzl", - "get_filename_config_settings", -) # buildifier: disable=bzl-visibility load( "//python/private/pypi:render_pkg_aliases.bzl", "get_whl_flag_versions", @@ -68,30 +64,24 @@ def _test_bzlmod_aliases(env): aliases = { "bar-baz": { whl_config_setting( - version = "3.2", - config_setting = "//:my_config_setting", + # Add one with micro version to mimic construction in the extension + version = "3.2.2", ): "pypi_32_bar_baz", whl_config_setting( version = "3.2", - config_setting = "//:my_config_setting", target_platforms = [ "cp32_linux_x86_64", ], ): "pypi_32_bar_baz_linux_x86_64", - whl_config_setting( - version = "3.2", - filename = "foo-0.0.0-py3-none-any.whl", - ): "filename_repo", - whl_config_setting( - version = "3.2", - filename = "foo-0.0.0-py3-none-any.whl", - target_platforms = [ - "cp32_linux_x86_64", - ], - ): "filename_repo_linux_x86_64", }, }, extra_hub_aliases = {"bar_baz": ["foo"]}, + platform_config_settings = { + "linux_x86_64": [ + "@platforms//os:linux", + "@platforms//cpu:x86_64", + ], + }, ) want_key = "bar_baz/BUILD.bazel" @@ -104,21 +94,11 @@ package(default_visibility = ["//visibility:public"]) pkg_aliases( name = "bar_baz", actual = { - "//:my_config_setting": "pypi_32_bar_baz", + "//_config:is_cp322": "pypi_32_bar_baz", whl_config_setting( target_platforms = ("cp32_linux_x86_64",), - config_setting = "//:my_config_setting", version = "3.2", ): "pypi_32_bar_baz_linux_x86_64", - whl_config_setting( - filename = "foo-0.0.0-py3-none-any.whl", - version = "3.2", - ): "filename_repo", - whl_config_setting( - filename = "foo-0.0.0-py3-none-any.whl", - target_platforms = ("cp32_linux_x86_64",), - version = "3.2", - ): "filename_repo_linux_x86_64", }, extra_aliases = ["foo"], )""" @@ -129,8 +109,13 @@ load("@rules_python//python/private/pypi:config_settings.bzl", "config_settings" config_settings( name = "config_settings", + platform_config_settings = { + "linux_x86_64": [ + "@platforms//os:linux", + "@platforms//cpu:x86_64", + ], + }, python_versions = ["3.2"], - target_platforms = ["linux_x86_64"], visibility = ["//:__subpackages__"], )""", ) @@ -244,256 +229,24 @@ def _test_get_python_versions_with_target_platforms(env): _tests.append(_test_get_python_versions_with_target_platforms) -def _test_get_python_versions_from_filenames(env): - got = get_whl_flag_versions( - settings = [ - whl_config_setting( - version = "3.3", - filename = "foo-0.0.0-py3-none-" + plat + ".whl", - ) - for plat in [ - "linux_x86_64", - "manylinux_2_17_x86_64", - "manylinux_2_14_aarch64.musllinux_1_1_aarch64", - "musllinux_1_0_x86_64", - "manylinux2014_x86_64.manylinux_2_17_x86_64", - "macosx_11_0_arm64", - "macosx_10_9_x86_64", - "macosx_10_9_universal2", - "windows_x86_64", - ] - ], - ) - want = { - "glibc_versions": [(2, 14), (2, 17)], - "muslc_versions": [(1, 0), (1, 1)], - "osx_versions": [(10, 9), (11, 0)], - "python_versions": ["3.3"], - "target_platforms": [ - "linux_aarch64", - "linux_x86_64", - "osx_aarch64", - "osx_x86_64", - "windows_x86_64", - ], - } - env.expect.that_dict(got).contains_exactly(want) - -_tests.append(_test_get_python_versions_from_filenames) - def _test_get_flag_versions_from_alias_target_platforms(env): got = get_whl_flag_versions( settings = [ + whl_config_setting(version = "3.3"), whl_config_setting( version = "3.3", - filename = "foo-0.0.0-py3-none-" + plat + ".whl", - ) - for plat in [ - "windows_x86_64", - ] - ] + [ - whl_config_setting( - version = "3.3", - filename = "foo-0.0.0-py3-none-any.whl", - target_platforms = [ - "cp33_linux_x86_64", - ], + target_platforms = ["cp33_linux_x86_64"], ), ], ) want = { "python_versions": ["3.3"], - "target_platforms": [ - "linux_x86_64", - "windows_x86_64", - ], + "target_platforms": ["linux_x86_64"], } env.expect.that_dict(got).contains_exactly(want) _tests.append(_test_get_flag_versions_from_alias_target_platforms) -def _test_config_settings( - env, - *, - filename, - want, - python_version, - want_versions = {}, - target_platforms = [], - glibc_versions = [], - muslc_versions = [], - osx_versions = []): - got, got_default_version_settings = get_filename_config_settings( - filename = filename, - target_platforms = target_platforms, - glibc_versions = glibc_versions, - muslc_versions = muslc_versions, - osx_versions = osx_versions, - python_version = python_version, - ) - env.expect.that_collection(got).contains_exactly(want) - env.expect.that_dict(got_default_version_settings).contains_exactly(want_versions) - -def _test_sdist(env): - # Do the first test for multiple extensions - for ext in [".tar.gz", ".zip"]: - _test_config_settings( - env, - filename = "foo-0.0.1" + ext, - python_version = "3.2", - want = [":is_cp3.2_sdist"], - ) - - ext = ".zip" - _test_config_settings( - env, - filename = "foo-0.0.1" + ext, - python_version = "3.2", - target_platforms = [ - "linux_aarch64", - "linux_x86_64", - ], - want = [ - ":is_cp3.2_sdist_linux_aarch64", - ":is_cp3.2_sdist_linux_x86_64", - ], - ) - -_tests.append(_test_sdist) - -def _test_py2_py3_none_any(env): - _test_config_settings( - env, - filename = "foo-0.0.1-py2.py3-none-any.whl", - python_version = "3.2", - want = [ - ":is_cp3.2_py_none_any", - ], - ) - - _test_config_settings( - env, - filename = "foo-0.0.1-py2.py3-none-any.whl", - python_version = "3.2", - target_platforms = [ - "osx_x86_64", - ], - want = [":is_cp3.2_py_none_any_osx_x86_64"], - ) - -_tests.append(_test_py2_py3_none_any) - -def _test_py3_none_any(env): - _test_config_settings( - env, - filename = "foo-0.0.1-py3-none-any.whl", - python_version = "3.1", - want = [":is_cp3.1_py3_none_any"], - ) - - _test_config_settings( - env, - filename = "foo-0.0.1-py3-none-any.whl", - python_version = "3.1", - target_platforms = ["linux_x86_64"], - want = [":is_cp3.1_py3_none_any_linux_x86_64"], - ) - -_tests.append(_test_py3_none_any) - -def _test_py3_none_macosx_10_9_universal2(env): - _test_config_settings( - env, - filename = "foo-0.0.1-py3-none-macosx_10_9_universal2.whl", - python_version = "3.1", - osx_versions = [ - (10, 9), - (11, 0), - ], - want = [], - want_versions = { - ":is_cp3.1_py3_none_osx_aarch64_universal2": { - (10, 9): ":is_cp3.1_py3_none_osx_10_9_aarch64_universal2", - (11, 0): ":is_cp3.1_py3_none_osx_11_0_aarch64_universal2", - }, - ":is_cp3.1_py3_none_osx_x86_64_universal2": { - (10, 9): ":is_cp3.1_py3_none_osx_10_9_x86_64_universal2", - (11, 0): ":is_cp3.1_py3_none_osx_11_0_x86_64_universal2", - }, - }, - ) - -_tests.append(_test_py3_none_macosx_10_9_universal2) - -def _test_cp37_abi3_linux_x86_64(env): - _test_config_settings( - env, - filename = "foo-0.0.1-cp37-abi3-linux_x86_64.whl", - python_version = "3.7", - want = [":is_cp3.7_cp3x_abi3_linux_x86_64"], - ) - -_tests.append(_test_cp37_abi3_linux_x86_64) - -def _test_cp37_abi3_windows_x86_64(env): - _test_config_settings( - env, - filename = "foo-0.0.1-cp37-abi3-windows_x86_64.whl", - python_version = "3.7", - want = [":is_cp3.7_cp3x_abi3_windows_x86_64"], - ) - -_tests.append(_test_cp37_abi3_windows_x86_64) - -def _test_cp37_abi3_manylinux_2_17_x86_64(env): - _test_config_settings( - env, - filename = "foo-0.0.1-cp37-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", - python_version = "3.7", - glibc_versions = [ - (2, 16), - (2, 17), - (2, 18), - ], - want = [], - want_versions = { - ":is_cp3.7_cp3x_abi3_manylinux_x86_64": { - (2, 17): ":is_cp3.7_cp3x_abi3_manylinux_2_17_x86_64", - (2, 18): ":is_cp3.7_cp3x_abi3_manylinux_2_18_x86_64", - }, - }, - ) - -_tests.append(_test_cp37_abi3_manylinux_2_17_x86_64) - -def _test_cp37_abi3_manylinux_2_17_musllinux_1_1_aarch64(env): - # I've seen such a wheel being built for `uv` - _test_config_settings( - env, - filename = "foo-0.0.1-cp37-cp37-manylinux_2_17_arm64.musllinux_1_1_arm64.whl", - python_version = "3.7", - glibc_versions = [ - (2, 16), - (2, 17), - (2, 18), - ], - muslc_versions = [ - (1, 1), - ], - want = [], - want_versions = { - ":is_cp3.7_cp3x_cp_manylinux_aarch64": { - (2, 17): ":is_cp3.7_cp3x_cp_manylinux_2_17_aarch64", - (2, 18): ":is_cp3.7_cp3x_cp_manylinux_2_18_aarch64", - }, - ":is_cp3.7_cp3x_cp_musllinux_aarch64": { - (1, 1): ":is_cp3.7_cp3x_cp_musllinux_1_1_aarch64", - }, - }, - ) - -_tests.append(_test_cp37_abi3_manylinux_2_17_musllinux_1_1_aarch64) - def render_pkg_aliases_test_suite(name): """Create the test suite. diff --git a/tests/pypi/requirements_files_by_platform/requirements_files_by_platform_tests.bzl b/tests/pypi/requirements_files_by_platform/requirements_files_by_platform_tests.bzl index b729b0eaf0..7f4260081b 100644 --- a/tests/pypi/requirements_files_by_platform/requirements_files_by_platform_tests.bzl +++ b/tests/pypi/requirements_files_by_platform/requirements_files_by_platform_tests.bzl @@ -15,21 +15,40 @@ "" load("@rules_testing//lib:test_suite.bzl", "test_suite") -load("//python/private/pypi:requirements_files_by_platform.bzl", "requirements_files_by_platform") # buildifier: disable=bzl-visibility +load("//python/private/pypi:requirements_files_by_platform.bzl", _sut = "requirements_files_by_platform") # buildifier: disable=bzl-visibility _tests = [] +requirements_files_by_platform = lambda **kwargs: _sut( + platforms = kwargs.pop( + "platforms", + [ + "linux_aarch64", + "linux_arm", + "linux_ppc", + "linux_s390x", + "linux_x86_64", + "osx_aarch64", + "osx_x86_64", + "windows_x86_64", + ], + ), + **kwargs +) + def _test_fail_no_requirements(env): + """Verify that omitting all requirements attributes produces an error.""" errors = [] requirements_files_by_platform( fail_fn = errors.append, ) env.expect.that_str(errors[0]).equals("""\ -A 'requirements_lock' attribute must be specified, a platform-specific lockfiles via 'requirements_by_platform' or an os-specific lockfiles must be specified via 'requirements_*' attributes""") +A 'requirements_lock' or 'uv_lock' attribute must be specified, a platform-specific lockfiles via 'requirements_by_platform' or an os-specific lockfiles must be specified via 'requirements_*' attributes""") _tests.append(_test_fail_no_requirements) def _test_fail_duplicate_platforms(env): + """Verify that a platform mapped to multiple requirements files errors.""" errors = [] requirements_files_by_platform( requirements_by_platform = { @@ -44,6 +63,7 @@ def _test_fail_duplicate_platforms(env): _tests.append(_test_fail_duplicate_platforms) def _test_fail_download_only_bad_attr(env): + """Verify that ``--platform`` pip args require a single ``requirements_lock``.""" errors = [] requirements_files_by_platform( requirements_linux = "requirements_linux", @@ -61,6 +81,7 @@ def _test_fail_download_only_bad_attr(env): _tests.append(_test_fail_download_only_bad_attr) def _test_simple(env): + """Test basic mapping of a single ``requirements_lock`` to all platforms.""" for got in [ requirements_files_by_platform( requirements_lock = "requirements_lock", @@ -86,7 +107,37 @@ def _test_simple(env): _tests.append(_test_simple) +def _test_simple_limited(env): + """Test that limiting the platform list restricts the output mapping.""" + for got in [ + requirements_files_by_platform( + requirements_lock = "requirements_lock", + platforms = ["linux_x86_64", "osx_x86_64"], + ), + requirements_files_by_platform( + requirements_by_platform = { + "requirements_lock": "*", + }, + platforms = ["linux_x86_64", "osx_x86_64"], + ), + requirements_files_by_platform( + requirements_by_platform = { + "requirements_lock": "linux_x86_64,osx_aarch64,osx_x86_64", + }, + platforms = ["linux_x86_64", "osx_x86_64", "windows_x86_64"], + ), + ]: + env.expect.that_dict(got).contains_exactly({ + "requirements_lock": [ + "linux_x86_64", + "osx_x86_64", + ], + }) + +_tests.append(_test_simple_limited) + def _test_simple_with_python_version(env): + """Test that ``python_version`` prefixes platform names with ``cpNNN_``.""" for got in [ requirements_files_by_platform( requirements_lock = "requirements_lock", @@ -124,6 +175,7 @@ def _test_simple_with_python_version(env): _tests.append(_test_simple_with_python_version) def _test_multi_os(env): + """Test per-OS requirements files mapping each OS group correctly.""" for got in [ requirements_files_by_platform( requirements_linux = "requirements_linux", @@ -158,6 +210,7 @@ def _test_multi_os(env): _tests.append(_test_multi_os) def _test_multi_os_download_only_platform(env): + """Test that ``--platform`` pip args narrow platforms to the host OS.""" got = requirements_files_by_platform( requirements_lock = "requirements_linux", extra_pip_args = [ @@ -174,12 +227,24 @@ def _test_multi_os_download_only_platform(env): _tests.append(_test_multi_os_download_only_platform) def _test_os_arch_requirements_with_default(env): + """Test combining specific OS/arch requirements with a fallback ``requirements_lock``.""" got = requirements_files_by_platform( requirements_by_platform = { "requirements_exotic": "linux_super_exotic", "requirements_linux": "linux_x86_64,linux_aarch64", }, requirements_lock = "requirements_lock", + platforms = [ + "linux_super_exotic", + "linux_x86_64", + "linux_aarch64", + "linux_arm", + "linux_ppc", + "linux_s390x", + "osx_aarch64", + "osx_x86_64", + "windows_x86_64", + ], ) env.expect.that_dict(got).contains_exactly({ "requirements_exotic": ["linux_super_exotic"], @@ -196,6 +261,83 @@ def _test_os_arch_requirements_with_default(env): _tests.append(_test_os_arch_requirements_with_default) +def _test_host_only_lockfile(env): + """Host-only: single requirements_lock with only the host platform. + + Verifies no extra empty-platform files leak into the return dict. + """ + got = requirements_files_by_platform( + requirements_lock = "requirements_lock", + platforms = ["osx_x86_64"], + ) + env.expect.that_dict(got).contains_exactly({ + "requirements_lock": ["osx_x86_64"], + }) + +_tests.append(_test_host_only_lockfile) + +def _test_host_only_multiple_os(env): + """Host-only with per-OS files but only host platform configured. + + Files with no matching platforms should appear with empty platform + lists so parse_requirements can read all packages for index URLs. + """ + got = requirements_files_by_platform( + requirements_linux = "requirements_linux", + requirements_osx = "requirements_osx", + requirements_windows = "requirements_windows", + platforms = ["osx_x86_64"], + ) + env.expect.that_dict(got).contains_exactly({ + # Per-OS files with no matching platforms get empty lists + "requirements_linux": [], + # The matching OS file gets its platforms + "requirements_osx": ["osx_x86_64"], + "requirements_windows": [], + }) + +_tests.append(_test_host_only_multiple_os) + +def _test_host_only_os_with_fallback(env): + """Host-only with per-OS files + fallback lock, host platform only. + + The fallback should not appear since the matching OS file covers + the only platform; unmatched files get empty lists. + """ + got = requirements_files_by_platform( + requirements_linux = "requirements_linux", + requirements_osx = "requirements_osx", + requirements_lock = "requirements_lock", + platforms = ["osx_x86_64"], + ) + env.expect.that_dict(got).contains_exactly({ + "requirements_linux": [], + "requirements_osx": ["osx_x86_64"], + # Fallback lock is not used because osx file already covers + # the only platform + }) + +_tests.append(_test_host_only_os_with_fallback) + +def _test_uv_lock_only(env): + """Verify that using only uv_lock without requirements_lock succeeds.""" + got = requirements_files_by_platform( + uv_lock = "uv.lock", + ) + env.expect.that_dict(got).contains_exactly({}) + +_tests.append(_test_uv_lock_only) + +def _test_uv_lock_with_platform_arg(env): + """Verify that using uv_lock with --platform argument succeeds.""" + got = requirements_files_by_platform( + uv_lock = "uv.lock", + extra_pip_args = ["--platform", "linux_x86_64"], + ) + env.expect.that_dict(got).contains_exactly({}) + +_tests.append(_test_uv_lock_with_platform_arg) + def requirements_files_by_platform_test_suite(name): """Create the test suite. diff --git a/tests/pypi/select_whl/BUILD.bazel b/tests/pypi/select_whl/BUILD.bazel new file mode 100644 index 0000000000..0ad8cba0cd --- /dev/null +++ b/tests/pypi/select_whl/BUILD.bazel @@ -0,0 +1,3 @@ +load(":select_whl_tests.bzl", "select_whl_test_suite") + +select_whl_test_suite(name = "select_whl_tests") diff --git a/tests/pypi/select_whl/select_whl_tests.bzl b/tests/pypi/select_whl/select_whl_tests.bzl new file mode 100644 index 0000000000..8d2170ed7f --- /dev/null +++ b/tests/pypi/select_whl/select_whl_tests.bzl @@ -0,0 +1,556 @@ +"" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "REPO_VERBOSITY_ENV_VAR", "repo_utils") # buildifier: disable=bzl-visibility +load("//python/private/pypi:select_whl.bzl", "select_whl") # buildifier: disable=bzl-visibility + +WHL_LIST = [ + "pkg-0.0.1-cp311-cp311-macosx_10_9_universal2.whl", + "pkg-0.0.1-cp311-cp311-macosx_10_9_x86_64.whl", + "pkg-0.0.1-cp311-cp311-macosx_11_0_arm64.whl", + "pkg-0.0.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", + "pkg-0.0.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", + "pkg-0.0.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", + "pkg-0.0.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", + "pkg-0.0.1-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", + "pkg-0.0.1-cp313-cp313t-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp313-cp313-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp313-abi3-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp313-none-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp311-cp311-musllinux_1_1_aarch64.whl", + "pkg-0.0.1-cp311-cp311-musllinux_1_1_i686.whl", + "pkg-0.0.1-cp311-cp311-musllinux_1_1_ppc64le.whl", + "pkg-0.0.1-cp311-cp311-musllinux_1_1_s390x.whl", + "pkg-0.0.1-cp311-cp311-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp311-cp311-win32.whl", + "pkg-0.0.1-cp311-cp311-win_amd64.whl", + "pkg-0.0.1-cp37-cp37m-macosx_10_9_x86_64.whl", + "pkg-0.0.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", + "pkg-0.0.1-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", + "pkg-0.0.1-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl", + "pkg-0.0.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", + "pkg-0.0.1-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", + "pkg-0.0.1-cp37-cp37m-musllinux_1_1_aarch64.whl", + "pkg-0.0.1-cp37-cp37m-musllinux_1_1_i686.whl", + "pkg-0.0.1-cp37-cp37m-musllinux_1_1_ppc64le.whl", + "pkg-0.0.1-cp37-cp37m-musllinux_1_1_s390x.whl", + "pkg-0.0.1-cp37-cp37m-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp37-cp37m-win32.whl", + "pkg-0.0.1-cp37-cp37m-win_amd64.whl", + "pkg-0.0.1-cp39-cp39-macosx_10_9_universal2.whl", + "pkg-0.0.1-cp39-cp39-macosx_10_9_x86_64.whl", + "pkg-0.0.1-cp39-cp39-macosx_11_0_arm64.whl", + "pkg-0.0.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", + "pkg-0.0.1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", + "pkg-0.0.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", + "pkg-0.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", + "pkg-0.0.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", + "pkg-0.0.1-cp39-cp39-musllinux_1_1_aarch64.whl", + "pkg-0.0.1-cp39-cp39-musllinux_1_1_i686.whl", + "pkg-0.0.1-cp39-cp39-musllinux_1_1_ppc64le.whl", + "pkg-0.0.1-cp39-cp39-musllinux_1_1_s390x.whl", + "pkg-0.0.1-cp39-cp39-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp39-cp39-win32.whl", + "pkg-0.0.1-cp39-cp39-win_amd64.whl", + "pkg-0.0.1-cp39-abi3-any.whl", + "pkg-0.0.1-py310-abi3-any.whl", + "pkg-0.0.1-py3-abi3-any.whl", + "pkg-0.0.1-py3-none-any.whl", + # Extra examples that should be discarded + "pkg-0.0.1-py27-cp27mu-win_amd64.whl", + "pkg-0.0.1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", +] + +def _match(env, got, *want_filenames): + if not want_filenames: + env.expect.that_collection(got).has_size(len(want_filenames)) + return + + got = [g for g in got if g] + got_filenames = [g.filename for g in got] + env.expect.that_collection(got_filenames).contains_exactly(want_filenames).in_order() + + if got: + # Check that we pass the original structs + env.expect.that_str(got[0].other).equals("dummy") + +def _select_whl(whls, debug = False, **kwargs): + return select_whl( + whls = [ + struct( + filename = f, + other = "dummy", + ) + for f in whls + ], + logger = repo_utils.logger(struct( + getenv = { + REPO_DEBUG_ENV_VAR: "1", + REPO_VERBOSITY_ENV_VAR: "TRACE" if debug else "INFO", + }.get, + ), "unit-test"), + **kwargs + ) + +_tests = [] + +def _test_not_select_py2(env): + # Check we prefer platform specific wheels + got = _select_whl( + whls = [ + "pkg-0.0.1-py2-none-any.whl", + "pkg-0.0.1-py3-none-any.whl", + "pkg-0.0.1-py312-none-any.whl", + ], + whl_platform_tags = ["any"], + whl_abi_tags = ["none"], + python_version = "3.13", + limit = 2, + ) + _match( + env, + got, + "pkg-0.0.1-py3-none-any.whl", + "pkg-0.0.1-py312-none-any.whl", + ) + +_tests.append(_test_not_select_py2) + +def _test_not_select_abi3(env): + # Check we prefer platform specific wheels + got = _select_whl( + whls = [ + "pkg-0.0.1-py3-none-any.whl", + # the following should be ignored + "pkg-0.0.1-py3-abi3-any.whl", + "pkg-0.0.1-py3-abi3-p1.p2.p2.whl", + ], + whl_platform_tags = ["any", "p1"], + whl_abi_tags = ["none"], + python_version = "3.13", + limit = 2, + ) + _match( + env, + got, + "pkg-0.0.1-py3-none-any.whl", + ) + +_tests.append(_test_not_select_abi3) + +def _test_select_cp312(env): + # Check we prefer platform specific wheels + got = _select_whl( + whls = [ + "pkg-0.0.1-py2-none-any.whl", + "pkg-0.0.1-py3-none-any.whl", + "pkg-0.0.1-py312-none-any.whl", + "pkg-0.0.1-cp39-none-any.whl", + "pkg-0.0.1-cp312-none-any.whl", + "pkg-0.0.1-cp314-none-any.whl", + ], + whl_platform_tags = ["any"], + whl_abi_tags = ["none"], + python_version = "3.13", + limit = 5, + ) + _match( + env, + got, + "pkg-0.0.1-py3-none-any.whl", + "pkg-0.0.1-py312-none-any.whl", + "pkg-0.0.1-cp39-none-any.whl", + "pkg-0.0.1-cp312-none-any.whl", + ) + +_tests.append(_test_select_cp312) + +def _test_simplest(env): + whls = [ + "pkg-0.0.1-py2.py3-abi3-any.whl", + "pkg-0.0.1-py3-abi3-any.whl", + "pkg-0.0.1-py3-none-any.whl", + ] + + got = _select_whl( + whls = whls, + whl_platform_tags = ["any"], + whl_abi_tags = ["abi3"], + python_version = "3.0", + ) + _match( + env, + [got], + "pkg-0.0.1-py3-abi3-any.whl", + ) + +_tests.append(_test_simplest) + +def _test_select_by_supported_py_version(env): + whls = [ + "pkg-0.0.1-py2.py3-abi3-any.whl", + "pkg-0.0.1-py3-abi3-any.whl", + "pkg-0.0.1-py311-abi3-any.whl", + ] + + for minor_version, match in { + 8: "pkg-0.0.1-py3-abi3-any.whl", + 11: "pkg-0.0.1-py311-abi3-any.whl", + }.items(): + got = _select_whl( + whls = whls, + whl_platform_tags = ["any"], + whl_abi_tags = ["abi3"], + python_version = "3.{}".format(minor_version), + ) + _match(env, [got], match) + +_tests.append(_test_select_by_supported_py_version) + +def _test_select_by_supported_cp_version(env): + whls = [ + "pkg-0.0.1-py2.py3-abi3-any.whl", + "pkg-0.0.1-py3-abi3-any.whl", + "pkg-0.0.1-py311-abi3-any.whl", + "pkg-0.0.1-cp311-abi3-any.whl", + ] + + for minor_version, match in { + 11: "pkg-0.0.1-cp311-abi3-any.whl", + 8: "pkg-0.0.1-py3-abi3-any.whl", + }.items(): + got = _select_whl( + whls = whls, + whl_platform_tags = ["any"], + whl_abi_tags = ["abi3"], + python_version = "3.{}".format(minor_version), + ) + _match(env, [got], match) + +_tests.append(_test_select_by_supported_cp_version) + +def _test_legacy_manylinux(env): + for legacy, replacement in { + "manylinux1": "manylinux_2_5", + "manylinux2010": "manylinux_2_12", + "manylinux2014": "manylinux_2_17", + }.items(): + for plat in [legacy, replacement]: + whls = [ + "pkg-0.0.1-py3-none-{}_x86_64.whl".format(plat), + "pkg-0.0.1-py3-none-any.whl", + ] + + got = _select_whl( + whls = whls, + whl_platform_tags = ["{}_x86_64".format(legacy)], + whl_abi_tags = ["none"], + python_version = "3.10", + ) + want = _select_whl( + whls = whls, + whl_platform_tags = ["{}_x86_64".format(replacement)], + whl_abi_tags = ["none"], + python_version = "3.10", + ) + _match(env, [got], want.filename) + +_tests.append(_test_legacy_manylinux) + +def _test_supported_cp_version_manylinux(env): + whls = [ + "pkg-0.0.1-py2.py3-none-manylinux_1_1_x86_64.whl", + "pkg-0.0.1-py3-none-manylinux_1_1_x86_64.whl", + "pkg-0.0.1-py311-none-manylinux_1_1_x86_64.whl", + "pkg-0.0.1-cp311-none-manylinux_1_1_x86_64.whl", + ] + + for minor_version, match in { + 8: "pkg-0.0.1-py3-none-manylinux_1_1_x86_64.whl", + 11: "pkg-0.0.1-cp311-none-manylinux_1_1_x86_64.whl", + }.items(): + got = _select_whl( + whls = whls, + whl_platform_tags = ["manylinux_1_1_x86_64"], + whl_abi_tags = ["none"], + python_version = "3.{}".format(minor_version), + ) + _match(env, [got], match) + +_tests.append(_test_supported_cp_version_manylinux) + +def _test_ignore_unsupported(env): + whls = ["pkg-0.0.1-xx3-abi3-any.whl"] + got = _select_whl( + whls = whls, + whl_platform_tags = ["any"], + whl_abi_tags = ["none"], + python_version = "3.0", + ) + if got: + _match(env, [got], None) + +_tests.append(_test_ignore_unsupported) + +def _test_match_abi_and_not_py_version(env): + # Check we match the ABI and not the py version + whls = WHL_LIST + whl_platform_tags = [ + "musllinux_*_x86_64", + "manylinux_*_x86_64", + ] + got = _select_whl( + whls = whls, + whl_platform_tags = whl_platform_tags, + whl_abi_tags = ["abi3", "cp37m"], + python_version = "3.7", + ) + _match( + env, + [got], + "pkg-0.0.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", + ) + + got = _select_whl( + whls = whls, + whl_platform_tags = whl_platform_tags[::-1], + whl_abi_tags = ["abi3", "cp37m"], + python_version = "3.7", + ) + _match( + env, + [got], + "pkg-0.0.1-cp37-cp37m-musllinux_1_1_x86_64.whl", + ) + +_tests.append(_test_match_abi_and_not_py_version) + +def _test_select_filename_with_many_tags(env): + # Check we can select a filename with many platform tags + got = _select_whl( + whls = WHL_LIST, + whl_platform_tags = [ + "any", + "musllinux_*_i686", + "manylinux_*_i686", + ], + whl_abi_tags = ["none", "abi3", "cp39"], + python_version = "3.9", + limit = 5, + ) + _match( + env, + got, + "pkg-0.0.1-py3-none-any.whl", + "pkg-0.0.1-py3-abi3-any.whl", + "pkg-0.0.1-cp39-abi3-any.whl", + "pkg-0.0.1-cp39-cp39-musllinux_1_1_i686.whl", + "pkg-0.0.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", + ) + +_tests.append(_test_select_filename_with_many_tags) + +def _test_freethreaded_wheels(env): + # Check we prefer platform specific wheels + got = _select_whl( + whls = WHL_LIST, + whl_platform_tags = [ + "any", + "musllinux_*_x86_64", + ], + whl_abi_tags = ["none", "abi3", "cp313", "cp313t"], + python_version = "3.13", + limit = 8, + ) + _match( + env, + got, + # The last item has the most priority + "pkg-0.0.1-py3-none-any.whl", + "pkg-0.0.1-py3-abi3-any.whl", + "pkg-0.0.1-py310-abi3-any.whl", + "pkg-0.0.1-cp39-abi3-any.whl", + "pkg-0.0.1-cp313-none-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp313-abi3-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp313-cp313-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp313-cp313t-musllinux_1_1_x86_64.whl", + ) + +_tests.append(_test_freethreaded_wheels) + +def _test_pytags_all_possible(env): + got = _select_whl( + whls = [ + "pkg-0.0.1-py2.py27.py3.py30.py31.py32.py33.py34.py35.py36.py37.py38.py39.py310.py311.py312.py313-none-win_amd64.whl", + ], + whl_platform_tags = ["win_amd64"], + whl_abi_tags = ["none"], + python_version = "3.12", + ) + _match( + env, + [got], + "pkg-0.0.1-py2.py27.py3.py30.py31.py32.py33.py34.py35.py36.py37.py38.py39.py310.py311.py312.py313-none-win_amd64.whl", + ) + +_tests.append(_test_pytags_all_possible) + +def _test_manylinx_musllinux_pref(env): + got = _select_whl( + whls = [ + "pkg-0.0.1-py3-none-manylinux_2_31_x86_64.musllinux_1_1_x86_64.whl", + ], + whl_platform_tags = [ + "manylinux_*_x86_64", + "musllinux_*_x86_64", + ], + whl_abi_tags = ["none"], + python_version = "3.12", + limit = 2, + ) + _match( + env, + got, + # there is only one wheel, just select that + "pkg-0.0.1-py3-none-manylinux_2_31_x86_64.musllinux_1_1_x86_64.whl", + ) + +_tests.append(_test_manylinx_musllinux_pref) + +def _test_multiple_musllinux(env): + got = _select_whl( + whls = [ + "pkg-0.0.1-py3-none-musllinux_1_2_x86_64.whl", + "pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl", + ], + whl_platform_tags = ["musllinux_*_x86_64"], + whl_abi_tags = ["none"], + python_version = "3.12", + limit = 2, + ) + _match( + env, + got, + # select the one with the lowest version that is matching because we want to + # increase the compatibility + "pkg-0.0.1-py3-none-musllinux_1_2_x86_64.whl", + "pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl", + ) + +_tests.append(_test_multiple_musllinux) + +def _test_multiple_musllinux_exact_params(env): + got = _select_whl( + whls = [ + "pkg-0.0.1-py3-none-musllinux_1_2_x86_64.whl", + "pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl", + ], + whl_platform_tags = ["musllinux_1_2_x86_64", "musllinux_1_1_x86_64"], + whl_abi_tags = ["none"], + python_version = "3.12", + limit = 2, + ) + _match( + env, + got, + # 1.2 is not within the candidates because it is not compatible + "pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl", + ) + +_tests.append(_test_multiple_musllinux_exact_params) + +def _test_multiple_mvs_match(env): + got = _select_whl( + whls = [ + "pkg-0.0.1-py3-none-musllinux_1_4_x86_64.whl", + "pkg-0.0.1-py3-none-musllinux_1_2_x86_64.whl", + "pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl", + ], + whl_platform_tags = ["musllinux_1_3_x86_64"], + whl_abi_tags = ["none"], + python_version = "3.12", + limit = 2, + ) + _match( + env, + got, + # select the one with the lowest version + "pkg-0.0.1-py3-none-musllinux_1_2_x86_64.whl", + "pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl", + ) + +_tests.append(_test_multiple_mvs_match) + +def _test_multiple_mvs_match_override_more_specific(env): + got = _select_whl( + whls = [ + "pkg-0.0.1-py3-none-musllinux_1_4_x86_64.whl", + "pkg-0.0.1-py3-none-musllinux_1_2_x86_64.whl", + "pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl", + ], + whl_platform_tags = [ + "musllinux_*_x86_64", # default to something + "musllinux_1_3_x86_64", # override the previous + ], + whl_abi_tags = ["none"], + python_version = "3.12", + limit = 2, + ) + _match( + env, + got, + # Should be the same as without the `*` match + "pkg-0.0.1-py3-none-musllinux_1_2_x86_64.whl", + "pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl", + ) + +_tests.append(_test_multiple_mvs_match_override_more_specific) + +def _test_multiple_mvs_match_override_less_specific(env): + got = _select_whl( + whls = [ + "pkg-0.0.1-py3-none-musllinux_1_4_x86_64.whl", + ], + whl_platform_tags = [ + "musllinux_1_3_x86_64", # default to 1.3 + "musllinux_*_x86_64", # then override to something less specific + ], + whl_abi_tags = ["none"], + python_version = "3.12", + limit = 2, + ) + _match( + env, + got, + "pkg-0.0.1-py3-none-musllinux_1_4_x86_64.whl", + ) + +_tests.append(_test_multiple_mvs_match_override_less_specific) + +def _test_android(env): + got = _select_whl( + whls = [ + "pkg-0.0.1-py3-none-android_4_x86_64.whl", + "pkg-0.0.1-py3-none-android_8_x86_64.whl", + ], + whl_platform_tags = ["android_5_x86_64"], + whl_abi_tags = ["none"], + python_version = "3.12", + limit = 2, + ) + _match( + env, + got, + # select the one with the highest version that is matching + "pkg-0.0.1-py3-none-android_4_x86_64.whl", + ) + +_tests.append(_test_android) + +def select_whl_test_suite(name): + """Create the test suite. + + Args: + name: the name of the test suite + """ + test_suite(name = name, basic_tests = _tests) diff --git a/tests/pypi/simpleapi_download/BUILD.bazel b/tests/pypi/simpleapi_download/BUILD.bazel new file mode 100644 index 0000000000..04747b6246 --- /dev/null +++ b/tests/pypi/simpleapi_download/BUILD.bazel @@ -0,0 +1,5 @@ +load("simpleapi_download_tests.bzl", "simpleapi_download_test_suite") + +simpleapi_download_test_suite( + name = "simpleapi_download_tests", +) diff --git a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl new file mode 100644 index 0000000000..dfa33ebe65 --- /dev/null +++ b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl @@ -0,0 +1,358 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private/pypi:pypi_cache.bzl", "pypi_cache") # buildifier: disable=bzl-visibility +load("//python/private/pypi:simpleapi_download.bzl", "simpleapi_download") # buildifier: disable=bzl-visibility +load("//tests/support/mocks:mocks.bzl", "mocks") + +_tests = [] + +def _test_simple(env): + calls = [] + + def read_simpleapi(ctx, url, versions, attr, cache, get_auth, block, parse_index): + if parse_index: + return struct( + success = True, + output = { + "bar": "/bar/", + "baz": "/baz/", + } if "main" in url else { + "foo": "/foo/", + }, + ) + + _ = ctx, attr, cache, get_auth, versions # buildifier: disable=unused-variable + env.expect.that_bool(block).equals(False) + calls.append(url) + return struct( + output = struct( + sdists = {"deadbeef": url.strip("/").split("/")[-1]}, + whls = {"deadb33f": url.strip("/").split("/")[-1]}, + hashes_by_version = {"fizz": url.strip("/").split("/")[-1]}, + ), + success = True, + ) + + contents = simpleapi_download( + ctx = mocks.mctx(), + attr = struct( + index_url_overrides = {}, + index_url = "https://main.com", + extra_index_urls = ["https://extra.com"], + sources = {"bar": None, "baz": None, "foo": None}, + envsubst = [], + ), + cache = pypi_cache(), + parallel_download = True, + read_simpleapi = read_simpleapi, + ) + + env.expect.that_collection(calls).contains_exactly([ + "https://extra.com/foo/", + "https://main.com/bar/", + "https://main.com/baz/", + ]) + env.expect.that_dict(contents).contains_exactly({ + "bar": struct( + index_url = "https://main.com/bar/", + sdists = {"deadbeef": "bar"}, + hashes_by_version = {"fizz": "bar"}, + whls = {"deadb33f": "bar"}, + ), + "baz": struct( + index_url = "https://main.com/baz/", + sdists = {"deadbeef": "baz"}, + hashes_by_version = {"fizz": "baz"}, + whls = {"deadb33f": "baz"}, + ), + "foo": struct( + index_url = "https://extra.com/foo/", + sdists = {"deadbeef": "foo"}, + hashes_by_version = {"fizz": "foo"}, + whls = {"deadb33f": "foo"}, + ), + }) + +_tests.append(_test_simple) + +def _test_index_overrides(env): + calls = [] + fails = [] + + def read_simpleapi(ctx, *, url, versions, attr, cache, get_auth, block, parse_index): + if parse_index: + return struct( + success = True, + output = { + # normalized + "ba_z": "/ba-z/", + "bar": "/bar/", + "foo": "/foo-should-fail/", + } if "main" in url else { + "foo": "/foo/", + }, + ) + + _ = ctx, attr, cache, get_auth, versions # buildifier: disable=unused-variable + env.expect.that_bool(block).equals(False) + calls.append(url) + return struct( + output = struct( + sdists = {"deadbeef": url.strip("/").split("/")[-1]}, + whls = {"deadb33f": url.strip("/").split("/")[-1]}, + hashes_by_version = {"fizz": url.strip("/").split("/")[-1]}, + ), + success = True, + ) + + contents = simpleapi_download( + ctx = mocks.mctx(), + attr = struct( + index_url_overrides = { + "foo": "https://extra.com", + }, + index_url = "https://main.com", + extra_index_urls = [], + sources = {"ba_z": None, "bar": None, "foo": None}, + envsubst = [], + ), + cache = pypi_cache(), + parallel_download = True, + read_simpleapi = read_simpleapi, + _fail = fails.append, + ) + + env.expect.that_collection(fails).contains_exactly([]) + env.expect.that_collection(calls).contains_exactly([ + "https://main.com/bar/", + "https://main.com/ba-z/", + "https://extra.com/foo/", + ]) + env.expect.that_dict(contents).contains_exactly({ + "ba_z": struct( + index_url = "https://main.com/ba-z/", + sdists = {"deadbeef": "ba-z"}, + hashes_by_version = {"fizz": "ba-z"}, + whls = {"deadb33f": "ba-z"}, + ), + "bar": struct( + index_url = "https://main.com/bar/", + sdists = {"deadbeef": "bar"}, + hashes_by_version = {"fizz": "bar"}, + whls = {"deadb33f": "bar"}, + ), + "foo": struct( + index_url = "https://extra.com/foo/", + sdists = {"deadbeef": "foo"}, + hashes_by_version = {"fizz": "foo"}, + whls = {"deadb33f": "foo"}, + ), + }) + +_tests.append(_test_index_overrides) + +def _test_download_url(env): + downloads = {} + reads = [ + "", + "", + "", + ] + + def download(url, output, **kwargs): + _ = kwargs # buildifier: disable=unused-variable + downloads[url[0]] = output + return struct(success = True) + + simpleapi_download( + ctx = struct( + getenv = {}.get, + download = download, + report_progress = lambda _: None, + # We will first add a download to the list, so this is a poor man's `next(foo)` + # implementation + read = lambda i: reads[len(downloads) - 1], + path = lambda i: "path/for/" + i, + ), + attr = struct( + index_url_overrides = {}, + index_url = "https://example.com/main/simple/", + extra_index_urls = [], + sources = {"bar": ["1.0"], "baz": ["1.0"], "foo": ["1.0"]}, + envsubst = [], + ), + cache = pypi_cache(), + parallel_download = False, + get_auth = lambda ctx, urls, ctx_attr: struct(), + ) + + env.expect.that_dict(downloads).contains_exactly({ + "https://example.com/main/simple/bar/": "path/for/https___example_com_main_simple_bar.html", + "https://example.com/main/simple/baz/": "path/for/https___example_com_main_simple_baz.html", + "https://example.com/main/simple/foo/": "path/for/https___example_com_main_simple_foo.html", + }) + +_tests.append(_test_download_url) + +def _test_download_url_parallel(env): + downloads = {} + reads = [ + # The first read is the index which seeds the downloads later + """ + bar + baz + foo + """, + "", + "", + "", + "", + ] + + def download(url, output, **kwargs): + _ = kwargs # buildifier: disable=unused-variable + downloads[url[0]] = output + return struct(wait = lambda: struct(success = True)) + + simpleapi_download( + ctx = struct( + getenv = {}.get, + download = download, + report_progress = lambda _: None, + # We will first add a download to the list, so this is a poor man's `next(foo)` + # implementation. We use 2 because we will enqueue 2 downloads in parallel. + read = lambda i: reads[len(downloads) - 2], + path = lambda i: "path/for/" + i, + ), + attr = struct( + index_url_overrides = {}, + index_url = "https://example.com/default/simple/", + extra_index_urls = ["https://example.com/extra/simple/"], + sources = {"bar": None, "baz": None, "foo": None}, + envsubst = [], + ), + cache = pypi_cache(), + parallel_download = True, + get_auth = lambda ctx, urls, ctx_attr: struct(), + ) + + env.expect.that_dict(downloads).contains_exactly({ + "https://example.com/default/simple/": "path/for/https___example_com_default_simple.html", + "https://example.com/extra/simple/": "path/for/https___example_com_extra_simple.html", + "https://example.com/extra/simple/bar/": "path/for/https___example_com_extra_simple_bar.html", + "https://example.com/extra/simple/baz/": "path/for/https___example_com_extra_simple_baz.html", + "https://example.com/extra/simple/foo/": "path/for/https___example_com_extra_simple_foo.html", + }) + +_tests.append(_test_download_url_parallel) + +def _test_download_url_parallel_with_overrides(env): + downloads = {} + reads = [ + "", + "", + "", + ] + + def download(url, output, **kwargs): + _ = kwargs # buildifier: disable=unused-variable + downloads[url[0]] = output + return struct(wait = lambda: struct(success = True)) + + simpleapi_download( + ctx = struct( + getenv = {}.get, + download = download, + report_progress = lambda _: None, + # We will first add a download to the list, so this is a poor man's `next(foo)` + # implementation. We use 2 because we will enqueue 2 downloads in parallel. + read = lambda i: reads[len(downloads) - 2], + path = lambda i: "path/for/" + i, + ), + attr = struct( + index_url_overrides = { + "bar": "https://example.com/extra/simple/", + }, + index_url = "https://example.com/default/simple/", + extra_index_urls = [], + sources = {"bar": None, "baz": None, "foo": None}, + envsubst = [], + ), + cache = pypi_cache(), + parallel_download = True, + get_auth = lambda ctx, urls, ctx_attr: struct(), + ) + + env.expect.that_dict(downloads).contains_exactly({ + "https://example.com/default/simple/baz/": "path/for/https___example_com_default_simple_baz.html", + "https://example.com/default/simple/foo/": "path/for/https___example_com_default_simple_foo.html", + "https://example.com/extra/simple/bar/": "path/for/https___example_com_extra_simple_bar.html", + }) + +_tests.append(_test_download_url_parallel_with_overrides) + +def _test_download_envsubst_url(env): + downloads = {} + reads = [ + "", + "", + "", + ] + + def download(url, output, **kwargs): + _ = kwargs # buildifier: disable=unused-variable + downloads[url[0]] = output + return struct(success = True) + + simpleapi_download( + ctx = struct( + getenv = {"INDEX_URL": "https://example.com/main/simple/"}.get, + download = download, + report_progress = lambda _: None, + # We will first add a download to the list, so this is a poor man's `next(foo)` + # implementation + read = lambda i: reads[len(downloads) - 1], + path = lambda i: "path/for/" + i, + ), + attr = struct( + index_url_overrides = {}, + index_url = "$INDEX_URL", + extra_index_urls = [], + sources = {"bar": None, "baz": None, "foo": None}, + envsubst = ["INDEX_URL"], + ), + cache = pypi_cache(), + parallel_download = False, + get_auth = lambda ctx, urls, ctx_attr: struct(), + ) + + env.expect.that_dict(downloads).contains_exactly({ + "https://example.com/main/simple/bar/": "path/for/~index_url~_bar.html", + "https://example.com/main/simple/baz/": "path/for/~index_url~_baz.html", + "https://example.com/main/simple/foo/": "path/for/~index_url~_foo.html", + }) + +_tests.append(_test_download_envsubst_url) + +def simpleapi_download_test_suite(name): + """Create the test suite. + + Args: + name: the name of the test suite + """ + test_suite(name = name, basic_tests = _tests) diff --git a/tests/pypi/urllib/BUILD.bazel b/tests/pypi/urllib/BUILD.bazel new file mode 100644 index 0000000000..a6405684a7 --- /dev/null +++ b/tests/pypi/urllib/BUILD.bazel @@ -0,0 +1,3 @@ +load(":urllib_tests.bzl", "urllib_test_suite") + +urllib_test_suite(name = "urllib_tests") diff --git a/tests/pypi/urllib/urllib_tests.bzl b/tests/pypi/urllib/urllib_tests.bzl new file mode 100644 index 0000000000..1a175dc120 --- /dev/null +++ b/tests/pypi/urllib/urllib_tests.bzl @@ -0,0 +1,56 @@ +"" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private/pypi:urllib.bzl", "urllib") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_absolute_url(env): + # Already absolute + for already_absolute in [ + "file:///foo", + "file:///c:/foo", + "https://foo.com", + "http://foo.com", + ]: + env.expect.that_str(urllib.absolute_url("https://ignored", already_absolute)).equals(already_absolute) + + # Simple with empty path segments + env.expect.that_str(urllib.absolute_url("https://example.com//", "file.whl")).equals("https://example.com/file.whl") + env.expect.that_str(urllib.absolute_url("https://example.com//a/b//", "../../file.whl")).equals("https://example.com/file.whl") + env.expect.that_str(urllib.absolute_url("https://example.com//a/b//", "/file.whl")).equals("https://example.com/file.whl") + + # Relative URLs + env.expect.that_str(urllib.absolute_url("https://example.com/relative", "file.whl")).equals("https://example.com/relative/file.whl") + env.expect.that_str(urllib.absolute_url("https://example.com/relative/", "file.whl")).equals("https://example.com/relative/file.whl") + env.expect.that_str(urllib.absolute_url("https://example.com/relative/", "../relative/file.whl")).equals("https://example.com/relative/file.whl") + + # Relative URL for files + env.expect.that_str(urllib.absolute_url("file://${PYPI_BAZEL_WORKSPACE_ROOT}", "vendor/distro/file.whl")).equals("file://${PYPI_BAZEL_WORKSPACE_ROOT}/vendor/distro/file.whl") + env.expect.that_str(urllib.absolute_url( + "file://${PYPI_BAZEL_WORKSPACE_ROOT}", + "vendor/distro/file.whl", + envsubst = ["PYPI_BAZEL_WORKSPACE_ROOT"], + getenv = {"PYPI_BAZEL_WORKSPACE_ROOT": "/some/dir"}.get, + )).equals("file:///some/dir/vendor/distro/file.whl") + +_tests.append(_test_absolute_url) + +def _test_strip_empty_path_segments(env): + env.expect.that_str(urllib.strip_empty_path_segments("no/scheme//is/unchanged")).equals("no/scheme//is/unchanged") + env.expect.that_str(urllib.strip_empty_path_segments("scheme://with/no/empty/segments")).equals("scheme://with/no/empty/segments") + env.expect.that_str(urllib.strip_empty_path_segments("scheme://with//empty/segments")).equals("scheme://with/empty/segments") + env.expect.that_str(urllib.strip_empty_path_segments("scheme://with///multiple//empty/segments")).equals("scheme://with/multiple/empty/segments") + env.expect.that_str(urllib.strip_empty_path_segments("scheme://with//trailing/slash/")).equals("scheme://with/trailing/slash/") + env.expect.that_str(urllib.strip_empty_path_segments("scheme://with/trailing/slashes///")).equals("scheme://with/trailing/slashes/") + env.expect.that_str(urllib.strip_empty_path_segments("file:///home/user//foo")).equals("file:///home/user/foo") + +_tests.append(_test_strip_empty_path_segments) + +def urllib_test_suite(name): + """Create the test suite. + + Args: + name: the name of the test suite + """ + test_suite(name = name, basic_tests = _tests) diff --git a/tests/pypi/version_from_filename/BUILD.bazel b/tests/pypi/version_from_filename/BUILD.bazel new file mode 100644 index 0000000000..e9d50dc6b8 --- /dev/null +++ b/tests/pypi/version_from_filename/BUILD.bazel @@ -0,0 +1,3 @@ +load(":version_from_filename_tests.bzl", "version_from_filename_test_suite") + +version_from_filename_test_suite(name = "version_from_filename_tests") diff --git a/tests/pypi/version_from_filename/version_from_filename_tests.bzl b/tests/pypi/version_from_filename/version_from_filename_tests.bzl new file mode 100644 index 0000000000..fab921fd4f --- /dev/null +++ b/tests/pypi/version_from_filename/version_from_filename_tests.bzl @@ -0,0 +1,56 @@ +"" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private/pypi:version_from_filename.bzl", "version_from_filename") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_wheel_version_extraction(env): + # Case 1: wheel + env.expect.that_str(version_from_filename("foo-1.2.3-py3-none-any.whl")).equals("1.2.3") + +_tests.append(_test_wheel_version_extraction) + +def _test_sdist_version_extraction(env): + # Case 1: Standard sdist + env.expect.that_str(version_from_filename("foo-1.2.3.tar.gz")).equals("1.2.3") + + # Case 2: PEP 625 - Project name has underscores (normalized from dashes) + # If the package is 'my-pkg', the sdist might be 'my_pkg-1.0.0.tar.gz' + env.expect.that_str(version_from_filename("my_pkg-1.0.0.tar.gz")).equals("1.0.0") + + # Case 3: Project name has multiple underscores + env.expect.that_str(version_from_filename("very_long_project_name-0.5.0.zip")).equals("0.5.0") + + # Case 4: Legacy sdist with hyphens in name + # Note: Modern tools normalize this, but we should support the hyphen split + env.expect.that_str(version_from_filename("complex-name-1.2.3.tar.gz")).equals("1.2.3") + + # Case 5: Version contains an underscore (e.g. local versions) + env.expect.that_str(version_from_filename("pkg-1.2.3_post1.tar.gz")).equals("1.2.3_post1") + + # Case 6: custom compression + env.expect.that_str(version_from_filename("pkg-1.2.3_post1.tar.xz")).equals("1.2.3_post1") + +_tests.append(_test_sdist_version_extraction) + +def _test_sdist_version_extraction_fail(env): + failures = [] + + # Case 1: 7z + env.expect.that_str(version_from_filename("foo-1.2.3.7z")).equals(None) + env.expect.that_str(version_from_filename("foo-1.2.3.7z", _fail = failures.append)).equals(None) + env.expect.that_collection(failures).contains_exactly(["Unsupported sdist extension: foo-1.2.3.7z"]) + + # Case 2: egg + failures.clear() + env.expect.that_str(version_from_filename("foo-1.2.3-py3.egg", _fail = failures.append)).equals(None) + env.expect.that_collection(failures).contains_exactly(["Unsupported sdist extension: foo-1.2.3-py3.egg"]) + +_tests.append(_test_sdist_version_extraction_fail) + +def version_from_filename_test_suite(name): + test_suite( + name = name, + basic_tests = _tests, + ) diff --git a/tests/pypi/whl_installer/BUILD.bazel b/tests/pypi/whl_installer/BUILD.bazel index e25c4a06a4..5a2efb1260 100644 --- a/tests/pypi/whl_installer/BUILD.bazel +++ b/tests/pypi/whl_installer/BUILD.bazel @@ -1,9 +1,4 @@ -load("//python:defs.bzl", "py_test") - -alias( - name = "lib", - actual = "//python/private/pypi/whl_installer:lib", -) +load("//python:py_test.bzl", "py_test") py_test( name = "arguments_test", @@ -12,53 +7,6 @@ py_test( "arguments_test.py", ], deps = [ - ":lib", - ], -) - -py_test( - name = "namespace_pkgs_test", - size = "small", - srcs = [ - "namespace_pkgs_test.py", - ], - deps = [ - ":lib", - ], -) - -py_test( - name = "platform_test", - size = "small", - srcs = [ - "platform_test.py", - ], - data = ["//examples/wheel:minimal_with_py_package"], - deps = [ - ":lib", - ], -) - -py_test( - name = "wheel_installer_test", - size = "small", - srcs = [ - "wheel_installer_test.py", - ], - data = ["//examples/wheel:minimal_with_py_package"], - deps = [ - ":lib", - ], -) - -py_test( - name = "wheel_test", - size = "small", - srcs = [ - "wheel_test.py", - ], - data = ["//examples/wheel:minimal_with_py_package"], - deps = [ - ":lib", + "//python/private/pypi/whl_installer:lib", ], ) diff --git a/tests/pypi/whl_installer/arguments_test.py b/tests/pypi/whl_installer/arguments_test.py index 5538054a59..c874445029 100644 --- a/tests/pypi/whl_installer/arguments_test.py +++ b/tests/pypi/whl_installer/arguments_test.py @@ -15,7 +15,7 @@ import json import unittest -from python.private.pypi.whl_installer import arguments, wheel +from python.private.pypi.whl_installer import arguments class ArgumentsTestCase(unittest.TestCase): @@ -36,7 +36,6 @@ def test_arguments(self) -> None: self.assertIn("requirement", args_dict) self.assertIn("extra_pip_args", args_dict) self.assertEqual(args_dict["pip_data_exclude"], []) - self.assertEqual(args_dict["enable_implicit_namespace_pkgs"], False) self.assertEqual(args_dict["extra_pip_args"], extra_pip_args) def test_deserialize_structured_args(self) -> None: @@ -49,18 +48,6 @@ def test_deserialize_structured_args(self) -> None: self.assertEqual(args["environment"], {"PIP_DO_SOMETHING": "True"}) self.assertEqual(args["extra_pip_args"], []) - def test_platform_aggregation(self) -> None: - parser = arguments.parser() - args = parser.parse_args( - args=[ - "--platform=linux_*", - "--platform=osx_*", - "--platform=windows_*", - "--requirement=foo", - ] - ) - self.assertEqual(set(wheel.Platform.all()), arguments.get_platforms(args)) - if __name__ == "__main__": unittest.main() diff --git a/tests/pypi/whl_installer/namespace_pkgs_test.py b/tests/pypi/whl_installer/namespace_pkgs_test.py deleted file mode 100644 index fbbd50926a..0000000000 --- a/tests/pypi/whl_installer/namespace_pkgs_test.py +++ /dev/null @@ -1,192 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -import os -import pathlib -import shutil -import tempfile -import unittest -from typing import Optional, Set - -from python.private.pypi.whl_installer import namespace_pkgs - - -class TempDir: - def __init__(self) -> None: - self.dir = tempfile.mkdtemp() - - def root(self) -> str: - return self.dir - - def add_dir(self, rel_path: str) -> None: - d = pathlib.Path(self.dir, rel_path) - d.mkdir(parents=True) - - def add_file(self, rel_path: str, contents: Optional[str] = None) -> None: - f = pathlib.Path(self.dir, rel_path) - f.parent.mkdir(parents=True, exist_ok=True) - if contents: - with open(str(f), "w") as writeable_f: - writeable_f.write(contents) - else: - f.touch() - - def remove(self) -> None: - shutil.rmtree(self.dir) - - -class TestImplicitNamespacePackages(unittest.TestCase): - def assertPathsEqual(self, actual: Set[pathlib.Path], expected: Set[str]) -> None: - self.assertEqual(actual, {pathlib.Path(p) for p in expected}) - - def test_in_current_directory(self) -> None: - directory = TempDir() - directory.add_file("foo/bar/biz.py") - directory.add_file("foo/bee/boo.py") - directory.add_file("foo/buu/__init__.py") - directory.add_file("foo/buu/bii.py") - cwd = os.getcwd() - os.chdir(directory.root()) - expected = { - "foo", - "foo/bar", - "foo/bee", - } - try: - actual = namespace_pkgs.implicit_namespace_packages(".") - self.assertPathsEqual(actual, expected) - finally: - os.chdir(cwd) - directory.remove() - - def test_finds_correct_namespace_packages(self) -> None: - directory = TempDir() - directory.add_file("foo/bar/biz.py") - directory.add_file("foo/bee/boo.py") - directory.add_file("foo/buu/__init__.py") - directory.add_file("foo/buu/bii.py") - - expected = { - directory.root() + "/foo", - directory.root() + "/foo/bar", - directory.root() + "/foo/bee", - } - actual = namespace_pkgs.implicit_namespace_packages(directory.root()) - self.assertPathsEqual(actual, expected) - - def test_ignores_empty_directories(self) -> None: - directory = TempDir() - directory.add_file("foo/bar/biz.py") - directory.add_dir("foo/cat") - - expected = { - directory.root() + "/foo", - directory.root() + "/foo/bar", - } - actual = namespace_pkgs.implicit_namespace_packages(directory.root()) - self.assertPathsEqual(actual, expected) - - def test_empty_case(self) -> None: - directory = TempDir() - directory.add_file("foo/__init__.py") - directory.add_file("foo/bar/__init__.py") - directory.add_file("foo/bar/biz.py") - - actual = namespace_pkgs.implicit_namespace_packages(directory.root()) - self.assertEqual(actual, set()) - - def test_ignores_non_module_files_in_directories(self) -> None: - directory = TempDir() - directory.add_file("foo/__init__.pyi") - directory.add_file("foo/py.typed") - - actual = namespace_pkgs.implicit_namespace_packages(directory.root()) - self.assertEqual(actual, set()) - - def test_parent_child_relationship_of_namespace_pkgs(self): - directory = TempDir() - directory.add_file("foo/bar/biff/my_module.py") - directory.add_file("foo/bar/biff/another_module.py") - - expected = { - directory.root() + "/foo", - directory.root() + "/foo/bar", - directory.root() + "/foo/bar/biff", - } - actual = namespace_pkgs.implicit_namespace_packages(directory.root()) - self.assertPathsEqual(actual, expected) - - def test_parent_child_relationship_of_namespace_and_standard_pkgs(self): - directory = TempDir() - directory.add_file("foo/bar/biff/__init__.py") - directory.add_file("foo/bar/biff/another_module.py") - - expected = { - directory.root() + "/foo", - directory.root() + "/foo/bar", - } - actual = namespace_pkgs.implicit_namespace_packages(directory.root()) - self.assertPathsEqual(actual, expected) - - def test_parent_child_relationship_of_namespace_and_nested_standard_pkgs(self): - directory = TempDir() - directory.add_file("foo/bar/__init__.py") - directory.add_file("foo/bar/biff/another_module.py") - directory.add_file("foo/bar/biff/__init__.py") - directory.add_file("foo/bar/boof/big_module.py") - directory.add_file("foo/bar/boof/__init__.py") - directory.add_file("fim/in_a_ns_pkg.py") - - expected = { - directory.root() + "/foo", - directory.root() + "/fim", - } - actual = namespace_pkgs.implicit_namespace_packages(directory.root()) - self.assertPathsEqual(actual, expected) - - def test_recognized_all_nonstandard_module_types(self): - directory = TempDir() - directory.add_file("ayy/my_module.pyc") - directory.add_file("bee/ccc/dee/eee.so") - directory.add_file("eff/jee/aych.pyd") - - expected = { - directory.root() + "/ayy", - directory.root() + "/bee", - directory.root() + "/bee/ccc", - directory.root() + "/bee/ccc/dee", - directory.root() + "/eff", - directory.root() + "/eff/jee", - } - actual = namespace_pkgs.implicit_namespace_packages(directory.root()) - self.assertPathsEqual(actual, expected) - - def test_skips_ignored_directories(self): - directory = TempDir() - directory.add_file("foo/boo/my_module.py") - directory.add_file("foo/bar/another_module.py") - - expected = { - directory.root() + "/foo", - directory.root() + "/foo/bar", - } - actual = namespace_pkgs.implicit_namespace_packages( - directory.root(), - ignored_dirnames=[directory.root() + "/foo/boo"], - ) - self.assertPathsEqual(actual, expected) - - -if __name__ == "__main__": - unittest.main() diff --git a/tests/pypi/whl_installer/platform_test.py b/tests/pypi/whl_installer/platform_test.py deleted file mode 100644 index 7ced1e9826..0000000000 --- a/tests/pypi/whl_installer/platform_test.py +++ /dev/null @@ -1,152 +0,0 @@ -import unittest -from random import shuffle - -from python.private.pypi.whl_installer.platform import ( - OS, - Arch, - Platform, - host_interpreter_minor_version, -) - - -class MinorVersionTest(unittest.TestCase): - def test_host(self): - host = host_interpreter_minor_version() - self.assertIsNotNone(host) - - -class PlatformTest(unittest.TestCase): - def test_can_get_host(self): - host = Platform.host() - self.assertIsNotNone(host) - self.assertEqual(1, len(Platform.from_string("host"))) - self.assertEqual(host, Platform.from_string("host")) - - def test_can_get_linux_x86_64_without_py_version(self): - got = Platform.from_string("linux_x86_64") - want = Platform(os=OS.linux, arch=Arch.x86_64) - self.assertEqual(want, got[0]) - - def test_can_get_specific_from_string(self): - got = Platform.from_string("cp33_linux_x86_64") - want = Platform(os=OS.linux, arch=Arch.x86_64, minor_version=3) - self.assertEqual(want, got[0]) - - def test_can_get_all_for_py_version(self): - cp39 = Platform.all(minor_version=9) - self.assertEqual(18, len(cp39), f"Got {cp39}") - self.assertEqual(cp39, Platform.from_string("cp39_*")) - - def test_can_get_all_for_os(self): - linuxes = Platform.all(OS.linux, minor_version=9) - self.assertEqual(6, len(linuxes)) - self.assertEqual(linuxes, Platform.from_string("cp39_linux_*")) - - def test_can_get_all_for_os_for_host_python(self): - linuxes = Platform.all(OS.linux) - self.assertEqual(6, len(linuxes)) - self.assertEqual(linuxes, Platform.from_string("linux_*")) - - def test_specific_version_specializations(self): - any_py33 = Platform(minor_version=3) - - # When - all_specializations = list(any_py33.all_specializations()) - - want = ( - [any_py33] - + [ - Platform(arch=arch, minor_version=any_py33.minor_version) - for arch in Arch - ] - + [Platform(os=os, minor_version=any_py33.minor_version) for os in OS] - + Platform.all(minor_version=any_py33.minor_version) - ) - self.assertEqual(want, all_specializations) - - def test_aarch64_specializations(self): - any_aarch64 = Platform(arch=Arch.aarch64) - all_specializations = list(any_aarch64.all_specializations()) - want = [ - Platform(os=None, arch=Arch.aarch64), - Platform(os=OS.linux, arch=Arch.aarch64), - Platform(os=OS.osx, arch=Arch.aarch64), - Platform(os=OS.windows, arch=Arch.aarch64), - ] - self.assertEqual(want, all_specializations) - - def test_linux_specializations(self): - any_linux = Platform(os=OS.linux) - all_specializations = list(any_linux.all_specializations()) - want = [ - Platform(os=OS.linux, arch=None), - Platform(os=OS.linux, arch=Arch.x86_64), - Platform(os=OS.linux, arch=Arch.x86_32), - Platform(os=OS.linux, arch=Arch.aarch64), - Platform(os=OS.linux, arch=Arch.ppc), - Platform(os=OS.linux, arch=Arch.s390x), - Platform(os=OS.linux, arch=Arch.arm), - ] - self.assertEqual(want, all_specializations) - - def test_osx_specializations(self): - any_osx = Platform(os=OS.osx) - all_specializations = list(any_osx.all_specializations()) - # NOTE @aignas 2024-01-14: even though in practice we would only have - # Python on osx aarch64 and osx x86_64, we return all arch posibilities - # to make the code simpler. - want = [ - Platform(os=OS.osx, arch=None), - Platform(os=OS.osx, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.x86_32), - Platform(os=OS.osx, arch=Arch.aarch64), - Platform(os=OS.osx, arch=Arch.ppc), - Platform(os=OS.osx, arch=Arch.s390x), - Platform(os=OS.osx, arch=Arch.arm), - ] - self.assertEqual(want, all_specializations) - - def test_platform_sort(self): - platforms = [ - Platform(os=OS.linux, arch=None), - Platform(os=OS.linux, arch=Arch.x86_64), - Platform(os=OS.osx, arch=None), - Platform(os=OS.osx, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.aarch64), - ] - shuffle(platforms) - platforms.sort() - want = [ - Platform(os=OS.linux, arch=None), - Platform(os=OS.linux, arch=Arch.x86_64), - Platform(os=OS.osx, arch=None), - Platform(os=OS.osx, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.aarch64), - ] - - self.assertEqual(want, platforms) - - def test_wheel_os_alias(self): - self.assertEqual("osx", str(OS.osx)) - self.assertEqual(str(OS.darwin), str(OS.osx)) - - def test_wheel_arch_alias(self): - self.assertEqual("x86_64", str(Arch.x86_64)) - self.assertEqual(str(Arch.amd64), str(Arch.x86_64)) - - def test_wheel_platform_alias(self): - give = Platform( - os=OS.darwin, - arch=Arch.amd64, - ) - alias = Platform( - os=OS.osx, - arch=Arch.x86_64, - ) - - self.assertEqual("osx_x86_64", str(give)) - self.assertEqual(str(alias), str(give)) - - -if __name__ == "__main__": - unittest.main() diff --git a/tests/pypi/whl_installer/wheel_installer_test.py b/tests/pypi/whl_installer/wheel_installer_test.py deleted file mode 100644 index 7139779c3e..0000000000 --- a/tests/pypi/whl_installer/wheel_installer_test.py +++ /dev/null @@ -1,105 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -import json -import os -import shutil -import tempfile -import unittest -from pathlib import Path - -from python.private.pypi.whl_installer import wheel_installer - - -class TestRequirementExtrasParsing(unittest.TestCase): - def test_parses_requirement_for_extra(self) -> None: - cases = [ - ("name[foo]", ("name", frozenset(["foo"]))), - ("name[ Foo123 ]", ("name", frozenset(["Foo123"]))), - (" name1[ foo ] ", ("name1", frozenset(["foo"]))), - ("Name[foo]", ("name", frozenset(["foo"]))), - ("name_foo[bar]", ("name-foo", frozenset(["bar"]))), - ( - "name [fred,bar] @ http://foo.com ; python_version=='2.7'", - ("name", frozenset(["fred", "bar"])), - ), - ( - "name[quux, strange];python_version<'2.7' and platform_version=='2'", - ("name", frozenset(["quux", "strange"])), - ), - ( - "name; (os_name=='a' or os_name=='b') and os_name=='c'", - (None, None), - ), - ( - "name@http://foo.com", - (None, None), - ), - ] - - for case, expected in cases: - with self.subTest(): - self.assertTupleEqual( - wheel_installer._parse_requirement_for_extra(case), expected - ) - - -class TestWhlFilegroup(unittest.TestCase): - def setUp(self) -> None: - self.wheel_name = "example_minimal_package-0.0.1-py3-none-any.whl" - self.wheel_dir = tempfile.mkdtemp() - self.wheel_path = os.path.join(self.wheel_dir, self.wheel_name) - shutil.copy(os.path.join("examples", "wheel", self.wheel_name), self.wheel_dir) - - def tearDown(self): - shutil.rmtree(self.wheel_dir) - - def test_wheel_exists(self) -> None: - wheel_installer._extract_wheel( - Path(self.wheel_path), - installation_dir=Path(self.wheel_dir), - extras={}, - enable_implicit_namespace_pkgs=False, - platforms=[], - ) - - want_files = [ - "metadata.json", - "site-packages", - self.wheel_name, - ] - self.assertEqual( - sorted(want_files), - sorted( - [ - str(p.relative_to(self.wheel_dir)) - for p in Path(self.wheel_dir).glob("*") - ] - ), - ) - with open("{}/metadata.json".format(self.wheel_dir)) as metadata_file: - metadata_file_content = json.load(metadata_file) - - want = dict( - version="0.0.1", - name="example-minimal-package", - deps=[], - deps_by_platform={}, - entry_points=[], - ) - self.assertEqual(want, metadata_file_content) - - -if __name__ == "__main__": - unittest.main() diff --git a/tests/pypi/whl_installer/wheel_test.py b/tests/pypi/whl_installer/wheel_test.py deleted file mode 100644 index 404218e12b..0000000000 --- a/tests/pypi/whl_installer/wheel_test.py +++ /dev/null @@ -1,371 +0,0 @@ -import unittest -from unittest import mock - -from python.private.pypi.whl_installer import wheel -from python.private.pypi.whl_installer.platform import OS, Arch, Platform - -_HOST_INTERPRETER_FN = ( - "python.private.pypi.whl_installer.wheel.host_interpreter_minor_version" -) - - -class DepsTest(unittest.TestCase): - def test_simple(self): - deps = wheel.Deps("foo", requires_dist=["bar"]) - - got = deps.build() - - self.assertIsInstance(got, wheel.FrozenDeps) - self.assertEqual(["bar"], got.deps) - self.assertEqual({}, got.deps_select) - - def test_can_add_os_specific_deps(self): - deps = wheel.Deps( - "foo", - requires_dist=[ - "bar", - "an_osx_dep; sys_platform=='darwin'", - "posix_dep; os_name=='posix'", - "win_dep; os_name=='nt'", - ], - platforms={ - Platform(os=OS.linux, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.aarch64), - Platform(os=OS.windows, arch=Arch.x86_64), - }, - ) - - got = deps.build() - - self.assertEqual(["bar"], got.deps) - self.assertEqual( - { - "@platforms//os:linux": ["posix_dep"], - "@platforms//os:osx": ["an_osx_dep", "posix_dep"], - "@platforms//os:windows": ["win_dep"], - }, - got.deps_select, - ) - - def test_can_add_os_specific_deps_with_specific_python_version(self): - deps = wheel.Deps( - "foo", - requires_dist=[ - "bar", - "an_osx_dep; sys_platform=='darwin'", - "posix_dep; os_name=='posix'", - "win_dep; os_name=='nt'", - ], - platforms={ - Platform(os=OS.linux, arch=Arch.x86_64, minor_version=8), - Platform(os=OS.osx, arch=Arch.x86_64, minor_version=8), - Platform(os=OS.osx, arch=Arch.aarch64, minor_version=8), - Platform(os=OS.windows, arch=Arch.x86_64, minor_version=8), - }, - ) - - got = deps.build() - - self.assertEqual(["bar"], got.deps) - self.assertEqual( - { - "@platforms//os:linux": ["posix_dep"], - "@platforms//os:osx": ["an_osx_dep", "posix_dep"], - "@platforms//os:windows": ["win_dep"], - }, - got.deps_select, - ) - - def test_deps_are_added_to_more_specialized_platforms(self): - got = wheel.Deps( - "foo", - requires_dist=[ - "m1_dep; sys_platform=='darwin' and platform_machine=='arm64'", - "mac_dep; sys_platform=='darwin'", - ], - platforms={ - Platform(os=OS.osx, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.aarch64), - }, - ).build() - - self.assertEqual( - wheel.FrozenDeps( - deps=[], - deps_select={ - "osx_aarch64": ["m1_dep", "mac_dep"], - "@platforms//os:osx": ["mac_dep"], - }, - ), - got, - ) - - def test_deps_from_more_specialized_platforms_are_propagated(self): - got = wheel.Deps( - "foo", - requires_dist=[ - "a_mac_dep; sys_platform=='darwin'", - "m1_dep; sys_platform=='darwin' and platform_machine=='arm64'", - ], - platforms={ - Platform(os=OS.osx, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.aarch64), - }, - ).build() - - self.assertEqual([], got.deps) - self.assertEqual( - { - "osx_aarch64": ["a_mac_dep", "m1_dep"], - "@platforms//os:osx": ["a_mac_dep"], - }, - got.deps_select, - ) - - def test_non_platform_markers_are_added_to_common_deps(self): - got = wheel.Deps( - "foo", - requires_dist=[ - "bar", - "baz; implementation_name=='cpython'", - "m1_dep; sys_platform=='darwin' and platform_machine=='arm64'", - ], - platforms={ - Platform(os=OS.linux, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.aarch64), - Platform(os=OS.windows, arch=Arch.x86_64), - }, - ).build() - - self.assertEqual(["bar", "baz"], got.deps) - self.assertEqual( - { - "osx_aarch64": ["m1_dep"], - }, - got.deps_select, - ) - - def test_self_is_ignored(self): - deps = wheel.Deps( - "foo", - requires_dist=[ - "bar", - "req_dep; extra == 'requests'", - "foo[requests]; extra == 'ssl'", - "ssl_lib; extra == 'ssl'", - ], - extras={"ssl"}, - ) - - got = deps.build() - - self.assertEqual(["bar", "req_dep", "ssl_lib"], got.deps) - self.assertEqual({}, got.deps_select) - - def test_self_dependencies_can_come_in_any_order(self): - deps = wheel.Deps( - "foo", - requires_dist=[ - "bar", - "baz; extra == 'feat'", - "foo[feat2]; extra == 'all'", - "foo[feat]; extra == 'feat2'", - "zdep; extra == 'all'", - ], - extras={"all"}, - ) - - got = deps.build() - - self.assertEqual(["bar", "baz", "zdep"], got.deps) - self.assertEqual({}, got.deps_select) - - def test_can_get_deps_based_on_specific_python_version(self): - requires_dist = [ - "bar", - "baz; python_version < '3.8'", - "posix_dep; os_name=='posix' and python_version >= '3.8'", - ] - - py38_deps = wheel.Deps( - "foo", - requires_dist=requires_dist, - platforms=[ - Platform(os=OS.linux, arch=Arch.x86_64, minor_version=8), - ], - ).build() - py37_deps = wheel.Deps( - "foo", - requires_dist=requires_dist, - platforms=[ - Platform(os=OS.linux, arch=Arch.x86_64, minor_version=7), - ], - ).build() - - self.assertEqual(["bar", "baz"], py37_deps.deps) - self.assertEqual({}, py37_deps.deps_select) - self.assertEqual(["bar"], py38_deps.deps) - self.assertEqual({"@platforms//os:linux": ["posix_dep"]}, py38_deps.deps_select) - - @mock.patch(_HOST_INTERPRETER_FN) - def test_no_version_select_when_single_version(self, mock_host_interpreter_version): - requires_dist = [ - "bar", - "baz; python_version >= '3.8'", - "posix_dep; os_name=='posix'", - "posix_dep_with_version; os_name=='posix' and python_version >= '3.8'", - "arch_dep; platform_machine=='x86_64' and python_version >= '3.8'", - ] - mock_host_interpreter_version.return_value = 7 - - self.maxDiff = None - - deps = wheel.Deps( - "foo", - requires_dist=requires_dist, - platforms=[ - Platform(os=os, arch=Arch.x86_64, minor_version=minor) - for minor in [8] - for os in [OS.linux, OS.windows] - ], - ) - got = deps.build() - - self.assertEqual(["bar", "baz"], got.deps) - self.assertEqual( - { - "@platforms//os:linux": ["posix_dep", "posix_dep_with_version"], - "linux_x86_64": ["arch_dep", "posix_dep", "posix_dep_with_version"], - "windows_x86_64": ["arch_dep"], - }, - got.deps_select, - ) - - @mock.patch(_HOST_INTERPRETER_FN) - def test_can_get_version_select(self, mock_host_interpreter_version): - requires_dist = [ - "bar", - "baz; python_version < '3.8'", - "baz_new; python_version >= '3.8'", - "posix_dep; os_name=='posix'", - "posix_dep_with_version; os_name=='posix' and python_version >= '3.8'", - "arch_dep; platform_machine=='x86_64' and python_version < '3.8'", - ] - mock_host_interpreter_version.return_value = 7 - - self.maxDiff = None - - deps = wheel.Deps( - "foo", - requires_dist=requires_dist, - platforms=[ - Platform(os=os, arch=Arch.x86_64, minor_version=minor) - for minor in [7, 8, 9] - for os in [OS.linux, OS.windows] - ], - ) - got = deps.build() - - self.assertEqual(["bar"], got.deps) - self.assertEqual( - { - "//conditions:default": ["baz"], - "@//python/config_settings:is_python_3.7": ["baz"], - "@//python/config_settings:is_python_3.8": ["baz_new"], - "@//python/config_settings:is_python_3.9": ["baz_new"], - "@platforms//os:linux": ["baz", "posix_dep"], - "cp37_linux_x86_64": ["arch_dep", "baz", "posix_dep"], - "cp37_windows_x86_64": ["arch_dep", "baz"], - "cp37_linux_anyarch": ["baz", "posix_dep"], - "cp38_linux_anyarch": [ - "baz_new", - "posix_dep", - "posix_dep_with_version", - ], - "cp39_linux_anyarch": [ - "baz_new", - "posix_dep", - "posix_dep_with_version", - ], - "linux_x86_64": ["arch_dep", "baz", "posix_dep"], - "windows_x86_64": ["arch_dep", "baz"], - }, - got.deps_select, - ) - - @mock.patch(_HOST_INTERPRETER_FN) - def test_deps_spanning_all_target_py_versions_are_added_to_common( - self, mock_host_version - ): - requires_dist = [ - "bar", - "baz (<2,>=1.11) ; python_version < '3.8'", - "baz (<2,>=1.14) ; python_version >= '3.8'", - ] - mock_host_version.return_value = 8 - - deps = wheel.Deps( - "foo", - requires_dist=requires_dist, - platforms=Platform.from_string(["cp37_*", "cp38_*", "cp39_*"]), - ) - got = deps.build() - - self.assertEqual(["bar", "baz"], got.deps) - self.assertEqual({}, got.deps_select) - - @mock.patch(_HOST_INTERPRETER_FN) - def test_deps_are_not_duplicated(self, mock_host_version): - mock_host_version.return_value = 7 - - # See an example in - # https://files.pythonhosted.org/packages/76/9e/db1c2d56c04b97981c06663384f45f28950a73d9acf840c4006d60d0a1ff/opencv_python-4.9.0.80-cp37-abi3-win32.whl.metadata - requires_dist = [ - "bar >=0.1.0 ; python_version < '3.7'", - "bar >=0.2.0 ; python_version >= '3.7'", - "bar >=0.4.0 ; python_version >= '3.6' and platform_system == 'Linux' and platform_machine == 'aarch64'", - "bar >=0.4.0 ; python_version >= '3.9'", - "bar >=0.5.0 ; python_version <= '3.9' and platform_system == 'Darwin' and platform_machine == 'arm64'", - "bar >=0.5.0 ; python_version >= '3.10' and platform_system == 'Darwin'", - "bar >=0.5.0 ; python_version >= '3.10'", - "bar >=0.6.0 ; python_version >= '3.11'", - ] - - deps = wheel.Deps( - "foo", - requires_dist=requires_dist, - platforms=Platform.from_string(["cp37_*", "cp310_*"]), - ) - got = deps.build() - - self.assertEqual(["bar"], got.deps) - self.assertEqual({}, got.deps_select) - - @mock.patch(_HOST_INTERPRETER_FN) - def test_deps_are_not_duplicated_when_encountering_platform_dep_first( - self, mock_host_version - ): - mock_host_version.return_value = 7 - - # Note, that we are sorting the incoming `requires_dist` and we need to ensure that we are not getting any - # issues even if the platform-specific line comes first. - requires_dist = [ - "bar >=0.4.0 ; python_version >= '3.6' and platform_system == 'Linux' and platform_machine == 'aarch64'", - "bar >=0.5.0 ; python_version >= '3.9'", - ] - - deps = wheel.Deps( - "foo", - requires_dist=requires_dist, - platforms=Platform.from_string(["cp37_*", "cp310_*"]), - ) - got = deps.build() - - self.assertEqual(["bar"], got.deps) - self.assertEqual({}, got.deps_select) - - -if __name__ == "__main__": - unittest.main() diff --git a/tests/pypi/whl_library/BUILD.bazel b/tests/pypi/whl_library/BUILD.bazel new file mode 100644 index 0000000000..cade0d2b8e --- /dev/null +++ b/tests/pypi/whl_library/BUILD.bazel @@ -0,0 +1,11 @@ +load("//python:py_test.bzl", "py_test") +load("//tests/support:support.bzl", "SUPPORTS_BZLMOD") + +py_test( + name = "whl_library_extras_test", + srcs = ["whl_library_extras_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, + deps = [ + "@whl_library_extras_direct_dep//:pkg", + ], +) diff --git a/tests/pypi/whl_library/testdata/BUILD.bazel b/tests/pypi/whl_library/testdata/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/pypi/whl_library/testdata/optional_dep/BUILD.bazel b/tests/pypi/whl_library/testdata/optional_dep/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/pypi/whl_library/testdata/optional_dep/optional-dep-1.0.dist-info/METADATA b/tests/pypi/whl_library/testdata/optional_dep/optional-dep-1.0.dist-info/METADATA new file mode 100644 index 0000000000..6495d1ba36 --- /dev/null +++ b/tests/pypi/whl_library/testdata/optional_dep/optional-dep-1.0.dist-info/METADATA @@ -0,0 +1,2 @@ +Name: optional-dep +Version: 1.0 diff --git a/tests/pypi/whl_library/testdata/optional_dep/optional-dep-1.0.dist-info/RECORD b/tests/pypi/whl_library/testdata/optional_dep/optional-dep-1.0.dist-info/RECORD new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/pypi/whl_library/testdata/optional_dep/optional-dep-1.0.dist-info/WHEEL b/tests/pypi/whl_library/testdata/optional_dep/optional-dep-1.0.dist-info/WHEEL new file mode 100644 index 0000000000..a64521a1cc --- /dev/null +++ b/tests/pypi/whl_library/testdata/optional_dep/optional-dep-1.0.dist-info/WHEEL @@ -0,0 +1 @@ +Wheel-Version: 1.0 diff --git a/tests/pypi/whl_library/testdata/optional_dep/optional_dep.py b/tests/pypi/whl_library/testdata/optional_dep/optional_dep.py new file mode 100644 index 0000000000..4af2718944 --- /dev/null +++ b/tests/pypi/whl_library/testdata/optional_dep/optional_dep.py @@ -0,0 +1 @@ +I_AM_OPTIONAL = True diff --git a/tests/pypi/whl_library/testdata/packages.bzl b/tests/pypi/whl_library/testdata/packages.bzl new file mode 100644 index 0000000000..e4a9b0af4c --- /dev/null +++ b/tests/pypi/whl_library/testdata/packages.bzl @@ -0,0 +1,6 @@ +"""A list of packages that this logical testdata hub repo contains.""" + +packages = [ + "optional_dep", + "pkg", +] diff --git a/tests/pypi/whl_library/testdata/pkg/BUILD.bazel b/tests/pypi/whl_library/testdata/pkg/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/pypi/whl_library/testdata/pkg/pkg-1.0.dist-info/METADATA b/tests/pypi/whl_library/testdata/pkg/pkg-1.0.dist-info/METADATA new file mode 100644 index 0000000000..712b44edbf --- /dev/null +++ b/tests/pypi/whl_library/testdata/pkg/pkg-1.0.dist-info/METADATA @@ -0,0 +1,4 @@ +Name: pkg +Version: 1.0 +Requires-Dist: optional_dep; extra == "optional" +Provides-Extra: optional diff --git a/tests/pypi/whl_library/testdata/pkg/pkg-1.0.dist-info/RECORD b/tests/pypi/whl_library/testdata/pkg/pkg-1.0.dist-info/RECORD new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/pypi/whl_library/testdata/pkg/pkg-1.0.dist-info/WHEEL b/tests/pypi/whl_library/testdata/pkg/pkg-1.0.dist-info/WHEEL new file mode 100644 index 0000000000..a64521a1cc --- /dev/null +++ b/tests/pypi/whl_library/testdata/pkg/pkg-1.0.dist-info/WHEEL @@ -0,0 +1 @@ +Wheel-Version: 1.0 diff --git a/tests/pypi/whl_library/testdata/pkg/pkg.py b/tests/pypi/whl_library/testdata/pkg/pkg.py new file mode 100644 index 0000000000..c5aca099c4 --- /dev/null +++ b/tests/pypi/whl_library/testdata/pkg/pkg.py @@ -0,0 +1,6 @@ +try: + import optional_dep + + WITH_EXTRAS = True +except ImportError: + WITH_EXTRAS = False diff --git a/tests/pypi/whl_library/whl_library_extras_test.py b/tests/pypi/whl_library/whl_library_extras_test.py new file mode 100644 index 0000000000..43cd5aec3f --- /dev/null +++ b/tests/pypi/whl_library/whl_library_extras_test.py @@ -0,0 +1,12 @@ +import unittest + + +class NamespacePackagesTest(unittest.TestCase): + def test_extras_propagated(self): + import pkg + + self.assertEqual(pkg.WITH_EXTRAS, True) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index e69eb0f0e9..d752159b32 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -15,29 +15,37 @@ "" load("@rules_testing//lib:test_suite.bzl", "test_suite") -load("//python/private:glob_excludes.bzl", "glob_excludes") # buildifier: disable=bzl-visibility -load("//python/private/pypi:whl_library_targets.bzl", "whl_library_targets") # buildifier: disable=bzl-visibility +load( + "//python/private/pypi:whl_library_targets.bzl", + "whl_library_deps_targets", + "whl_library_srcs", +) # buildifier: disable=bzl-visibility +load("//tests/support/mocks:mocks.bzl", "mocks") _tests = [] def _test_filegroups(env): calls = [] - def glob(match, *, allow_empty): + def glob(include, *, exclude = [], allow_empty): + _ = exclude # @unused env.expect.that_bool(allow_empty).equals(True) - return match + if include == ["rewrite-bin/*"] or include == ["bin/*"]: + return [] + return include - whl_library_targets( + whl_library_srcs( name = "", - dep_template = "", native = struct( filegroup = lambda **kwargs: calls.append(kwargs), glob = glob, ), - rules = struct(), + rules = struct( + venv_rewrite_shebang = lambda **kwargs: None, + ), ) - env.expect.that_collection(calls).contains_exactly([ + env.expect.that_collection(calls, expr = "filegroup calls").contains_exactly([ { "name": "dist_info", "srcs": ["site-packages/*.dist-info/**"], @@ -45,94 +53,37 @@ def _test_filegroups(env): }, { "name": "data", - "srcs": ["data/**"], + "srcs": ["data/**", "bin/**", "include/**"], "visibility": ["//visibility:public"], }, { - "name": "whl", - "srcs": [""], - "data": [], + "name": "extracted_whl_files", + "srcs": ["**"], "visibility": ["//visibility:public"], }, - ]) # buildifier: @unsorted-dict-items - -_tests.append(_test_filegroups) - -def _test_platforms(env): - calls = [] - - whl_library_targets( - name = "", - dep_template = None, - dependencies_by_platform = { - "@//python/config_settings:is_python_3.9": ["py39_dep"], - "@platforms//cpu:aarch64": ["arm_dep"], - "@platforms//os:windows": ["win_dep"], - "cp310_linux_ppc": ["py310_linux_ppc_dep"], - "cp39_anyos_aarch64": ["py39_arm_dep"], - "cp39_linux_anyarch": ["py39_linux_dep"], - "linux_x86_64": ["linux_intel_dep"], - }, - filegroups = {}, - native = struct( - config_setting = lambda **kwargs: calls.append(kwargs), - ), - rules = struct(), - ) - - env.expect.that_collection(calls).contains_exactly([ { - "name": "is_python_3.10_linux_ppc", - "flag_values": { - "@rules_python//python/config_settings:python_version_major_minor": "3.10", - }, - "constraint_values": [ - "@platforms//cpu:ppc", - "@platforms//os:linux", - ], - "visibility": ["//visibility:private"], - }, - { - "name": "is_python_3.9_anyos_aarch64", - "flag_values": { - "@rules_python//python/config_settings:python_version_major_minor": "3.9", - }, - "constraint_values": ["@platforms//cpu:aarch64"], - "visibility": ["//visibility:private"], - }, - { - "name": "is_python_3.9_linux_anyarch", - "flag_values": { - "@rules_python//python/config_settings:python_version_major_minor": "3.9", - }, - "constraint_values": ["@platforms//os:linux"], - "visibility": ["//visibility:private"], - }, - { - "name": "is_linux_x86_64", - "constraint_values": [ - "@platforms//cpu:x86_64", - "@platforms//os:linux", - ], - "visibility": ["//visibility:private"], + "name": "whl_file", + "srcs": [""], + "visibility": ["//visibility:public"], }, ]) # buildifier: @unsorted-dict-items -_tests.append(_test_platforms) +_tests.append(_test_filegroups) def _test_copy(env): calls = [] - whl_library_targets( + whl_library_srcs( name = "", - dep_template = None, - dependencies_by_platform = {}, filegroups = {}, copy_files = {"file_src": "file_dest"}, copy_executables = {"exec_src": "exec_dest"}, - native = struct(), + native = struct( + glob = lambda *args, **kwargs: [], + ), rules = struct( copy_file = lambda **kwargs: calls.append(kwargs), + venv_rewrite_shebang = lambda **kwargs: None, ), ) @@ -154,202 +105,231 @@ def _test_copy(env): _tests.append(_test_copy) -def _test_entrypoints(env): - calls = [] - - whl_library_targets( - name = "", - dep_template = None, - dependencies_by_platform = {}, - filegroups = {}, - entry_points = { - "fizz": "buzz.py", - }, - native = struct(), +def _test_whl_library_deps_targets(env): + filegroup_calls = [] + py_library_calls = [] + env_marker_setting_calls = [] + + m_glob = mocks.glob() + + m_glob.results.append([]) # bin + m_glob.results.append([]) # rewrite-bin + m_glob.results.append(["site-packages/foo/SRCS.py"]) # srcs + m_glob.results.append(["site-packages/foo/DATA.txt"]) # data + m_glob.results.append(["site-packages/foo/PYI.pyi"]) # pyi + + whl_library_deps_targets( + name = "foo-0-py3-none-any.whl", + metadata_name = "Foo", + dep_template = "@pypi//{name}:{target}", + requires_dist = [ + "foo", # this self-edge will be ignored + "bar", + "bar-baz; python_version < \"8.2\"", + "booo", # this is effectively excluded due to the list below + ], + include = ["foo", "bar", "bar_baz"], + # Overrides for testing + repo = None, + aliases = None, + extras = [], + native = struct( + filegroup = lambda **kwargs: filegroup_calls.append(kwargs), + alias = lambda **kwargs: None, + config_setting = lambda **_: None, + glob = m_glob.glob, + ), rules = struct( - py_binary = lambda **kwargs: calls.append(kwargs), + py_library = lambda **kwargs: py_library_calls.append(kwargs), + env_marker_setting = lambda **kwargs: env_marker_setting_calls.append(kwargs), + create_inits = lambda *args, **kwargs: ["_create_inits_target"], + venv_rewrite_shebang = lambda **kwargs: None, ), ) - env.expect.that_collection(calls).contains_exactly([ + env.expect.that_collection(filegroup_calls).contains_exactly([ { - "name": "rules_python_wheel_entry_point_fizz", - "srcs": ["buzz.py"], - "deps": [":pkg"], - "imports": ["."], + "name": "whl", + # NOTE @aignas 2026-07-25: depending on the brackets position one may get different + # results in the expectation. + "srcs": ["whl_file"], + "data": ["@pypi//bar:whl"] + select({ + ":is_include_bar_baz_true": ["@pypi//bar_baz:whl"], + "//conditions:default": [], + }), "visibility": ["//visibility:public"], }, ]) # buildifier: @unsorted-dict-items -_tests.append(_test_entrypoints) + env.expect.that_collection(py_library_calls).has_size(1) + if len(py_library_calls) != 1: + return + py_library_call = py_library_calls[0] + + env.expect.that_dict(py_library_call).contains_exactly({ + "name": "pkg", + "srcs": ["srcs"], + "deps": ["srcs", "@pypi//bar:pkg"] + select({ + ":is_include_bar_baz_true": ["@pypi//bar_baz:pkg"], + "//conditions:default": [], + }), + "tags": [], + "visibility": ["//visibility:public"], + }) # buildifier: @unsorted-dict-items + + env.expect.that_collection(m_glob.calls).contains_exactly([]) + + env.expect.that_collection(env_marker_setting_calls).contains_exactly([ + { + "name": "include_bar_baz", + "expression": "python_version < \"8.2\"", + "visibility": ["//visibility:private"], + }, + ]) # buildifier: @unsorted-dict-items + +_tests.append(_test_whl_library_deps_targets) -def _test_whl_and_library_deps(env): +def _test_whl_library_deps_targets_no_deps(env): + alias_calls = [] filegroup_calls = [] py_library_calls = [] + env_marker_setting_calls = [] - whl_library_targets( - name = "foo.whl", - dep_template = "@pypi_{name}//:{target}", - dependencies = ["foo", "bar-baz"], - dependencies_by_platform = { - "@//python/config_settings:is_python_3.9": ["py39_dep"], - "@platforms//cpu:aarch64": ["arm_dep"], - "@platforms//os:windows": ["win_dep"], - "cp310_linux_ppc": ["py310_linux_ppc_dep"], - "cp39_anyos_aarch64": ["py39_arm_dep"], - "cp39_linux_anyarch": ["py39_linux_dep"], - "linux_x86_64": ["linux_intel_dep"], - }, - data_exclude = [], - tags = ["tag1", "tag2"], - # Overrides for testing - filegroups = {}, + whl_library_deps_targets( + name = "foo-0-py3-none-any.whl", + metadata_name = "Foo", + dep_template = "@pypi//{name}:{target}", + requires_dist = [], + group_name = "qux", + repo = None, + aliases = {}, + extras = [], native = struct( filegroup = lambda **kwargs: filegroup_calls.append(kwargs), + alias = lambda **kwargs: alias_calls.append(kwargs), config_setting = lambda **_: None, - glob = _glob, - select = _select, + glob = lambda **_: [], ), rules = struct( py_library = lambda **kwargs: py_library_calls.append(kwargs), + env_marker_setting = lambda **kwargs: env_marker_setting_calls.append(kwargs), ), ) - env.expect.that_collection(filegroup_calls).contains_exactly([ + # If the package is in a group but has no deps, then the public labels should be aliases + # to the srcs targets and no other targets should be created. + env.expect.that_collection(alias_calls).contains_exactly([ { - "name": "whl", - "srcs": ["foo.whl"], - "data": [ - "@pypi_bar_baz//:whl", - "@pypi_foo//:whl", - ] + _select( - { - Label("//python/config_settings:is_python_3.9"): ["@pypi_py39_dep//:whl"], - "@platforms//cpu:aarch64": ["@pypi_arm_dep//:whl"], - "@platforms//os:windows": ["@pypi_win_dep//:whl"], - ":is_python_3.10_linux_ppc": ["@pypi_py310_linux_ppc_dep//:whl"], - ":is_python_3.9_anyos_aarch64": ["@pypi_py39_arm_dep//:whl"], - ":is_python_3.9_linux_anyarch": ["@pypi_py39_linux_dep//:whl"], - ":is_linux_x86_64": ["@pypi_linux_intel_dep//:whl"], - "//conditions:default": [], - }, - ), + "name": "pkg", + "actual": "srcs", "visibility": ["//visibility:public"], }, - ]) # buildifier: @unsorted-dict-items - env.expect.that_collection(py_library_calls).contains_exactly([ { - "name": "pkg", - "srcs": _glob( - ["site-packages/**/*.py"], - exclude = [], - allow_empty = True, - ), - "data": [] + _glob( - ["site-packages/**/*"], - exclude = [ - "**/*.py", - "**/*.pyc", - "**/*.pyc.*", - "**/*.dist-info/RECORD", - ] + glob_excludes.version_dependent_exclusions(), - ), - "imports": ["site-packages"], - "deps": [ - "@pypi_bar_baz//:pkg", - "@pypi_foo//:pkg", - ] + _select( - { - Label("//python/config_settings:is_python_3.9"): ["@pypi_py39_dep//:pkg"], - "@platforms//cpu:aarch64": ["@pypi_arm_dep//:pkg"], - "@platforms//os:windows": ["@pypi_win_dep//:pkg"], - ":is_python_3.10_linux_ppc": ["@pypi_py310_linux_ppc_dep//:pkg"], - ":is_python_3.9_anyos_aarch64": ["@pypi_py39_arm_dep//:pkg"], - ":is_python_3.9_linux_anyarch": ["@pypi_py39_linux_dep//:pkg"], - ":is_linux_x86_64": ["@pypi_linux_intel_dep//:pkg"], - "//conditions:default": [], - }, - ), - "tags": ["tag1", "tag2"], + "name": "whl", + "actual": "whl_file", "visibility": ["//visibility:public"], }, ]) # buildifier: @unsorted-dict-items -_tests.append(_test_whl_and_library_deps) + env.expect.that_collection(filegroup_calls).contains_exactly([]) + env.expect.that_collection(py_library_calls).contains_exactly([]) + env.expect.that_collection(env_marker_setting_calls).contains_exactly([]) -def _test_group(env): - alias_calls = [] - py_library_calls = [] +_tests.append(_test_whl_library_deps_targets_no_deps) - whl_library_targets( +def _test_sdist_excludes_record(env): + py_library_calls = [] + m_glob = mocks.glob() + m_glob.results.append([]) # bin + m_glob.results.append([]) # rewrite-bin + m_glob.results.append([]) # srcs + m_glob.results.append([]) # data + m_glob.results.append([]) # pyi + + whl_library_srcs( name = "foo.whl", - dep_template = "@pypi_{name}//:{target}", - dependencies = ["foo", "bar-baz", "qux"], - dependencies_by_platform = { - "linux_x86_64": ["box", "box-amd64"], - "windows_x86_64": ["fox"], - "@platforms//os:linux": ["box"], # buildifier: disable=unsorted-dict-items to check that we sort inside the test - }, - tags = [], - entry_points = {}, - data_exclude = [], - group_name = "qux", - group_deps = ["foo", "fox", "qux"], - # Overrides for testing + sdist_filename = "foo.tar.gz", filegroups = {}, native = struct( + filegroup = lambda **_: None, config_setting = lambda **_: None, - glob = _glob, - alias = lambda **kwargs: alias_calls.append(kwargs), - select = _select, + glob = m_glob.glob, ), rules = struct( py_library = lambda **kwargs: py_library_calls.append(kwargs), + create_inits = lambda **kwargs: [], + venv_rewrite_shebang = lambda **kwargs: None, ), ) - env.expect.that_collection(alias_calls).contains_exactly([ - {"name": "pkg", "actual": "@pypi__groups//:qux_pkg", "visibility": ["//visibility:public"]}, - {"name": "whl", "actual": "@pypi__groups//:qux_whl", "visibility": ["//visibility:public"]}, - ]) # buildifier: @unsorted-dict-items - env.expect.that_collection(py_library_calls).contains_exactly([ - { - "name": "_pkg", - "srcs": _glob(["site-packages/**/*.py"], exclude = [], allow_empty = True), - "data": [] + _glob( - ["site-packages/**/*"], - exclude = [ - "**/*.py", - "**/*.pyc", - "**/*.pyc.*", - "**/*.dist-info/RECORD", - ] + glob_excludes.version_dependent_exclusions(), - ), - "imports": ["site-packages"], - "deps": ["@pypi_bar_baz//:pkg"] + _select({ - "@platforms//os:linux": ["@pypi_box//:pkg"], - ":is_linux_x86_64": ["@pypi_box//:pkg", "@pypi_box_amd64//:pkg"], - "//conditions:default": [], - }), - "tags": [], - "visibility": ["@pypi__groups//:__pkg__"], - }, - ]) # buildifier: @unsorted-dict-items - -_tests.append(_test_group) + env.expect.that_collection(m_glob.calls).contains_at_least([ + mocks.glob_call( + ["site-packages/**/*"], + exclude = [ + "**/*.py", + "**/*.pyc", + "**/*.pyc.*", + "**/*.dist-info/RECORD", + ], + allow_empty = True, + ), + ]) -def _glob(*args, **kwargs): - return [struct( - glob = args, - kwargs = kwargs, - )] +_tests.append(_test_sdist_excludes_record) + +def _test_exclude_bazel_files(env): + # Regression test: the `extracted_whl_files` glob must always exclude the + # Bazel repo files, even when the wheel is not built from an sdist. + for sdist_filename in [None, "foo.tar.gz"]: + m_glob = mocks.glob() + m_glob.results.append([]) # bin + m_glob.results.append([]) # rewrite-bin + m_glob.results.append([]) # extracted_whl_files + m_glob.results.append([]) # dist_info + m_glob.results.append([]) # data + + whl_library_srcs( + name = "foo.whl", + sdist_filename = sdist_filename, + native = struct( + filegroup = lambda **_: None, + glob = m_glob.glob, + ), + rules = struct( + venv_rewrite_shebang = lambda **kwargs: None, + ), + ) + + expected_exclude = [ + "BUILD", + "BUILD.bazel", + "REPO.bazel", + "WORKSPACE", + "WORKSPACE.bzlmod", + "WORKSPACE.bazel", + ] + if sdist_filename: + expected_exclude.append(sdist_filename) + + env.expect.that_collection(m_glob.calls).contains_exactly([ + mocks.glob_call(["bin/*"], allow_empty = True), + mocks.glob_call(["rewrite-bin/*"], allow_empty = True), + mocks.glob_call( + include = ["**"], + exclude = expected_exclude, + allow_empty = True, + ), + mocks.glob_call( + include = ["site-packages/*.dist-info/**"], + allow_empty = True, + ), + mocks.glob_call( + include = ["data/**", "bin/**", "include/**"], + allow_empty = True, + ), + ]) -def _select(*args, **kwargs): - """We need to have this mock select because we still need to support bazel 6.""" - return [struct( - select = args, - kwargs = kwargs, - )] +_tests.append(_test_exclude_bazel_files) def whl_library_targets_test_suite(name): """create the test suite. diff --git a/tests/pypi/whl_metadata/BUILD.bazel b/tests/pypi/whl_metadata/BUILD.bazel new file mode 100644 index 0000000000..3f1d665dd2 --- /dev/null +++ b/tests/pypi/whl_metadata/BUILD.bazel @@ -0,0 +1,5 @@ +load(":whl_metadata_tests.bzl", "whl_metadata_test_suite") + +whl_metadata_test_suite( + name = "whl_metadata_tests", +) diff --git a/tests/pypi/whl_metadata/whl_metadata_tests.bzl b/tests/pypi/whl_metadata/whl_metadata_tests.bzl new file mode 100644 index 0000000000..8131b0f452 --- /dev/null +++ b/tests/pypi/whl_metadata/whl_metadata_tests.bzl @@ -0,0 +1,253 @@ +"" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("@rules_testing//lib:truth.bzl", "subjects") +load( + "//python/private/pypi:whl_metadata.bzl", + "find_whl_metadata", + "parse_entry_points", + "parse_whl_metadata", +) # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_empty(env): + fake_path = struct( + basename = "site-packages", + readdir = lambda watch = None: [], + ) + fail_messages = [] + find_whl_metadata(install_dir = fake_path, logger = struct( + fail = fail_messages.append, + )) + env.expect.that_collection(fail_messages).contains_exactly([ + "The '*.dist-info' directory could not be found in 'site-packages'", + ]) + +_tests.append(_test_empty) + +def _test_contains_dist_info_but_no_metadata(env): + fake_path = struct( + basename = "site-packages", + readdir = lambda watch = None: [ + struct( + basename = "something.dist-info", + is_dir = True, + get_child = lambda basename: struct( + basename = basename, + exists = False, + ), + ), + ], + ) + fail_messages = [] + find_whl_metadata(install_dir = fake_path, logger = struct( + fail = fail_messages.append, + )) + env.expect.that_collection(fail_messages).contains_exactly([ + "The METADATA file for the wheel could not be found in 'site-packages/something.dist-info'", + ]) + +_tests.append(_test_contains_dist_info_but_no_metadata) + +def _test_contains_metadata(env): + fake_path = struct( + basename = "site-packages", + readdir = lambda watch = None: [ + struct( + basename = "something.dist-info", + is_dir = True, + get_child = lambda basename: struct( + basename = basename, + exists = True, + ), + ), + ], + ) + fail_messages = [] + got = find_whl_metadata(install_dir = fake_path, logger = struct( + fail = fail_messages.append, + )) + env.expect.that_collection(fail_messages).contains_exactly([]) + env.expect.that_str(got.basename).equals("METADATA") + +_tests.append(_test_contains_metadata) + +def _parse_whl_metadata(env, **kwargs): + result = parse_whl_metadata(**kwargs) + + return env.expect.that_struct( + struct( + name = result.name, + version = result.version, + requires_dist = result.requires_dist, + provides_extra = result.provides_extra, + ), + attrs = dict( + name = subjects.str, + version = subjects.str, + requires_dist = subjects.collection, + provides_extra = subjects.collection, + ), + ) + +def _test_parse_metadata_invalid(env): + got = _parse_whl_metadata( + env, + contents = "", + ) + got.name().equals("") + got.version().equals("") + got.requires_dist().contains_exactly([]) + got.provides_extra().contains_exactly([]) + +_tests.append(_test_parse_metadata_invalid) + +def _test_parse_metadata_basic(env): + got = _parse_whl_metadata( + env, + contents = """\ +Name: foo +Version: 0.0.1 +""", + ) + got.name().equals("foo") + got.version().equals("0.0.1") + got.requires_dist().contains_exactly([]) + got.provides_extra().contains_exactly([]) + +_tests.append(_test_parse_metadata_basic) + +def _test_parse_metadata_all(env): + got = _parse_whl_metadata( + env, + contents = """\ +Name: foo +Version: 0.0.1 +Requires-Dist: bar; extra == "all" +Provides-Extra: all + +Requires-Dist: this will be ignored +""", + ) + got.name().equals("foo") + got.version().equals("0.0.1") + got.requires_dist().contains_exactly([ + "bar; extra == \"all\"", + ]) + got.provides_extra().contains_exactly([ + "all", + ]) + +_tests.append(_test_parse_metadata_all) + +def _test_parse_metadata_multiline_license(env): + got = _parse_whl_metadata( + env, + # NOTE: The trailing whitespace here is meaningful as an empty line + # denotes the end of the header. + contents = """\ +Name: foo +Version: 0.0.1 +License: some License + + some line + + another line + +Requires-Dist: bar; extra == "all" +Provides-Extra: all + +Requires-Dist: this will be ignored +""", + ) + got.name().equals("foo") + got.version().equals("0.0.1") + got.requires_dist().contains_exactly([ + "bar; extra == \"all\"", + ]) + got.provides_extra().contains_exactly([ + "all", + ]) + +_tests.append(_test_parse_metadata_multiline_license) + +def _test_parse_entry_points(env): + got = parse_entry_points("""\ +[something] +interesting # with comments + +[console_scripts] +foo = foomod:main +# One which depends on extras: +foobar = importable.foomod:main_bar [bar, baz] + + # With a comment at the end +foobarbaz = foomod:main.attr # comment + +# With extra and comment +foo_extra_comment = foomod:main [extra] # comment + +[something else] +not very much interesting +""") + env.expect.that_dict(got).contains_exactly({ + "foo": { + "attribute": "main", + "extras": "", + "group": "console_scripts", + "module": "foomod", + "name": "foo", + }, + "foo_extra_comment": { + "attribute": "main", + "extras": "extra", + "group": "console_scripts", + "module": "foomod", + "name": "foo_extra_comment", + }, + "foobar": { + "attribute": "main_bar", + "extras": "bar, baz", + "group": "console_scripts", + "module": "importable.foomod", + "name": "foobar", + }, + "foobarbaz": { + "attribute": "main.attr", + "extras": "", + "group": "console_scripts", + "module": "foomod", + "name": "foobarbaz", + }, + }) + +_tests.append(_test_parse_entry_points) + +def _test_parse_entry_points_deduplicate(env): + got = parse_entry_points("""\ +[console_scripts] +FooBar = foomod:main +foobar = othermod:main +fooBAR = another:main + +[gui_scripts] +FOOBAR = guimod:main +""") + env.expect.that_dict(got).contains_exactly({ + "FooBar": { + "attribute": "main", + "extras": "", + "group": "console_scripts", + "module": "foomod", + "name": "FooBar", + }, + }) + +_tests.append(_test_parse_entry_points_deduplicate) + +def whl_metadata_test_suite(name): # buildifier: disable=function-docstring + test_suite( + name = name, + basic_tests = _tests, + ) diff --git a/tests/pypi/whl_repo_name/whl_repo_name_tests.bzl b/tests/pypi/whl_repo_name/whl_repo_name_tests.bzl index 8b7df83530..968bdc3d2a 100644 --- a/tests/pypi/whl_repo_name/whl_repo_name_tests.bzl +++ b/tests/pypi/whl_repo_name/whl_repo_name_tests.bzl @@ -20,29 +20,64 @@ load("//python/private/pypi:whl_repo_name.bzl", "whl_repo_name") # buildifier: _tests = [] def _test_simple(env): - got = whl_repo_name("prefix", "foo-1.2.3-py3-none-any.whl", "deadbeef") - env.expect.that_str(got).equals("prefix_foo_py3_none_any_deadbeef") + got = whl_repo_name("foo-1.2.3-py3-none-any.whl", "deadbeef") + env.expect.that_str(got).equals("foo_py3_none_any_deadbeef") _tests.append(_test_simple) +def _test_simple_canonical_digest(env): + got = whl_repo_name("foo-1.2.3-py3-none-any.whl", "sha256:deadbeef") + env.expect.that_str(got).equals("foo_py3_none_any_deadbeef") + +_tests.append(_test_simple_canonical_digest) + +def _test_simple_canonical_digest_other_algo(env): + got = whl_repo_name("foo-1.2.3-py3-none-any.whl", "sha512:deadbeef000deadbeef") + env.expect.that_str(got).equals("foo_py3_none_any_deadbeef") + +_tests.append(_test_simple_canonical_digest_other_algo) + +def _test_simple_no_sha(env): + got = whl_repo_name("foo-1.2.3-py3-none-any.whl", "") + env.expect.that_str(got).equals("foo_1_2_3_py3_none_any") + +_tests.append(_test_simple_no_sha) + def _test_sdist(env): - got = whl_repo_name("prefix", "foo-1.2.3.tar.gz", "deadbeef000deadbeef") - env.expect.that_str(got).equals("prefix_foo_sdist_deadbeef") + got = whl_repo_name("foo-1.2.3.tar.gz", "deadbeef000deadbeef") + env.expect.that_str(got).equals("foo_sdist_deadbeef") _tests.append(_test_sdist) +def _test_sdist_no_sha(env): + got = whl_repo_name("foo-1.2.3.tar.gz", "") + env.expect.that_str(got).equals("foo_1_2_3") + +_tests.append(_test_sdist_no_sha) + def _test_platform_whl(env): got = whl_repo_name( - "prefix", "foo-1.2.3-cp39.cp310-abi3-manylinux1_x86_64.manylinux_2_17_x86_64.whl", "deadbeef000deadbeef", ) # We only need the first segment of each - env.expect.that_str(got).equals("prefix_foo_cp39_abi3_manylinux_2_5_x86_64_deadbeef") + env.expect.that_str(got).equals("foo_cp39_abi3_manylinux_2_5_x86_64_deadbeef") _tests.append(_test_platform_whl) +def _test_name_with_plus(env): + got = whl_repo_name("gptqmodel-2.0.0+cu126torch2.6-cp312-cp312-linux_x86_64.whl", "") + env.expect.that_str(got).equals("gptqmodel_2_0_0_cu126torch2_6_cp312_cp312_linux_x86_64") + +_tests.append(_test_name_with_plus) + +def _test_name_with_percent(env): + got = whl_repo_name("gptqmodel-2.0.0%2Bcu126torch2.6-cp312-cp312-linux_x86_64.whl", "") + env.expect.that_str(got).equals("gptqmodel_2_0_0_2Bcu126torch2_6_cp312_cp312_linux_x86_64") + +_tests.append(_test_name_with_percent) + def whl_repo_name_test_suite(name): """Create the test suite. diff --git a/tests/pypi/whl_target_platforms/BUILD.bazel b/tests/pypi/whl_target_platforms/BUILD.bazel index 6c35b08d32..fec25af033 100644 --- a/tests/pypi/whl_target_platforms/BUILD.bazel +++ b/tests/pypi/whl_target_platforms/BUILD.bazel @@ -12,9 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. -load(":select_whl_tests.bzl", "select_whl_test_suite") load(":whl_target_platforms_tests.bzl", "whl_target_platforms_test_suite") -select_whl_test_suite(name = "select_whl_tests") - whl_target_platforms_test_suite(name = "whl_target_platforms_tests") diff --git a/tests/pypi/whl_target_platforms/select_whl_tests.bzl b/tests/pypi/whl_target_platforms/select_whl_tests.bzl deleted file mode 100644 index 2994bd513f..0000000000 --- a/tests/pypi/whl_target_platforms/select_whl_tests.bzl +++ /dev/null @@ -1,278 +0,0 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"" - -load("@rules_testing//lib:test_suite.bzl", "test_suite") -load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "REPO_VERBOSITY_ENV_VAR", "repo_utils") # buildifier: disable=bzl-visibility -load("//python/private/pypi:whl_target_platforms.bzl", "select_whls") # buildifier: disable=bzl-visibility - -WHL_LIST = [ - "pkg-0.0.1-cp311-cp311-macosx_10_9_universal2.whl", - "pkg-0.0.1-cp311-cp311-macosx_10_9_x86_64.whl", - "pkg-0.0.1-cp311-cp311-macosx_11_0_arm64.whl", - "pkg-0.0.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - "pkg-0.0.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", - "pkg-0.0.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", - "pkg-0.0.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "pkg-0.0.1-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", - "pkg-0.0.1-cp311-cp311-musllinux_1_1_aarch64.whl", - "pkg-0.0.1-cp311-cp311-musllinux_1_1_i686.whl", - "pkg-0.0.1-cp311-cp311-musllinux_1_1_ppc64le.whl", - "pkg-0.0.1-cp311-cp311-musllinux_1_1_s390x.whl", - "pkg-0.0.1-cp311-cp311-musllinux_1_1_x86_64.whl", - "pkg-0.0.1-cp311-cp311-win32.whl", - "pkg-0.0.1-cp311-cp311-win_amd64.whl", - "pkg-0.0.1-cp37-cp37m-macosx_10_9_x86_64.whl", - "pkg-0.0.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - "pkg-0.0.1-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", - "pkg-0.0.1-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl", - "pkg-0.0.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "pkg-0.0.1-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", - "pkg-0.0.1-cp37-cp37m-musllinux_1_1_aarch64.whl", - "pkg-0.0.1-cp37-cp37m-musllinux_1_1_i686.whl", - "pkg-0.0.1-cp37-cp37m-musllinux_1_1_ppc64le.whl", - "pkg-0.0.1-cp37-cp37m-musllinux_1_1_s390x.whl", - "pkg-0.0.1-cp37-cp37m-musllinux_1_1_x86_64.whl", - "pkg-0.0.1-cp37-cp37m-win32.whl", - "pkg-0.0.1-cp37-cp37m-win_amd64.whl", - "pkg-0.0.1-cp39-cp39-macosx_10_9_universal2.whl", - "pkg-0.0.1-cp39-cp39-macosx_10_9_x86_64.whl", - "pkg-0.0.1-cp39-cp39-macosx_11_0_arm64.whl", - "pkg-0.0.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - "pkg-0.0.1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", - "pkg-0.0.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", - "pkg-0.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "pkg-0.0.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", - "pkg-0.0.1-cp39-cp39-musllinux_1_1_aarch64.whl", - "pkg-0.0.1-cp39-cp39-musllinux_1_1_i686.whl", - "pkg-0.0.1-cp39-cp39-musllinux_1_1_ppc64le.whl", - "pkg-0.0.1-cp39-cp39-musllinux_1_1_s390x.whl", - "pkg-0.0.1-cp39-cp39-musllinux_1_1_x86_64.whl", - "pkg-0.0.1-cp39-cp39-win32.whl", - "pkg-0.0.1-cp39-cp39-win_amd64.whl", - "pkg-0.0.1-cp39-abi3-any.whl", - "pkg-0.0.1-py310-abi3-any.whl", - "pkg-0.0.1-py3-abi3-any.whl", - "pkg-0.0.1-py3-none-any.whl", -] - -def _match(env, got, *want_filenames): - if not want_filenames: - env.expect.that_collection(got).has_size(len(want_filenames)) - return - - got_filenames = [g.filename for g in got] - env.expect.that_collection(got_filenames).contains_exactly(want_filenames) - - if got: - # Check that we pass the original structs - env.expect.that_str(got[0].other).equals("dummy") - -def _select_whls(whls, debug = False, **kwargs): - return select_whls( - whls = [ - struct( - filename = f, - other = "dummy", - ) - for f in whls - ], - logger = repo_utils.logger(struct( - os = struct( - environ = { - REPO_DEBUG_ENV_VAR: "1", - REPO_VERBOSITY_ENV_VAR: "TRACE" if debug else "INFO", - }, - ), - ), "unit-test"), - **kwargs - ) - -_tests = [] - -def _test_simplest(env): - got = _select_whls( - whls = [ - "pkg-0.0.1-py2.py3-abi3-any.whl", - "pkg-0.0.1-py3-abi3-any.whl", - "pkg-0.0.1-py3-none-any.whl", - ], - want_platforms = ["cp30_ignored"], - ) - _match( - env, - got, - "pkg-0.0.1-py3-abi3-any.whl", - "pkg-0.0.1-py3-none-any.whl", - ) - -_tests.append(_test_simplest) - -def _test_select_by_supported_py_version(env): - for minor_version, match in { - 8: "pkg-0.0.1-py3-abi3-any.whl", - 11: "pkg-0.0.1-py311-abi3-any.whl", - }.items(): - got = _select_whls( - whls = [ - "pkg-0.0.1-py2.py3-abi3-any.whl", - "pkg-0.0.1-py3-abi3-any.whl", - "pkg-0.0.1-py311-abi3-any.whl", - ], - want_platforms = ["cp3{}_ignored".format(minor_version)], - ) - _match(env, got, match) - -_tests.append(_test_select_by_supported_py_version) - -def _test_select_by_supported_cp_version(env): - for minor_version, match in { - 11: "pkg-0.0.1-cp311-abi3-any.whl", - 8: "pkg-0.0.1-py3-abi3-any.whl", - }.items(): - got = _select_whls( - whls = [ - "pkg-0.0.1-py2.py3-abi3-any.whl", - "pkg-0.0.1-py3-abi3-any.whl", - "pkg-0.0.1-py311-abi3-any.whl", - "pkg-0.0.1-cp311-abi3-any.whl", - ], - want_platforms = ["cp3{}_ignored".format(minor_version)], - ) - _match(env, got, match) - -_tests.append(_test_select_by_supported_cp_version) - -def _test_supported_cp_version_manylinux(env): - for minor_version, match in { - 8: "pkg-0.0.1-py3-none-manylinux_x86_64.whl", - 11: "pkg-0.0.1-cp311-none-manylinux_x86_64.whl", - }.items(): - got = _select_whls( - whls = [ - "pkg-0.0.1-py2.py3-none-manylinux_x86_64.whl", - "pkg-0.0.1-py3-none-manylinux_x86_64.whl", - "pkg-0.0.1-py311-none-manylinux_x86_64.whl", - "pkg-0.0.1-cp311-none-manylinux_x86_64.whl", - ], - want_platforms = ["cp3{}_linux_x86_64".format(minor_version)], - ) - _match(env, got, match) - -_tests.append(_test_supported_cp_version_manylinux) - -def _test_ignore_unsupported(env): - got = _select_whls( - whls = [ - "pkg-0.0.1-xx3-abi3-any.whl", - ], - want_platforms = ["cp30_ignored"], - ) - _match(env, got) - -_tests.append(_test_ignore_unsupported) - -def _test_match_abi_and_not_py_version(env): - # Check we match the ABI and not the py version - got = _select_whls(whls = WHL_LIST, want_platforms = ["cp37_linux_x86_64"]) - _match( - env, - got, - "pkg-0.0.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "pkg-0.0.1-cp37-cp37m-musllinux_1_1_x86_64.whl", - "pkg-0.0.1-py3-abi3-any.whl", - "pkg-0.0.1-py3-none-any.whl", - ) - -_tests.append(_test_match_abi_and_not_py_version) - -def _test_select_filename_with_many_tags(env): - # Check we can select a filename with many platform tags - got = _select_whls(whls = WHL_LIST, want_platforms = ["cp39_linux_x86_32"]) - _match( - env, - got, - "pkg-0.0.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", - "pkg-0.0.1-cp39-cp39-musllinux_1_1_i686.whl", - "pkg-0.0.1-cp39-abi3-any.whl", - "pkg-0.0.1-py3-none-any.whl", - ) - -_tests.append(_test_select_filename_with_many_tags) - -def _test_osx_prefer_arch_specific(env): - # Check that we prefer the specific wheel - got = _select_whls( - whls = WHL_LIST, - want_platforms = ["cp311_osx_x86_64", "cp311_osx_x86_32"], - ) - _match( - env, - got, - "pkg-0.0.1-cp311-cp311-macosx_10_9_universal2.whl", - "pkg-0.0.1-cp311-cp311-macosx_10_9_x86_64.whl", - "pkg-0.0.1-cp39-abi3-any.whl", - "pkg-0.0.1-py3-none-any.whl", - ) - - got = _select_whls(whls = WHL_LIST, want_platforms = ["cp311_osx_aarch64"]) - _match( - env, - got, - "pkg-0.0.1-cp311-cp311-macosx_10_9_universal2.whl", - "pkg-0.0.1-cp311-cp311-macosx_11_0_arm64.whl", - "pkg-0.0.1-cp39-abi3-any.whl", - "pkg-0.0.1-py3-none-any.whl", - ) - -_tests.append(_test_osx_prefer_arch_specific) - -def _test_osx_fallback_to_universal2(env): - # Check that we can use the universal2 if the arm wheel is not available - got = _select_whls( - whls = [w for w in WHL_LIST if "arm64" not in w], - want_platforms = ["cp311_osx_aarch64"], - ) - _match( - env, - got, - "pkg-0.0.1-cp311-cp311-macosx_10_9_universal2.whl", - "pkg-0.0.1-cp39-abi3-any.whl", - "pkg-0.0.1-py3-none-any.whl", - ) - -_tests.append(_test_osx_fallback_to_universal2) - -def _test_prefer_manylinux_wheels(env): - # Check we prefer platform specific wheels - got = _select_whls(whls = WHL_LIST, want_platforms = ["cp39_linux_x86_64"]) - _match( - env, - got, - "pkg-0.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "pkg-0.0.1-cp39-cp39-musllinux_1_1_x86_64.whl", - "pkg-0.0.1-cp39-abi3-any.whl", - "pkg-0.0.1-py3-none-any.whl", - ) - -_tests.append(_test_prefer_manylinux_wheels) - -def select_whl_test_suite(name): - """Create the test suite. - - Args: - name: the name of the test suite - """ - test_suite(name = name, basic_tests = _tests) diff --git a/tests/pypi/whl_target_platforms/whl_target_platforms_tests.bzl b/tests/pypi/whl_target_platforms/whl_target_platforms_tests.bzl index a72bdc275f..6bec26c10c 100644 --- a/tests/pypi/whl_target_platforms/whl_target_platforms_tests.bzl +++ b/tests/pypi/whl_target_platforms/whl_target_platforms_tests.bzl @@ -32,7 +32,10 @@ def _test_simple(env): struct(os = "linux", cpu = "x86_32", abi = None, target_platform = "linux_x86_32", version = (2, 17)), ], "musllinux_1_1_ppc64le": [ - struct(os = "linux", cpu = "ppc", abi = None, target_platform = "linux_ppc", version = (1, 1)), + struct(os = "linux", cpu = "ppc64le", abi = None, target_platform = "linux_ppc64le", version = (1, 1)), + ], + "musllinux_1_2_riscv64": [ + struct(os = "linux", cpu = "riscv64", abi = None, target_platform = "linux_riscv64", version = (1, 2)), ], "win_amd64": [ struct(os = "windows", cpu = "x86_64", abi = None, target_platform = "windows_x86_64", version = (0, 0)), @@ -60,9 +63,15 @@ def _test_with_abi(env): "manylinux1_i686.manylinux_2_17_i686": [ struct(os = "linux", cpu = "x86_32", abi = "cp38", target_platform = "cp38_linux_x86_32", version = (0, 0)), ], - "musllinux_1_1_ppc64le": [ + "musllinux_1_1_ppc64": [ struct(os = "linux", cpu = "ppc", abi = "cp311", target_platform = "cp311_linux_ppc", version = (1, 1)), ], + "musllinux_1_1_ppc64le": [ + struct(os = "linux", cpu = "ppc64le", abi = "cp311", target_platform = "cp311_linux_ppc64le", version = (1, 1)), + ], + "musllinux_1_2_riscv64": [ + struct(os = "linux", cpu = "riscv64", abi = "cp311", target_platform = "cp311_linux_riscv64", version = (1, 2)), + ], "win_amd64": [ struct(os = "windows", cpu = "x86_64", abi = "cp311", target_platform = "cp311_windows_x86_64", version = (0, 0)), ], @@ -100,6 +109,7 @@ def _can_parse_existing_tags(env): "manylinux_11_12_i686": 1, "manylinux_11_12_ppc64": 1, "manylinux_11_12_ppc64le": 1, + "manylinux_11_12_riscv64": 1, "manylinux_11_12_s390x": 1, "manylinux_11_12_x86_64": 1, "manylinux_1_2_aarch64": 1, @@ -108,6 +118,7 @@ def _can_parse_existing_tags(env): "musllinux_11_12_armv7l": 1, "musllinux_11_12_i686": 1, "musllinux_11_12_ppc64le": 1, + "musllinux_11_12_riscv64": 1, "musllinux_11_12_s390x": 1, "musllinux_11_12_x86_64": 1, "win32": 1, diff --git a/tests/pytest_test/BUILD.bazel b/tests/pytest_test/BUILD.bazel new file mode 100644 index 0000000000..b15094a615 --- /dev/null +++ b/tests/pytest_test/BUILD.bazel @@ -0,0 +1,22 @@ +load("//tests/support:support.bzl", "SUPPORTS_BZLMOD") +load("//tests/support/pytest_test:pytest_test.bzl", "pytest_test") + +pytest_test( + name = "pytest_script_venv_test", + srcs = [ + "basic_test.py", + ], + config_settings = { + "@rules_python//python/config_settings:bootstrap_impl": "script", + "@rules_python//python/config_settings:venvs_site_packages": "yes", + }, + target_compatible_with = SUPPORTS_BZLMOD, +) + +pytest_test( + name = "pytest_default_test", + srcs = [ + "basic_test.py", + ], + target_compatible_with = SUPPORTS_BZLMOD, +) diff --git a/tests/pytest_test/basic_test.py b/tests/pytest_test/basic_test.py new file mode 100644 index 0000000000..30e2c41ab9 --- /dev/null +++ b/tests/pytest_test/basic_test.py @@ -0,0 +1,2 @@ +def test_foo(): + assert True diff --git a/tests/python/BUILD.bazel b/tests/python/BUILD.bazel index 2553536b63..887fe969b5 100644 --- a/tests/python/BUILD.bazel +++ b/tests/python/BUILD.bazel @@ -12,6 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. -load(":python_tests.bzl", "python_test_suite") +load(":python_tests.bzl", "register_python_tests") -python_test_suite(name = "python_tests") +register_python_tests(name = "python_tests") diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index 40504302d1..cbef5637fd 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -12,121 +12,33 @@ # See the License for the specific language governing permissions and # limitations under the License. -"" +"""Unit tests for //python/extensions:python.bzl bzlmod extension.""" load("@pythons_hub//:versions.bzl", "MINOR_MAPPING") load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility load("//python/private:python.bzl", "parse_modules") # buildifier: disable=bzl-visibility +load("//python/private:repo_utils.bzl", "repo_utils") # buildifier: disable=bzl-visibility +load("//tests/support/mocks:python_ext.bzl", "python_ext") _tests = [] -def _mock_mctx(*modules, environ = {}): - return struct( - os = struct(environ = environ), - modules = [ - struct( - name = modules[0].name, - tags = modules[0].tags, - is_root = modules[0].is_root, - ), - ] + [ - struct( - name = mod.name, - tags = mod.tags, - is_root = False, - ) - for mod in modules[1:] - ], - ) - -def _mod(*, name, toolchain = [], override = [], single_version_override = [], single_version_platform_override = [], is_root = True): - return struct( - name = name, - tags = struct( - toolchain = toolchain, - override = override, - single_version_override = single_version_override, - single_version_platform_override = single_version_platform_override, - ), +def _rules_python_module(is_root = False): + """A mock of what the real rules_python MODULE.bazel looks like.""" + return python_ext.module( + name = "rules_python", + defaults = [python_ext.defaults(python_version = "3.11")], + toolchain = [python_ext.toolchain(python_version = "3.11")], is_root = is_root, ) -def _toolchain(python_version, *, is_default = False, **kwargs): - return struct( - is_default = is_default, - python_version = python_version, - **kwargs - ) - -def _override( - auth_patterns = {}, - available_python_versions = [], - base_url = "", - ignore_root_user_error = False, - minor_mapping = {}, - netrc = "", - register_all_versions = False): - return struct( - auth_patterns = auth_patterns, - available_python_versions = available_python_versions, - base_url = base_url, - ignore_root_user_error = ignore_root_user_error, - minor_mapping = minor_mapping, - netrc = netrc, - register_all_versions = register_all_versions, - ) - -def _single_version_override( - python_version = "", - sha256 = {}, - urls = [], - patch_strip = 0, - patches = [], - strip_prefix = "python", - distutils_content = "", - distutils = None): - if not python_version: - fail("missing mandatory args: python_version ({})".format(python_version)) - - return struct( - python_version = python_version, - sha256 = sha256, - urls = urls, - patch_strip = patch_strip, - patches = patches, - strip_prefix = strip_prefix, - distutils_content = distutils_content, - distutils = distutils, - ) - -def _single_version_platform_override( - coverage_tool = None, - patch_strip = 0, - patches = [], - platform = "", - python_version = "", - sha256 = "", - strip_prefix = "python", - urls = []): - if not platform or not python_version: - fail("missing mandatory args: platform ({}) and python_version ({})".format(platform, python_version)) - - return struct( - sha256 = sha256, - urls = urls, - strip_prefix = strip_prefix, - platform = platform, - coverage_tool = coverage_tool, - python_version = python_version, - patch_strip = patch_strip, - patches = patches, - ) - -def _test_default(env): +def _test_default_from_rules_python_when_rules_python_is_root(env): + """Verify that rules_python (as root module) default is applied.""" py = parse_modules( - module_ctx = _mock_mctx( - _mod(name = "rules_python", toolchain = [_toolchain("3.11")]), + module_ctx = python_ext.mctx( + _rules_python_module(is_root = True), ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) # The value there should be consistent in bzlmod with the automatically @@ -135,11 +47,10 @@ def _test_default(env): env.expect.that_dict(py.config.minor_mapping).contains_exactly(MINOR_MAPPING) env.expect.that_collection(py.config.kwargs).has_size(0) env.expect.that_collection(py.config.default.keys()).contains_exactly([ - "base_url", - "ignore_root_user_error", + "base_urls", "tool_versions", + "platforms", ]) - env.expect.that_bool(py.config.default["ignore_root_user_error"]).equals(False) env.expect.that_str(py.default_python_version).equals("3.11") want_toolchain = struct( @@ -149,13 +60,15 @@ def _test_default(env): ) env.expect.that_collection(py.toolchains).contains_exactly([want_toolchain]) -_tests.append(_test_default) +_tests.append(_test_default_from_rules_python_when_rules_python_is_root) -def _test_default_some_module(env): +def _test_default_from_rules_python_when_rules_python_is_not_root(env): + """Verify that rules_python default applies when rules_python is not the root module.""" py = parse_modules( - module_ctx = _mock_mctx( - _mod(name = "rules_python", toolchain = [_toolchain("3.11")], is_root = False), + module_ctx = python_ext.mctx( + _rules_python_module(), ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_str(py.default_python_version).equals("3.11") @@ -167,13 +80,21 @@ def _test_default_some_module(env): ) env.expect.that_collection(py.toolchains).contains_exactly([want_toolchain]) -_tests.append(_test_default_some_module) +_tests.append(_test_default_from_rules_python_when_rules_python_is_not_root) def _test_default_with_patch_version(env): py = parse_modules( - module_ctx = _mock_mctx( - _mod(name = "rules_python", toolchain = [_toolchain("3.11.2")]), + module_ctx = python_ext.mctx( + modules = [ + python_ext.module( + name = "alpha", + is_root = True, + toolchain = [python_ext.toolchain(python_version = "3.11.2")], + ), + _rules_python_module(is_root = False), + ], ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_str(py.default_python_version).equals("3.11.2") @@ -183,138 +104,249 @@ def _test_default_with_patch_version(env): python_version = "3.11.2", register_coverage_tool = False, ) - env.expect.that_collection(py.toolchains).contains_exactly([want_toolchain]) + env.expect.that_collection(py.toolchains).contains_at_least([want_toolchain]) _tests.append(_test_default_with_patch_version) -def _test_default_non_rules_python(env): +def _test_toolchain_ordering(env): py = parse_modules( - module_ctx = _mock_mctx( - # NOTE @aignas 2024-09-06: the first item in the module_ctx.modules - # could be a non-root module, which is the case if the root module - # does not make any calls to the extension. - _mod(name = "rules_python", toolchain = [_toolchain("3.11")], is_root = False), + module_ctx = python_ext.mctx( + python_ext.module( + name = "my_module", + is_root = True, + toolchain = [ + python_ext.toolchain(python_version = "3.10"), + python_ext.toolchain(python_version = "3.10.15"), + python_ext.toolchain(python_version = MINOR_MAPPING["3.10"]), + python_ext.toolchain(python_version = "3.10.13"), + python_ext.toolchain(python_version = "3.11.1"), + python_ext.toolchain(python_version = "3.11.10"), + python_ext.toolchain( + python_version = MINOR_MAPPING["3.11"], + is_default = True, + ), + ], + ), + _rules_python_module(), ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) + got_versions = [ + t.python_version + for t in py.toolchains + ] - env.expect.that_str(py.default_python_version).equals("3.11") - rules_python_toolchain = struct( - name = "python_3_11", - python_version = "3.11", - register_coverage_tool = False, + env.expect.that_str(py.default_python_version).equals(MINOR_MAPPING["3.11"]) + env.expect.that_dict(py.config.minor_mapping).contains_exactly(MINOR_MAPPING) + env.expect.that_collection(got_versions).contains_exactly([ + # First the full-version toolchains that are in minor_mapping + # so that they get matched first if only the `python_version` is in MINOR_MAPPING + # + # The default version is always set in the `python_version` flag, so know, that + # the default match will be somewhere in the first bunch. + "3.10", + MINOR_MAPPING["3.10"], + "3.11", + MINOR_MAPPING["3.11"], + # Next, the rest, where we will match things based on the `python_version` being + # the same + "3.10.15", + "3.10.13", + "3.11.1", + "3.11.10", + ]).in_order() + +_tests.append(_test_toolchain_ordering) + +def _test_default_from_defaults(env): + py = parse_modules( + module_ctx = python_ext.mctx( + python_ext.module( + name = "my_root_module", + defaults = [python_ext.defaults(python_version = "3.11")], + is_root = True, + toolchain = [ + python_ext.toolchain(python_version = "3.10"), + python_ext.toolchain(python_version = "3.11"), + python_ext.toolchain(python_version = "3.12"), + ], + ), + ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) - env.expect.that_collection(py.toolchains).contains_exactly([rules_python_toolchain]) -_tests.append(_test_default_non_rules_python) + env.expect.that_str(py.default_python_version).equals("3.11") + + want_toolchains = [ + struct( + name = "python_3_" + minor_version, + python_version = "3." + minor_version, + register_coverage_tool = False, + ) + for minor_version in ["10", "11", "12"] + ] + env.expect.that_collection(py.toolchains).contains_exactly(want_toolchains) -def _test_default_non_rules_python_ignore_root_user_error(env): +_tests.append(_test_default_from_defaults) + +def _test_default_from_defaults_env(env): py = parse_modules( - module_ctx = _mock_mctx( - _mod( - name = "my_module", - toolchain = [_toolchain("3.12", ignore_root_user_error = True)], + module_ctx = python_ext.mctx( + python_ext.module( + name = "my_root_module", + defaults = [ + python_ext.defaults( + python_version = "3.11", + python_version_env = "PYENV_VERSION", + ), + ], + is_root = True, + toolchain = [ + python_ext.toolchain(python_version = "3.10"), + python_ext.toolchain(python_version = "3.11"), + python_ext.toolchain(python_version = "3.12"), + ], ), - _mod(name = "rules_python", toolchain = [_toolchain("3.11")]), + environ = {"PYENV_VERSION": "3.12"}, ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) - env.expect.that_bool(py.config.default["ignore_root_user_error"]).equals(True) env.expect.that_str(py.default_python_version).equals("3.12") - my_module_toolchain = struct( - name = "python_3_12", - python_version = "3.12", - register_coverage_tool = False, - ) - rules_python_toolchain = struct( - name = "python_3_11", - python_version = "3.11", - register_coverage_tool = False, - ) - env.expect.that_collection(py.toolchains).contains_exactly([ - rules_python_toolchain, - my_module_toolchain, - ]).in_order() + want_toolchains = [ + struct( + name = "python_3_" + minor_version, + python_version = "3." + minor_version, + register_coverage_tool = False, + ) + for minor_version in ["10", "11", "12"] + ] + env.expect.that_collection(py.toolchains).contains_exactly(want_toolchains) -_tests.append(_test_default_non_rules_python_ignore_root_user_error) +_tests.append(_test_default_from_defaults_env) -def _test_default_non_rules_python_ignore_root_user_error_override(env): +def _test_default_from_defaults_file(env): py = parse_modules( - module_ctx = _mock_mctx( - _mod( - name = "my_module", - toolchain = [_toolchain("3.12")], - override = [_override(ignore_root_user_error = True)], + module_ctx = python_ext.mctx( + python_ext.module( + name = "my_root_module", + defaults = [ + python_ext.defaults( + python_version_file = "@@//:.python-version", + ), + ], + is_root = True, + toolchain = [ + python_ext.toolchain(python_version = "3.10"), + python_ext.toolchain(python_version = "3.11"), + python_ext.toolchain(python_version = "3.12"), + ], ), - _mod(name = "rules_python", toolchain = [_toolchain("3.11")]), + mock_files = {"@@//:.python-version": "3.12\n"}, ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) - env.expect.that_bool(py.config.default["ignore_root_user_error"]).equals(True) env.expect.that_str(py.default_python_version).equals("3.12") - my_module_toolchain = struct( - name = "python_3_12", - python_version = "3.12", - register_coverage_tool = False, - ) - rules_python_toolchain = struct( - name = "python_3_11", - python_version = "3.11", - register_coverage_tool = False, - ) - env.expect.that_collection(py.toolchains).contains_exactly([ - rules_python_toolchain, - my_module_toolchain, - ]).in_order() + want_toolchains = [ + struct( + name = "python_3_" + minor_version, + python_version = "3." + minor_version, + register_coverage_tool = False, + ) + for minor_version in ["10", "11", "12"] + ] + env.expect.that_collection(py.toolchains).contains_exactly(want_toolchains) -_tests.append(_test_default_non_rules_python_ignore_root_user_error_override) +_tests.append(_test_default_from_defaults_file) -def _test_default_non_rules_python_ignore_root_user_error_non_root_module(env): +def _test_default_from_single_toolchain(env): py = parse_modules( - module_ctx = _mock_mctx( - _mod(name = "my_module", toolchain = [_toolchain("3.13")]), - _mod(name = "some_module", toolchain = [_toolchain("3.12", ignore_root_user_error = True)]), - _mod(name = "rules_python", toolchain = [_toolchain("3.11")]), + module_ctx = python_ext.mctx( + python_ext.module( + name = "my_root_module", + is_root = True, + toolchain = [python_ext.toolchain(python_version = "3.12")], + ), + _rules_python_module(), ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) + env.expect.that_str(py.default_python_version).equals("3.12") - env.expect.that_str(py.default_python_version).equals("3.13") - env.expect.that_bool(py.config.default["ignore_root_user_error"]).equals(False) +_tests.append(_test_default_from_single_toolchain) - my_module_toolchain = struct( - name = "python_3_13", - python_version = "3.13", - register_coverage_tool = False, - ) - some_module_toolchain = struct( - name = "python_3_12", - python_version = "3.12", - register_coverage_tool = False, +def _test_defaults_overrides_single_toolchain(env): + py = parse_modules( + module_ctx = python_ext.mctx( + python_ext.module( + name = "my_root_module", + defaults = [ + # This relies on rules_python registering 3.11 + python_ext.defaults(python_version = "3.11"), + ], + is_root = True, + toolchain = [python_ext.toolchain(python_version = "3.12")], + ), + _rules_python_module(), + ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) - rules_python_toolchain = struct( - name = "python_3_11", - python_version = "3.11", - register_coverage_tool = False, + env.expect.that_str(py.default_python_version).equals("3.11") + +_tests.append(_test_defaults_overrides_single_toolchain) + +def _test_defaults_overrides_toolchains_setting_is_default(env): + py = parse_modules( + module_ctx = python_ext.mctx( + python_ext.module( + name = "my_root_module", + defaults = [python_ext.defaults(python_version = "3.13")], + is_root = True, + toolchain = [ + python_ext.toolchain(python_version = "3.13"), + python_ext.toolchain( + python_version = "3.12", + is_default = True, + ), + ], + ), + _rules_python_module(), + ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) - env.expect.that_collection(py.toolchains).contains_exactly([ - some_module_toolchain, - rules_python_toolchain, - my_module_toolchain, # this was the only toolchain, default to that - ]).in_order() + env.expect.that_str(py.default_python_version).equals("3.13") -_tests.append(_test_default_non_rules_python_ignore_root_user_error_non_root_module) +_tests.append(_test_defaults_overrides_toolchains_setting_is_default) def _test_first_occurance_of_the_toolchain_wins(env): py = parse_modules( - module_ctx = _mock_mctx( - _mod(name = "my_module", toolchain = [_toolchain("3.12")]), - _mod(name = "some_module", toolchain = [_toolchain("3.12", configure_coverage_tool = True)]), - _mod(name = "rules_python", toolchain = [_toolchain("3.11")]), + module_ctx = python_ext.mctx( + modules = [ + python_ext.module( + name = "my_module", + is_root = True, + toolchain = [python_ext.toolchain(python_version = "3.12")], + ), + python_ext.module( + name = "some_module", + is_root = False, + toolchain = [ + python_ext.toolchain( + python_version = "3.12", + configure_coverage_tool = True, + ), + ], + ), + _rules_python_module(), + ], environ = { "RULES_PYTHON_BZLMOD_DEBUG": "1", }, ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_str(py.default_python_version).equals("3.12") @@ -338,8 +370,8 @@ def _test_first_occurance_of_the_toolchain_wins(env): env.expect.that_dict(py.debug_info).contains_exactly({ "toolchains_registered": [ - {"ignore_root_user_error": False, "module": {"is_root": True, "name": "my_module"}, "name": "python_3_12"}, - {"ignore_root_user_error": False, "module": {"is_root": False, "name": "rules_python"}, "name": "python_3_11"}, + {"module": {"is_root": True, "name": "my_module"}, "name": "python_3_12"}, + {"module": {"is_root": False, "name": "rules_python"}, "name": "python_3_11"}, ], }) @@ -347,24 +379,25 @@ _tests.append(_test_first_occurance_of_the_toolchain_wins) def _test_auth_overrides(env): py = parse_modules( - module_ctx = _mock_mctx( - _mod( + module_ctx = python_ext.mctx( + python_ext.module( name = "my_module", - toolchain = [_toolchain("3.12")], + is_root = True, override = [ - _override( - netrc = "/my/netrc", + python_ext.override( auth_patterns = {"foo": "bar"}, + netrc = "/my/netrc", ), ], + toolchain = [python_ext.toolchain(python_version = "3.12")], ), - _mod(name = "rules_python", toolchain = [_toolchain("3.11")]), + _rules_python_module(), ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_dict(py.config.default).contains_at_least({ "auth_patterns": {"foo": "bar"}, - "ignore_root_user_error": False, "netrc": "/my/netrc", }) env.expect.that_str(py.default_python_version).equals("3.12") @@ -386,49 +419,82 @@ def _test_auth_overrides(env): _tests.append(_test_auth_overrides) +def _test_add_target_settings(env): + py = parse_modules( + module_ctx = python_ext.mctx( + python_ext.module( + name = "my_module", + is_root = True, + override = [ + python_ext.override( + add_target_settings = [ + "@@//my:custom_setting", + ], + ), + ], + toolchain = [python_ext.toolchain(python_version = "3.12")], + ), + _rules_python_module(), + ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), + ) + + env.expect.that_collection( + py.config.add_target_settings, + ).contains_exactly(["@@//my:custom_setting"]) + +_tests.append(_test_add_target_settings) + def _test_add_new_version(env): py = parse_modules( - module_ctx = _mock_mctx( - _mod( + module_ctx = python_ext.mctx( + python_ext.module( name = "my_module", - toolchain = [_toolchain("3.13")], + is_root = True, + override = [ + python_ext.override( + available_python_versions = [ + "3.12.4", + "3.13.0", + "3.13.1", + "3.13.99", + ], + base_urls = [], + minor_mapping = { + "3.13": "3.13.99", + }, + ), + ], single_version_override = [ - _single_version_override( + python_ext.single_version_override( + distutils = None, + distutils_content = "", + patch_strip = 0, + patches = [], python_version = "3.13.0", sha256 = { "aarch64-unknown-linux-gnu": "deadbeef", }, - urls = ["example.org"], - patch_strip = 0, - patches = [], strip_prefix = "prefix", - distutils_content = "", - distutils = None, + urls = ["example.org"], ), ], single_version_platform_override = [ - _single_version_platform_override( - sha256 = "deadb00f", - urls = ["something.org", "else.org"], - strip_prefix = "python", - platform = "aarch64-unknown-linux-gnu", + python_ext.single_version_platform_override( coverage_tool = "specific_cov_tool", - python_version = "3.13.1", patch_strip = 2, patches = ["specific-patch.txt"], + platform = "aarch64-unknown-linux-gnu", + python_version = "3.13.99", + sha256 = "deadb00f", + strip_prefix = "python", + urls = ["something.org", "else.org"], ), ], - override = [ - _override( - base_url = "", - available_python_versions = ["3.12.4", "3.13.0", "3.13.1"], - minor_mapping = { - "3.13": "3.13.0", - }, - ), - ], + toolchain = [python_ext.toolchain(python_version = "3.13")], ), ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_str(py.default_python_version).equals("3.13") @@ -436,13 +502,14 @@ def _test_add_new_version(env): "3.12.4", "3.13.0", "3.13.1", + "3.13.99", ]) env.expect.that_dict(py.config.default["tool_versions"]["3.13.0"]).contains_exactly({ "sha256": {"aarch64-unknown-linux-gnu": "deadbeef"}, "strip_prefix": {"aarch64-unknown-linux-gnu": "prefix"}, "url": {"aarch64-unknown-linux-gnu": ["example.org"]}, }) - env.expect.that_dict(py.config.default["tool_versions"]["3.13.1"]).contains_exactly({ + env.expect.that_dict(py.config.default["tool_versions"]["3.13.99"]).contains_exactly({ "coverage_tool": {"aarch64-unknown-linux-gnu": "specific_cov_tool"}, "patch_strip": {"aarch64-unknown-linux-gnu": 2}, "patches": {"aarch64-unknown-linux-gnu": ["specific-patch.txt"]}, @@ -452,7 +519,7 @@ def _test_add_new_version(env): }) env.expect.that_dict(py.config.minor_mapping).contains_exactly({ "3.12": "3.12.4", # The `minor_mapping` will be overriden only for the missing keys - "3.13": "3.13.0", + "3.13": "3.13.99", }) env.expect.that_collection(py.toolchains).contains_exactly([ struct( @@ -466,12 +533,24 @@ _tests.append(_test_add_new_version) def _test_register_all_versions(env): py = parse_modules( - module_ctx = _mock_mctx( - _mod( + module_ctx = python_ext.mctx( + python_ext.module( name = "my_module", - toolchain = [_toolchain("3.13")], + is_root = True, + override = [ + python_ext.override( + available_python_versions = [ + "3.12.4", + "3.13.0", + "3.13.1", + "3.13.99", + ], + base_urls = [], + register_all_versions = True, + ), + ], single_version_override = [ - _single_version_override( + python_ext.single_version_override( python_version = "3.13.0", sha256 = { "aarch64-unknown-linux-gnu": "deadbeef", @@ -480,22 +559,17 @@ def _test_register_all_versions(env): ), ], single_version_platform_override = [ - _single_version_platform_override( + python_ext.single_version_platform_override( + platform = "aarch64-unknown-linux-gnu", + python_version = "3.13.99", sha256 = "deadb00f", urls = ["something.org"], - platform = "aarch64-unknown-linux-gnu", - python_version = "3.13.1", - ), - ], - override = [ - _override( - base_url = "", - available_python_versions = ["3.12.4", "3.13.0", "3.13.1"], - register_all_versions = True, ), ], + toolchain = [python_ext.toolchain(python_version = "3.13")], ), ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_str(py.default_python_version).equals("3.13") @@ -503,11 +577,12 @@ def _test_register_all_versions(env): "3.12.4", "3.13.0", "3.13.1", + "3.13.99", ]) env.expect.that_dict(py.config.minor_mapping).contains_exactly({ # The mapping is calculated automatically "3.12": "3.12.4", - "3.13": "3.13.1", + "3.13": "3.13.99", }) env.expect.that_collection(py.toolchains).contains_exactly([ struct( @@ -521,55 +596,136 @@ def _test_register_all_versions(env): "python_3_13": "3.13", "python_3_13_0": "3.13.0", "python_3_13_1": "3.13.1", + "python_3_13_99": "3.13.99", }.items() ]) _tests.append(_test_register_all_versions) -def _test_add_patches(env): +def _test_ignore_unsupported_versions(env): py = parse_modules( - module_ctx = _mock_mctx( - _mod( + module_ctx = python_ext.mctx( + python_ext.module( name = "my_module", - toolchain = [_toolchain("3.13")], + is_root = True, + override = [ + python_ext.override( + available_python_versions = [ + "3.12.4", + "3.13.0", + "3.13.1", + ], + base_urls = [], + minor_mapping = { + "3.12": "3.12.4", + "3.13": "3.13.1", + }, + ), + ], single_version_override = [ - _single_version_override( + python_ext.single_version_override( python_version = "3.13.0", sha256 = { - "aarch64-apple-darwin": "deadbeef", "aarch64-unknown-linux-gnu": "deadbeef", }, urls = ["example.org"], - patch_strip = 1, - patches = ["common.txt"], - strip_prefix = "prefix", - distutils_content = "", - distutils = None, ), ], single_version_platform_override = [ - _single_version_platform_override( - sha256 = "deadb00f", - urls = ["something.org", "else.org"], - strip_prefix = "python", + python_ext.single_version_platform_override( platform = "aarch64-unknown-linux-gnu", - coverage_tool = "specific_cov_tool", - python_version = "3.13.0", - patch_strip = 2, - patches = ["specific-patch.txt"], + python_version = "3.13.99", + sha256 = "deadb00f", + urls = ["something.org"], + ), + ], + toolchain = [ + python_ext.toolchain(python_version = "3.11"), + python_ext.toolchain(python_version = "3.12"), + python_ext.toolchain( + python_version = "3.13", + is_default = True, ), ], + ), + ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), + ) + + env.expect.that_str(py.default_python_version).equals("3.13") + env.expect.that_collection(py.config.default["tool_versions"].keys()).contains_exactly([ + "3.12.4", + "3.13.0", + "3.13.1", + ]) + env.expect.that_dict(py.config.minor_mapping).contains_exactly({ + # The mapping is calculated automatically + "3.12": "3.12.4", + "3.13": "3.13.1", + }) + env.expect.that_collection(py.toolchains).contains_exactly([ + struct( + name = name, + python_version = version, + register_coverage_tool = False, + ) + for name, version in { + # NOTE: that '3.11' wont be actually registered and present in the + # `tool_versions` above. + "python_3_11": "3.11", + "python_3_12": "3.12", + "python_3_13": "3.13", + }.items() + ]) + +_tests.append(_test_ignore_unsupported_versions) + +def _test_add_patches(env): + py = parse_modules( + module_ctx = python_ext.mctx( + python_ext.module( + name = "my_module", + is_root = True, override = [ - _override( - base_url = "", + python_ext.override( available_python_versions = ["3.13.0"], + base_urls = [], minor_mapping = { "3.13": "3.13.0", }, ), ], + single_version_override = [ + python_ext.single_version_override( + distutils = None, + distutils_content = "", + patch_strip = 1, + patches = ["common.txt"], + python_version = "3.13.0", + sha256 = { + "aarch64-apple-darwin": "deadbeef", + "aarch64-unknown-linux-gnu": "deadbeef", + }, + strip_prefix = "prefix", + urls = ["example.org"], + ), + ], + single_version_platform_override = [ + python_ext.single_version_platform_override( + coverage_tool = "specific_cov_tool", + patch_strip = 2, + patches = ["specific-patch.txt"], + platform = "aarch64-unknown-linux-gnu", + python_version = "3.13.0", + sha256 = "deadb00f", + strip_prefix = "python", + urls = ["something.org", "else.org"], + ), + ], + toolchain = [python_ext.toolchain(python_version = "3.13")], ), ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_str(py.default_python_version).equals("3.13") @@ -605,17 +761,19 @@ _tests.append(_test_add_patches) def _test_fail_two_overrides(env): errors = [] parse_modules( - module_ctx = _mock_mctx( - _mod( + module_ctx = python_ext.mctx( + python_ext.module( name = "my_module", - toolchain = [_toolchain("3.13")], + is_root = True, override = [ - _override(base_url = "foo"), - _override(base_url = "bar"), + python_ext.override(base_urls = ["foo"]), + python_ext.override(base_urls = ["bar"]), ], + toolchain = [python_ext.toolchain(python_version = "3.13")], ), ), _fail = errors.append, + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_collection(errors).contains_exactly([ "Only a single 'python.override' can be present", @@ -627,28 +785,30 @@ def _test_single_version_override_errors(env): for test in [ struct( overrides = [ - _single_version_override(python_version = "3.12.4", distutils_content = "foo"), - _single_version_override(python_version = "3.12.4", distutils_content = "foo"), + python_ext.single_version_override( + distutils_content = "foo", + python_version = "3.12.4", + ), + python_ext.single_version_override( + distutils_content = "foo", + python_version = "3.12.4", + ), ], want_error = "Only a single 'python.single_version_override' can be present for '3.12.4'", ), - struct( - overrides = [ - _single_version_override(python_version = "3.12.4+3", distutils_content = "foo"), - ], - want_error = "The 'python_version' attribute needs to specify an 'X.Y.Z' semver-compatible version, got: '3.12.4+3'", - ), ]: errors = [] parse_modules( - module_ctx = _mock_mctx( - _mod( + module_ctx = python_ext.mctx( + python_ext.module( name = "my_module", - toolchain = [_toolchain("3.13")], + is_root = True, single_version_override = test.overrides, + toolchain = [python_ext.toolchain(python_version = "3.13")], ), ), _fail = errors.append, + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_collection(errors).contains_exactly([test.want_error]) @@ -658,34 +818,50 @@ def _test_single_version_platform_override_errors(env): for test in [ struct( overrides = [ - _single_version_platform_override(python_version = "3.12.4", platform = "foo", coverage_tool = "foo"), - _single_version_platform_override(python_version = "3.12.4", platform = "foo", coverage_tool = "foo"), + python_ext.single_version_platform_override( + coverage_tool = "foo", + platform = "foo", + python_version = "3.12.4", + ), + python_ext.single_version_platform_override( + coverage_tool = "foo", + platform = "foo", + python_version = "3.12.4", + ), ], want_error = "Only a single 'python.single_version_platform_override' can be present for '(\"3.12.4\", \"foo\")'", ), struct( overrides = [ - _single_version_platform_override(python_version = "3.12", platform = "foo"), + python_ext.single_version_platform_override( + platform = "foo", + python_version = "3.12", + ), ], - want_error = "The 'python_version' attribute needs to specify an 'X.Y.Z' semver-compatible version, got: '3.12'", + want_error = "The 'python_version' attribute needs to specify the full version in at least 'X.Y.Z' format, got: '3.12'", ), struct( overrides = [ - _single_version_platform_override(python_version = "3.12.1+my_build", platform = "foo"), + python_ext.single_version_platform_override( + platform = "foo", + python_version = "foo", + ), ], - want_error = "The 'python_version' attribute needs to specify an 'X.Y.Z' semver-compatible version, got: '3.12.1+my_build'", + want_error = "Failed to parse PEP 440 version identifier 'foo'. Parse error at 'foo'", ), ]: errors = [] parse_modules( - module_ctx = _mock_mctx( - _mod( + module_ctx = python_ext.mctx( + python_ext.module( name = "my_module", - toolchain = [_toolchain("3.13")], + is_root = True, single_version_platform_override = test.overrides, + toolchain = [python_ext.toolchain(python_version = "3.13")], ), ), - _fail = errors.append, + _fail = lambda *a: errors.append(" ".join(a)), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_collection(errors).contains_exactly([test.want_error]) @@ -702,3 +878,17 @@ def python_test_suite(name): name: the name of the test suite """ test_suite(name = name, basic_tests = _tests) + +def register_python_tests(name): + """Registers the python tests if Bzlmod is enabled, otherwise defines an empty test_suite. + + Args: + name: The name of the test target. + """ + if BZLMOD_ENABLED: + python_test_suite(name = name) + else: + native.test_suite( + name = name, + tests = [], + ) diff --git a/tests/python_bzlmod_ext/BUILD.bazel b/tests/python_bzlmod_ext/BUILD.bazel new file mode 100644 index 0000000000..e266c01b9c --- /dev/null +++ b/tests/python_bzlmod_ext/BUILD.bazel @@ -0,0 +1,7 @@ +load(":test_helpers.bzl", "register_python_bzlmod_ext_tests") + +register_python_bzlmod_ext_tests( + name = "python_bzlmod_ext_tests", + parse_runtime_manifest_name = "parse_runtime_manifest_tests", + runtime_manifests_name = "runtime_manifests_tests", +) diff --git a/tests/python_bzlmod_ext/parse_runtime_manifest_tests.bzl b/tests/python_bzlmod_ext/parse_runtime_manifest_tests.bzl new file mode 100644 index 0000000000..4493576147 --- /dev/null +++ b/tests/python_bzlmod_ext/parse_runtime_manifest_tests.bzl @@ -0,0 +1,183 @@ +"""Tests for manifest parsing Starlark functions.""" + +load("@bazel_skylib//lib:structs.bzl", "structs") +load("@rules_testing//lib:analysis_test.bzl", "analysis_test") +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("@rules_testing//lib:util.bzl", rt_util = "util") +load("//python/private:pbs_manifest.bzl", "parse_filename", "parse_runtime_manifest") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_parse_filename_baseline(name): + """Sets up the baseline filename parsing test. + + Args: + name: The name of the test. + """ + rt_util.helper_target( + native.filegroup, + name = name + "_subject", + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_parse_filename_baseline_impl, + ) + +def _test_parse_filename_baseline_impl(env, target): + _ = target # @unused + + # 1. Baseline + parsed1 = parse_filename("cpython-3.11.15+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz") + env.expect.that_dict(parsed1).contains_exactly({ + "arch": "x86_64", + "archive_flavor": "install_only", + "build_flavor": "", + "build_version": "20260414", + "freethreaded": False, + "libc": "gnu", + "location": "cpython-3.11.15+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz", + "microarch": "", + "os": "linux", + "python_version": "3.11.15", + "vendor": "unknown", + }) + + # 2. Microarch + parsed2 = parse_filename("cpython-3.10.20+20260414-x86_64_v2-unknown-linux-musl-lto-full.tar.zst") + env.expect.that_dict(parsed2).contains_exactly({ + "arch": "x86_64", + "archive_flavor": "full", + "build_flavor": "lto", + "build_version": "20260414", + "freethreaded": False, + "libc": "musl", + "location": "cpython-3.10.20+20260414-x86_64_v2-unknown-linux-musl-lto-full.tar.zst", + "microarch": "v2", + "os": "linux", + "python_version": "3.10.20", + "vendor": "unknown", + }) + + # 3. Freethreaded + parsed3 = parse_filename("cpython-3.13.13+20260414-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst") + env.expect.that_dict(parsed3).contains_exactly({ + "arch": "aarch64", + "archive_flavor": "full", + "build_flavor": "pgo+lto", + "build_version": "20260414", + "freethreaded": True, + "libc": "", + "location": "cpython-3.13.13+20260414-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst", + "microarch": "", + "os": "darwin", + "python_version": "3.13.13", + "vendor": "apple", + }) + + # 4. Invalid + parsed4 = parse_filename("invalid-filename.tar.gz") + env.expect.that_bool(parsed4 == None).equals(True) + + # 5. Full URL (should return the original URL as location) + parsed5 = parse_filename("https://github.com/astral-sh/python-build-standalone/releases/download/20260414/cpython-3.11.15+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz") + env.expect.that_dict(parsed5).contains_exactly({ + "arch": "x86_64", + "archive_flavor": "install_only", + "build_flavor": "", + "build_version": "20260414", + "freethreaded": False, + "libc": "gnu", + "location": "https://github.com/astral-sh/python-build-standalone/releases/download/20260414/cpython-3.11.15+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz", + "microarch": "", + "os": "linux", + "python_version": "3.11.15", + "vendor": "unknown", + }) + +_tests.append(_test_parse_filename_baseline) + +def _test_parse_runtime_manifest(name): + """Sets up the manifest file parsing test. + + Args: + name: The name of the test. + """ + rt_util.helper_target( + native.filegroup, + name = name + "_subject", + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_parse_runtime_manifest_impl, + ) + +def _test_parse_runtime_manifest_impl(env, target): + _ = target # @unused + content = """ +8b14030dd3af9ea7f7c51b4c90feb04afd8a8f45435727e67b875270bd08f3bc cpython-3.11.15+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +a57ffd435652092d16b30e783f9826c55e9c64b0f0a72cbae0a9f39e663137fb cpython-3.11.15+20260414-aarch64-apple-darwin-install_only.tar.gz +ce18fdfd47c66830a40ea9b9e314a14b1636bbfd684501bc5ca1fc6d55a7933f https://example.com/cpython-3.10.20+20260414-x86_64_v2-unknown-linux-musl-lto-full.tar.zst +1111111111111111111111111111111111111111111111111111111111111111 cpython-3.13.13+20260414-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +""" + parsed = parse_runtime_manifest(content) + env.expect.that_collection(parsed).has_size(4) + + env.expect.that_dict(structs.to_dict(parsed[0])).contains_exactly({ + "arch": "x86_64", + "archive_flavor": "install_only", + "build_flavor": "", + "build_version": "20260414", + "freethreaded": False, + "libc": "gnu", + "location": "cpython-3.11.15+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz", + "microarch": "", + "os": "linux", + "python_version": "3.11.15", + "sha256": "8b14030dd3af9ea7f7c51b4c90feb04afd8a8f45435727e67b875270bd08f3bc", + "vendor": "unknown", + }) + + env.expect.that_dict(structs.to_dict(parsed[2])).contains_exactly({ + "arch": "x86_64", + "archive_flavor": "full", + "build_flavor": "lto", + "build_version": "20260414", + "freethreaded": False, + "libc": "musl", + "location": "https://example.com/cpython-3.10.20+20260414-x86_64_v2-unknown-linux-musl-lto-full.tar.zst", + "microarch": "v2", + "os": "linux", + "python_version": "3.10.20", + "sha256": "ce18fdfd47c66830a40ea9b9e314a14b1636bbfd684501bc5ca1fc6d55a7933f", + "vendor": "unknown", + }) + + env.expect.that_dict(structs.to_dict(parsed[3])).contains_exactly({ + "arch": "aarch64", + "archive_flavor": "full", + "build_flavor": "pgo+lto", + "build_version": "20260414", + "freethreaded": True, + "libc": "", + "location": "cpython-3.13.13+20260414-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst", + "microarch": "", + "os": "darwin", + "python_version": "3.13.13", + "sha256": "1111111111111111111111111111111111111111111111111111111111111111", + "vendor": "apple", + }) + +_tests.append(_test_parse_runtime_manifest) + +def parse_runtime_manifest_test_suite(name): + """Defines the test suite for manifest parsing. + + Args: + name: The name of the test suite. + """ + test_suite( + name = name, + tests = _tests, + ) diff --git a/tests/python_bzlmod_ext/runtime_manifests_tests.bzl b/tests/python_bzlmod_ext/runtime_manifests_tests.bzl new file mode 100644 index 0000000000..6de0ff93c2 --- /dev/null +++ b/tests/python_bzlmod_ext/runtime_manifests_tests.bzl @@ -0,0 +1,162 @@ +"""Starlark unit tests for dynamic toolchain registration via manifests.""" + +load("@rules_testing//lib:analysis_test.bzl", "analysis_test") +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("@rules_testing//lib:util.bzl", rt_util = "util") +load("//python/private:python.bzl", "parse_modules") # buildifier: disable=bzl-visibility +load("//python/private:repo_utils.bzl", "repo_utils") # buildifier: disable=bzl-visibility +load("//tests/support/mocks:python_ext.bzl", "python_ext") # buildifier: disable=bzl-visibility + +_tests = [] + +_mock_logger = repo_utils.logger( + name = "mock", + verbosity_level = "ERROR", +) + +def _test_dynamic_manifest_toolchains(name): + rt_util.helper_target( + native.filegroup, + name = name + "_subject", + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_dynamic_manifest_toolchains_impl, + ) + +def _test_dynamic_manifest_toolchains_impl(env, target): + _ = target # @unused + + # Construct Bzlmod mock module locally inside the test execution block. + # We test using virtual patch version "3.11.99" (not present in TOOL_VERSIONS) + # so that the populated config contains ONLY our dynamically parsed manifest keys + # without any pre-populated multi-platform templates, allowing exact dictionary match! + root_module = python_ext.module( + name = "runtime_manifests", + override = [ + python_ext.override( + add_runtime_manifest_urls = [ + "https://github.com/astral-sh/python-build-standalone/releases/download/20260414/SHA256SUMS", + ], + runtime_manifest_sha = "ce18fdfd47c66830a40ea9b9e314a14b1636bbfd684501bc5ca1fc6d55a7933f", + register_all_versions = True, + ), + ], + defaults = [ + python_ext.defaults( + python_version = "3.11.99", + ), + ], + ) + + # Pre-populate mock_files directly to bypass download output struct key mismatch in mock read lookups. + mock_mctx = python_ext.mctx( + modules = [root_module], + mock_files = { + "runtime_manifest": """ +01e607cf764b97d4d5d6f69fd1ff3d8a9a162513dde5c39e98260fce40fe220a cpython-3.11.99+20260414-x86_64-unknown-linux-gnu-pgo+lto-full.tar.zst +8b14030dd3af9ea7f7c51b4c90feb04afd8a8f45435727e67b875270bd08f3bc cpython-3.11.99+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +""", + }, + ) + + res = parse_modules( + module_ctx = mock_mctx, + logger = _mock_logger, + ) + + tool_versions = res.config.default["tool_versions"] + env.expect.that_bool("3.11.99" in tool_versions).equals(True) + + version_info = tool_versions["3.11.99"] + + # Assert on the entire dictionary at once! + env.expect.that_dict(version_info).contains_exactly({ + "sha256": { + "x86_64-unknown-linux-gnu": "8b14030dd3af9ea7f7c51b4c90feb04afd8a8f45435727e67b875270bd08f3bc", + }, + "strip_prefix": { + "x86_64-unknown-linux-gnu": "python", + }, + "url": { + "x86_64-unknown-linux-gnu": [ + "https://github.com/astral-sh/python-build-standalone/releases/download/20260414/cpython-3.11.99+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz", + ], + }, + }) + +_tests.append(_test_dynamic_manifest_toolchains) + +def _test_dynamic_manifest_files(name): + rt_util.helper_target( + native.filegroup, + name = name + "_subject", + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_dynamic_manifest_files_impl, + ) + +def _test_dynamic_manifest_files_impl(env, target): + _ = target # @unused + + root_module = python_ext.module( + name = "runtime_manifests", + override = [ + python_ext.override( + add_runtime_manifest_files = [ + Label("//:SHA256SUMS"), + ], + base_urls = ["https://example.com/dl"], + register_all_versions = True, + ), + ], + defaults = [ + python_ext.defaults( + python_version = "3.12.99", + ), + ], + ) + + mock_mctx = python_ext.mctx( + modules = [root_module], + mock_files = { + str(Label("//:SHA256SUMS")): """ +01e607cf764b97d4d5d6f69fd1ff3d8a9a162513dde5c39e98260fce40fe220a cpython-3.12.99+20260414-x86_64-unknown-linux-gnu-pgo+lto-full.tar.zst +""", + }, + ) + + res = parse_modules( + module_ctx = mock_mctx, + logger = _mock_logger, + ) + + tool_versions = res.config.default["tool_versions"] + env.expect.that_bool("3.12.99" in tool_versions).equals(True) + + version_info = tool_versions["3.12.99"] + + env.expect.that_dict(version_info).contains_exactly({ + "sha256": { + "x86_64-unknown-linux-gnu": "01e607cf764b97d4d5d6f69fd1ff3d8a9a162513dde5c39e98260fce40fe220a", + }, + "strip_prefix": { + "x86_64-unknown-linux-gnu": "python/install", + }, + "url": { + "x86_64-unknown-linux-gnu": [ + "https://example.com/dl/cpython-3.12.99+20260414-x86_64-unknown-linux-gnu-pgo+lto-full.tar.zst", + ], + }, + }) + +_tests.append(_test_dynamic_manifest_files) + +def runtime_manifests_test_suite(name): + test_suite( + name = name, + tests = _tests, + ) diff --git a/tests/python_bzlmod_ext/test_helpers.bzl b/tests/python_bzlmod_ext/test_helpers.bzl new file mode 100644 index 0000000000..78ad57c110 --- /dev/null +++ b/tests/python_bzlmod_ext/test_helpers.bzl @@ -0,0 +1,48 @@ +# Copyright 2026 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Helpers to conditionally register tests depending on Bzlmod enablement.""" + +load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility +load(":parse_runtime_manifest_tests.bzl", "parse_runtime_manifest_test_suite") +load(":runtime_manifests_tests.bzl", "runtime_manifests_test_suite") + +def register_python_bzlmod_ext_tests(name, parse_runtime_manifest_name, runtime_manifests_name): + """Registers the Bzlmod extension tests if Bzlmod is enabled, otherwise defines empty test_suites. + + Args: + name: The name of the master test_suite target. + parse_runtime_manifest_name: The name of the parse_runtime_manifest test target. + runtime_manifests_name: The name of the runtime_manifests test target. + """ + if BZLMOD_ENABLED: + parse_runtime_manifest_test_suite(name = parse_runtime_manifest_name) + runtime_manifests_test_suite(name = runtime_manifests_name) + else: + native.test_suite( + name = parse_runtime_manifest_name, + tests = [], + ) + native.test_suite( + name = runtime_manifests_name, + tests = [], + ) + + native.test_suite( + name = name, + tests = [ + parse_runtime_manifest_name, + runtime_manifests_name, + ], + ) diff --git a/tests/repl/BUILD.bazel b/tests/repl/BUILD.bazel new file mode 100644 index 0000000000..b3986cc023 --- /dev/null +++ b/tests/repl/BUILD.bazel @@ -0,0 +1,44 @@ +load("//python:py_library.bzl", "py_library") +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") + +# A library that adds a special import path only when this is specified as a +# dependency. This makes it easy for a dependency to have this import path +# available without the top-level target being able to import the module. +py_library( + name = "helper/test_module", + srcs = [ + "helper/test_module.py", + ], + imports = [ + "helper", + ], +) + +py_reconfig_test( + name = "repl_without_dep_test", + srcs = ["repl_test.py"], + data = [ + "//python/bin:repl", + ], + env = { + # The helper/test_module should _not_ be importable for this test. + "EXPECT_TEST_MODULE_IMPORTABLE": "0", + }, + main = "repl_test.py", + python_version = "3.12", +) + +py_reconfig_test( + name = "repl_with_dep_test", + srcs = ["repl_test.py"], + data = [ + "//python/bin:repl", + ], + env = { + # The helper/test_module _should_ be importable for this test. + "EXPECT_TEST_MODULE_IMPORTABLE": "1", + }, + main = "repl_test.py", + python_version = "3.12", + repl_dep = ":helper/test_module", +) diff --git a/tests/repl/helper/test_module.py b/tests/repl/helper/test_module.py new file mode 100644 index 0000000000..0c4a309b01 --- /dev/null +++ b/tests/repl/helper/test_module.py @@ -0,0 +1,5 @@ +"""This is a file purely intended for validating //python/bin:repl.""" + + +def print_hello(): + print("Hello World") diff --git a/tests/repl/repl_test.py b/tests/repl/repl_test.py new file mode 100644 index 0000000000..2b3d5c7a4d --- /dev/null +++ b/tests/repl/repl_test.py @@ -0,0 +1,159 @@ +import os +import subprocess +import sys # noqa: F401 +import tempfile +import unittest +from pathlib import Path +from typing import Iterable + +from python import runfiles + +rfiles = runfiles.Create() + +# Signals the tests below whether we should be expecting the import of +# helpers/test_module.py on the REPL to work or not. +EXPECT_TEST_MODULE_IMPORTABLE = os.environ["EXPECT_TEST_MODULE_IMPORTABLE"] == "1" + + +# An arbitrary piece of code that sets some kind of variable. The variable needs to persist into the +# actual shell. +PYTHONSTARTUP_SETS_VAR = """\ +foo = 1234 +""" + +IS_WINDOWS = os.name == "nt" + + +class ReplTest(unittest.TestCase): + def setUp(self): + rpath = "rules_python/python/bin/repl" + if IS_WINDOWS: + rpath += ".exe" + self.repl = rfiles.Rlocation(rpath) + assert self.repl + if IS_WINDOWS: + self.repl = os.path.normpath(self.repl) + + def run_code_in_repl(self, lines: Iterable[str], *, env=None) -> str: + """Runs the lines of code in the REPL and returns the text output.""" + input = "\n".join(lines) + try: + return subprocess.check_output( + [self.repl], + text=True, + stderr=subprocess.STDOUT, + input=input, + env=env, + ).strip() + except subprocess.CalledProcessError as error: + raise RuntimeError(f"Failed to run the REPL:\n{error.stdout}") from error + except Exception as exc: + if env: + env_str = "\n".join( + f"{key}={value!r}" for key, value in sorted(env.items()) + ) + else: + env_str = "" + if isinstance(exc, subprocess.CalledProcessError): + stdout = exc.stdout + else: + stdout = "" + exc.add_note( + f""" +===== env start ===== +{env_str} +===== env end ===== +===== input start ===== +{input} +===== input end ===== +commmand: {self.repl} +===== stdout start ===== +{stdout} +===== stdout end ===== +""" + ) + raise + + def test_repl_version(self): + """Validates that we can successfully execute arbitrary code on the REPL.""" + + result = self.run_code_in_repl( + [ + "import sys", + "v = sys.version_info", + "print(f'version: {v.major}.{v.minor}')", + ] + ) + self.assertIn("version: 3.12", result) + + def test_cannot_import_test_module_directly(self): + """Validates that we cannot import helper/test_module.py since it's not a direct dep.""" + with self.assertRaises(ModuleNotFoundError): + import test_module # noqa: F401 + + @unittest.skipIf( + not EXPECT_TEST_MODULE_IMPORTABLE, "test only works without repl_dep set" + ) + def test_import_test_module_success(self): + """Validates that we can import helper/test_module.py when repl_dep is set.""" + result = self.run_code_in_repl( + [ + "import test_module", + "test_module.print_hello()", + ] + ) + self.assertIn("Hello World", result) + + @unittest.skipIf( + EXPECT_TEST_MODULE_IMPORTABLE, "test only works without repl_dep set" + ) + def test_import_test_module_failure(self): + """Validates that we cannot import helper/test_module.py when repl_dep isn't set.""" + result = self.run_code_in_repl( + [ + "import test_module", + ] + ) + self.assertIn("ModuleNotFoundError: No module named 'test_module'", result) + + def test_pythonstartup_gets_executed(self): + """Validates that we can use the variables from PYTHONSTARTUP in the console itself.""" + with tempfile.TemporaryDirectory() as tempdir: + pythonstartup = Path(tempdir) / "pythonstartup.py" + pythonstartup.write_text(PYTHONSTARTUP_SETS_VAR) + + env = os.environ.copy() + env["PYTHONSTARTUP"] = str(pythonstartup) + + result = self.run_code_in_repl( + [ + "print(f'The value of foo is {foo}')", + ], + env=env, + ) + + self.assertIn("The value of foo is 1234", result) + + def test_pythonstartup_doesnt_leak(self): + """Validates that we don't accidentally leak code into the console. + + This test validates that a few of the variables we use in the template and stub are not + accessible in the REPL itself. + """ + with tempfile.TemporaryDirectory() as tempdir: + pythonstartup = Path(tempdir) / "pythonstartup.py" + pythonstartup.write_text(PYTHONSTARTUP_SETS_VAR) + + env = os.environ.copy() + env["PYTHONSTARTUP"] = str(pythonstartup) + + for var_name in ("exitmsg", "sys", "code", "bazel_runfiles", "STUB_PATH"): + with self.subTest(var_name=var_name): + result = self.run_code_in_repl([f"print({var_name})"], env=env) + self.assertIn( + f"NameError: name '{var_name}' is not defined", result + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/repo_utils/BUILD.bazel b/tests/repo_utils/BUILD.bazel new file mode 100644 index 0000000000..74e8e37489 --- /dev/null +++ b/tests/repo_utils/BUILD.bazel @@ -0,0 +1,3 @@ +load(":repo_utils_test.bzl", "repo_utils_test_suite") + +repo_utils_test_suite(name = "repo_utils_tests") diff --git a/tests/repo_utils/repo_utils_test.bzl b/tests/repo_utils/repo_utils_test.bzl new file mode 100644 index 0000000000..56d125724d --- /dev/null +++ b/tests/repo_utils/repo_utils_test.bzl @@ -0,0 +1,132 @@ +"""Unit tests for repo_utils.bzl.""" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private:repo_utils.bzl", "repo_utils") # buildifier: disable=bzl-visibility +load("//tests/support/mocks:mocks.bzl", "mocks") + +_tests = [] + +def _make_rctx(*, os_name, mock_extracts = None, mock_files = None): + return mocks.rctx( + attr = { + "name": "unit", + "_rule_name": "test_rule", + }, + mock_files = mock_files, + mock_extracts = mock_extracts, + os_name = os_name, + ) + +def _test_get_platforms_os_name(env): + mock_mrctx = mocks.rctx(os_name = "Mac OS X") + got = repo_utils.get_platforms_os_name(mock_mrctx) + env.expect.that_str(got).equals("osx") + +_tests.append(_test_get_platforms_os_name) + +def _test_relative_to(env): + mock_mrctx_linux = mocks.rctx(os_name = "linux") + mock_mrctx_win = mocks.rctx(os_name = "windows") + + # Case-sensitive matching (Linux) + got = repo_utils.relative_to(mock_mrctx_linux, "foo/bar/baz", "foo/bar") + env.expect.that_str(got).equals("baz") + + # Case-insensitive matching (Windows) + got = repo_utils.relative_to(mock_mrctx_win, "C:/Foo/Bar/Baz", "c:/foo/bar") + env.expect.that_str(got).equals("Baz") + + # Failure case + failures = [] + + def _mock_fail(msg): + failures.append(msg) + + repo_utils.relative_to(mock_mrctx_linux, "foo/bar/baz", "qux", fail = _mock_fail) + env.expect.that_collection(failures).contains_exactly(["foo/bar/baz is not relative to qux"]) + +_tests.append(_test_relative_to) + +def _test_is_relative_to(env): + mock_mrctx_linux = mocks.rctx(os_name = "linux") + mock_mrctx_win = mocks.rctx(os_name = "windows") + + # Case-sensitive matching (Linux) + env.expect.that_bool(repo_utils.is_relative_to(mock_mrctx_linux, "foo/bar/baz", "foo/bar")).equals(True) + env.expect.that_bool(repo_utils.is_relative_to(mock_mrctx_linux, "foo/bar/baz", "qux")).equals(False) + + # Case-insensitive matching (Windows) + env.expect.that_bool(repo_utils.is_relative_to(mock_mrctx_win, "C:/Foo/Bar/Baz", "c:/foo/bar")).equals(True) + env.expect.that_bool(repo_utils.is_relative_to(mock_mrctx_win, "C:/Foo/Bar/Baz", "D:/Foo")).equals(False) + +_tests.append(_test_is_relative_to) + +def _test_extract_calls_chmod_when_enabled(env): + mock_rctx = _make_rctx( + os_name = "linux", + mock_extracts = {"test.whl": {"f1": "c1"}}, + ) + + repo_utils.extract( + mock_rctx, + archive = mock_rctx.path("test.whl"), + output = "out", + supports_whl_extraction = True, + extract_needs_chmod = True, + ) + + env.expect.that_bool(len(mock_rctx.execute_calls) > 0).equals(True) + env.expect.that_str(mock_rctx.execute_calls[0][0]).equals("chmod") + +_tests.append(_test_extract_calls_chmod_when_enabled) + +def _test_extract_skips_chmod_when_disabled(env): + mock_rctx = _make_rctx( + os_name = "linux", + mock_extracts = {"test.whl": {"f1": "c1"}}, + ) + + repo_utils.extract( + mock_rctx, + archive = mock_rctx.path("test.whl"), + output = "out", + supports_whl_extraction = True, + extract_needs_chmod = False, + ) + + env.expect.that_collection(mock_rctx.execute_calls).contains_exactly([]) + +_tests.append(_test_extract_skips_chmod_when_disabled) + +def _test_maybe_fix_permissions_calls_chmod_on_linux(env): + mock_rctx = _make_rctx(os_name = "linux") + + repo_utils.maybe_fix_permissions( + mock_rctx, + whl_path = mock_rctx.path("test.whl"), + ) + + env.expect.that_bool(len(mock_rctx.execute_calls) > 0).equals(True) + env.expect.that_str(mock_rctx.execute_calls[0][0]).equals("chmod") + +_tests.append(_test_maybe_fix_permissions_calls_chmod_on_linux) + +def _test_maybe_fix_permissions_skips_on_windows(env): + mock_rctx = _make_rctx(os_name = "windows") + + repo_utils.maybe_fix_permissions( + mock_rctx, + whl_path = mock_rctx.path("test.whl"), + ) + + env.expect.that_collection(mock_rctx.execute_calls).contains_exactly([]) + +_tests.append(_test_maybe_fix_permissions_skips_on_windows) + +def repo_utils_test_suite(name): + """Create the test suite. + + Args: + name: the name of the test suite + """ + test_suite(name = name, basic_tests = _tests) diff --git a/tests/repos/BUILD.bazel b/tests/repos/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/repos/pkgutil_nspkg1/BUILD.bazel b/tests/repos/pkgutil_nspkg1/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/repos/pkgutil_nspkg1/nspkg/__init__.py b/tests/repos/pkgutil_nspkg1/nspkg/__init__.py new file mode 100644 index 0000000000..c4da2cf0b7 --- /dev/null +++ b/tests/repos/pkgutil_nspkg1/nspkg/__init__.py @@ -0,0 +1,2 @@ +# __init__.py +__path__ = __import__("pkgutil").extend_path(__path__, __name__) diff --git a/tests/repos/pkgutil_nspkg1/nspkg/one/a.txt b/tests/repos/pkgutil_nspkg1/nspkg/one/a.txt new file mode 100644 index 0000000000..f4dbe63934 --- /dev/null +++ b/tests/repos/pkgutil_nspkg1/nspkg/one/a.txt @@ -0,0 +1 @@ +dummy content \ No newline at end of file diff --git a/tests/repos/pkgutil_nspkg1/pkgutil_nspkg1-1.0.dist-info/METADATA b/tests/repos/pkgutil_nspkg1/pkgutil_nspkg1-1.0.dist-info/METADATA new file mode 100644 index 0000000000..7ffb60b701 --- /dev/null +++ b/tests/repos/pkgutil_nspkg1/pkgutil_nspkg1-1.0.dist-info/METADATA @@ -0,0 +1,3 @@ +Metadata-Version: 2.1 +Name: pkgutil-nspkg1 +Version: 1.0 diff --git a/tests/repos/pkgutil_nspkg1/pkgutil_nspkg1-1.0.dist-info/RECORD b/tests/repos/pkgutil_nspkg1/pkgutil_nspkg1-1.0.dist-info/RECORD new file mode 100644 index 0000000000..e039fee1ae --- /dev/null +++ b/tests/repos/pkgutil_nspkg1/pkgutil_nspkg1-1.0.dist-info/RECORD @@ -0,0 +1,5 @@ +nspkg/__init__.py,sha256=d10f14d9ce938ae14416c3c1f4b516c4f40dfe0c9bf973a833e4ef40517ba7d0,81 +nspkg/one/a.txt,sha256=bf0ecbdb9b814248d086c9b69cf26182d9d4138f2ad3d0637c4555fc8cbf68e5,13 +pkgutil_nspkg1-1.0.dist-info/METADATA,sha256=49525c3e6f1fc8f46d9c92c996e35f81327f9fe417df2d9d10784fa4e9cfe84b,59 +pkgutil_nspkg1-1.0.dist-info/WHEEL,sha256=d652ec50af6f144788dc1ffef052e8833b6704e98818e6cf78d80a625ad498fb,105 +pkgutil_nspkg1-1.0.dist-info/RECORD,, diff --git a/tests/repos/pkgutil_nspkg1/pkgutil_nspkg1-1.0.dist-info/WHEEL b/tests/repos/pkgutil_nspkg1/pkgutil_nspkg1-1.0.dist-info/WHEEL new file mode 100644 index 0000000000..dad2b72544 --- /dev/null +++ b/tests/repos/pkgutil_nspkg1/pkgutil_nspkg1-1.0.dist-info/WHEEL @@ -0,0 +1,4 @@ +Wheel-Version: 1.0 +Generator: rules_python_whl_from_dir_repo +Root-Is-Purelib: true +Tag: py3-none-any diff --git a/tests/repos/pkgutil_nspkg2/BUILD.bazel b/tests/repos/pkgutil_nspkg2/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/repos/pkgutil_nspkg2/nspkg/__init__.py b/tests/repos/pkgutil_nspkg2/nspkg/__init__.py new file mode 100644 index 0000000000..c4da2cf0b7 --- /dev/null +++ b/tests/repos/pkgutil_nspkg2/nspkg/__init__.py @@ -0,0 +1,2 @@ +# __init__.py +__path__ = __import__("pkgutil").extend_path(__path__, __name__) diff --git a/tests/repos/pkgutil_nspkg2/nspkg/two/b.txt b/tests/repos/pkgutil_nspkg2/nspkg/two/b.txt new file mode 100644 index 0000000000..f4dbe63934 --- /dev/null +++ b/tests/repos/pkgutil_nspkg2/nspkg/two/b.txt @@ -0,0 +1 @@ +dummy content \ No newline at end of file diff --git a/tests/repos/pkgutil_nspkg2/pkgutil_nspkg2-1.0.dist-info/METADATA b/tests/repos/pkgutil_nspkg2/pkgutil_nspkg2-1.0.dist-info/METADATA new file mode 100644 index 0000000000..368e64ce19 --- /dev/null +++ b/tests/repos/pkgutil_nspkg2/pkgutil_nspkg2-1.0.dist-info/METADATA @@ -0,0 +1,3 @@ +Metadata-Version: 2.1 +Name: pkgutil-nspkg2 +Version: 1.0 diff --git a/tests/repos/pkgutil_nspkg2/pkgutil_nspkg2-1.0.dist-info/RECORD b/tests/repos/pkgutil_nspkg2/pkgutil_nspkg2-1.0.dist-info/RECORD new file mode 100644 index 0000000000..c93970cb26 --- /dev/null +++ b/tests/repos/pkgutil_nspkg2/pkgutil_nspkg2-1.0.dist-info/RECORD @@ -0,0 +1,5 @@ +nspkg/__init__.py,sha256=d10f14d9ce938ae14416c3c1f4b516c4f40dfe0c9bf973a833e4ef40517ba7d0,81 +nspkg/two/b.txt,sha256=bf0ecbdb9b814248d086c9b69cf26182d9d4138f2ad3d0637c4555fc8cbf68e5,13 +pkgutil_nspkg2-1.0.dist-info/METADATA,sha256=9a72654b480f17c55df07fe27d026c4ea461c493babcc675f26fbcfce40828b6,59 +pkgutil_nspkg2-1.0.dist-info/WHEEL,sha256=d652ec50af6f144788dc1ffef052e8833b6704e98818e6cf78d80a625ad498fb,105 +pkgutil_nspkg2-1.0.dist-info/RECORD,, diff --git a/tests/repos/pkgutil_nspkg2/pkgutil_nspkg2-1.0.dist-info/WHEEL b/tests/repos/pkgutil_nspkg2/pkgutil_nspkg2-1.0.dist-info/WHEEL new file mode 100644 index 0000000000..dad2b72544 --- /dev/null +++ b/tests/repos/pkgutil_nspkg2/pkgutil_nspkg2-1.0.dist-info/WHEEL @@ -0,0 +1,4 @@ +Wheel-Version: 1.0 +Generator: rules_python_whl_from_dir_repo +Root-Is-Purelib: true +Tag: py3-none-any diff --git a/tests/repos/whl_with_data1/BUILD.bazel b/tests/repos/whl_with_data1/BUILD.bazel new file mode 100644 index 0000000000..7ef8ba4cd9 --- /dev/null +++ b/tests/repos/whl_with_data1/BUILD.bazel @@ -0,0 +1 @@ +exports_files(glob(["**"])) diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/bin/data_overlap.sh b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/bin/data_overlap.sh new file mode 100644 index 0000000000..b47ce4e9f3 --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/bin/data_overlap.sh @@ -0,0 +1 @@ +echo data_bin diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/include/data_overlap.h b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/include/data_overlap.h new file mode 100644 index 0000000000..299c39d0a7 --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/include/data_overlap.h @@ -0,0 +1 @@ +/* data_include */ diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/overlap/both.txt b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/overlap/both.txt new file mode 100644 index 0000000000..771c76ed7b --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/overlap/both.txt @@ -0,0 +1 @@ +both1 diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/overlap/data1.txt b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/overlap/data1.txt new file mode 100644 index 0000000000..d760283f59 --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/overlap/data1.txt @@ -0,0 +1 @@ +data1 diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/site-packages/data_overlap.py b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/site-packages/data_overlap.py new file mode 100644 index 0000000000..d3ee4d8a3f --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/site-packages/data_overlap.py @@ -0,0 +1 @@ +# data_site_packages diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/whl_with_data1/data_data_file.txt b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/whl_with_data1/data_data_file.txt new file mode 100644 index 0000000000..39ec676600 --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/whl_with_data1/data_data_file.txt @@ -0,0 +1 @@ +from .data/data diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/data_overlap.h b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/data_overlap.h new file mode 100644 index 0000000000..ffd49d0cee --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/data_overlap.h @@ -0,0 +1 @@ +/* headers */ diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/overlap/both.h b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/overlap/both.h new file mode 100644 index 0000000000..49f33a8c6e --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/overlap/both.h @@ -0,0 +1 @@ +both diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/overlap/header1.h b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/overlap/header1.h new file mode 100644 index 0000000000..412e9ed7df --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/overlap/header1.h @@ -0,0 +1 @@ +header1 diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/whl_with_data1/header_file.h b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/whl_with_data1/header_file.h new file mode 100644 index 0000000000..59c9bf78c2 --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/whl_with_data1/header_file.h @@ -0,0 +1 @@ +from .data/headers diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/platlib/whl_with_data1/platlib_file.txt b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/platlib/whl_with_data1/platlib_file.txt new file mode 100644 index 0000000000..b27295614f --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/platlib/whl_with_data1/platlib_file.txt @@ -0,0 +1 @@ +from .data/platlib diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/purelib/data_overlap.py b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/purelib/data_overlap.py new file mode 100644 index 0000000000..f82e46670f --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/purelib/data_overlap.py @@ -0,0 +1 @@ +# purelib diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/purelib/whl_with_data1/__init__.py b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/purelib/whl_with_data1/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/purelib/whl_with_data1/data_file.txt b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/purelib/whl_with_data1/data_file.txt new file mode 100644 index 0000000000..e547fe48ed --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/purelib/whl_with_data1/data_file.txt @@ -0,0 +1 @@ +from .data diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/data_overlap.sh b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/data_overlap.sh new file mode 100644 index 0000000000..d6eb28dc3d --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/data_overlap.sh @@ -0,0 +1 @@ +echo scripts diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/overlap/both.sh b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/overlap/both.sh new file mode 100644 index 0000000000..49f33a8c6e --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/overlap/both.sh @@ -0,0 +1 @@ +both diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/overlap/script1.sh b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/overlap/script1.sh new file mode 100644 index 0000000000..4d68a2e3e0 --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/overlap/script1.sh @@ -0,0 +1 @@ +script1 diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/whl_script.sh b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/whl_script.sh new file mode 100644 index 0000000000..1a2485251c --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/whl_script.sh @@ -0,0 +1 @@ +#!/bin/sh diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/whl_with_data1_pythonw b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/whl_with_data1_pythonw new file mode 100755 index 0000000000..6c7b3434c5 --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/whl_with_data1_pythonw @@ -0,0 +1,9 @@ +#!pythonw +import sys + +# On Windows, pythonw doesn't have stdout/stderr streams, +# so output has to be written to a file. +with open(sys.argv[1], "w") as fp: + fp.write("hello from whl_with_data1_pythonw\n") + fp.write(sys.executable) + fp.write("\n") diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/whl_with_data1_script b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/whl_with_data1_script new file mode 100755 index 0000000000..8af40a9a55 --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/whl_with_data1_script @@ -0,0 +1,5 @@ +#!python +import sys + +print("hello from whl_with_data1_script") +print(sys.executable) diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/METADATA b/tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/METADATA new file mode 100644 index 0000000000..f403970d7a --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/METADATA @@ -0,0 +1,3 @@ +Metadata-Version: 2.1 +Name: whl-with-data1 +Version: 1.0 diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/RECORD b/tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/RECORD new file mode 100644 index 0000000000..10307c76a0 --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/RECORD @@ -0,0 +1,19 @@ +whl_with_data1-1.0.data/platlib/whl_with_data1/platlib_file.txt,sha256=123,123 +whl_with_data1-1.0.data/scripts/whl_with_data1_script,sha256=123,123 +whl_with_data1-1.0.data/scripts/whl_script.sh,sha256=123,123 +whl_with_data1-1.0.data/headers/whl_with_data1/header_file.h,sha256=123,123 +whl_with_data1-1.0.data/purelib/whl_with_data1/data_file.txt,sha256=123,123 +whl_with_data1-1.0.data/data/whl_with_data1/data_data_file.txt,sha256=123,123 +whl_with_data1-1.0.data/data/whl_with_data1/data_data_file.txt,sha256=123,123 +whl_with_data1-1.0.data/data/overlap/both.txt,sha256=123,123 +whl_with_data1-1.0.data/data/overlap/data1.txt,sha256=123,123 +whl_with_data1-1.0.data/scripts/overlap/both.sh,sha256=123,123 +whl_with_data1-1.0.data/scripts/overlap/script1.sh,sha256=123,123 +whl_with_data1-1.0.data/headers/overlap/both.h,sha256=123,123 +whl_with_data1-1.0.data/headers/overlap/header1.h,sha256=123,123 +whl_with_data1-1.0.data/scripts/data_overlap.sh,sha256=123,123 +whl_with_data1-1.0.data/data/bin/data_overlap.sh,sha256=123,123 +whl_with_data1-1.0.data/headers/data_overlap.h,sha256=123,123 +whl_with_data1-1.0.data/data/include/data_overlap.h,sha256=123,123 +whl_with_data1-1.0.data/purelib/data_overlap.py,sha256=123,123 +whl_with_data1-1.0.data/data/site-packages/data_overlap.py,sha256=123,123 diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/WHEEL b/tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/WHEEL new file mode 100644 index 0000000000..a64521a1cc --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/WHEEL @@ -0,0 +1 @@ +Wheel-Version: 1.0 diff --git a/tests/repos/whl_with_data2/BUILD.bazel b/tests/repos/whl_with_data2/BUILD.bazel new file mode 100644 index 0000000000..af49d1ebbf --- /dev/null +++ b/tests/repos/whl_with_data2/BUILD.bazel @@ -0,0 +1 @@ +exports_files(glob(["*"])) diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/data/overlap/both.txt b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/data/overlap/both.txt new file mode 100644 index 0000000000..1a8aa8b533 --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/data/overlap/both.txt @@ -0,0 +1 @@ +both2 diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/data/overlap/data2.txt b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/data/overlap/data2.txt new file mode 100644 index 0000000000..98d81a2ec6 --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/data/overlap/data2.txt @@ -0,0 +1 @@ +data2 diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/data/whl_with_data2/data_data_file.txt b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/data/whl_with_data2/data_data_file.txt new file mode 100644 index 0000000000..39ec676600 --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/data/whl_with_data2/data_data_file.txt @@ -0,0 +1 @@ +from .data/data diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/headers/overlap/both.h b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/headers/overlap/both.h new file mode 100644 index 0000000000..49f33a8c6e --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/headers/overlap/both.h @@ -0,0 +1 @@ +both diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/headers/overlap/header2.h b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/headers/overlap/header2.h new file mode 100644 index 0000000000..da0a719745 --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/headers/overlap/header2.h @@ -0,0 +1 @@ +header2 diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/headers/whl_with_data2/header_file.h b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/headers/whl_with_data2/header_file.h new file mode 100644 index 0000000000..59c9bf78c2 --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/headers/whl_with_data2/header_file.h @@ -0,0 +1 @@ +from .data/headers diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/platlib/whl_with_data2/platlib_file.txt b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/platlib/whl_with_data2/platlib_file.txt new file mode 100644 index 0000000000..b27295614f --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/platlib/whl_with_data2/platlib_file.txt @@ -0,0 +1 @@ +from .data/platlib diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/purelib/whl_with_data2/__init__.py b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/purelib/whl_with_data2/__init__.py new file mode 100644 index 0000000000..45132c14d7 --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/purelib/whl_with_data2/__init__.py @@ -0,0 +1,6 @@ +import sys + + +def main(): + print("hello from whl_with_data2_bin") + print(sys.executable) diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/purelib/whl_with_data2/data_file.txt b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/purelib/whl_with_data2/data_file.txt new file mode 100644 index 0000000000..e547fe48ed --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/purelib/whl_with_data2/data_file.txt @@ -0,0 +1 @@ +from .data diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/scripts/overlap/both.sh b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/scripts/overlap/both.sh new file mode 100644 index 0000000000..49f33a8c6e --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/scripts/overlap/both.sh @@ -0,0 +1 @@ +both diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/scripts/overlap/script2.sh b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/scripts/overlap/script2.sh new file mode 100644 index 0000000000..026ed8b62a --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/scripts/overlap/script2.sh @@ -0,0 +1 @@ +script2 diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/scripts/whl_script.sh b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/scripts/whl_script.sh new file mode 100644 index 0000000000..1a2485251c --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/scripts/whl_script.sh @@ -0,0 +1 @@ +#!/bin/sh diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/METADATA b/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/METADATA new file mode 100644 index 0000000000..c762d184fb --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/METADATA @@ -0,0 +1,3 @@ +Metadata-Version: 2.1 +Name: whl-with-data2 +Version: 1.0 diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/RECORD b/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/RECORD new file mode 100644 index 0000000000..55c70740c8 --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/RECORD @@ -0,0 +1,14 @@ +whl_with_data2-1.0.data/platlib/whl_with_data2/platlib_file.txt,sha256=123,123 +whl_with_data2-1.0.data/scripts/whl_script.sh,sha256=123,123 +whl_with_data2-1.0.data/headers/whl_with_data2/header_file.h,sha256=123,123 +whl_with_data2-1.0.data/purelib/whl_with_data2/data_file.txt,sha256=123,123 +whl_with_data2-1.0.data/data/whl_with_data2/data_data_file.txt,sha256=123,123 +whl_with_data2-1.0.data/data/whl_with_data2/data_data_file.txt,sha256=123,123 +whl_with_data2-1.0.data/data/overlap/both.txt,sha256=123,123 +whl_with_data2-1.0.data/data/overlap/data2.txt,sha256=123,123 +whl_with_data2-1.0.data/scripts/overlap/both.sh,sha256=123,123 +whl_with_data2-1.0.data/scripts/overlap/script2.sh,sha256=123,123 +whl_with_data2-1.0.data/headers/overlap/both.h,sha256=123,123 +whl_with_data2-1.0.data/headers/overlap/header2.h,sha256=123,123 +whl_with_data2-1.0.data/purelib/whl_with_data2/__init__.py,sha256=123,123 +whl_with_data2-1.0.dist-info/entry_points.txt,sha256=123,123 diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/WHEEL b/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/WHEEL new file mode 100644 index 0000000000..a64521a1cc --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/WHEEL @@ -0,0 +1 @@ +Wheel-Version: 1.0 diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/entry_points.txt b/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/entry_points.txt new file mode 100644 index 0000000000..8389a8a826 --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/entry_points.txt @@ -0,0 +1,2 @@ +[console_scripts] +whl_with_data2_bin = whl_with_data2:main diff --git a/tests/runfiles/BUILD.bazel b/tests/runfiles/BUILD.bazel index 5c92026082..505b3c17c5 100644 --- a/tests/runfiles/BUILD.bazel +++ b/tests/runfiles/BUILD.bazel @@ -5,9 +5,40 @@ load("@rules_python//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # bu py_test( name = "runfiles_test", srcs = ["runfiles_test.py"], + data = [ + "//tests/support:current_build_settings", + ], + env = { + "BZLMOD_ENABLED": "1" if BZLMOD_ENABLED else "0", + }, + deps = ["//python/runfiles"], +) + +py_test( + name = "runfiles_min_python_test", + srcs = ["runfiles_test.py"], + data = [ + "//tests/support:current_build_settings", + ], env = { "BZLMOD_ENABLED": "1" if BZLMOD_ENABLED else "0", }, + main = "runfiles_test.py", + python_version = "3.10", + deps = ["//python/runfiles"], +) + +py_test( + name = "pathlib_test", + srcs = ["pathlib_test.py"], + deps = ["//python/runfiles"], +) + +py_test( + name = "pathlib_min_python_test", + srcs = ["pathlib_test.py"], + main = "pathlib_test.py", + python_version = "3.10", deps = ["//python/runfiles"], ) diff --git a/tests/runfiles/pathlib_test.py b/tests/runfiles/pathlib_test.py new file mode 100644 index 0000000000..a959138235 --- /dev/null +++ b/tests/runfiles/pathlib_test.py @@ -0,0 +1,193 @@ +import os +import pathlib +import tempfile +import unittest + +from python.runfiles import runfiles + + +class PathlibTest(unittest.TestCase): + def setUp(self) -> None: + self.tmpdir = tempfile.TemporaryDirectory(dir=os.environ.get("TEST_TMPDIR")) + # Runfiles paths are expected to be posix paths internally when we + # construct the strings for assertions + self.root_path = pathlib.Path(self.tmpdir.name).resolve() + self.root_dir = self.root_path.as_posix() + + # Create dummy files for I/O tests + self.repo_dir = self.root_path / "my_repo" + self.repo_dir.mkdir() + self.test_file = self.repo_dir / "data.txt" + self.test_file.write_text("hello runfiles", encoding="utf-8") + self.sub_dir = self.repo_dir / "subdir" + self.sub_dir.mkdir() + (self.sub_dir / "other.txt").write_text("other content", encoding="utf-8") + + def _create_runfiles(self) -> runfiles.Runfiles: + r = runfiles.Create({"RUNFILES_DIR": self.root_dir}) + assert r is not None + return r + + def tearDown(self) -> None: + self.tmpdir.cleanup() + + def test_path_api(self) -> None: + r = self._create_runfiles() + root = r.root() + + # Test basic joining + p = root / "repo/pkg/file.txt" + self.assertEqual(str(p), f"{self.root_dir}/repo/pkg/file.txt") + + # Test PurePath API + self.assertEqual(p.name, "file.txt") + self.assertEqual(p.suffix, ".txt") + self.assertEqual(p.parent.name, "pkg") + self.assertEqual(p.parts, ("repo", "pkg", "file.txt")) + self.assertEqual(p.stem, "file") + self.assertEqual(p.suffixes, [".txt"]) + + # Test multiple joins + p2 = root / "repo" / "pkg" / "file.txt" + self.assertEqual(p, p2) + + # Test joins with pathlib objects + p3 = root / pathlib.PurePath("repo/pkg/file.txt") + self.assertEqual(p, p3) + + def test_root(self) -> None: + r = self._create_runfiles() + self.assertEqual(str(r.root()), self.root_dir) + + def test_runfiles_root_method(self) -> None: + r = self._create_runfiles() + p = r.root() / "foo/bar" + self.assertEqual(p.runfiles_root(), r.root()) + self.assertEqual(str(p.runfiles_root()), self.root_dir) + + def test_os_path_like(self) -> None: + r = self._create_runfiles() + p = r.root() / "foo" + self.assertEqual(os.fspath(p), f"{self.root_dir}/foo") + + def test_equality_and_hash(self) -> None: + r = self._create_runfiles() + p1 = r.root() / "foo" + p2 = r.root() / "foo" + p3 = r.root() / "bar" + + self.assertEqual(p1, p2) + self.assertNotEqual(p1, p3) + self.assertEqual(hash(p1), hash(p2)) + + def test_join_path(self) -> None: + r = self._create_runfiles() + p = r.root().joinpath("repo", "file") + self.assertEqual(str(p), f"{self.root_dir}/repo/file") + + def test_parents(self) -> None: + r = self._create_runfiles() + p = r.root() / "a/b/c" + parents = list(p.parents) + self.assertEqual(len(parents), 3) + self.assertEqual(str(parents[0]), f"{self.root_dir}/a/b") + self.assertEqual(str(parents[1]), f"{self.root_dir}/a") + self.assertEqual(str(parents[2]), self.root_dir) + + def test_with_methods(self) -> None: + r = self._create_runfiles() + p = r.root() / "foo/bar.txt" + self.assertEqual(str(p.with_name("baz.py")), f"{self.root_dir}/foo/baz.py") + self.assertEqual(str(p.with_suffix(".dat")), f"{self.root_dir}/foo/bar.dat") + + def test_match(self) -> None: + r = self._create_runfiles() + p = r.root() / "foo/bar.txt" + self.assertTrue(p.match("*.txt")) + self.assertTrue(p.match("foo/*.txt")) + self.assertFalse(p.match("bar/*.txt")) + + def test_reading_api(self) -> None: + r = self._create_runfiles() + root = r.root() + p = root / "my_repo/data.txt" + + self.assertTrue(p.exists()) + self.assertTrue(p.is_file()) + self.assertEqual(p.read_text(encoding="utf-8"), "hello runfiles") + self.assertEqual(p.read_bytes(), b"hello runfiles") + + with p.open("r", encoding="utf-8") as f: + self.assertEqual(f.read(), "hello runfiles") + + def test_stat_api(self) -> None: + r = self._create_runfiles() + root = r.root() + p = root / "my_repo/data.txt" + + st = p.stat() + self.assertEqual(st.st_size, len("hello runfiles")) + + def test_iteration_api(self) -> None: + r = self._create_runfiles() + root = r.root() + p = root / "my_repo" + + self.assertTrue(p.is_dir()) + contents = {c.name for c in p.iterdir()} + self.assertEqual(contents, {"data.txt", "subdir"}) + # Ensure they are still runfiles.Path + for c in p.iterdir(): + self.assertIsInstance(c, runfiles.Path) + + def test_glob(self) -> None: + r = self._create_runfiles() + root = r.root() + p = root / "my_repo" + + glob_results = { + pathlib.PurePath(c).relative_to(pathlib.PurePath(p)).as_posix() + for c in p.glob("*.txt") + } + self.assertEqual(glob_results, {"data.txt"}) + + rglob_results = { + pathlib.PurePath(c).relative_to(pathlib.PurePath(p)).as_posix() + for c in p.rglob("*.txt") + } + self.assertEqual(rglob_results, {"data.txt", "subdir/other.txt"}) + + for c in p.rglob("*.txt"): + self.assertIsInstance(c, runfiles.Path) + + def test_resolve_and_absolute(self) -> None: + r = self._create_runfiles() + root = r.root() + p = root / "my_repo/data.txt" + + resolved = p.resolve() + self.assertIsInstance(resolved, runfiles.Path) + self.assertTrue(resolved.exists()) + self.assertEqual(resolved.read_text(encoding="utf-8"), "hello runfiles") + + absoluted = p.absolute() + self.assertIsInstance(absoluted, runfiles.Path) + self.assertTrue(absoluted.exists()) + self.assertEqual(absoluted.read_text(encoding="utf-8"), "hello runfiles") + + def test_runfile_path(self) -> None: + r = self._create_runfiles() + root = r.root() + p = root / "my_repo/data.txt" + self.assertEqual(p.runfile_path, "my_repo/data.txt") + + p2 = root / "foo" / "bar.txt" + self.assertEqual(p2.runfile_path, "foo/bar.txt") + + self.assertEqual(root.runfile_path, "") + self.assertEqual(repr(root), "runfiles.Path('')") + self.assertEqual(repr(p), "runfiles.Path('my_repo/data.txt')") + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/runfiles/runfiles_test.py b/tests/runfiles/runfiles_test.py index cf6a70a020..38a89ede7e 100644 --- a/tests/runfiles/runfiles_test.py +++ b/tests/runfiles/runfiles_test.py @@ -12,12 +12,15 @@ # See the License for the specific language governing permissions and # limitations under the License. +import json import os +import pathlib import tempfile import unittest from typing import Any, List, Optional from python.runfiles import runfiles +from python.runfiles.runfiles import _RepositoryMapping class RunfilesTest(unittest.TestCase): @@ -62,6 +65,16 @@ def testRlocationArgumentValidation(self) -> None: lambda: r.Rlocation("\\foo"), ) + def testRlocationWithData(self) -> None: + r = runfiles.Create() + assert r is not None # mypy doesn't understand the unittest api. + settings_path = r.Rlocation( + "rules_python/tests/support/current_build_settings.json" + ) + assert settings_path is not None + settings = json.loads(pathlib.Path(settings_path).read_text()) + self.assertIn("bootstrap_impl", settings) + def testCreatesManifestBasedRunfiles(self) -> None: with _MockFile(contents=["a/b c/d"]) as mf: r = runfiles.Create( @@ -182,6 +195,62 @@ def testFailsToCreateAnyRunfilesBecauseEnvvarsAreNotDefined(self) -> None: self.assertIsNone(runfiles.Create({"TEST_SRCDIR": "always ignored"})) self.assertIsNone(runfiles.Create({"FOO": "bar"})) + def testCreatesManifestBasedRunfilesWithCreateOrRaise(self) -> None: + with _MockFile(contents=["a/b c/d"]) as mf: + r = runfiles.CreateOrRaise( + { + "RUNFILES_MANIFEST_FILE": mf.Path(), + "RUNFILES_DIR": "ignored when RUNFILES_MANIFEST_FILE has a value", + "TEST_SRCDIR": "always ignored", + } + ) + self.assertEqual(r.Rlocation("a/b"), "c/d") + self.assertIsNone(r.Rlocation("foo")) + + r_class = runfiles.Runfiles.CreateOrRaise( + { + "RUNFILES_MANIFEST_FILE": mf.Path(), + } + ) + self.assertEqual(r_class.Rlocation("a/b"), "c/d") + + def testCreatesDirectoryBasedRunfilesWithCreateOrRaise(self) -> None: + r = runfiles.CreateOrRaise( + { + "RUNFILES_DIR": "runfiles/dir", + "TEST_SRCDIR": "always ignored", + } + ) + self.assertEqual(r.Rlocation("a/b"), "runfiles/dir/a/b") + self.assertEqual(r.Rlocation("foo"), "runfiles/dir/foo") + + r_class = runfiles.Runfiles.CreateOrRaise( + { + "RUNFILES_DIR": "runfiles/dir", + } + ) + self.assertEqual(r_class.Rlocation("a/b"), "runfiles/dir/a/b") + + def testFailsToCreateManifestBasedBecauseManifestDoesNotExistWithCreateOrRaise( + self, + ) -> None: + def _Run(): + runfiles.CreateOrRaise({"RUNFILES_MANIFEST_FILE": "non-existing path"}) + + self.assertRaisesRegex(IOError, "non-existing path", _Run) + + def testFailsToCreateAnyRunfilesWithCreateOrRaise(self) -> None: + with self.assertRaises(RuntimeError): + runfiles.CreateOrRaise({"TEST_SRCDIR": "always ignored"}) + with self.assertRaises(RuntimeError): + runfiles.CreateOrRaise({"FOO": "bar"}) + with self.assertRaises(RuntimeError): + runfiles.CreateOrRaise({}) + with self.assertRaises(RuntimeError): + runfiles.Runfiles.CreateOrRaise({"TEST_SRCDIR": "always ignored"}) + with self.assertRaises(RuntimeError): + runfiles.Runfiles.CreateOrRaise({}) + def testManifestBasedRlocation(self) -> None: with _MockFile( contents=[ @@ -525,6 +594,165 @@ def testDirectoryBasedRlocationWithRepoMappingFromOtherRepo(self) -> None: r.Rlocation("config.json", "protobuf~3.19.2"), dir + "/config.json" ) + def testDirectoryBasedRlocationWithCompactRepoMappingFromMain(self) -> None: + """Test repository mapping with prefix-based entries (compact format).""" + with _MockFile( + name="_repo_mapping", + contents=[ + # Exact mappings (no asterisk) + "_,config.json,config.json~1.2.3", + ",my_module,_main", + ",my_workspace,_main", + # Prefixed mappings (with asterisk) - these apply to any repo starting with the prefix + "deps+*,external_dep,external_dep~1.0.0", + "test_deps+*,test_lib,test_lib~2.1.0", + ], + ) as rm: + dir = os.path.dirname(rm.Path()) + r = runfiles.CreateDirectoryBased(dir) + + # Test exact mappings still work + self.assertEqual( + r.Rlocation("my_module/bar/runfile", ""), dir + "/_main/bar/runfile" + ) + self.assertEqual( + r.Rlocation("my_workspace/bar/runfile", ""), dir + "/_main/bar/runfile" + ) + + # Test prefixed mappings - should match any repo starting with "deps+" + self.assertEqual( + r.Rlocation("external_dep/foo/file", "deps+dep1"), + dir + "/external_dep~1.0.0/foo/file", + ) + self.assertEqual( + r.Rlocation("external_dep/bar/file", "deps+dep2"), + dir + "/external_dep~1.0.0/bar/file", + ) + self.assertEqual( + r.Rlocation("external_dep/nested/path/file", "deps+some_long_dep_name"), + dir + "/external_dep~1.0.0/nested/path/file", + ) + + # Test that prefixed mappings work for test_deps+ prefix too + self.assertEqual( + r.Rlocation("test_lib/test/file", "test_deps+junit"), + dir + "/test_lib~2.1.0/test/file", + ) + + # Test that non-matching prefixes don't match + self.assertEqual( + r.Rlocation("external_dep/foo/file", "other_prefix"), + dir + "/external_dep/foo/file", # No mapping applied, use as-is + ) + + def testDirectoryBasedRlocationWithCompactRepoMappingPrecedence(self) -> None: + """Test that exact mappings take precedence over prefixed mappings.""" + with _MockFile( + name="_repo_mapping", + contents=[ + # Exact mapping for a specific source repo + "deps+specific_repo,external_dep,external_dep~exact", + # Prefixed mapping for repos starting with "deps+" + "deps+*,external_dep,external_dep~prefix", + # Another prefixed mapping with different prefix + "other+*,external_dep,external_dep~other", + ], + ) as rm: + dir = os.path.dirname(rm.Path()) + r = runfiles.CreateDirectoryBased(dir) + + # Exact mapping should take precedence over prefix + self.assertEqual( + r.Rlocation("external_dep/foo/file", "deps+specific_repo"), + dir + "/external_dep~exact/foo/file", + ) + + # Other repos with deps+ prefix should use the prefixed mapping + self.assertEqual( + r.Rlocation("external_dep/foo/file", "deps+other_repo"), + dir + "/external_dep~prefix/foo/file", + ) + + # Different prefix should use its own mapping + self.assertEqual( + r.Rlocation("external_dep/foo/file", "other+some_repo"), + dir + "/external_dep~other/foo/file", + ) + + def testDirectoryBasedRlocationWithCompactRepoMappingOrderMatters(self) -> None: + """Test that order matters for prefixed mappings (first match wins).""" + with _MockFile( + name="_repo_mapping", + contents=[ + # More specific prefix comes first + "deps+specific+*,lib,lib~specific", + # More general prefix comes second + "deps+*,lib,lib~general", + ], + ) as rm: + dir = os.path.dirname(rm.Path()) + r = runfiles.CreateDirectoryBased(dir) + + # Should match the more specific prefix first + self.assertEqual( + r.Rlocation("lib/foo/file", "deps+specific+repo"), + dir + "/lib~specific/foo/file", + ) + + # Should match the general prefix for non-specific repos + self.assertEqual( + r.Rlocation("lib/foo/file", "deps+other_repo"), + dir + "/lib~general/foo/file", + ) + + def testRepositoryMappingLookup(self) -> None: + """Test _RepositoryMapping.lookup() method for both exact and prefix-based mappings.""" + exact_mappings = { + ("", "my_workspace"): "_main", + ("", "config_lib"): "config_lib~1.0.0", + ("deps+specific_repo", "external_dep"): "external_dep~exact", + } + prefixed_mappings = { + ("deps+", "external_dep"): "external_dep~prefix", + ("test_deps+", "test_lib"): "test_lib~2.1.0", + } + + repo_mapping = _RepositoryMapping(exact_mappings, prefixed_mappings) + + # Test exact lookups + self.assertEqual(repo_mapping.lookup("", "my_workspace"), "_main") + self.assertEqual(repo_mapping.lookup("", "config_lib"), "config_lib~1.0.0") + self.assertEqual( + repo_mapping.lookup("deps+specific_repo", "external_dep"), + "external_dep~exact", + ) + + # Test prefix-based lookups + self.assertEqual( + repo_mapping.lookup("deps+some_repo", "external_dep"), "external_dep~prefix" + ) + self.assertEqual( + repo_mapping.lookup("test_deps+another_repo", "test_lib"), "test_lib~2.1.0" + ) + + # Test that exact takes precedence over prefix + self.assertEqual( + repo_mapping.lookup("deps+specific_repo", "external_dep"), + "external_dep~exact", + ) + + # Test non-existent mapping + self.assertIsNone(repo_mapping.lookup("nonexistent", "repo")) + self.assertIsNone(repo_mapping.lookup("unknown+repo", "missing")) + + # Test empty mapping + empty_mapping = _RepositoryMapping({}, {}) + self.assertIsNone(empty_mapping.lookup("any", "repo")) + + # Test is_empty() method + self.assertFalse(repo_mapping.is_empty()) # Should have mappings + self.assertTrue(empty_mapping.is_empty()) # Should be empty + def testCurrentRepository(self) -> None: # Under bzlmod, the current repository name is the empty string instead # of the name in the workspace file. @@ -532,7 +760,7 @@ def testCurrentRepository(self) -> None: expected = "" else: expected = "rules_python" - r = runfiles.Create({"RUNFILES_DIR": "whatever"}) + r = runfiles.Create() assert r is not None # mypy doesn't understand the unittest api. self.assertEqual(r.CurrentRepository(), expected) @@ -552,8 +780,8 @@ def __init__( def __enter__(self) -> Any: tmpdir = os.environ.get("TEST_TMPDIR") self._path = os.path.join(tempfile.mkdtemp(dir=tmpdir), self._name) - with open(self._path, "wt") as f: - f.writelines(l + "\n" for l in self._contents) + with open(self._path, "wt", encoding="utf-8", newline="\n") as f: + f.writelines(l + "\n" for l in self._contents) # noqa: E741 return self def __exit__( diff --git a/tests/runtime_env_toolchain/BUILD.bazel b/tests/runtime_env_toolchain/BUILD.bazel index afc6b587f0..f1bda251f9 100644 --- a/tests/runtime_env_toolchain/BUILD.bazel +++ b/tests/runtime_env_toolchain/BUILD.bazel @@ -12,7 +12,8 @@ # See the License for the specific language governing permissions and # limitations under the License. -load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test") +load("@rules_python_runtime_env_tc_info//:info.bzl", "PYTHON_VERSION") +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") load("//tests/support:support.bzl", "CC_TOOLCHAIN") load(":runtime_env_toolchain_tests.bzl", "runtime_env_toolchain_test_suite") @@ -30,5 +31,35 @@ py_reconfig_test( CC_TOOLCHAIN, ], main = "toolchain_runs_test.py", + # With bootstrap=script, the build version must match the runtime version + # because the venv has the version in the lib/site-packages dir name. + python_version = PYTHON_VERSION, + # Our RBE has Python 3.6, which is too old for the language features + # we use now. Using the runtime-env toolchain on RBE is pretty + # questionable anyways. + tags = ["no-remote-exec"], + deps = ["//python/runfiles"], +) + +py_reconfig_test( + name = "bootstrap_script_test", + srcs = ["toolchain_runs_test.py"], + bootstrap_impl = "script", + data = [ + "//tests/support:current_build_settings", + ], + extra_toolchains = [ + "//python/runtime_env_toolchains:all", + # Necessary for RBE CI + CC_TOOLCHAIN, + ], + main = "toolchain_runs_test.py", + # With bootstrap=script, the build version must match the runtime version + # because the venv has the version in the lib/site-packages dir name. + python_version = PYTHON_VERSION, + # Our RBE has Python 3.6, which is too old for the language features + # we use now. Using the runtime-env toolchain on RBE is pretty + # questionable anyways. + tags = ["no-remote-exec"], deps = ["//python/runfiles"], ) diff --git a/tests/runtime_env_toolchain/runtime_env_toolchain_tests.bzl b/tests/runtime_env_toolchain/runtime_env_toolchain_tests.bzl index 9885a1ef9b..527448ebbc 100644 --- a/tests/runtime_env_toolchain/runtime_env_toolchain_tests.bzl +++ b/tests/runtime_env_toolchain/runtime_env_toolchain_tests.bzl @@ -17,14 +17,14 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:util.bzl", rt_util = "util") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load( "//python/private:toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", "PY_CC_TOOLCHAIN_TYPE", "TARGET_TOOLCHAIN_TYPE", ) # buildifier: disable=bzl-visibility -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility -load("//tests/support:support.bzl", "CC_TOOLCHAIN", "EXEC_TOOLS_TOOLCHAIN", "VISIBLE_FOR_TESTING") +load("//tests/support:support.bzl", "CC_TOOLCHAIN") _LookupInfo = provider() # buildifier: disable=provider-params @@ -54,33 +54,22 @@ def _test_runtime_env_toolchain_matches(name): name = name + "_subject", ) extra_toolchains = [ + # We have to add a cc toolchain because py_cc toolchain depends on it. + # However, that package also defines a different fake py_cc toolchain we + # don't want to use, so we need to ensure the runtime_env toolchain has + # higher precendence. + CC_TOOLCHAIN, str(Label("//python/runtime_env_toolchains:all")), ] - # We have to add a cc toolchain because py_cc toolchain depends on it. - # However, that package also defines a different fake py_cc toolchain we - # don't want to use, so we need to ensure the runtime_env toolchain has - # higher precendence. - # However, Bazel 6 and Bazel 7 process --extra_toolchains in different - # orders: - # * Bazel 6 goes left to right - # * Bazel 7 goes right to left - # We could just put our preferred toolchain before *and* after - # the undesired toolchain... - # However, Bazel 7 has a bug where *duplicate* entries are ignored, - # and only the *first* entry is respected. - if IS_BAZEL_7_OR_HIGHER: - extra_toolchains.insert(0, CC_TOOLCHAIN) - else: - extra_toolchains.append(CC_TOOLCHAIN) analysis_test( name = name, impl = _test_runtime_env_toolchain_matches_impl, target = name + "_subject", config_settings = { "//command_line_option:extra_toolchains": extra_toolchains, - EXEC_TOOLS_TOOLCHAIN: "enabled", - VISIBLE_FOR_TESTING: True, + labels.EXEC_TOOLS_TOOLCHAIN: "enabled", + labels.VISIBLE_FOR_TESTING: True, }, ) diff --git a/tests/runtime_env_toolchain/toolchain_runs_test.py b/tests/runtime_env_toolchain/toolchain_runs_test.py index 7be2472e8b..13b5775ff0 100644 --- a/tests/runtime_env_toolchain/toolchain_runs_test.py +++ b/tests/runtime_env_toolchain/toolchain_runs_test.py @@ -1,6 +1,7 @@ import json import pathlib import platform +import sys import unittest from python.runfiles import runfiles @@ -13,6 +14,7 @@ def test_ran(self): "rules_python/tests/support/current_build_settings.json" ) settings = json.loads(pathlib.Path(settings_path).read_text()) + if platform.system() == "Windows": self.assertEqual( "/_magic_pyruntime_sentinel_do_not_use", settings["interpreter_path"] @@ -23,6 +25,14 @@ def test_ran(self): settings["interpreter"]["short_path"], ) + if settings["bootstrap_impl"] == "script": + # Verify we're running in a venv + self.assertNotEqual(sys.prefix, sys.base_prefix) + # .venv/ occurs for a build-time venv. + # For a runtime created venv, it goes into a temp dir, so + # look for the /bin/ dir as an indicator. + self.assertRegex(sys.executable, r"[.]venv/|/bin/") + if __name__ == "__main__": unittest.main() diff --git a/tests/semver/semver_test.bzl b/tests/semver/semver_test.bzl deleted file mode 100644 index 9d13402c92..0000000000 --- a/tests/semver/semver_test.bzl +++ /dev/null @@ -1,113 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"" - -load("@rules_testing//lib:test_suite.bzl", "test_suite") -load("//python/private:semver.bzl", "semver") # buildifier: disable=bzl-visibility - -_tests = [] - -def _test_semver_from_major(env): - actual = semver("3") - env.expect.that_int(actual.major).equals(3) - env.expect.that_int(actual.minor).equals(None) - env.expect.that_int(actual.patch).equals(None) - env.expect.that_str(actual.build).equals("") - -_tests.append(_test_semver_from_major) - -def _test_semver_from_major_minor_version(env): - actual = semver("4.9") - env.expect.that_int(actual.major).equals(4) - env.expect.that_int(actual.minor).equals(9) - env.expect.that_int(actual.patch).equals(None) - env.expect.that_str(actual.build).equals("") - -_tests.append(_test_semver_from_major_minor_version) - -def _test_semver_with_build_info(env): - actual = semver("1.2.3+mybuild") - env.expect.that_int(actual.major).equals(1) - env.expect.that_int(actual.minor).equals(2) - env.expect.that_int(actual.patch).equals(3) - env.expect.that_str(actual.build).equals("mybuild") - -_tests.append(_test_semver_with_build_info) - -def _test_semver_with_build_info_multiple_pluses(env): - actual = semver("1.2.3-rc0+build+info") - env.expect.that_int(actual.major).equals(1) - env.expect.that_int(actual.minor).equals(2) - env.expect.that_int(actual.patch).equals(3) - env.expect.that_str(actual.pre_release).equals("rc0") - env.expect.that_str(actual.build).equals("build+info") - -_tests.append(_test_semver_with_build_info_multiple_pluses) - -def _test_semver_alpha_beta(env): - actual = semver("1.2.3-alpha.beta") - env.expect.that_int(actual.major).equals(1) - env.expect.that_int(actual.minor).equals(2) - env.expect.that_int(actual.patch).equals(3) - env.expect.that_str(actual.pre_release).equals("alpha.beta") - -_tests.append(_test_semver_alpha_beta) - -def _test_semver_sort(env): - want = [ - semver(item) - for item in [ - # The items are sorted from lowest to highest version - "0.0.1", - "0.1.0-rc", - "0.1.0", - "0.9.11", - "0.9.12", - "1.0.0-alpha", - "1.0.0-alpha.1", - "1.0.0-alpha.beta", - "1.0.0-beta", - "1.0.0-beta.2", - "1.0.0-beta.11", - "1.0.0-rc.1", - "1.0.0-rc.2", - "1.0.0", - # Also handle missing minor and patch version strings - "2.0", - "3", - # Alphabetic comparison for different builds - "3.0.0+build0", - "3.0.0+build1", - ] - ] - actual = sorted(want, key = lambda x: x.key()) - env.expect.that_collection(actual).contains_exactly(want).in_order() - for i, greater in enumerate(want[1:]): - smaller = actual[i] - if greater.key() <= smaller.key(): - env.fail("Expected '{}' to be smaller than '{}', but got otherwise".format( - smaller.str(), - greater.str(), - )) - -_tests.append(_test_semver_sort) - -def semver_test_suite(name): - """Create the test suite. - - Args: - name: the name of the test suite - """ - test_suite(name = name, basic_tests = _tests) diff --git a/tests/support/BUILD.bazel b/tests/support/BUILD.bazel index 9fb5cd0760..45f43c89e2 100644 --- a/tests/support/BUILD.bazel +++ b/tests/support/BUILD.bazel @@ -12,81 +12,25 @@ # See the License for the specific language governing permissions and # limitations under the License. -# ==================== -# NOTE: You probably want to use the constants in test_platforms.bzl -# Otherwise, you'll probably have to manually call Label() on these targets -# to force them to resolve in the proper context. -# ==================== - +load("@bazel_skylib//rules:common_settings.bzl", "string_flag") load(":sh_py_run_test.bzl", "current_build_settings") package( default_visibility = ["//:__subpackages__"], ) -platform( - name = "mac", - constraint_values = [ - "@platforms//os:macos", - ], -) - -platform( - name = "linux", - constraint_values = [ - "@platforms//os:linux", - ], -) - -platform( - name = "windows", - constraint_values = [ - "@platforms//os:windows", - ], -) - -# Used when testing downloading of toolchains for a different platform - -platform( - name = "linux_x86_64", - constraint_values = [ - "@platforms//cpu:x86_64", - "@platforms//os:linux", - ], -) - -platform( - name = "linux_aarch64", - constraint_values = [ - "@platforms//cpu:aarch64", - "@platforms//os:linux", - ], -) - -platform( - name = "mac_x86_64", - constraint_values = [ - "@platforms//cpu:x86_64", - "@platforms//os:macos", - ], -) - -platform( - name = "windows_x86_64", - constraint_values = [ - "@platforms//cpu:x86_64", - "@platforms//os:windows", - ], +current_build_settings( + name = "current_build_settings", ) -platform( - name = "win_aarch64", - constraint_values = [ - "@platforms//os:windows", - "@platforms//cpu:aarch64", - ], +string_flag( + name = "custom_runtime", + build_setting_default = "", ) -current_build_settings( - name = "current_build_settings", +config_setting( + name = "is_custom_runtime_linux-x86-install-only-stripped", + flag_values = { + ":custom_runtime": "linux-x86-install-only-stripped", + }, ) diff --git a/tests/support/cc_info_subject.bzl b/tests/support/cc_info_subject.bzl index e33ccb8262..433a3da72d 100644 --- a/tests/support/cc_info_subject.bzl +++ b/tests/support/cc_info_subject.bzl @@ -81,6 +81,7 @@ def _compilation_context_subject_new(info, *, meta): # buildifier: disable=uninitialized public = struct( + actual = info, # go/keep-sorted start direct_headers = lambda *a, **k: _compilation_context_subject_direct_headers(self, *a, **k), direct_public_headers = lambda *a, **k: _compilation_context_subject_direct_public_headers(self, *a, **k), @@ -156,6 +157,7 @@ def _linking_context_subject_new(info, meta): # buildifier: disable=uninitialized public = struct( + actual = info, # go/keep-sorted start linker_inputs = lambda *a, **k: _linking_context_subject_linker_inputs(self, *a, **k), # go/keep-sorted end diff --git a/tests/support/cc_toolchains/BUILD.bazel b/tests/support/cc_toolchains/BUILD.bazel index f6e6654d09..a74a8016fc 100644 --- a/tests/support/cc_toolchains/BUILD.bazel +++ b/tests/support/cc_toolchains/BUILD.bazel @@ -12,6 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. +load("@rules_cc//cc:cc_library.bzl", "cc_library") load("@rules_cc//cc/toolchains:cc_toolchain.bzl", "cc_toolchain") load("@rules_cc//cc/toolchains:cc_toolchain_suite.bzl", "cc_toolchain_suite") load("@rules_testing//lib:util.bzl", "PREVENT_IMPLICIT_BUILDING_TAGS") @@ -20,7 +21,23 @@ load(":fake_cc_toolchain_config.bzl", "fake_cc_toolchain_config") package(default_visibility = ["//:__subpackages__"]) -exports_files(["fake_header.h"]) +# Factored out for testing +filegroup( + name = "py_headers_files", + srcs = [ + "py_header.h", + ":py_headers_abi3_files", + ], +) + +# Factored out for testing +filegroup( + name = "py_headers_abi3_files", + srcs = [ + "py_abi3_header.h", + "py_include/py_include.h", + ], +) filegroup( name = "libpython", @@ -37,22 +54,30 @@ toolchain( py_cc_toolchain( name = "fake_py_cc_toolchain_impl", - headers = ":fake_headers", + headers = ":py_headers", + headers_abi3 = ":py_headers_abi3", libs = ":fake_libs", + platform_machine = "x86_64", python_version = "3.999", + sys_platform = "linux", tags = PREVENT_IMPLICIT_BUILDING_TAGS, ) -# buildifier: disable=native-cc cc_library( - name = "fake_headers", - hdrs = ["fake_header.h"], + name = "py_headers_abi3", + hdrs = [":py_headers_abi3_files"], data = ["data.txt"], - includes = ["fake_include"], + includes = ["py_include"], + tags = PREVENT_IMPLICIT_BUILDING_TAGS, +) + +cc_library( + name = "py_headers", + hdrs = [":py_headers_files"], tags = PREVENT_IMPLICIT_BUILDING_TAGS, + deps = [":py_headers_abi3"], ) -# buildifier: disable=native-cc cc_library( name = "fake_libs", srcs = ["libpython3.so"], diff --git a/tests/support/cc_toolchains/fake_cc_toolchain_config.bzl b/tests/support/cc_toolchains/fake_cc_toolchain_config.bzl index 8240f09e04..b70c79cfd8 100644 --- a/tests/support/cc_toolchains/fake_cc_toolchain_config.bzl +++ b/tests/support/cc_toolchains/fake_cc_toolchain_config.bzl @@ -15,6 +15,7 @@ """Fake for providing CcToolchainConfigInfo.""" load("@rules_cc//cc/common:cc_common.bzl", "cc_common") +load("@rules_cc//cc/toolchains:cc_toolchain_config_info.bzl", "CcToolchainConfigInfo") def _impl(ctx): return cc_common.create_cc_toolchain_config_info( diff --git a/tests/support/cc_toolchains/py_abi3_header.h b/tests/support/cc_toolchains/py_abi3_header.h new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/support/cc_toolchains/py_header.h b/tests/support/cc_toolchains/py_header.h new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/support/cc_toolchains/py_include/py_include.h b/tests/support/cc_toolchains/py_include/py_include.h new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/support/copy_file.bzl b/tests/support/copy_file.bzl new file mode 100644 index 0000000000..bd9bb218f3 --- /dev/null +++ b/tests/support/copy_file.bzl @@ -0,0 +1,33 @@ +"""Copies a file to a directory.""" + +def _copy_file_to_dir_impl(ctx): + out_file = ctx.actions.declare_file( + "{}/{}".format(ctx.attr.out_dir, ctx.file.src.basename), + ) + ctx.actions.run_shell( + inputs = [ctx.file.src], + outputs = [out_file], + arguments = [ctx.file.src.path, out_file.path], + # Perform a copy to better match how a file install from + # a repo-phase (e.g. whl extraction) looks. + command = 'cp -f "$1" "$2"', + progress_message = "Copying %{input} to %{output}", + ) + return [DefaultInfo(files = depset([out_file]))] + +copy_file_to_dir = rule( + implementation = _copy_file_to_dir_impl, + doc = """ +This allows copying a file whose name is platform-dependent to a directory. + +While bazel_skylib has a copy_file rule, you must statically specify the +output file name. +""", + attrs = { + "out_dir": attr.string(mandatory = True), + "src": attr.label( + allow_single_file = True, + mandatory = True, + ), + }, +) diff --git a/tests/support/empty_toolchain/BUILD.bazel b/tests/support/empty_toolchain/BUILD.bazel new file mode 100644 index 0000000000..cab5f800ec --- /dev/null +++ b/tests/support/empty_toolchain/BUILD.bazel @@ -0,0 +1,3 @@ +load(":empty.bzl", "empty_toolchain") + +empty_toolchain(name = "empty") diff --git a/tests/support/empty_toolchain/empty.bzl b/tests/support/empty_toolchain/empty.bzl new file mode 100644 index 0000000000..e2839283c7 --- /dev/null +++ b/tests/support/empty_toolchain/empty.bzl @@ -0,0 +1,23 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Defines an empty toolchain that returns just ToolchainInfo.""" + +def _empty_toolchain_impl(ctx): + # Include the label so e.g. tests can identify what the target was. + return [platform_common.ToolchainInfo(label = ctx.label)] + +empty_toolchain = rule( + implementation = _empty_toolchain_impl, +) diff --git a/tests/support/mocks/BUILD.bazel b/tests/support/mocks/BUILD.bazel new file mode 100644 index 0000000000..07bdb2bffc --- /dev/null +++ b/tests/support/mocks/BUILD.bazel @@ -0,0 +1,19 @@ +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") +load(":mocks_tests.bzl", "mocks_test_suite") + +package( + default_visibility = ["//:__subpackages__"], +) + +bzl_library( + name = "mocks_bzl", + srcs = ["mocks.bzl"], +) + +bzl_library( + name = "python_ext_bzl", + srcs = ["python_ext.bzl"], + deps = [":mocks_bzl"], +) + +mocks_test_suite(name = "mocks_tests") diff --git a/tests/support/mocks/mocks.bzl b/tests/support/mocks/mocks.bzl new file mode 100644 index 0000000000..6c79b1f4ff --- /dev/null +++ b/tests/support/mocks/mocks.bzl @@ -0,0 +1,582 @@ +"""Mocks for repository_ctx, module_ctx, and File objects.""" + +def _path_new(path, mock_files = None): + """Create a mock path object. + + Args: + path: {type}`string` The path string. + mock_files: {type}`dict[string, string]` A dict of mocked files. + + Returns: + {type}`MockPath` A struct mocking a path object. + """ + mock_files = mock_files or {} + return struct( + exists = path in mock_files, + basename = path.split("/")[-1], + dirname = "/".join(path.split("/")[:-1]), + _path = path, + ) + +def _file_new(short_path, *, path = None, is_source = True, owner = None): + """Create a mock File object. + + Args: + short_path: {type}`string` The short path to the file. + path: {type}`string` The full path to the file. Defaults to a made + up exec-root path or the short path if is_source. + is_source: {type}`bool` Whether the file is a source file. + owner: {type}`Label|string` The owner label of the file. + + Returns: + {type}`MockFile` A struct mocking a File object. + """ + if owner == None: + owner = Label("//:mock") + + owner_str = str(owner) + repo_name = owner_str.split("//")[0] + + is_main_repo = repo_name in ("", "@", "@@") + + actual_short_path = short_path + if not is_main_repo: + repo_name = repo_name.lstrip("@") + if not actual_short_path.startswith("../"): + actual_short_path = "../{}/{}".format(repo_name, short_path) + + if path == None: + rel_path = short_path + if rel_path.startswith("../"): + parts = rel_path.split("/") + rel_path = "/".join(parts[2:]) + + if is_source: + path = "external/{}/{}".format(repo_name, rel_path) + else: + path = "bazel-out/k9-deadbeef/bin/external/{}/{}".format( + repo_name, + rel_path, + ) + elif path == None: + if is_source: + path = short_path + else: + path = "bazel-out/k9-deadbeef/bin/{}".format(short_path) + + return struct( + path = path, + basename = path.split("/")[-1], + dirname = "/".join(path.split("/")[:-1]), + extension = path.split(".")[-1] if "." in path else "", + is_source = is_source, + owner = owner, + short_path = actual_short_path, + ) + +def _tag_new(**kwargs): + """Create a mock tag. + + Args: + **kwargs: {type}`dict` The tag attributes. + + Returns: + {type}`MockTag` A mock tag object. + """ + return struct(**kwargs) + +def _module_new(name, *, is_root = False, **tags): + """Create a mock module object. + + Args: + name: {type}`string` The name of the module. + is_root: {type}`bool` Whether this is the root module. + **tags: {type}`list[MockTag]` Lists of tag objects. + + Returns: + {type}`MockModule` A mock module object. + """ + return struct( + name = name, + tags = struct(**tags), + is_root = is_root, + ) + +def _mctx_read(self, x, watch = None): + _ = watch # @unused + path_str = x._path if hasattr(x, "_path") else str(x) + if path_str in self.mock_files: + return self.mock_files[path_str] + for k, v in self.mock_files.items(): + if k in path_str or k in path_str.replace(":", "/"): + return v + fail("File not found in mock_files: " + path_str) + +def _mctx_path(self, x): + return _path_new(str(x), self.mock_files) + +def _get_download_file_name(url, output = ""): + """Compute the download file name. + + Args: + url: {type}`string` The URL being downloaded. + output: {type}`string` The explicit output path, if any. + + Returns: + {type}`string` The file name. + """ + if output: + return str(output) + return str(url).split("?")[0].split("/")[-1] + +def _mctx_download( + self, + url, + output = "", + sha256 = "", + executable = False, + allow_fail = False, + canonical_id = "", + auth = {}, + headers = {}, + integrity = "", + block = True): + _ = ( + sha256, + executable, + allow_fail, + canonical_id, + auth, + headers, + integrity, + block, + ) # @unused + urls = url if type(url) == "list" else [url] + for u in urls: + content = None + if u in self.mock_downloads: + content = self.mock_downloads[u] + elif "*" in self.mock_downloads: + content = self.mock_downloads["*"] + + if content != None: + if type(content) == "string": + out = _get_download_file_name(u, output) + self.mock_files[out] = content + return struct( + success = True, + wait = lambda: struct(success = True), + ) + else: + return content( + self, + u, + output, + sha256, + executable, + allow_fail, + canonical_id, + auth, + headers, + integrity, + ) + + if not self.mock_downloads: + return struct(success = True, wait = lambda: struct(success = True)) + return struct(success = False, wait = lambda: struct(success = False)) + +def _mrctx_report_progress(self, message): + self.report_progress_calls.append(message) + return None + +def _mctx_add_module(self, **kwargs): + """Add a module to the mock module_ctx. + + Args: + self: The mock module_ctx. + **kwargs: Arguments to pass to _module_new. + + Returns: + {type}`MockModuleCtx` The mock module_ctx. + """ + module = _module_new(**kwargs) + if module.is_root and len(self.modules) > 0: + fail("is_root=True can only be set on the first module in the " + + "modules list.") + self.modules.append(module) + return self + +def _mctx_new( + *args, + modules = None, + environ = None, + mock_files = None, + mock_downloads = None, + os_name = "linux", + arch_name = "x86_64", + facts = None): + """Create a mock module_ctx object. + + Args: + *args: {type}`list[MockModule]` Mock modules passed positionally. + modules: {type}`list[MockModule]` List of mock modules (alternative + to positional args). + environ: {type}`dict[string, string]` Dict of environment variables. + mock_files: {type}`dict[string, string]` Dict mapping path strings + to content. + mock_downloads: {type}`dict[string, string|callable]` Dict mapping + url to string or callable. + os_name: {type}`string` The OS name. + arch_name: {type}`string` The architecture name. + facts: {type}`dict` Optional facts dict. + + Returns: + {type}`MockModuleCtx` A struct mocking a module_ctx object. + """ + modules = list(args) + (modules or []) + + for i, mod in enumerate(modules): + if getattr(mod, "is_root", False) and i != 0: + fail("is_root=True can only be set on the first module in the " + + "modules list.") + + environ = environ or {} + mock_files = mock_files or {} + mock_downloads = mock_downloads or {} + + # buildifier: disable=uninitialized + self = struct( + mock_files = mock_files, + mock_downloads = mock_downloads, + report_progress_calls = [], + getenv = environ.get, + facts = facts, + os = struct( + name = os_name, + arch = arch_name, + ), + modules = list(modules), + path = lambda *a, **k: _mctx_path(self, *a, **k), + read = lambda *a, **k: _mctx_read(self, *a, **k), + download = lambda *a, **k: _mctx_download(self, *a, **k), + report_progress = lambda *a, **k: _mrctx_report_progress(self, *a, **k), + add_module = lambda **k: _mctx_add_module(self, **k), + ) + return self + +def _rctx_read(self, x): + path_str = x._path if hasattr(x, "_path") else str(x) + if path_str not in self.mock_files: + fail("File not found in mock_files: " + path_str) + + val = self.mock_files[path_str] + for _ in range(10): + if type(val) == "dict" and val.get("type") == "symlink": + path_str = val["target"] + if path_str not in self.mock_files: + fail("Symlink target not found in mock_files: " + path_str) + val = self.mock_files[path_str] + else: + break + + if type(val) == "dict" and val.get("type") == "symlink": + fail("Too many symlinks followed") + + return val + +def _rctx_path(self, x): + return _path_new(str(x), self.mock_files) + +def _rctx_file(self, path, content = "", executable = True, legacy_utf8 = True): + _ = executable, legacy_utf8 # @unused + self.mock_files[str(path)] = content + +def _rctx_template(self, path, template, substitutions = {}, executable = True): + _ = executable # @unused + template_str = str(template) + if template_str not in self.mock_files: + fail("Template file not found: " + template_str) + + content = self.mock_files[template_str] + for key, value in substitutions.items(): + content = content.replace(key, value) + + self.mock_files[str(path)] = content + +def _rctx_which(self, program): + prog_str = str(program) + if prog_str in self.mock_which: + res = self.mock_which[prog_str] + if res == None: + return None + return _path_new(res, self.mock_files) + return None + +def _rctx_download( + self, + url, + output = "", + sha256 = "", + executable = False, + allow_fail = False, + canonical_id = "", + auth = {}, + headers = {}, + integrity = ""): + _ = ( + sha256, + executable, + allow_fail, + canonical_id, + auth, + headers, + integrity, + ) # @unused + + urls = url if type(url) == "list" else [url] + + for u in urls: + if u in self.mock_downloads: + res = self.mock_downloads[u] + if type(res) == "string": + out = _get_download_file_name(u, output) + self.mock_files[out] = res + return struct(success = True, sha256 = "mocksha256") + else: + return res( + self, + u, + output, + sha256, + executable, + allow_fail, + canonical_id, + auth, + headers, + integrity, + ) + + if not allow_fail: + fail("Download not mocked for url: " + str(urls)) + return struct(success = False) + +def _rctx_extract( + self, + archive, + output = "", + stripPrefix = "", + rename_files = {}, + *, + watch_archive = "auto"): + _ = ( + stripPrefix, + rename_files, + watch_archive, + ) # @unused + + archive_str = str(archive) + if archive_str in self.mock_extracts: + for f, c in self.mock_extracts[archive_str].items(): + out_path = "{}/{}".format(output, f) if output else str(f) + self.mock_files[out_path] = c + +def _rctx_download_and_extract( + self, + url, + output = "", + sha256 = "", + type = "", + stripPrefix = "", + allow_fail = False, + canonical_id = "", + auth = {}, + headers = {}, + integrity = "", + rename_files = {}): + _ = type # @unused + + res = self.download( + url = url, + output = "", + sha256 = sha256, + allow_fail = allow_fail, + canonical_id = canonical_id, + auth = auth, + headers = headers, + integrity = integrity, + ) + if not res.success: + return res + + urls = url if type(url) == "list" else [url] + for u in urls: + downloaded_file = _get_download_file_name(u) + if downloaded_file in self.mock_extracts: + self.extract( + archive = downloaded_file, + output = output, + stripPrefix = stripPrefix, + rename_files = rename_files, + ) + break + + return res + +def _rctx_execute( + self, + arguments, + timeout = 600, + quiet = True, + working_directory = "", + environment = {}, + custom_reporter = ""): + _ = ( + self, + arguments, + timeout, + quiet, + working_directory, + environment, + custom_reporter, + ) # @unused + self.execute_calls.append(arguments) + return struct(return_code = 0, stdout = "", stderr = "") + +def _rctx_symlink(self, target, link_name): + self.mock_files[str(link_name)] = {"target": str(target), "type": "symlink"} + +def _rctx_new( + attr = None, + environ = None, + mock_files = None, + mock_which = None, + mock_downloads = None, + mock_extracts = None, + os_name = "linux", + arch_name = "x86_64"): + """Create a mock repository_ctx object. + + Args: + attr: {type}`dict` Dict of attributes. + environ: {type}`dict[string, string]` Dict of environment variables. + mock_files: {type}`dict[string, string]` Dict mapping path strings + to content. + mock_which: {type}`dict[string, string]` Dict mapping program + name to path string. + mock_downloads: {type}`dict[string, string|callable]` Dict mapping + url to string or callable. + mock_extracts: {type}`dict[string, dict[string, string]]` Dict mapping + downloaded filename to a dict of extracted filename to content. + os_name: {type}`string` The OS name. + arch_name: {type}`string` The architecture name. + + Returns: + {type}`MockRepositoryCtx` A struct mocking a repository_ctx object. + """ + attr = attr or {} + environ = environ or {} + mock_files = mock_files or {} + mock_which = mock_which or {} + mock_downloads = mock_downloads or {} + mock_extracts = mock_extracts or {} + + # buildifier: disable=uninitialized + self = struct( + mock_files = mock_files, + mock_which = mock_which, + mock_downloads = mock_downloads, + mock_extracts = mock_extracts, + execute_calls = [], + report_progress_calls = [], + attr = struct(**attr), + name = attr.get("name", "mock"), + os = struct( + name = os_name, + arch = arch_name, + ), + os_environ = environ, + path = lambda *a, **k: _rctx_path(self, *a, **k), + read = lambda *a, **k: _rctx_read(self, *a, **k), + file = lambda *a, **k: _rctx_file(self, *a, **k), + getenv = environ.get, + template = lambda *a, **k: _rctx_template(self, *a, **k), + which = lambda *a, **k: _rctx_which(self, *a, **k), + download = lambda *a, **k: _rctx_download(self, *a, **k), + extract = lambda *a, **k: _rctx_extract(self, *a, **k), + delete = lambda x: True, + download_and_extract = lambda *a, **k: _rctx_download_and_extract( + self, + *a, + **k + ), + execute = lambda *a, **k: _rctx_execute(self, *a, **k), + symlink = lambda *a, **k: _rctx_symlink(self, *a, **k), + report_progress = lambda *a, **k: _mrctx_report_progress(self, *a, **k), + ) + + return self + +def _glob_call_new(*args, **kwargs): + """Create a struct representing a glob call. + + Args: + *args: {type}`tuple` Positional arguments to glob. + **kwargs: {type}`dict` Keyword arguments to glob. + + Returns: + {type}`MockGlobCall` A struct with glob and kwargs fields. + """ + return struct( + glob = args, + kwargs = kwargs, + ) + +def _glob_new(): + """Create a mock glob object. + + Returns: + {type}`MockGlob` A struct with calls and results lists, and a + glob function. + """ + calls = [] + results = [] + + def _glob_fn(*args, **kwargs): + calls.append(_glob_call_new(*args, **kwargs)) + if not results: + fail("Mock glob missing for invocation: args={} kwargs={}".format( + args, + kwargs, + )) + return results.pop(0) + + return struct( + calls = calls, + results = results, + glob = _glob_fn, + ) + +def _select_new(value, no_match_error = None): + """A mock select function that returns the value. + + Args: + value: {type}`Any` The value to return. + no_match_error: {type}`string` Ignored. + + Returns: + {type}`MockSelect` The value. + """ + _ = no_match_error # @unused + return value + +mocks = struct( + file = _file_new, + glob = _glob_new, + glob_call = _glob_call_new, + mctx = _mctx_new, + module = _module_new, + path = _path_new, + rctx = _rctx_new, + select = _select_new, + tag = _tag_new, +) diff --git a/tests/support/mocks/mocks_tests.bzl b/tests/support/mocks/mocks_tests.bzl new file mode 100644 index 0000000000..1dc495b342 --- /dev/null +++ b/tests/support/mocks/mocks_tests.bzl @@ -0,0 +1,139 @@ +"""Tests for mocks.bzl""" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//tests/support/mocks:mocks.bzl", "mocks") + +_tests = [] + +def _test_path(env): + p1 = mocks.path("a/b/c", mock_files = {"a/b/c": "data"}) + env.expect.that_bool(p1.exists).equals(True) + env.expect.that_str(p1.basename).equals("c") + env.expect.that_str(p1.dirname).equals("a/b") + env.expect.that_str(p1._path).equals("a/b/c") + + p2 = mocks.path("d/e/f", mock_files = {}) + env.expect.that_bool(p2.exists).equals(False) + +_tests.append(_test_path) + +def _test_file(env): + # Default main repo + f1 = mocks.file("a/b.txt", is_source = True) + env.expect.that_str(f1.path).equals("a/b.txt") + env.expect.that_str(f1.short_path).equals("a/b.txt") + env.expect.that_str(f1.basename).equals("b.txt") + env.expect.that_str(f1.dirname).equals("a") + env.expect.that_str(f1.extension).equals("txt") + env.expect.that_bool(f1.is_source).equals(True) + env.expect.that_str(str(f1.owner)).equals(str(Label("//:mock"))) + + # External repo + f2 = mocks.file("a/b.txt", is_source = True, owner = "@foo//:mock") + env.expect.that_str(f2.path).equals("external/foo/a/b.txt") + env.expect.that_str(f2.short_path).equals("../foo/a/b.txt") + + # External repo generated file + f3 = mocks.file("a/b.txt", is_source = False, owner = "@foo//:mock") + env.expect.that_str(f3.path).equals( + "bazel-out/k9-deadbeef/bin/external/foo/a/b.txt", + ) + env.expect.that_str(f3.short_path).equals("../foo/a/b.txt") + +_tests.append(_test_file) + +def _test_mctx(env): + mctx = mocks.mctx( + environ = {"FOO": "bar"}, + mock_files = {"file.txt": "content"}, + mock_downloads = {"http://example.com": "downloaded"}, + os_name = "windows", + arch_name = "x86_64", + ) + env.expect.that_str(mctx.getenv("FOO")).equals("bar") + env.expect.that_str(mctx.read(mocks.path("file.txt"))).equals("content") + env.expect.that_str(mctx.os.name).equals("windows") + env.expect.that_str(mctx.os.arch).equals("x86_64") + + # Test download + res = mctx.download("http://example.com", "out.txt") + env.expect.that_bool(res.success).equals(True) + env.expect.that_str(mctx.read(mocks.path("out.txt"))).equals("downloaded") + + # Test report progress + mctx.report_progress("doing something") + env.expect.that_collection(mctx.report_progress_calls).contains_exactly([ + "doing something", + ]) + +_tests.append(_test_mctx) + +def _test_rctx(env): + rctx = mocks.rctx( + environ = {"FOO": "bar"}, + mock_files = {"file.txt": "content"}, + mock_which = {"mycmd": "path/to/mycmd"}, + ) + env.expect.that_str(rctx.os_environ["FOO"]).equals("bar") + env.expect.that_str(rctx.read(mocks.path("file.txt"))).equals("content") + + # Test which + w = rctx.which("mycmd") + env.expect.that_str(w._path).equals("path/to/mycmd") + env.expect.that_bool(rctx.which("not_found") == None).equals(True) + + # Test file writing + rctx.file("new.txt", "new content") + env.expect.that_str(rctx.read(mocks.path("new.txt"))).equals("new content") + + # Test template + rctx.file("template.txt", "Hello {name}") + rctx.template("rendered.txt", "template.txt", {"{name}": "World"}) + env.expect.that_str(rctx.read(mocks.path("rendered.txt"))).equals( + "Hello World", + ) + + # Test symlink reading + rctx.symlink("rendered.txt", "link.txt") + env.expect.that_str(rctx.read(mocks.path("link.txt"))).equals("Hello World") + +_tests.append(_test_rctx) + +def _test_glob(env): + g = mocks.glob() + g.results.append(["a.txt", "b.txt"]) + res = g.glob(["*.txt"], exclude = ["c.txt"]) + env.expect.that_collection(res).contains_exactly(["a.txt", "b.txt"]) + env.expect.that_collection(g.calls).has_size(1) + env.expect.that_collection(g.calls[0].glob[0]).contains_exactly(["*.txt"]) + env.expect.that_collection(g.calls[0].kwargs["exclude"]).contains_exactly([ + "c.txt", + ]) + +_tests.append(_test_glob) + +def _test_module_and_tags(env): + mod = mocks.module( + "my_mod", + is_root = True, + my_tag = [mocks.tag(attr1 = "val1")], + ) + env.expect.that_str(mod.name).equals("my_mod") + env.expect.that_bool(mod.is_root).equals(True) + env.expect.that_str(mod.tags.my_tag[0].attr1).equals("val1") + +_tests.append(_test_module_and_tags) + +def _test_select(env): + res = mocks.select({"//conditions:default": "val"}) + env.expect.that_dict(res).contains_exactly({"//conditions:default": "val"}) + +_tests.append(_test_select) + +def mocks_test_suite(name): + """Create the test suite. + + Args: + name: the name of the test suite + """ + test_suite(name = name, basic_tests = _tests) diff --git a/tests/support/mocks/python_ext.bzl b/tests/support/mocks/python_ext.bzl new file mode 100644 index 0000000000..dc3ac41f8d --- /dev/null +++ b/tests/support/mocks/python_ext.bzl @@ -0,0 +1,140 @@ +"""Helper for defining a mock module for the python bzlmod extension.""" + +load(":mocks.bzl", "mocks") + +def _module(name = "rules_python", is_root = True, **tags): + """Creates a mock Bzlmod module struct with defaulted tag lists. + + Args: + name: The module name. + is_root: Whether this is the root module. + **tags: Lists of tag objects. + + Returns: + A mock module struct. + """ + defaulted_tags = { + "defaults": [], + "override": [], + "single_version_override": [], + "single_version_platform_override": [], + "toolchain": [], + } + defaulted_tags.update(tags) + return mocks.module(name = name, is_root = is_root, **defaulted_tags) + +def _override(**kwargs): + """Creates a mock python.override tag with default values.""" + attrs = { + "add_runtime_manifest_files": [], + "add_runtime_manifest_urls": [], + "add_target_settings": [], + "available_python_versions": [], + "base_urls": ["https://github.com/astral-sh/python-build-standalone/releases/download"], + "ignore_root_user_error": True, + "minor_mapping": {}, + "register_all_versions": False, + "runtime_manifest_sha": "", + } + attrs.update(kwargs) + return mocks.tag(**attrs) + +def _defaults(**kwargs): + """Creates a mock python.defaults tag with default values.""" + attrs = { + "python_version": "", + "python_version_env": "", + } + attrs.update(kwargs) + return mocks.tag(**attrs) + +def _single_version_override(**kwargs): + """Creates a mock python.single_version_override tag with default values.""" + attrs = { + "distutils": None, + "distutils_content": "", + "patch_strip": 0, + "patches": [], + "python_version": "", + "sha256": {}, + "strip_prefix": "python", + "urls": [], + } + attrs.update(kwargs) + return mocks.tag(**attrs) + +def _single_version_platform_override(**kwargs): + """Creates a mock python.single_version_platform_override tag with default values.""" + attrs = { + "arch": "", + "coverage_tool": None, + "os_name": "", + "patch_strip": 0, + "patches": [], + "platform": "", + "python_version": "", + "sha256": "", + "strip_prefix": "python", + "target_compatible_with": [], + "target_settings": [], + "urls": [], + } + attrs.update(kwargs) + return mocks.tag(**attrs) + +def _toolchain(**kwargs): + """Creates a mock python.toolchain tag with default values.""" + attrs = { + "configure_coverage_tool": False, + "ignore_root_user_error": True, + "is_default": False, + "python_version": "", + } + attrs.update(kwargs) + return mocks.tag(**attrs) + +_DEFAULT_RUNTIMES_MANIFEST = """ +87275619c2706affa4d1090d2ca3dad354b6d69f8b85dbfafe38785870751b9a 20251031/cpython-3.9.25+20251031-x86_64-unknown-linux-gnu-install_only.tar.gz +6112d46355857680b81849764a6cf9f38cc4cd0d1cf29d432bc12fe5aeedf9d0 20260414/cpython-3.10.20+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +1111111111111111111111111111111111111111111111111111111111111111 20241016/cpython-3.10.15+20241016-x86_64-unknown-linux-gnu-install_only.tar.gz +2222222222222222222222222222222222222222222222222222222222222222 20240224/cpython-3.10.13+20240224-x86_64-unknown-linux-gnu-install_only.tar.gz +0000000000000000000000000000000000000000000000000000000000000000 20260414/cpython-3.11.15+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +3333333333333333333333333333333333333333333333333333333333333333 20241016/cpython-3.11.10+20241016-x86_64-unknown-linux-gnu-install_only.tar.gz +4444444444444444444444444444444444444444444444444444444444444444 20230116/cpython-3.11.2+20230116-x86_64-unknown-linux-gnu-install_only.tar.gz +5555555555555555555555555555555555555555555555555555555555555555 20230116/cpython-3.11.1+20230116-x86_64-unknown-linux-gnu-install_only.tar.gz +6666666666666666666666666666666666666666666666666666666666666666 20260414/cpython-3.12.13+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +7777777777777777777777777777777777777777777777777777777777777777 20240726/cpython-3.12.4+20240726-x86_64-unknown-linux-gnu-install_only.tar.gz +8888888888888888888888888888888888888888888888888888888888888888 20260414/cpython-3.13.13+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +9999999999999999999999999999999999999999999999999999999999999999 20241016/cpython-3.13.0+20241016-x86_64-unknown-linux-gnu-install_only.tar.gz +aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 20260414/cpython-3.14.4+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb 20251031/cpython-3.14.0+20251031-x86_64-unknown-linux-gnu-install_only.tar.gz +cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc 20260414/cpython-3.15.0a8+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd 20241205/cpython-3.13.1+20241205-x86_64-unknown-linux-gnu-install_only.tar.gz +""" + +def _mctx(*args, **kwargs): + """Creates a mock module_ctx pre-populated with the default runtimes manifest. + + Args: + *args: Positional arguments to pass to mocks.mctx. + **kwargs: Keyword arguments to pass to mocks.mctx. + + Returns: + A mock module_ctx struct. + """ + mock_files = { + "python/private/runtimes_manifest.txt": _DEFAULT_RUNTIMES_MANIFEST, + } + mock_files.update(kwargs.pop("mock_files", {})) + kwargs["mock_files"] = mock_files + return mocks.mctx(*args, **kwargs) + +python_ext = struct( + defaults = _defaults, + mctx = _mctx, + module = _module, + override = _override, + single_version_override = _single_version_override, + single_version_platform_override = _single_version_platform_override, + toolchain = _toolchain, +) diff --git a/tests/support/platforms/BUILD.bazel b/tests/support/platforms/BUILD.bazel new file mode 100644 index 0000000000..eeb7ccb597 --- /dev/null +++ b/tests/support/platforms/BUILD.bazel @@ -0,0 +1,85 @@ +package( + default_visibility = ["//:__subpackages__"], +) + +# ==================== +# NOTE: You probably want to use the constants in test_platforms.bzl +# Otherwise, you'll probably have to manually call Label() on these targets +# to force them to resolve in the proper context. +# ==================== +platform( + name = "mac", + constraint_values = [ + "@platforms//os:macos", + ], +) + +platform( + name = "linux", + constraint_values = [ + "@platforms//os:linux", + ], +) + +platform( + name = "windows", + constraint_values = [ + "@platforms//os:windows", + ], +) + +platform( + name = "linux_x86_64", + constraint_values = [ + "@platforms//cpu:x86_64", + "@platforms//os:linux", + ], +) + +platform( + name = "linux_aarch64", + constraint_values = [ + "@platforms//cpu:aarch64", + "@platforms//os:linux", + ], +) + +platform( + name = "mac_x86_64", + constraint_values = [ + "@platforms//cpu:x86_64", + "@platforms//os:macos", + ], +) + +platform( + name = "mac_aarch64", + constraint_values = [ + "@platforms//cpu:aarch64", + "@platforms//os:macos", + ], +) + +platform( + name = "windows_x86_64", + constraint_values = [ + "@platforms//cpu:x86_64", + "@platforms//os:windows", + ], +) + +platform( + name = "windows_aarch64", + constraint_values = [ + "@platforms//os:windows", + "@platforms//cpu:aarch64", + ], +) + +platform( + name = "exotic_unix", + constraint_values = [ + "@platforms//os:linux", + "@platforms//cpu:s390x", + ], +) diff --git a/tests/support/platforms/platforms.bzl b/tests/support/platforms/platforms.bzl new file mode 100644 index 0000000000..92a1d61844 --- /dev/null +++ b/tests/support/platforms/platforms.bzl @@ -0,0 +1,17 @@ +"""Constants and utilities for platforms used for testing.""" + +platform_targets = struct( + LINUX = Label("//tests/support/platforms:linux"), + LINUX_AARCH64 = Label("//tests/support/platforms:linux_aarch64"), + LINUX_X86_64 = Label("//tests/support/platforms:linux_x86_64"), + MAC = Label("//tests/support/platforms:mac"), + MAC_X86_64 = Label("//tests/support/platforms:mac_x86_64"), + MAC_AARCH64 = Label("//tests/support/platforms:mac_aarch64"), + WINDOWS = Label("//tests/support/platforms:windows"), + WINDOWS_AARCH64 = Label("//tests/support/platforms:windows_aarch64"), + WINDOWS_X86_64 = Label("//tests/support/platforms:windows_x86_64"), + + # Unspecified Unix platform that is unlikely to be the host platform in CI, + # but still provides a Python toolchain. + EXOTIC_UNIX = Label("//tests/support/platforms:exotic_unix"), +) diff --git a/tests/support/py_cc_toolchain_info_subject.bzl b/tests/support/py_cc_toolchain_info_subject.bzl index 4d3647c53e..d6aa258a64 100644 --- a/tests/support/py_cc_toolchain_info_subject.bzl +++ b/tests/support/py_cc_toolchain_info_subject.bzl @@ -18,14 +18,26 @@ load("@rules_testing//lib:truth.bzl", "subjects") def _py_cc_toolchain_info_subject_new(info, *, meta): # buildifier: disable=uninitialized public = struct( + abi_flags = lambda *a, **k: _py_cc_toolchain_info_subject_abi_flags(self, *a, **k), headers = lambda *a, **k: _py_cc_toolchain_info_subject_headers(self, *a, **k), + headers_abi3 = lambda *a, **k: _py_cc_toolchain_info_subject_headers_abi3(self, *a, **k), libs = lambda *a, **k: _py_cc_toolchain_info_subject_libs(self, *a, **k), + platform_machine = lambda *a, **k: _py_cc_toolchain_info_subject_platform_machine(self, *a, **k), + platform_tag = lambda *a, **k: _py_cc_toolchain_info_subject_platform_tag(self, *a, **k), python_version = lambda *a, **k: _py_cc_toolchain_info_subject_python_version(self, *a, **k), + soabi = lambda *a, **k: _py_cc_toolchain_info_subject_soabi(self, *a, **k), + sys_platform = lambda *a, **k: _py_cc_toolchain_info_subject_sys_platform(self, *a, **k), actual = info, ) self = struct(actual = info, meta = meta) return public +def _py_cc_toolchain_info_subject_abi_flags(self): + return subjects.str( + self.actual.abi_flags, + meta = self.meta.derive("abi_flags()"), + ) + def _py_cc_toolchain_info_subject_headers(self): return subjects.struct( self.actual.headers, @@ -35,6 +47,15 @@ def _py_cc_toolchain_info_subject_headers(self): ), ) +def _py_cc_toolchain_info_subject_headers_abi3(self): + return subjects.struct( + self.actual.headers_abi3, + meta = self.meta.derive("headers_abi3()"), + attrs = dict( + providers_map = subjects.dict, + ), + ) + def _py_cc_toolchain_info_subject_libs(self): return subjects.struct( self.actual.libs, @@ -44,12 +65,36 @@ def _py_cc_toolchain_info_subject_libs(self): ), ) +def _py_cc_toolchain_info_subject_platform_machine(self): + return subjects.str( + self.actual.platform_machine, + meta = self.meta.derive("platform_machine()"), + ) + +def _py_cc_toolchain_info_subject_platform_tag(self): + return subjects.str( + self.actual.platform_tag, + meta = self.meta.derive("platform_tag()"), + ) + def _py_cc_toolchain_info_subject_python_version(self): return subjects.str( self.actual.python_version, meta = self.meta.derive("python_version()"), ) +def _py_cc_toolchain_info_subject_soabi(self): + return subjects.str( + self.actual.soabi, + meta = self.meta.derive("soabi()"), + ) + +def _py_cc_toolchain_info_subject_sys_platform(self): + return subjects.str( + self.actual.sys_platform, + meta = self.meta.derive("sys_platform()"), + ) + # Disable this to aid doc generation # buildifier: disable=name-conventions PyCcToolchainInfoSubject = struct( diff --git a/tests/support/py_info_subject.bzl b/tests/support/py_info_subject.bzl index bfed0b335d..9122eaa9fd 100644 --- a/tests/support/py_info_subject.bzl +++ b/tests/support/py_info_subject.bzl @@ -31,11 +31,15 @@ def py_info_subject(info, *, meta): # buildifier: disable=uninitialized public = struct( # go/keep-sorted start + direct_original_sources = lambda *a, **k: _py_info_subject_direct_original_sources(self, *a, **k), direct_pyc_files = lambda *a, **k: _py_info_subject_direct_pyc_files(self, *a, **k), + direct_pyi_files = lambda *a, **k: _py_info_subject_direct_pyi_files(self, *a, **k), has_py2_only_sources = lambda *a, **k: _py_info_subject_has_py2_only_sources(self, *a, **k), has_py3_only_sources = lambda *a, **k: _py_info_subject_has_py3_only_sources(self, *a, **k), imports = lambda *a, **k: _py_info_subject_imports(self, *a, **k), + transitive_original_sources = lambda *a, **k: _py_info_subject_transitive_original_sources(self, *a, **k), transitive_pyc_files = lambda *a, **k: _py_info_subject_transitive_pyc_files(self, *a, **k), + transitive_pyi_files = lambda *a, **k: _py_info_subject_transitive_pyi_files(self, *a, **k), transitive_sources = lambda *a, **k: _py_info_subject_transitive_sources(self, *a, **k), uses_shared_libraries = lambda *a, **k: _py_info_subject_uses_shared_libraries(self, *a, **k), # go/keep-sorted end @@ -46,6 +50,14 @@ def py_info_subject(info, *, meta): ) return public +def _py_info_subject_direct_original_sources(self): + """Returns a `DepsetFileSubject` for the `direct_original_sources` attribute. + """ + return subjects.depset_file( + self.actual.direct_original_sources, + meta = self.meta.derive("direct_original_sources()"), + ) + def _py_info_subject_direct_pyc_files(self): """Returns a `DepsetFileSubject` for the `direct_pyc_files` attribute. @@ -56,6 +68,14 @@ def _py_info_subject_direct_pyc_files(self): meta = self.meta.derive("direct_pyc_files()"), ) +def _py_info_subject_direct_pyi_files(self): + """Returns a `DepsetFileSubject` for the `direct_pyi_files` attribute. + """ + return subjects.depset_file( + self.actual.direct_pyi_files, + meta = self.meta.derive("direct_pyi_files()"), + ) + def _py_info_subject_has_py2_only_sources(self): """Returns a `BoolSubject` for the `has_py2_only_sources` attribute. @@ -86,6 +106,16 @@ def _py_info_subject_imports(self): meta = self.meta.derive("imports()"), ) +def _py_info_subject_transitive_original_sources(self): + """Returns a `DepsetFileSubject` for the `transitive_original_sources` attribute. + + Method: PyInfoSubject.transitive_original_sources + """ + return subjects.depset_file( + self.actual.transitive_original_sources, + meta = self.meta.derive("transitive_original_sources()"), + ) + def _py_info_subject_transitive_pyc_files(self): """Returns a `DepsetFileSubject` for the `transitive_pyc_files` attribute. @@ -96,6 +126,14 @@ def _py_info_subject_transitive_pyc_files(self): meta = self.meta.derive("transitive_pyc_files()"), ) +def _py_info_subject_transitive_pyi_files(self): + """Returns a `DepsetFileSubject` for the `transitive_pyi_files` attribute. + """ + return subjects.depset_file( + self.actual.transitive_pyi_files, + meta = self.meta.derive("transitive_pyi_files()"), + ) + def _py_info_subject_transitive_sources(self): """Returns a `DepsetFileSubject` for the `transitive_sources` attribute. diff --git a/tests/support/py_reconfig.bzl b/tests/support/py_reconfig.bzl new file mode 100644 index 0000000000..9bbfdb1104 --- /dev/null +++ b/tests/support/py_reconfig.bzl @@ -0,0 +1,114 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Run a py_binary/py_test with altered config settings. + +This facilitates verify running binaries with different configuration settings +without the overhead of a bazel-in-bazel integration test. +""" + +load("@rules_python_internal//:rules_python_config.bzl", "config") +load("//python/private:attr_builders.bzl", "attrb") # buildifier: disable=bzl-visibility +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility +load("//python/private:py_binary_macro.bzl", "py_binary_macro") # buildifier: disable=bzl-visibility +load("//python/private:py_binary_rule.bzl", "create_py_binary_rule_builder") # buildifier: disable=bzl-visibility +load("//python/private:py_test_macro.bzl", "py_test_macro") # buildifier: disable=bzl-visibility +load("//python/private:py_test_rule.bzl", "create_py_test_rule_builder") # buildifier: disable=bzl-visibility +load("//tests/support:support.bzl", "CUSTOM_RUNTIME") + +def _perform_transition_impl(input_settings, attr, base_impl): + settings = {k: input_settings[k] for k in _RECONFIG_INHERITED_OUTPUTS if k in input_settings} + settings.update(base_impl(input_settings, attr)) + + settings[labels.VISIBLE_FOR_TESTING] = True + + if _BUILTIN_BUILD_PYTHON_ZIP: + settings["//command_line_option:build_python_zip"] = str(attr.build_python_zip) + settings[labels.BUILD_PYTHON_ZIP] = attr.build_python_zip + if attr.bootstrap_impl: + settings[labels.BOOTSTRAP_IMPL] = attr.bootstrap_impl + if attr.extra_toolchains: + settings["//command_line_option:extra_toolchains"] = attr.extra_toolchains + if attr.python_src: + settings[labels.PYTHON_SRC] = attr.python_src + if attr.repl_dep: + settings[labels.REPL_DEP] = attr.repl_dep + if attr.venvs_use_declare_symlink: + settings[labels.VENVS_USE_DECLARE_SYMLINK] = attr.venvs_use_declare_symlink + if attr.venvs_site_packages: + settings[labels.VENVS_SITE_PACKAGES] = attr.venvs_site_packages + return settings + +_BUILTIN_BUILD_PYTHON_ZIP = [] if config.bazel_10_or_later else [ + "//command_line_option:build_python_zip", +] + +_RECONFIG_INPUTS = [ + "//command_line_option:build_runfile_links", + "//command_line_option:extra_toolchains", + "//command_line_option:stamp", + CUSTOM_RUNTIME, + labels.BOOTSTRAP_IMPL, + labels.PYTHON_SRC, + labels.REPL_DEP, + labels.VENVS_SITE_PACKAGES, + labels.VENVS_USE_DECLARE_SYMLINK, +] +_RECONFIG_OUTPUTS = _RECONFIG_INPUTS + [ + labels.BUILD_PYTHON_ZIP, + labels.VISIBLE_FOR_TESTING, +] + _BUILTIN_BUILD_PYTHON_ZIP +_RECONFIG_INHERITED_OUTPUTS = [v for v in _RECONFIG_OUTPUTS if v in _RECONFIG_INPUTS] + +_RECONFIG_ATTRS = { + "bootstrap_impl": attrb.String(), + "build_python_zip": attrb.Bool(default = config.build_python_zip_default), + "config_settings": attrb.LabelKeyedStringDict(), + "extra_toolchains": attrb.StringList( + doc = """ +Value for the --extra_toolchains flag. + +NOTE: You'll likely have to also specify //tests/support/cc_toolchains:all (or some CC toolchain) +to make the RBE presubmits happy, which disable auto-detection of a CC +toolchain. +""", + ), + "python_src": attrb.Label(), + "repl_dep": attrb.Label(), + "venvs_site_packages": attrb.String(), + "venvs_use_declare_symlink": attrb.String(), +} + +def _create_reconfig_rule(builder): + builder.attrs.update(_RECONFIG_ATTRS) + + base_cfg_impl = builder.cfg.implementation() + builder.cfg.set_implementation(lambda *args: _perform_transition_impl(base_impl = base_cfg_impl, *args)) + builder.cfg.update_inputs(_RECONFIG_INPUTS) + builder.cfg.update_outputs(_RECONFIG_OUTPUTS) + return builder.build() + +_py_reconfig_binary = _create_reconfig_rule(create_py_binary_rule_builder()) + +_py_reconfig_test = _create_reconfig_rule(create_py_test_rule_builder()) + +def py_reconfig_test(**kwargs): + """Create a py_test with customized build settings for testing. + + Args: + **kwargs: kwargs to pass along to _py_reconfig_test. + """ + py_test_macro(_py_reconfig_test, **kwargs) + +def py_reconfig_binary(**kwargs): + py_binary_macro(_py_reconfig_binary, **kwargs) diff --git a/tests/support/py_runtime_info_subject.bzl b/tests/support/py_runtime_info_subject.bzl index 541d4d9e18..0c6c672743 100644 --- a/tests/support/py_runtime_info_subject.bzl +++ b/tests/support/py_runtime_info_subject.bzl @@ -37,6 +37,9 @@ def py_runtime_info_subject(info, *, meta): coverage_tool = lambda *a, **k: _py_runtime_info_subject_coverage_tool(self, *a, **k), files = lambda *a, **k: _py_runtime_info_subject_files(self, *a, **k), interpreter = lambda *a, **k: _py_runtime_info_subject_interpreter(self, *a, **k), + interpreter_files_to_run = lambda *a, **k: ( + _py_runtime_info_subject_interpreter_files_to_run(self, *a, **k) + ), interpreter_path = lambda *a, **k: _py_runtime_info_subject_interpreter_path(self, *a, **k), interpreter_version_info = lambda *a, **k: _py_runtime_info_subject_interpreter_version_info(self, *a, **k), python_version = lambda *a, **k: _py_runtime_info_subject_python_version(self, *a, **k), @@ -84,6 +87,15 @@ def _py_runtime_info_subject_interpreter(self): meta = self.meta.derive("interpreter()"), ) +def _py_runtime_info_subject_interpreter_files_to_run(self): + return subjects.struct( + self.actual.interpreter_files_to_run, + attrs = dict( + executable = subjects.file, + ), + meta = self.meta.derive("interpreter_files_to_run()"), + ) + def _py_runtime_info_subject_interpreter_path(self): return subjects.str( self.actual.interpreter_path, diff --git a/tests/support/pyrefly/BUILD.bazel b/tests/support/pyrefly/BUILD.bazel new file mode 100644 index 0000000000..447af06f8f --- /dev/null +++ b/tests/support/pyrefly/BUILD.bazel @@ -0,0 +1,8 @@ +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") + +package(default_visibility = ["//:__subpackages__"]) + +bzl_library( + name = "pyrefly", + srcs = ["pyrefly.bzl"], +) diff --git a/tests/support/pyrefly/pyrefly.bzl b/tests/support/pyrefly/pyrefly.bzl new file mode 100644 index 0000000000..bcb791403f --- /dev/null +++ b/tests/support/pyrefly/pyrefly.bzl @@ -0,0 +1,7 @@ +"""Aspect definition for Pyrefly static type checking.""" + +load("@rules_pyrefly//pyrefly:pyrefly.bzl", "pyrefly") + +pyrefly_aspect = pyrefly( + opt_in_tags = ["pyrefly"], +) diff --git a/tests/support/pytest_test/BUILD.bazel b/tests/support/pytest_test/BUILD.bazel new file mode 100644 index 0000000000..4e6f6dd168 --- /dev/null +++ b/tests/support/pytest_test/BUILD.bazel @@ -0,0 +1,29 @@ +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") + +package(default_visibility = ["//:__subpackages__"]) + +filegroup( + name = "bootstrap_template", + srcs = ["pytest_bootstrap_template.py"], +) + +# keep +bzl_library( + name = "pytest_test", + srcs = ["pytest_test.bzl"], + deps = [ + "//python:py_test", + ], +) + +# These aliases are used to avoid duplicate targets in the deps list +alias( + name = "default_pytest", + actual = "@pypi//pytest", +) + +# These aliases are used to avoid duplicate targets in the deps list +alias( + name = "default_pytest_bazel", + actual = "@pypi//pytest_bazel", +) diff --git a/tests/support/pytest_test/pytest_bootstrap_template.py b/tests/support/pytest_test/pytest_bootstrap_template.py new file mode 100644 index 0000000000..9769531f47 --- /dev/null +++ b/tests/support/pytest_test/pytest_bootstrap_template.py @@ -0,0 +1,8 @@ +import sys + +import pytest_bazel + +TEST_FILES = """%TEST_FILES%""".splitlines() + +args = sys.argv[1:] + TEST_FILES +sys.exit(pytest_bazel.main(args)) diff --git a/tests/support/pytest_test/pytest_test.bzl b/tests/support/pytest_test/pytest_test.bzl new file mode 100644 index 0000000000..893b22e5ce --- /dev/null +++ b/tests/support/pytest_test/pytest_test.bzl @@ -0,0 +1,74 @@ +"""pytest_test rule implementation.""" + +load("//python:py_test.bzl", "py_test") + +_DEFAULT_PYTEST = Label("//tests/support/pytest_test:default_pytest") +_DEFAULT_PYTEST_BAZEL = Label("//tests/support/pytest_test:default_pytest_bazel") + +def pytest_test( + *, + name, + srcs, + pytest = None, + pytest_bazel = None, + **kwargs): + """Run pytest tests. + + Args: + name: A unique name for this target. + srcs: List of source files (test files). These are the files that + pytest will run as tests. + pytest: The pytest target to use. Defaults to @pypi//pytest. + pytest_bazel: The pytest-bazel target to use. Defaults to + @pypi//pytest_bazel. + **kwargs: Additional arguments passed to py_test. Note that `main` is + not a supported argument. + """ + if pytest == None: + pytest = _DEFAULT_PYTEST + if pytest_bazel == None: + pytest_bazel = _DEFAULT_PYTEST_BAZEL + + bootstrap_target = name + "_bootstrap" + main_file = name + "_boot.py" + _write_pytest_bootstrap( + name = bootstrap_target, + srcs = srcs, + output_name = main_file, + ) + + py_test( + name = name, + main = main_file, + srcs = [bootstrap_target] + srcs, + deps = kwargs.pop("deps", []) + [ + pytest, + pytest_bazel, + ], + **kwargs + ) + +def _write_pytest_bootstrap_impl(ctx): + output = ctx.actions.declare_file(ctx.attr.output_name) + test_files = "\n".join([f.short_path for f in ctx.files.srcs]) + + ctx.actions.expand_template( + output = output, + template = ctx.file._bootstrap_template, + substitutions = { + "%TEST_FILES%": test_files, + }, + ) + return [DefaultInfo(files = depset([output]))] + +_write_pytest_bootstrap = rule( + implementation = _write_pytest_bootstrap_impl, + attrs = { + "output_name": attr.string(mandatory = True), + "srcs": attr.label_list(allow_files = True), + "_bootstrap_template": attr.label( + default = "//tests/support/pytest_test:bootstrap_template", + allow_single_file = True, + ), + }, +) diff --git a/tests/support/sh_py_run_test.bzl b/tests/support/sh_py_run_test.bzl index 7fb7016eec..7dff6673ce 100644 --- a/tests/support/sh_py_run_test.bzl +++ b/tests/support/sh_py_run_test.bzl @@ -18,135 +18,9 @@ without the overhead of a bazel-in-bazel integration test. """ load("@rules_shell//shell:sh_test.bzl", "sh_test") -load("//python:py_binary.bzl", "py_binary") -load("//python:py_test.bzl", "py_test") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:toolchain_types.bzl", "TARGET_TOOLCHAIN_TYPE") # buildifier: disable=bzl-visibility -load("//tests/support:support.bzl", "VISIBLE_FOR_TESTING") - -def _perform_transition_impl(input_settings, attr): - settings = dict(input_settings) - settings[VISIBLE_FOR_TESTING] = True - settings["//command_line_option:build_python_zip"] = attr.build_python_zip - if attr.bootstrap_impl: - settings["//python/config_settings:bootstrap_impl"] = attr.bootstrap_impl - if attr.extra_toolchains: - settings["//command_line_option:extra_toolchains"] = attr.extra_toolchains - if attr.python_version: - settings["//python/config_settings:python_version"] = attr.python_version - return settings - -_perform_transition = transition( - implementation = _perform_transition_impl, - inputs = [ - "//python/config_settings:bootstrap_impl", - "//command_line_option:extra_toolchains", - "//python/config_settings:python_version", - ], - outputs = [ - "//command_line_option:build_python_zip", - "//command_line_option:extra_toolchains", - "//python/config_settings:bootstrap_impl", - "//python/config_settings:python_version", - VISIBLE_FOR_TESTING, - ], -) - -def _py_reconfig_impl(ctx): - default_info = ctx.attr.target[DefaultInfo] - exe_ext = default_info.files_to_run.executable.extension - if exe_ext: - exe_ext = "." + exe_ext - exe_name = ctx.label.name + exe_ext - - executable = ctx.actions.declare_file(exe_name) - ctx.actions.symlink(output = executable, target_file = default_info.files_to_run.executable) - - default_outputs = [executable] - - # todo: could probably check target.owner vs src.owner to check if it should - # be symlinked or included as-is - # For simplicity of implementation, we're assuming the target being run is - # py_binary-like. In order for Windows to work, we need to make sure the - # file that the .exe launcher runs (the .zip or underlying non-exe - # executable) is a sibling of the .exe file with the same base name. - for src in default_info.files.to_list(): - if src.extension in ("", "zip"): - ext = ("." if src.extension else "") + src.extension - output = ctx.actions.declare_file(ctx.label.name + ext) - ctx.actions.symlink(output = output, target_file = src) - default_outputs.append(output) - - return [ - DefaultInfo( - executable = executable, - files = depset(default_outputs), - # On windows, the other default outputs must also be included - # in runfiles so the exe launcher can find the backing file. - runfiles = ctx.runfiles(default_outputs).merge( - default_info.default_runfiles, - ), - ), - testing.TestEnvironment( - environment = ctx.attr.env, - ), - ] - -def _make_reconfig_rule(**kwargs): - attrs = { - "bootstrap_impl": attr.string(), - "build_python_zip": attr.string(default = "auto"), - "env": attr.string_dict(), - "extra_toolchains": attr.string_list( - doc = """ -Value for the --extra_toolchains flag. - -NOTE: You'll likely have to also specify //tests/support/cc_toolchains:all (or some CC toolchain) -to make the RBE presubmits happy, which disable auto-detection of a CC -toolchain. -""", - ), - "python_version": attr.string(), - "target": attr.label(executable = True, cfg = "target"), - "_allowlist_function_transition": attr.label( - default = "@bazel_tools//tools/allowlists/function_transition_allowlist", - ), - } - return rule( - implementation = _py_reconfig_impl, - attrs = attrs, - cfg = _perform_transition, - **kwargs - ) - -_py_reconfig_binary = _make_reconfig_rule(executable = True) - -_py_reconfig_test = _make_reconfig_rule(test = True) - -def py_reconfig_test(*, name, **kwargs): - """Create a py_test with customized build settings for testing. - - Args: - name: str, name of teset target. - **kwargs: kwargs to pass along to _py_reconfig_test and py_test. - """ - reconfig_kwargs = {} - reconfig_kwargs["bootstrap_impl"] = kwargs.pop("bootstrap_impl", None) - reconfig_kwargs["extra_toolchains"] = kwargs.pop("extra_toolchains", None) - reconfig_kwargs["python_version"] = kwargs.pop("python_version", None) - reconfig_kwargs["env"] = kwargs.get("env") - reconfig_kwargs["target_compatible_with"] = kwargs.get("target_compatible_with") - - inner_name = "_{}_inner".format(name) - _py_reconfig_test( - name = name, - target = inner_name, - **reconfig_kwargs - ) - py_test( - name = inner_name, - tags = ["manual"], - **kwargs - ) +load(":py_reconfig.bzl", "py_reconfig_binary") def sh_py_run_test(*, name, sh_src, py_src, **kwargs): """Run a py_binary within a sh_test. @@ -169,26 +43,12 @@ def sh_py_run_test(*, name, sh_src, py_src, **kwargs): "BIN_RLOCATION": "$(rlocationpaths {})".format(bin_name), }, ) - - py_binary_kwargs = { - key: kwargs.pop(key) - for key in ("imports", "deps") - if key in kwargs - } - - _py_reconfig_binary( + py_reconfig_binary( name = bin_name, - tags = ["manual"], - target = "_{}_plain_bin".format(name), - **kwargs - ) - - py_binary( - name = "_{}_plain_bin".format(name), srcs = [py_src], main = py_src, tags = ["manual"], - **py_binary_kwargs + **kwargs ) def _current_build_settings_impl(ctx): @@ -199,6 +59,7 @@ def _current_build_settings_impl(ctx): ctx.actions.write( output = info, content = json.encode({ + "bootstrap_impl": ctx.attr._bootstrap_impl_flag[config_common.FeatureFlagInfo].value, "interpreter": { "short_path": runtime.interpreter.short_path if runtime.interpreter else None, }, @@ -217,6 +78,11 @@ Writes information about the current build config to JSON for testing. This is so tests can verify information about the build config used for them. """, implementation = _current_build_settings_impl, + attrs = { + "_bootstrap_impl_flag": attr.label( + default = labels.BOOTSTRAP_IMPL, + ), + }, toolchains = [ TARGET_TOOLCHAIN_TYPE, ], diff --git a/tests/support/support.bzl b/tests/support/support.bzl index 7358a6b1ee..64f77d76bc 100644 --- a/tests/support/support.bzl +++ b/tests/support/support.bzl @@ -19,12 +19,8 @@ # rules_testing or as config_setting values, which don't support Label in some # places. -MAC = Label("//tests/support:mac") -MAC_X86_64 = Label("//tests/support:mac_x86_64") -LINUX = Label("//tests/support:linux") -LINUX_X86_64 = Label("//tests/support:linux_x86_64") -WINDOWS = Label("//tests/support:windows") -WINDOWS_X86_64 = Label("//tests/support:windows_x86_64") +load("@rules_python_internal//:rules_python_config.bzl", "config") +load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility PY_TOOLCHAINS = str(Label("//tests/support/py_toolchains:all")) CC_TOOLCHAIN = str(Label("//tests/support/cc_toolchains:all")) @@ -32,10 +28,18 @@ CROSSTOOL_TOP = Label("//tests/support/cc_toolchains:cc_toolchain_suite") # str() around Label() is necessary because rules_testing's config_settings # doesn't accept yet Label objects. -ADD_SRCS_TO_RUNFILES = str(Label("//python/config_settings:add_srcs_to_runfiles")) -EXEC_TOOLS_TOOLCHAIN = str(Label("//python/config_settings:exec_tools_toolchain")) -PRECOMPILE = str(Label("//python/config_settings:precompile")) -PRECOMPILE_SOURCE_RETENTION = str(Label("//python/config_settings:precompile_source_retention")) -PYC_COLLECTION = str(Label("//python/config_settings:pyc_collection")) -PYTHON_VERSION = str(Label("//python/config_settings:python_version")) -VISIBLE_FOR_TESTING = str(Label("//python/private:visible_for_testing")) +CUSTOM_RUNTIME = str(Label("//tests/support:custom_runtime")) + +SUPPORTS_BOOTSTRAP_SCRIPT = select({ + "@platforms//os:windows": ["@platforms//:incompatible"], + "//conditions:default": [], +}) + +SUPPORTS_BZLMOD = [] if BZLMOD_ENABLED else ["@platforms//:incompatible"] + +NOT_WINDOWS = select({ + "@platforms//os:windows": ["@platforms//:incompatible"], + "//conditions:default": [], +}) + +BAZEL_8_OR_LATER = [] if config.bazel_8_or_later else ["@platforms//:incompatible"] diff --git a/tests/support/whl_from_dir/BUILD.bazel b/tests/support/whl_from_dir/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/support/whl_from_dir/whl_from_dir_repo.bzl b/tests/support/whl_from_dir/whl_from_dir_repo.bzl new file mode 100644 index 0000000000..c827c8a0a0 --- /dev/null +++ b/tests/support/whl_from_dir/whl_from_dir_repo.bzl @@ -0,0 +1,75 @@ +"""Creates a whl file from a directory tree. + +Used to test wheels. Avoids checking in prebuilt files and their associated +security risks. +""" + +load("//python/private:repo_utils.bzl", "repo_utils") # buildifier: disable=bzl-visibility + +def _whl_from_dir_repo(rctx): + root = rctx.path(rctx.attr.root).dirname + rctx.watch_tree(root) + + output = rctx.path(rctx.attr.output) + if repo_utils.get_platforms_os_name(rctx) == "windows": + powershell_exe = rctx.which("powershell.exe") or rctx.which("powershell") + if not powershell_exe: + fail("powershell not found on PATH") + + zip_script = rctx.path(rctx.attr._zip_script) + + repo_utils.execute_checked( + rctx, + op = "WhlFromDir", + arguments = [ + powershell_exe, + "-NoProfile", + "-File", + str(zip_script), + str(output), + str(root), + ], + # zip.ps1 handles relativizing paths. + ) + else: + repo_utils.execute_checked( + rctx, + # cd to root so zip recursively takes everything there. + working_directory = str(root), + op = "WhlFromDir", + arguments = [ + "zip", + "-0", # Skip compressing + "-X", # Don't store file time or metadata + str(output), + "-r", + ".", + ], + ) + rctx.file("BUILD.bazel", 'exports_files(glob(["*"]))') + +whl_from_dir_repo = repository_rule( + implementation = _whl_from_dir_repo, + attrs = { + "output": attr.string( + doc = """ +Output file name to write. Should match the wheel filename format: +`pkg-version-pyversion-abi-platform.whl`. Typically a value like +`mypkg-1.0-any-none-any.whl` is whats used for testing. + +For the full format, see +https://packaging.python.org/en/latest/specifications/binary-distribution-format/#file-name-convention +""", + ), + "root": attr.label( + doc = """ +A file whose directory will be put into the output wheel. All files +are included verbatim. + """, + ), + "_zip_script": attr.label( + default = "//tests/support/whl_from_dir:zip.ps1", + allow_single_file = True, + ), + }, +) diff --git a/tests/support/whl_from_dir/zip.ps1 b/tests/support/whl_from_dir/zip.ps1 new file mode 100644 index 0000000000..1e8c199cde --- /dev/null +++ b/tests/support/whl_from_dir/zip.ps1 @@ -0,0 +1,45 @@ +param ( + [Parameter(Position=0, Mandatory=$true)] + [string]$Output, + + [Parameter(Position=1, Mandatory=$true)] + [string]$Root +) + +Add-Type -AssemblyName System.IO.Compression + +$fixedTime = [datetime]"1980-01-01T00:00:00" +$RootFull = (Resolve-Path $Root).Path + +$stream = [System.IO.File]::Open($Output, [System.IO.FileMode]::Create) +try { + $archive = [System.IO.Compression.ZipArchive]::new($stream, [System.IO.Compression.ZipArchiveMode]::Create) + try { + $files = Get-ChildItem -Path $RootFull -Recurse -File + foreach ($file in $files) { + # Relativize path and normalize separators + $relPath = $file.FullName.Substring($RootFull.Length).TrimStart('\', '/') + $relPath = $relPath -replace '\\', '/' + + $entry = $archive.CreateEntry($relPath, [System.IO.Compression.CompressionLevel]::NoCompression) + $entry.LastWriteTime = $fixedTime + + $entryStream = $entry.Open() + try { + $fileStream = [System.IO.File]::OpenRead($file.FullName) + try { + $fileStream.CopyTo($entryStream) + } finally { + $fileStream.Dispose() + } + } finally { + $entryStream.Dispose() + } + } + } finally { + $archive.Dispose() + } +} finally { + $stream.Dispose() +} + diff --git a/tests/toolchains/BUILD.bazel b/tests/toolchains/BUILD.bazel index c55dc92a7d..b336a06269 100644 --- a/tests/toolchains/BUILD.bazel +++ b/tests/toolchains/BUILD.bazel @@ -12,8 +12,45 @@ # See the License for the specific language governing permissions and # limitations under the License. +load("@bazel_skylib//rules:build_test.bzl", "build_test") +load("@bazel_skylib//rules:diff_test.bzl", "diff_test") +load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") load(":defs.bzl", "define_toolchain_tests") define_toolchain_tests( name = "toolchain_tests", ) + +py_reconfig_test( + name = "custom_platform_toolchain_test", + srcs = ["custom_platform_toolchain_test.py"], + config_settings = {"//tests/support:custom_runtime": "linux-x86-install-only-stripped"}, + python_version = "3.13.1", + target_compatible_with = [ + "@platforms//os:linux", + "@platforms//cpu:x86_64", + ] if BZLMOD_ENABLED else ["@platforms//:incompatible"], +) + +build_test( + name = "build_test", + targets = [ + "@python_3_11//:python_headers", + ], +) + +# Verify that runtimes_manifest_workspace.bzl exactly matches runtimes_manifest.txt +genrule( + name = "gen_expected_runtimes_manifest_workspace", + srcs = ["//python/private:runtimes_manifest.txt"], + outs = ["expected_runtimes_manifest_workspace.bzl"], + cmd = "$(execpath //python/private:sync_runtimes_manifest_workspace) $(location //python/private:runtimes_manifest.txt) $@", + tools = ["//python/private:sync_runtimes_manifest_workspace"], +) + +diff_test( + name = "runtimes_manifest_workspace_sync_test", + file1 = "//python/private:runtimes_manifest_workspace.bzl", + file2 = ":expected_runtimes_manifest_workspace.bzl", +) diff --git a/tests/toolchains/custom_platform_toolchain_test.py b/tests/toolchains/custom_platform_toolchain_test.py new file mode 100644 index 0000000000..2769d4bcf9 --- /dev/null +++ b/tests/toolchains/custom_platform_toolchain_test.py @@ -0,0 +1,15 @@ +import sys +import unittest + + +class VerifyCustomPlatformToolchainTest(unittest.TestCase): + def test_custom_platform_interpreter_used(self): + # For lack of a better option, check the version. Identifying the + self.assertEqual( + "3.13.1", + f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}", + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/toolchains/defs.bzl b/tests/toolchains/defs.bzl index fbb70820c9..fb4b3beb94 100644 --- a/tests/toolchains/defs.bzl +++ b/tests/toolchains/defs.bzl @@ -15,7 +15,8 @@ "" load("//python:versions.bzl", "PLATFORMS", "TOOL_VERSIONS") -load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test") +load("//python/private:version.bzl", "version") # buildifier: disable=bzl-visibility +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") def define_toolchain_tests(name): """Define the toolchain tests. @@ -38,15 +39,23 @@ def define_toolchain_tests(name): is_platform = "_is_{}".format(platform_key) target_compatible_with[is_platform] = [] + parsed = version.parse(python_version, strict = True) + expect_python_version = "{0}.{1}.{2}".format(*parsed.release) + if parsed.pre: + expect_python_version = "{0}{1}{2}".format( + expect_python_version, + *parsed.pre + ) py_reconfig_test( name = "python_{}_test".format(python_version), srcs = ["python_toolchain_test.py"], main = "python_toolchain_test.py", python_version = python_version, env = { - "EXPECT_PYTHON_VERSION": python_version, + "EXPECT_PYTHON_VERSION": expect_python_version, }, deps = ["//python/runfiles"], data = ["//tests/support:current_build_settings"], target_compatible_with = select(target_compatible_with), + size = "large", ) diff --git a/tests/toolchains/multi_platform_resolution/BUILD.bazel b/tests/toolchains/multi_platform_resolution/BUILD.bazel new file mode 100644 index 0000000000..35f18e98ed --- /dev/null +++ b/tests/toolchains/multi_platform_resolution/BUILD.bazel @@ -0,0 +1,3 @@ +load(":resolution_tests.bzl", "resolution_test_suite") + +resolution_test_suite(name = "resolution_tests") diff --git a/tests/toolchains/multi_platform_resolution/resolution_tests.bzl b/tests/toolchains/multi_platform_resolution/resolution_tests.bzl new file mode 100644 index 0000000000..a48d815789 --- /dev/null +++ b/tests/toolchains/multi_platform_resolution/resolution_tests.bzl @@ -0,0 +1,177 @@ +"""Tests to verify toolchain resolution of different config variants. + +NOTE: This test relies on the project toolchain configuration. This is +intentional because it wants to verify that, using the toolchains as +rules_python configures them, the different implementations can be used +by setting the appropriate flags. +""" + +load("@bazel_skylib//lib:structs.bzl", "structs") +load("@rules_testing//lib:analysis_test.bzl", "analysis_test") +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python:versions.bzl", "TOOL_VERSIONS") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility +load("//python/private:toolchain_types.bzl", "TARGET_TOOLCHAIN_TYPE") # buildifier: disable=bzl-visibility +load("//python/private:version.bzl", "version") # buildifier: disable=bzl-visibility +load("//tests/support/platforms:platforms.bzl", "platform_targets") + +_PLATFORM_TARGET_MAP = { + "linux": { + "aarch64": platform_targets.LINUX_AARCH64, + "x86_64": platform_targets.LINUX_X86_64, + }, + "osx": { + "aarch64": platform_targets.MAC_AARCH64, + "x86_64": platform_targets.MAC_X86_64, + }, + "windows": { + "aarch64": platform_targets.WINDOWS_AARCH64, + "x86_64": platform_targets.WINDOWS_X86_64, + }, +} +_PLATFORM_TRIPLES = { + ("linux", "glibc"): "unknown_linux_gnu", + ("linux", "musl"): "unknown_linux_musl", + "osx": "apple_darwin", + "windows": "pc_windows_msvc", +} + +_ResolvedToolchainsInfo = provider( + doc = "Tell what toolchain was found", + fields = { + "target": "ToolchainInfo for //python:toolchain_type", + }, +) + +def _current_toolchain_impl(ctx): + # todo: also return current settings for various config flags + # to help identify state + return [_ResolvedToolchainsInfo( + target = ctx.toolchains[TARGET_TOOLCHAIN_TYPE], + )] + +_current_toolchain = rule( + implementation = _current_toolchain_impl, + toolchains = [ + TARGET_TOOLCHAIN_TYPE, + ], +) + +def _platform(os, arch, libc, *, ft): + if os == "linux": + platform_triple = _PLATFORM_TRIPLES[os, libc] + else: + platform_triple = _PLATFORM_TRIPLES[os] + return struct( + arch = arch, + freethreaded = ft, + libc = libc, + os = os, + platform_triple = platform_triple, + platform_target = _PLATFORM_TARGET_MAP[os][arch], + ) + +# There's many exceptions to the full `os x arch x libc x threading` matrix, +# so just list the specific combinations that are supported. +# We also omit some more esoteric archs to reduce the matrix size (and +# thus how many runtimes get downloaded). +# The important quality is to have at least 2 for every dimension +_PLATFORMS = [ + _platform("linux", "aarch64", "glibc", ft = "no"), + _platform("linux", "aarch64", "glibc", ft = "yes"), + _platform("linux", "x86_64", "glibc", ft = "no"), + _platform("linux", "x86_64", "glibc", ft = "yes"), + _platform("linux", "x86_64", "musl", ft = "no"), + _platform("osx", "aarch64", None, ft = "no"), + _platform("osx", "aarch64", None, ft = "yes"), + _platform("osx", "x86_64", None, ft = "no"), + _platform("osx", "x86_64", None, ft = "yes"), + _platform("windows", "aarch64", None, ft = "no"), + _platform("windows", "aarch64", None, ft = "yes"), + _platform("windows", "x86_64", None, ft = "no"), + _platform("windows", "x86_64", None, ft = "yes"), +] + +def _compute_runtimes(): + runtimes = [] + + # Limit to the two most recent versions. This helps ensure that multiple + # versions are matching correctly. Limit to two because the disk/download + # isn't worth the marginal coverage improvment. + selected_versions = sorted( + TOOL_VERSIONS.keys(), + key = lambda v: version.parse(v).key(), + )[-2:] + + for python_version in selected_versions: + for platform in _PLATFORMS: + runtimes.append(struct( + name = "{python_version}_{arch}_{triple}{threading}".format( + python_version = python_version.replace(".", "_"), + arch = platform.arch, + triple = platform.platform_triple, + threading = "_freethreaded" if platform.freethreaded == "yes" else "", + ), + python_version = python_version, + **structs.to_dict(platform) + )) + + return sorted(runtimes, key = lambda v: v.name) + +def _test_toolchains_impl(env, target): + target_tc = target[_ResolvedToolchainsInfo].target + toolchain_str = str(target_tc.toolchain_label).replace("-", "_") + env.expect.that_str(toolchain_str).contains(env.ctx.attr.expected_toolchain_name) + +def _test_toolchains(name): + _current_toolchain( + name = name + "_current_toolchain", + ) + test_names = [] + for runtime in _compute_runtimes(): + test_name = "test_{}".format(runtime.name) + test_names.append(test_name) + config_settings = { + "//command_line_option:platforms": [runtime.platform_target], + labels.VISIBLE_FOR_TESTING: True, + labels.PY_FREETHREADED: runtime.freethreaded, + labels.PYTHON_VERSION: runtime.python_version, + } + if runtime.libc: + config_settings[labels.PY_LINUX_LIBC] = runtime.libc + + analysis_test( + name = test_name, + target = name + "_current_toolchain", + impl = _test_toolchains_impl, + config_settings = config_settings, + attrs = {"expected_toolchain_name": attr.string()}, + attr_values = { + "expected_toolchain_name": runtime.name, + # A lot of tests are generated, so set tags to make selecting + # subsets easier + "tags": [ + "python-version={}".format(runtime.python_version), + "libc={}".format(runtime.libc), + "freethreaded={}".format(runtime.freethreaded), + "os={}".format(runtime.os), + "arch={}".format(runtime.arch), + ], + }, + ) + + # We have to return a target for `name`. + native.test_suite( + name = name, + tests = test_names, + ) + +_tests = [ + _test_toolchains, +] + +def resolution_test_suite(name): + test_suite( + name = name, + tests = _tests, + ) diff --git a/tests/toolchains/python_toolchain_test.py b/tests/toolchains/python_toolchain_test.py index 371b252a4a..ff45fc0863 100644 --- a/tests/toolchains/python_toolchain_test.py +++ b/tests/toolchains/python_toolchain_test.py @@ -1,6 +1,7 @@ import json import os import pathlib +import pprint import sys import unittest @@ -18,9 +19,22 @@ def test_expected_toolchain_matches(self): settings = json.loads(pathlib.Path(settings_path).read_text()) expected = "python_{}".format(expect_version.replace(".", "_")) - self.assertIn(expected, settings["toolchain_label"], str(settings)) + msg = ( + "Expected toolchain not found\n" + + f"Expected toolchain label to contain: {expected}\n" + + "Actual build settings:\n" + + pprint.pformat(settings) + ) + self.assertIn(expected, settings["toolchain_label"], msg) actual = "{v.major}.{v.minor}.{v.micro}".format(v=sys.version_info) + if sys.version_info.releaselevel != "final": + release_prefix = ( + "rc" + if sys.version_info.releaselevel == "candidate" + else sys.version_info.releaselevel[0] + ) + actual = f"{actual}{release_prefix}{sys.version_info.serial}" self.assertEqual(actual, expect_version) diff --git a/tests/toolchains/transitions/BUILD.bazel b/tests/toolchains/transitions/BUILD.bazel new file mode 100644 index 0000000000..a7bef8c0e5 --- /dev/null +++ b/tests/toolchains/transitions/BUILD.bazel @@ -0,0 +1,5 @@ +load(":transitions_tests.bzl", "transitions_test_suite") + +transitions_test_suite( + name = "transitions_tests", +) diff --git a/tests/toolchains/transitions/transitions_tests.bzl b/tests/toolchains/transitions/transitions_tests.bzl new file mode 100644 index 0000000000..81ce1e68cc --- /dev/null +++ b/tests/toolchains/transitions/transitions_tests.bzl @@ -0,0 +1,188 @@ +# Copyright 2022 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"" + +load("@pythons_hub//:versions.bzl", "DEFAULT_PYTHON_VERSION", "MINOR_MAPPING", "PYTHON_VERSIONS") +load("@rules_testing//lib:analysis_test.bzl", "analysis_test") +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("@rules_testing//lib:util.bzl", rt_util = "util") +load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility +load("//python/private:full_version.bzl", "full_version") # buildifier: disable=bzl-visibility +load("//python/private:toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE") # buildifier: disable=bzl-visibility + +_analysis_tests = [] + +def _transition_impl(input_settings, attr): + """Transition based on python_version flag. + + This is a simple transition impl that a user of rules_python may implement + for their own rule. + """ + settings = { + labels.PYTHON_VERSION: input_settings[labels.PYTHON_VERSION], + } + if attr.python_version: + settings[labels.PYTHON_VERSION] = attr.python_version + return settings + +_python_version_transition = transition( + implementation = _transition_impl, + inputs = [labels.PYTHON_VERSION], + outputs = [labels.PYTHON_VERSION], +) + +TestInfo = provider( + doc = "A simple test provider to forward the values for the assertion.", + fields = {"got": "", "want": ""}, +) + +def _impl(ctx): + if ctx.attr.skip: + return [TestInfo(got = "", want = "")] + + exec_tools = ctx.toolchains[EXEC_TOOLS_TOOLCHAIN_TYPE].exec_tools + got_version = exec_tools.exec_interpreter[platform_common.ToolchainInfo].py3_runtime.interpreter_version_info + got = "{}.{}.{}".format( + got_version.major, + got_version.minor, + got_version.micro, + ) + if got_version.releaselevel != "final": + got = "{}{}{}".format( + got, + "rc" if got_version.releaselevel == "candidate" else got_version.releaselevel[0], + got_version.serial, + ) + + return [ + TestInfo( + got = got, + want = ctx.attr.want_version, + ), + ] + +_simple_transition = rule( + implementation = _impl, + attrs = { + "python_version": attr.string( + doc = "The input python version which we transition on.", + ), + "skip": attr.bool( + doc = "Whether to skip the test", + ), + "want_version": attr.string( + doc = "The python version that we actually expect to receive.", + ), + "_allowlist_function_transition": attr.label( + default = "@bazel_tools//tools/allowlists/function_transition_allowlist", + ), + }, + toolchains = [ + config_common.toolchain_type( + EXEC_TOOLS_TOOLCHAIN_TYPE, + mandatory = False, + ), + ], + cfg = _python_version_transition, +) + +def _test_transitions(*, name, tests, skip = False): + """A reusable rule so that we can split the tests.""" + targets = {} + for test_name, (input_version, want_version) in tests.items(): + target_name = "{}_{}".format(name, test_name) + targets["python_" + test_name] = target_name + rt_util.helper_target( + _simple_transition, + name = target_name, + python_version = input_version, + want_version = want_version, + skip = skip, + ) + + analysis_test( + name = name, + impl = _test_transition_impl, + targets = targets, + ) + +def _test_transition_impl(env, targets): + # Check that the forwarded version from the PyRuntimeInfo is correct + for target in dir(targets): + if not target.startswith("python"): + # Skip other attributes that might be not the ones we set (e.g. to_json, to_proto). + continue + + test_info = env.expect.that_target(getattr(targets, target)).provider( + TestInfo, + factory = lambda v, meta: v, + ) + env.expect.that_str(test_info.got).equals(test_info.want) + +def _test_full_version(name): + """Check that python_version transitions work. + + Expectation is to get the same full version that we input. + """ + _test_transitions( + name = name, + tests = { + v.replace(".", "_"): (v, v) + for v in PYTHON_VERSIONS + }, + ) + +_analysis_tests.append(_test_full_version) + +def _test_minor_versions(name): + """Ensure that MINOR_MAPPING versions are correctly selected.""" + _test_transitions( + name = name, + skip = not BZLMOD_ENABLED, + tests = { + minor.replace(".", "_"): (minor, full) + for minor, full in MINOR_MAPPING.items() + }, + ) + +_analysis_tests.append(_test_minor_versions) + +def _test_default(name): + """Check the default version. + + Lastly, if we don't provide any version to the transition, we should + get the default version + """ + default_version = full_version( + version = DEFAULT_PYTHON_VERSION, + minor_mapping = MINOR_MAPPING, + ) if DEFAULT_PYTHON_VERSION else "" + + _test_transitions( + name = name, + skip = not BZLMOD_ENABLED, + tests = { + "default": (None, default_version), + }, + ) + +_analysis_tests.append(_test_default) + +def transitions_test_suite(name): + test_suite( + name = name, + tests = _analysis_tests, + ) diff --git a/tests/tools/BUILD.bazel b/tests/tools/BUILD.bazel new file mode 100644 index 0000000000..4d163f19f1 --- /dev/null +++ b/tests/tools/BUILD.bazel @@ -0,0 +1,23 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +load("//python:py_test.bzl", "py_test") + +licenses(["notice"]) + +py_test( + name = "wheelmaker_test", + size = "small", + srcs = ["wheelmaker_test.py"], + deps = ["//tools:wheelmaker"], +) diff --git a/tests/tools/private/release/BUILD.bazel b/tests/tools/private/release/BUILD.bazel new file mode 100644 index 0000000000..dcef2ab53a --- /dev/null +++ b/tests/tools/private/release/BUILD.bazel @@ -0,0 +1,212 @@ +load("//python:py_library.bzl", "py_library") +load("//tests/support:support.bzl", "NOT_WINDOWS") +load("//tests/support/pytest_test:pytest_test.bzl", "pytest_test") + +py_library( + name = "conftest", + testonly = True, + srcs = ["conftest.py"], + target_compatible_with = NOT_WINDOWS, + deps = [ + ":release_test_helper", + "//tools/private/release:release_lib", + "@pypi//pytest_mock", + ], +) + +py_library( + name = "release_test_helper", + srcs = ["release_test_helper.py"], + target_compatible_with = NOT_WINDOWS, + deps = [ + "//tools/private/release:mock_gh", + "//tools/private/release:release_lib", + ], +) + +pytest_test( + name = "add_backports_test", + srcs = ["add_backports_test.py"], + target_compatible_with = NOT_WINDOWS, + deps = [ + ":conftest", + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +pytest_test( + name = "changelog_news_test", + srcs = ["changelog_news_test.py"], + target_compatible_with = NOT_WINDOWS, + deps = [ + ":conftest", + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +pytest_test( + name = "complete_prepare_test", + srcs = ["complete_prepare_test.py"], + target_compatible_with = NOT_WINDOWS, + deps = [ + ":conftest", + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +pytest_test( + name = "complete_sync_changelog_test", + srcs = ["complete_sync_changelog_test.py"], + target_compatible_with = NOT_WINDOWS, + deps = [ + ":conftest", + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +pytest_test( + name = "create_release_branch_test", + srcs = ["create_release_branch_test.py"], + target_compatible_with = NOT_WINDOWS, + deps = [ + ":conftest", + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +pytest_test( + name = "git_test", + srcs = ["git_test.py"], + target_compatible_with = NOT_WINDOWS, + deps = [ + ":conftest", + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +pytest_test( + name = "on_pr_merged_test", + srcs = ["on_pr_merged_test.py"], + target_compatible_with = NOT_WINDOWS, + deps = [ + ":conftest", + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +pytest_test( + name = "promote_test", + srcs = ["promote_test.py"], + target_compatible_with = NOT_WINDOWS, + deps = [ + ":conftest", + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +pytest_test( + name = "release_issue_test", + srcs = ["release_issue_test.py"], + target_compatible_with = NOT_WINDOWS, + deps = [ + ":conftest", + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +pytest_test( + name = "release_test", + srcs = ["release_test.py"], + target_compatible_with = NOT_WINDOWS, + deps = [ + ":conftest", + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +pytest_test( + name = "backport_create_releases_test", + srcs = ["backport_create_releases_test.py"], + target_compatible_with = NOT_WINDOWS, + deps = [ + ":conftest", + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +pytest_test( + name = "prepare_test", + srcs = ["prepare_test.py"], + target_compatible_with = NOT_WINDOWS, + deps = [ + ":conftest", + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +pytest_test( + name = "gh_test", + srcs = ["gh_test.py"], + target_compatible_with = NOT_WINDOWS, + deps = [ + ":conftest", + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +pytest_test( + name = "backport_prepare_test", + srcs = ["backport_prepare_test.py"], + target_compatible_with = NOT_WINDOWS, + deps = [ + ":conftest", + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +pytest_test( + name = "process_backports_test", + srcs = ["process_backports_test.py"], + target_compatible_with = NOT_WINDOWS, + deps = [ + ":conftest", + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +pytest_test( + name = "create_rc_test", + srcs = ["create_rc_test.py"], + target_compatible_with = NOT_WINDOWS, + deps = [ + ":conftest", + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +pytest_test( + name = "utils_test", + srcs = ["utils_test.py"], + target_compatible_with = NOT_WINDOWS, + deps = [ + ":conftest", + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) diff --git a/tests/tools/private/release/add_backports_test.py b/tests/tools/private/release/add_backports_test.py new file mode 100644 index 0000000000..c1c7b5cc0a --- /dev/null +++ b/tests/tools/private/release/add_backports_test.py @@ -0,0 +1,101 @@ +import argparse + +from tools.private.release.add_backports import AddBackports + +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +def test_add_backports_explicit_issue(mock_gh): + args = argparse.Namespace(issue=123, prs=["124", "125"]) + mock_gh.issues[123] = { + "title": "Release 2.1.0", + "body": """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Tag Final + +## Backports +""", + "labels": ["type: release"], + "number": 123, + "url": "https://github.com/bazel-contrib/rules_python/issues/123", + } + result = AddBackports(args, mock_gh).run() + + assert result == 0 + updated_body = mock_gh.get_issue_body(123) + assert "- [ ] #124" in updated_body + assert "- [ ] #125" in updated_body + assert "- [ ] Tag RC0" in updated_body + assert "- [ ] Sync Changelog #124" in updated_body + assert "- [ ] Sync Changelog #125" in updated_body + + +def test_add_backports_auto_discover_success(mock_gh): + args = argparse.Namespace(issue=None, prs=["124"]) + issue_num = mock_gh.create_issue( + title="Release 2.1.0", + body=""" +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Tag Final + +## Backports +""", + labels=["type: release"], + ) + result = AddBackports(args, mock_gh).run() + + assert result == 0 + updated_body = mock_gh.get_issue_body(issue_num) + assert "- [ ] #124" in updated_body + + +def test_add_backports_auto_discover_no_issues(mock_gh): + args = argparse.Namespace(issue=None, prs=["124"]) + + result = AddBackports(args, mock_gh).run() + + assert result == 1 + + +def test_add_backports_auto_discover_multiple_issues(mock_gh): + args = argparse.Namespace(issue=None, prs=["124"]) + mock_gh.create_issue( + title="Release 2.1.0", body="## Backports\n", labels=["type: release"] + ) + mock_gh.create_issue( + title="Release 2.2.0", body="## Backports\n", labels=["type: release"] + ) + + result = AddBackports(args, mock_gh).run() + + assert result == 1 + + +def test_add_backports_no_auto_add_rc_if_pending(mock_gh): + args = argparse.Namespace(issue=123, prs=["124"]) + mock_gh.issues[123] = { + "title": "Release 2.1.0", + "body": """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Tag RC0 +- [ ] Tag Final + +## Backports +""", + "labels": ["type: release"], + "number": 123, + "url": "https://github.com/bazel-contrib/rules_python/issues/123", + } + result = AddBackports(args, mock_gh).run() + + assert result == 0 + updated_body = mock_gh.get_issue_body(123) + assert "Tag RC1" not in updated_body + assert "- [ ] Tag RC0" in updated_body + assert "- [ ] Sync Changelog #124" in updated_body diff --git a/tests/tools/private/release/backport_create_releases_test.py b/tests/tools/private/release/backport_create_releases_test.py new file mode 100644 index 0000000000..81dc7e00ce --- /dev/null +++ b/tests/tools/private/release/backport_create_releases_test.py @@ -0,0 +1,174 @@ +import argparse + +from tools.private.release.backport_create_releases import BackportCreateReleases + +# Register pytest fixtures (such as release_tool_env) from release_test_helper +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +def test_create_releases_all_success(release_tool_env, mock_gh): + # Arrange + args = argparse.Namespace(issue=123, dry_run=False) + gh = mock_gh + + # Setup backport issue in mock GH + backport_body = """* PR: #456 +* From version: 1.7 +* To version: 1.9 + +## Tasks + +- [x] Verify apply 1.7 | status=success +- [x] Verify apply 1.8 | status=success +- [x] Verify apply 1.9 | status=success +- [ ] Track Release 1.7.2 +- [ ] Track Release 1.8.1 +- [ ] Track Release 1.9.0""" + + gh.issues[123] = { + "title": "Backport: #456", + "body": backport_body, + "labels": ["type:backport-pr"], + "number": 123, + "url": "https://github.com/.../issues/123", + } + + # Act + result = BackportCreateReleases(args, gh).run() + + # Assert + assert result == 0 + + # Verify release issues created (IDs 1001, 1002, 1003) + assert 1001 in gh.issues + assert 1002 in gh.issues + assert 1003 in gh.issues + + # 1.7.2 (patch) should not have Tag RC0 + assert gh.issues[1001]["title"] == "Release 1.7.2" + assert "Tag RC0" not in gh.issues[1001]["body"] + assert "## Backports\n- [ ] #456" in gh.issues[1001]["body"] + + # 1.9.0 (minor) should have Tag RC0 + assert gh.issues[1003]["title"] == "Release 1.9.0" + assert "Tag RC0" in gh.issues[1003]["body"] + + # Verify backport issue updated + expected_updated_backport_body = """* PR: #456 +* From version: 1.7 +* To version: 1.9 + +## Tasks + +- [x] Verify apply 1.7 | status=success +- [x] Verify apply 1.8 | status=success +- [x] Verify apply 1.9 | status=success +- [x] Track Release 1.7.2 | status=success release_issue=#1001 +- [x] Track Release 1.8.1 | status=success release_issue=#1002 +- [x] Track Release 1.9.0 | status=success release_issue=#1003""" + + assert gh.issues[123]["body"] == expected_updated_backport_body + + +def test_create_releases_dependency_blocking(release_tool_env, mock_gh): + # Arrange + args = argparse.Namespace(issue=123, dry_run=False) + gh = mock_gh + + # 1.8 failed, 1.7 and 1.9 succeeded + backport_body = """* PR: #456 +* From version: 1.7 +* To version: 1.9 + +## Tasks + +- [x] Verify apply 1.7 | status=success +- [ ] Verify apply 1.8 | status=failed-conflict +- [x] Verify apply 1.9 | status=success +- [ ] Track Release 1.7.2 +- [ ] Track Release 1.8.1 +- [ ] Track Release 1.9.0""" + + gh.issues[123] = { + "title": "Backport: #456", + "body": backport_body, + "labels": ["type:backport-pr"], + "number": 123, + "url": "https://github.com/.../issues/123", + } + + # Act + result = BackportCreateReleases(args, gh).run() + + # Assert + assert result == 0 + + # Only 1.9.0 (ID 1001) should be created. 1.7.2 and 1.8.1 are blocked by 1.8 failure. + assert len(gh.issues) == 2 # Backport issue + 1 release issue + assert 1001 in gh.issues + assert gh.issues[1001]["title"] == "Release 1.9.0" + + # Verify backport issue updated with block statuses + expected_updated_backport_body = """* PR: #456 +* From version: 1.7 +* To version: 1.9 + +## Tasks + +- [x] Verify apply 1.7 | status=success +- [ ] Verify apply 1.8 | status=failed-conflict +- [x] Verify apply 1.9 | status=success +- [ ] Track Release 1.7.2 | status=error-later-release-did-not-apply +- [ ] Track Release 1.8.1 | status=error-later-release-did-not-apply +- [x] Track Release 1.9.0 | status=success release_issue=#1001""" + + assert gh.issues[123]["body"] == expected_updated_backport_body + + +def test_create_releases_already_exists(release_tool_env, mock_gh): + # Arrange + args = argparse.Namespace(issue=123, dry_run=False) + gh = mock_gh + + backport_body = """* PR: #456 +* From version: 1.7 +* To version: 1.7 + +## Tasks + +- [x] Verify apply 1.7 | status=success +- [ ] Track Release 1.7.2""" + + gh.issues[123] = { + "title": "Backport: #456", + "body": backport_body, + "labels": ["type:backport-pr"], + "number": 123, + "url": "https://github.com/.../issues/123", + } + + # Pre-create the release issue to simulate it already existing + gh.create_issue( + title="Release 1.7.2", + body="existing body\n## Backports\n", + labels=["type: release"], + ) # Will get ID 1001 + + # Act + result = BackportCreateReleases(args, gh).run() + + # Assert + assert result == 0 + assert len(gh.issues) == 2 + + # Should link existing issue 1001 + expected_updated_backport_body = """* PR: #456 +* From version: 1.7 +* To version: 1.7 + +## Tasks + +- [x] Verify apply 1.7 | status=success +- [x] Track Release 1.7.2 | status=success release_issue=#1001""" + + assert gh.issues[123]["body"] == expected_updated_backport_body diff --git a/tests/tools/private/release/backport_prepare_test.py b/tests/tools/private/release/backport_prepare_test.py new file mode 100644 index 0000000000..ddef4885b9 --- /dev/null +++ b/tests/tools/private/release/backport_prepare_test.py @@ -0,0 +1,238 @@ +import argparse +from unittest.mock import ANY, call + +from tools.private.release.backport_prepare import BackportPrepare +from tools.private.release.gh import BACKPORT_LABEL + +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +def test_prepare_from_issue_success(mocker, mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + issue=123, + pr=None, + from_minor=None, + to_minor=None, + remote="my-remote", + dry_run=False, + ) + + # Setup backport issue in mock GH + backport_body = "* PR: #456\n* From version: 1.7\n* To version: 1.9\n" + mock_gh.issues[123] = { + "title": "Backport: #456", + "body": backport_body, + "labels": ["type: backport-pr"], + "number": 123, + "url": "https://github.com/.../issues/123", + } + + # Setup PR info in mock GH + mock_gh.prs[456] = { + "state": "MERGED", + "mergeCommit": {"oid": "pr_merge_sha_12345"}, + } + + # Mock remote branches + mock_git.get_remote_branches.return_value = [ + "main", + "release/1.6", + "release/1.7", + "release/1.8", + "release/1.9", + "release/2.0", + ] + mock_git.get_current_branch.return_value = "work-branch" + + mock_news = mocker.patch("tools.private.release.backport_prepare.changelog_news") + mock_det = mocker.patch( + "tools.private.release.backport_prepare.determine_next_version" + ) + mock_det.side_effect = ["1.7.2", "1.8.1", "1.9.0"] + + # Act + result = BackportPrepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.fetch.assert_called_once_with("my-remote", tags=True, force=True) + + # Verify checkouts and cherry-picks + mock_git.checkout.assert_has_calls( + [ + call("release/1.7", track_remote="my-remote"), + call("release/1.8", track_remote="my-remote"), + call("release/1.9", track_remote="my-remote"), + call("work-branch"), # Restored branch + ] + ) + + mock_git.cherry_pick.assert_has_calls( + [ + call("pr_merge_sha_12345"), + call("pr_merge_sha_12345"), + call("pr_merge_sha_12345"), + ] + ) + + # Verify changelog updates + mock_news.update_changelog.assert_has_calls( + [ + call("1.7.2", ANY), + call("1.8.1", ANY), + call("1.9.0", ANY), + ] + ) + + # Verify issue body update + expected_body = ( + "* PR: #456\n" + "* From version: 1.7\n" + "* To version: 1.9\n" + "\n" + "## Tasks\n" + "\n" + "- [x] Verify apply 1.7 | status=success\n" + "- [x] Verify apply 1.8 | status=success\n" + "- [x] Verify apply 1.9 | status=success\n" + "- [ ] Track Release 1.7.2\n" + "- [ ] Track Release 1.8.1\n" + "- [ ] Track Release 1.9.0" + ) + assert mock_gh.issues[123]["body"] == expected_body + + +def test_prepare_manual_success(mocker, mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + issue=None, + pr="#456", + from_minor="1.7", + to_minor="1.8", + remote="my-remote", + dry_run=False, + ) + mock_gh.prs[456] = { + "state": "MERGED", + "mergeCommit": {"oid": "pr_merge_sha_12345"}, + } + mock_git.get_remote_branches.return_value = [ + "release/1.7", + "release/1.8", + ] + mock_git.get_current_branch.return_value = "work-branch" + + mocker.patch("tools.private.release.backport_prepare.changelog_news") + mock_det = mocker.patch( + "tools.private.release.backport_prepare.determine_next_version" + ) + mock_det.side_effect = ["1.7.2", "1.8.1"] + + # Act + result = BackportPrepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + assert 1001 in mock_gh.issues + issue = mock_gh.issues[1001] + assert issue["title"] == "Backport: #456" + assert issue["labels"] == [BACKPORT_LABEL] + body = issue["body"] + assert "- [x] Verify apply 1.7 | status=success" in body + assert "- [x] Verify apply 1.8 | status=success" in body + assert "- [ ] Track Release 1.7.2" in body + assert "- [ ] Track Release 1.8.1" in body + + +def test_prepare_manual_with_patch_versions(mocker, mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + issue=None, + pr="#456", + from_minor="1.7.0", + to_minor="1.8.0", + remote="my-remote", + dry_run=False, + ) + mock_gh.prs[456] = { + "state": "MERGED", + "mergeCommit": {"oid": "pr_merge_sha_12345"}, + } + mock_git.get_remote_branches.return_value = [ + "release/1.7", + "release/1.8", + ] + mock_git.get_current_branch.return_value = "work-branch" + + mocker.patch("tools.private.release.backport_prepare.changelog_news") + mock_det = mocker.patch( + "tools.private.release.backport_prepare.determine_next_version" + ) + mock_det.side_effect = ["1.7.2", "1.8.1"] + + # Act + result = BackportPrepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + assert 1001 in mock_gh.issues + body = mock_gh.issues[1001]["body"] + assert "- [x] Verify apply 1.7 | status=success" in body + assert "- [x] Verify apply 1.8 | status=success" in body + + +def test_prepare_verify_failed(mocker, mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + issue=123, + pr=None, + from_minor=None, + to_minor=None, + remote="my-remote", + dry_run=False, + ) + issue_body = "* PR: #456\n* From version: 1.7\n* To version: 1.8\n" + mock_gh.issues[123] = { + "title": "Backport: #456", + "body": issue_body, + "labels": ["type: backport-pr"], + "number": 123, + "url": "https://github.com/.../issues/123", + } + mock_gh.prs[456] = { + "state": "MERGED", + "mergeCommit": {"oid": "pr_merge_sha_12345"}, + } + mock_git.get_remote_branches.return_value = [ + "release/1.7", + "release/1.8", + ] + mock_git.get_current_branch.return_value = "work-branch" + + mock_news = mocker.patch("tools.private.release.backport_prepare.changelog_news") + mock_det = mocker.patch( + "tools.private.release.backport_prepare.determine_next_version" + ) + mock_det.side_effect = ["1.7.2", "1.8.1"] + mock_git.cherry_pick.side_effect = [Exception("Conflict"), None] + mock_news.update_changelog.side_effect = [Exception("Changelog error")] + + # Act + result = BackportPrepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + expected_body = ( + "* PR: #456\n" + "* From version: 1.7\n" + "* To version: 1.8\n" + "\n" + "## Tasks\n" + "\n" + "- [ ] Verify apply 1.7 | status=failed-conflict\n" + "- [ ] Verify apply 1.8 | status=failed-changelog\n" + "- [ ] Track Release 1.7.2\n" + "- [ ] Track Release 1.8.1" + ) + assert mock_gh.issues[123]["body"] == expected_body diff --git a/tests/tools/private/release/changelog_news_test.py b/tests/tools/private/release/changelog_news_test.py new file mode 100644 index 0000000000..dd2bd6df79 --- /dev/null +++ b/tests/tools/private/release/changelog_news_test.py @@ -0,0 +1,555 @@ +import pathlib + +import pytest + +from tools.private.release import changelog_news + + +def test_update_changelog_with_news(tmp_path): + # Arrange + changelog = """# Changelog + +{#unreleased} +## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +Unreleased changes are tracked as individual files in the [news/](./news) +directory, or view the [latest generated +changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). + +{#v2-0-2} +## [2.0.2] - 2026-05-14 + +[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 + +{#v2-0-2-added} +### Added +* (toolchains) Some older change. +""" + changelog_path = tmp_path / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = tmp_path / "news" + news_dir.mkdir() + + # Create news files + (news_dir / "123.fixed.md").write_text("Fixed a bug in the compiler") + # Test that it handles prefixing "* " if not present + (news_dir / "456.added.md").write_text("* Added a new feature for Python 3.13") + # Empty file should be ignored + (news_dir / "789.changed.md").write_text("") + # Invalid name should be ignored + (news_dir / "invalid_name.md").write_text("Should be ignored") + + # Act + changelog_news.update_changelog( + "3.0.0", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + # 1. News files matching the pattern should be deleted (even empty ones) + assert not (news_dir / "123.fixed.md").exists() + assert not (news_dir / "456.added.md").exists() + assert not (news_dir / "789.changed.md").exists() + # Invalid name does not match pattern -> NOT deleted + assert (news_dir / "invalid_name.md").exists() + + new_content = changelog_path.read_text() + + # 2. A fresh active Unreleased section should be present + assert "{#unreleased}" in new_content + assert "## Unreleased" in new_content + assert ( + "Unreleased changes are tracked as individual files in the [news/](./news)\n" + "directory, or view the [latest generated\n" + "changelog](https://rules-python.readthedocs.io/en/latest/changelog.html)." + in new_content + ) + + # 3. The new release section should be present + assert "{#v3-0-0}" in new_content + assert "## [3.0.0] - 2026-06-16" in new_content + assert ( + "[3.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/3.0.0" + in new_content + ) + + # 4. Correct categories and content + assert "{#v3-0-0-fixed}\n### Fixed\n* Fixed a bug in the compiler" in new_content + assert ( + "{#v3-0-0-added}\n### Added\n* Added a new feature for Python 3.13" + in new_content + ) + + # 5. Omitted categories should NOT be present in the new release + assert "{#v3-0-0-removed}" not in new_content + assert "{#v3-0-0-changed}" not in new_content + + # 6. Old release should still be there + assert "{#v2-0-2}" in new_content + assert "## [2.0.2] - 2026-05-14" in new_content + + +def test_update_changelog_sorting(tmp_path): + # Arrange + changelog = """# Changelog + +{#unreleased} +## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +Unreleased changes are tracked as individual files in the [news/](./news) +directory, or view the [latest generated +changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). + +{#v2-0-2} +## [2.0.2] - 2026-05-14 + +[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 + +{#v2-0-2-added} +### Added +* (toolchains) Some older change. +""" + changelog_path = tmp_path / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = tmp_path / "news" + news_dir.mkdir() + + # Create news files with different sub-categories and some without + (news_dir / "1.fixed.md").write_text("* (zebra) Zebra fix") + (news_dir / "2.fixed.md").write_text("* (apple) Apple fix") + (news_dir / "3.fixed.md").write_text("No subcategory B") + (news_dir / "4.fixed.md").write_text("* (apple) Another apple fix") + (news_dir / "5.fixed.md").write_text("No subcategory A") + + # Act + changelog_news.update_changelog( + "3.0.0", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + new_content = changelog_path.read_text() + + expected_fixed_section = ( + "### Fixed\n" + "* No subcategory A\n" + "* No subcategory B\n" + "* (apple) Another apple fix\n" + "* (apple) Apple fix\n" + "* (zebra) Zebra fix\n" + ) + + assert expected_fixed_section in new_content + + +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +def test_update_changelog_read_failure(mocker, tmp_path): + # Arrange + original_read_text = pathlib.Path.read_text + + mock_read_text = mocker.patch.object(pathlib.Path, "read_text", autospec=True) + + def side_effect(path_self, *args, **kwargs): + if "bad_file.fixed.md" in str(path_self): + raise IOError("Simulated read error") + return original_read_text(path_self, *args, **kwargs) + + mock_read_text.side_effect = side_effect + + changelog = """# Changelog + +{#unreleased} +## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +Unreleased changes are tracked as individual files in the [news/](./news) +directory, or view the [latest generated +changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). + +{#v2-0-2} +## [2.0.2] - 2026-05-14 + +[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 + +{#v2-0-2-added} +### Added +* (toolchains) Some older change. +""" + changelog_path = tmp_path / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = tmp_path / "news" + news_dir.mkdir() + + # Create the bad file (must exist so it is found by iterdir) + bad_file = news_dir / "bad_file.fixed.md" + bad_file.write_text("some content that won't be read") + + # Create a good file too + good_file = news_dir / "good_file.fixed.md" + good_file.write_text("* (sub) Good fix") + + # Act & Assert + with pytest.raises(IOError): + changelog_news.update_changelog( + "3.0.0", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Both files should still exist (no deletion on failure!) + assert bad_file.exists() + assert good_file.exists() + + # Changelog should not be modified + new_content = changelog_path.read_text() + assert changelog == new_content + + +def test_update_changelog_merge_existing(tmp_path): + # Arrange + changelog = """# Changelog + +{#unreleased} +## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +Unreleased changes are tracked as individual files in the [news/](./news) +directory, or view the [latest generated +changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). + +{#v2-0-3} +## [2.0.3] - 2026-06-15 + +[2.0.3]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.3 + +{#v2-0-3-fixed} +### Fixed +* (pypi) Old fix + multi-line detail + * nested bullet item +* (pypi) Z old fix +""" + changelog_path = tmp_path / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = tmp_path / "news" + news_dir.mkdir() + + # Create news files to merge + (news_dir / "1.fixed.md").write_text("(pypi) New fix") + (news_dir / "2.added.md").write_text("(toolchains) New feature") + + # Act + changelog_news.update_changelog( + "2.0.3", + "2026-06-15", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + assert not (news_dir / "1.fixed.md").exists() + assert not (news_dir / "2.added.md").exists() + + new_content = changelog_path.read_text() + + expected_fixed_section = ( + "### Fixed\n" + "* (pypi) New fix\n" + "* (pypi) Old fix\n" + " multi-line detail\n" + " * nested bullet item\n" + "* (pypi) Z old fix\n" + ) + assert expected_fixed_section in new_content + + expected_added_section = "### Added\n* (toolchains) New feature\n" + assert expected_added_section in new_content + assert "Unreleased changes are tracked as individual files" in new_content + + +def test_update_changelog_does_not_leak(tmp_path): + # Arrange + changelog = """# Changelog + +{#unreleased} +## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +Unreleased changes are tracked as individual files in the [news/](./news) +directory, or view the [latest generated +changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). + +{#v2-0-2} +## [2.0.2] - 2026-05-14 + +[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 + +This release body mentions the word unreleased and {#unreleased} anchor to test leaks. +""" + changelog_path = tmp_path / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = tmp_path / "news" + news_dir.mkdir() + (news_dir / "1.fixed.md").write_text("Some fix") + + # Act + changelog_news.update_changelog( + "3.0.0", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + new_content = changelog_path.read_text() + assert ( + "This release body mentions the word unreleased and {#unreleased} anchor to test leaks." + in new_content + ) + + +def test_update_changelog_empty_news(tmp_path): + # Arrange + changelog = """# Changelog + +{#unreleased} +## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +Unreleased changes are tracked as individual files in the [news/](./news) +directory, or view the [latest generated +changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). + +{#v2-0-2} +## [2.0.2] - 2026-05-14 + +[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 + +{#v2-0-2-added} +### Added +* (toolchains) Some older change. +""" + changelog_path = tmp_path / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = tmp_path / "news" + news_dir.mkdir() + + # Act + changelog_news.update_changelog( + "3.0.0", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + new_content = changelog_path.read_text() + + assert "{#v3-0-0}" in new_content + assert "## [3.0.0] - 2026-06-16" in new_content + assert ( + "[3.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/3.0.0" + in new_content + ) + assert "No notable changes." in new_content + assert "{#v3-0-0-fixed}" not in new_content + assert "{#v3-0-0-added}" not in new_content + + +def test_update_changelog_selective_news_files(tmp_path): + # Arrange + changelog = """# Changelog + +{#unreleased} +## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +{#v2-0-2} +## [2.0.2] - 2026-05-14 + +[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 +""" + changelog_path = tmp_path / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = tmp_path / "news" + news_dir.mkdir() + + # Create news files + (news_dir / "123.fixed.md").write_text("Fix A") + (news_dir / "456.fixed.md").write_text("Fix B") + + # Act: Only process 123.fixed.md + changelog_news.update_changelog( + "2.0.3", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + news_files=[news_dir / "123.fixed.md"], + ) + + # Assert + assert not (news_dir / "123.fixed.md").exists() + assert (news_dir / "456.fixed.md").exists() + + new_content = changelog_path.read_text() + assert "Fix A" in new_content + assert "Fix B" not in new_content + + +def test_update_changelog_insertion_point(tmp_path): + # Arrange + changelog = """# Changelog + +{#unreleased} +## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +{#v2-2-0} +## [2.2.0] - 2026-06-30 + +[2.2.0]: https://github.com/bazel-contrib/rules_python/releases/tag/2.2.0 + +{#v2-0-0} +## [2.0.0] - 2026-04-09 + +[2.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0 +""" + changelog_path = tmp_path / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = tmp_path / "news" + news_dir.mkdir() + (news_dir / "123.fixed.md").write_text("Fix in 2.1.0") + + # Act: Insert 2.1.0 + changelog_news.update_changelog( + "2.1.0", + "2026-06-17", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + new_content = changelog_path.read_text() + + idx_2_2_0 = new_content.index("{#v2-2-0}") + idx_2_1_0 = new_content.index("{#v2-1-0}") + idx_2_0_0 = new_content.index("{#v2-0-0}") + + assert idx_2_2_0 < idx_2_1_0 < idx_2_0_0 + assert "Fix in 2.1.0" in new_content + + +def test_update_changelog_insertion_point_too_small(tmp_path): + # Arrange + changelog = """# Changelog + +{#unreleased} +## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +{#v2-0-0} +## [2.0.0] - 2026-04-09 + +[2.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0 +""" + changelog_path = tmp_path / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = tmp_path / "news" + news_dir.mkdir() + (news_dir / "123.fixed.md").write_text("Fix in 1.0.0") + + # Act & Assert + with pytest.raises( + ValueError, match="Could not find a version in CHANGELOG.md smaller than 1.0.0" + ): + changelog_news.update_changelog( + "1.0.0", + "2026-01-01", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + +def test_format_entry(): + # Multi-line without leading bullet + raw = "(foo) bla bla\nblabl bla\nblabla" + expected = "* (foo) bla bla\n blabl bla\n blabla" + assert changelog_news._format_entry(raw) == expected + + # Multi-line with leading bullet + raw_with_bullet = "* (foo) bla bla\nblabl bla\nblabla" + assert changelog_news._format_entry(raw_with_bullet) == expected + + # Multi-line already indented + raw_indented = "* (foo) bla bla\n blabl bla\n blabla" + assert changelog_news._format_entry(raw_indented) == expected + + # Empty string + assert changelog_news._format_entry("") == "" + + +def test_update_changelog_multiline_indentation(tmp_path): + # Arrange + changelog = """# Changelog + +{#unreleased} +## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +{#v2-0-0} +## [2.0.0] - 2026-04-09 + +[2.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0 +""" + changelog_path = tmp_path / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = tmp_path / "news" + news_dir.mkdir() + (news_dir / "1.fixed.md").write_text("(foo) bla bla\nblabl bla\nblabla") + (news_dir / "2.added.md").write_text("* (bar) feature one\nsecond line of feature") + + # Act + changelog_news.update_changelog( + "2.1.0", + "2026-06-17", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + new_content = changelog_path.read_text() + + expected_fixed = "### Fixed\n* (foo) bla bla\n blabl bla\n blabla" + expected_added = "### Added\n* (bar) feature one\n second line of feature" + + assert expected_fixed in new_content + assert expected_added in new_content diff --git a/tests/tools/private/release/complete_prepare_test.py b/tests/tools/private/release/complete_prepare_test.py new file mode 100644 index 0000000000..b710c0d0c9 --- /dev/null +++ b/tests/tools/private/release/complete_prepare_test.py @@ -0,0 +1,138 @@ +import argparse + +from tools.private.release.complete_prepare import CompletePrepare + +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +def test_complete_prepare_with_pr_success(mock_gh): + args = argparse.Namespace(pr=456, issue=None) + mock_gh.prs[456] = { + "state": "MERGED", + "body": "Prepare release for v2.0.0\n\nWork towards #123", + "mergeCommit": {"oid": "abcdef1234567890"}, + } + issue_body = """ +## Checklist +- [ ] Prepare Release | status=pending pr=#456 +- [ ] Create Release branch +- [ ] Tag Final +""" + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": issue_body, + "labels": ["type: release"], + "number": 123, + "url": "https://github.com/bazel-contrib/rules_python/issues/123", + } + + result = CompletePrepare(args, mock_gh).run() + + assert result == 0 + updated_body = mock_gh.get_issue_body(123) + assert ( + "- [x] Prepare Release | status=done pr=#456 commit= abcdef12" in updated_body + ) + + +def test_complete_prepare_writes_github_output(release_tool_env, mock_gh): + args = argparse.Namespace(pr=456, issue=None) + mock_gh.prs[456] = { + "state": "MERGED", + "body": "Prepare release for v2.0.0\n\nWork towards #123", + "mergeCommit": {"oid": "abcdef1234567890"}, + } + issue_body = """ +## Checklist +- [ ] Prepare Release | status=pending pr=#456 +- [ ] Create Release branch +- [ ] Tag Final +""" + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": issue_body, + "labels": ["type: release"], + "number": 123, + "url": "https://github.com/bazel-contrib/rules_python/issues/123", + } + + result = CompletePrepare(args, mock_gh).run() + assert result == 0 + assert release_tool_env.github_output_file.exists() + assert ( + release_tool_env.github_output_file.read_text(encoding="utf-8") == "issue=123\n" + ) + + +def test_complete_prepare_with_issue_success(mock_gh): + args = argparse.Namespace(pr=None, issue=123) + mock_gh.prs[456] = { + "state": "MERGED", + "body": "Prepare release for v2.0.0", + "mergeCommit": {"oid": "1234567890abcdef"}, + } + issue_body = """ +## Checklist +- [ ] Prepare Release | status=pending pr=#456 +- [ ] Create Release branch +- [ ] Tag Final +""" + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": issue_body, + "labels": ["type: release"], + "number": 123, + "url": "https://github.com/bazel-contrib/rules_python/issues/123", + } + + result = CompletePrepare(args, mock_gh).run() + + assert result == 0 + updated_body = mock_gh.get_issue_body(123) + assert ( + "- [x] Prepare Release | status=done pr=#456 commit= 12345678" in updated_body + ) + + +def test_complete_prepare_no_args(mock_gh): + args = argparse.Namespace(pr=None, issue=None) + result = CompletePrepare(args, mock_gh).run() + assert result == 1 + + +def test_complete_prepare_not_merged(mock_gh): + args = argparse.Namespace(pr=456, issue=123) + mock_gh.prs[456] = { + "state": "OPEN", + "body": "Prepare release for v2.0.0", + } + issue_body = """ +## Checklist +- [ ] Prepare Release | status=pending pr=#456 +""" + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": issue_body, + "labels": ["type: release"], + "number": 123, + } + + result = CompletePrepare(args, mock_gh).run() + assert result == 1 + + +def test_complete_prepare_issue_missing_pr_task(mock_gh): + args = argparse.Namespace(pr=None, issue=123) + issue_body = """ +## Checklist +- [ ] Create Release branch +""" + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": issue_body, + "labels": ["type: release"], + "number": 123, + } + + result = CompletePrepare(args, mock_gh).run() + assert result == 1 diff --git a/tests/tools/private/release/complete_sync_changelog_test.py b/tests/tools/private/release/complete_sync_changelog_test.py new file mode 100644 index 0000000000..d0eb148b57 --- /dev/null +++ b/tests/tools/private/release/complete_sync_changelog_test.py @@ -0,0 +1,105 @@ +import argparse + +from tools.private.release.complete_sync_changelog import CompleteSyncChangelog + +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +def test_complete_sync_changelog_success(mock_gh): + args = argparse.Namespace(pr=999) + mock_gh.prs[999] = { + "state": "MERGED", + "body": "Updates CHANGELOG.md\n\nRelease-Tracking-Issue: #123", + "mergeCommit": {"oid": "abcdef1234567890"}, + } + issue_body = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Sync Changelog #124 | status=pending pr=#999 +- [ ] Sync Changelog #125 | status=pending pr=#999 +- [ ] Sync Changelog #126 | status=pending pr=#888 +- [ ] Tag Final + +## Backports +""" + mock_gh.issues[123] = { + "title": "Release 2.1.0", + "body": issue_body, + "labels": ["type: release"], + "number": 123, + "url": "https://github.com/bazel-contrib/rules_python/issues/123", + } + + result = CompleteSyncChangelog(args, mock_gh).run() + + assert result == 0 + updated_body = mock_gh.get_issue_body(123) + + # Check that only tasks pointing to #999 were marked checked=True and status=done + assert ( + "- [x] Sync Changelog #124 | status=done pr=#999 commit= abcdef12" + in updated_body + ) + assert ( + "- [x] Sync Changelog #125 | status=done pr=#999 commit= abcdef12" + in updated_body + ) + # Task pointing to #888 should remain unchanged + assert "- [ ] Sync Changelog #126 | status=pending pr=#888" in updated_body + + +def test_complete_sync_changelog_not_merged(mock_gh): + args = argparse.Namespace(pr=999) + mock_gh.prs[999] = { + "state": "OPEN", + "body": "Updates CHANGELOG.md\n\nRelease-Tracking-Issue: #123", + } + + result = CompleteSyncChangelog(args, mock_gh).run() + + assert result == 1 + + +def test_complete_sync_changelog_missing_tracking_issue_link(mock_gh): + args = argparse.Namespace(pr=999) + mock_gh.prs[999] = { + "state": "MERGED", + "body": "Updates CHANGELOG.md without tracking issue link", + "mergeCommit": {"oid": "abcdef1234567890"}, + } + + result = CompleteSyncChangelog(args, mock_gh).run() + + assert result == 1 + + +def test_complete_sync_changelog_no_matching_tasks(mock_gh): + args = argparse.Namespace(pr=999) + mock_gh.prs[999] = { + "state": "MERGED", + "body": "Updates CHANGELOG.md\n\nRelease-Tracking-Issue: #123", + "mergeCommit": {"oid": "abcdef1234567890"}, + } + # Checklist has no tasks pointing to #999 + issue_body = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Sync Changelog #124 | status=pending pr=#888 +- [ ] Tag Final + +## Backports +""" + mock_gh.issues[123] = { + "title": "Release 2.1.0", + "body": issue_body, + "labels": ["type: release"], + "number": 123, + "url": "https://github.com/bazel-contrib/rules_python/issues/123", + } + + result = CompleteSyncChangelog(args, mock_gh).run() + + assert result == 0 + assert mock_gh.get_issue_body(123) == issue_body diff --git a/tests/tools/private/release/conftest.py b/tests/tools/private/release/conftest.py new file mode 100644 index 0000000000..f8798c11c7 --- /dev/null +++ b/tests/tools/private/release/conftest.py @@ -0,0 +1,50 @@ +import dataclasses +from unittest.mock import MagicMock + +import pytest + +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +@dataclasses.dataclass +class AutoPatchCmdHelpers: + """Dataclass holding mocked command helpers.""" + + run_cmd: MagicMock + run_git: MagicMock + run_gh: MagicMock + + +@pytest.fixture(name="mock_run_cmd") +def fixture_mock_run_cmd(mocker): + """Fixture to patch shell.run_cmd and its imports in git and gh modules.""" + mock = mocker.patch("tools.private.release.shell.run_cmd") + mocker.patch("tools.private.release.git.run_cmd", mock) + mocker.patch("tools.private.release.gh.run_cmd", mock) + return mock + + +@pytest.fixture(name="mock_run_git") +def fixture_mock_run_git(mocker): + """Fixture to patch Git._run_git.""" + return mocker.patch("tools.private.release.git.Git._run_git") + + +@pytest.fixture(name="mock_run_gh") +def fixture_mock_run_gh(mocker): + """Fixture to patch GitHub._run_gh.""" + return mocker.patch("tools.private.release.gh.GitHub._run_gh") + + +@pytest.fixture(name="auto_patch_cmd_helpers", autouse=True) +def fixture_auto_patch_cmd_helpers(mock_run_cmd, mock_run_git, mock_run_gh): + """Automatically patches run_cmd, Git, and GitHub CLI helpers. + + This prevents tests from executing command line tools that could have + side-effects. + """ + return AutoPatchCmdHelpers( + run_cmd=mock_run_cmd, + run_git=mock_run_git, + run_gh=mock_run_gh, + ) diff --git a/tests/tools/private/release/create_rc_test.py b/tests/tools/private/release/create_rc_test.py new file mode 100644 index 0000000000..6aee746dbb --- /dev/null +++ b/tests/tools/private/release/create_rc_test.py @@ -0,0 +1,420 @@ +import argparse +import os +import pathlib +import tempfile +from unittest.mock import call + +from tools.private.release.create_rc import CreateRc + +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +def test_create_rc_success_first_rc(mocker, mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=None, dry_run=False + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [ ] Tag RC0 | status=pending +""", + "labels": ["type: release"], + } + mock_git.get_remote_tags.return_value = [] + mock_git.get_commit_sha.return_value = "1234567890" + + # Act + with tempfile.TemporaryDirectory() as tmpdir: + github_output_file = pathlib.Path(tmpdir) / "github_output" + mocker.patch.dict(os.environ, {"GITHUB_OUTPUT": str(github_output_file)}) + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + assert github_output_file.exists() + assert github_output_file.read_text() == "tag_name=2.0.0-rc0\n" + + mock_git.fetch.assert_has_calls( + [call("my-remote"), call("my-remote", tags=True, force=True)] + ) + mock_git.checkout.assert_not_called() + mock_git.tag.assert_called_once_with("2.0.0-rc0", "my-remote/release/2.0") + mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc0") + mock_git.get_commit_sha.assert_called_once_with("my-remote/release/2.0") + + updated_body = mock_gh.get_issue_body(123) + assert "tag=2.0.0-rc0" in updated_body + assert "commit= 12345678" in updated_body + + assert 123 in mock_gh.issue_comments + comment_text = mock_gh.issue_comments[123][0] + assert "**New Release Candidate Tagged!** 🐍🌿" in comment_text + assert ( + "tagged on branch [`release/2.0`](https://github.com/bazel-contrib/rules_python/tree/release/2.0)" + in comment_text + ) + assert ( + "- [Github Release 2.0.0-rc0](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0-rc0)" + in comment_text + ) + assert ( + "- [BCR Entry 2.0.0-rc0](https://registry.bazel.build/modules/rules_python/2.0.0-rc0)" + in comment_text + ) + assert ( + "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+2.0.0-rc0)" + in comment_text + ) + assert ( + "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_create_rc.yaml)" + in comment_text + ) + + +def test_create_rc_success_with_run_id(mocker, mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=None, dry_run=False + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [ ] Tag RC0 | status=pending +""", + "labels": ["type: release"], + } + mock_git.get_remote_tags.return_value = [] + mock_git.get_commit_sha.return_value = "1234567890" + + # Act + mocker.patch.dict(os.environ, {"GITHUB_RUN_ID": "987654321"}) + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + assert 123 in mock_gh.issue_comments + comment_text = mock_gh.issue_comments[123][0] + assert ( + "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/runs/987654321)" + in comment_text + ) + assert ( + "tagged on branch [`release/2.0`](https://github.com/bazel-contrib/rules_python/tree/release/2.0)" + in comment_text + ) + + +def test_create_rc_success_next_rc(mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=None, dry_run=False + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [x] Tag RC0 | status=done tag=2.0.0-rc0 commit=abcdef12 +- [ ] Tag RC1 | status=pending +""", + "labels": ["type: release"], + } + mock_git.get_remote_tags.return_value = ["2.0.0-rc0"] + mock_git.get_commit_sha.return_value = "1234567890" + + # Act + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.fetch.assert_has_calls( + [call("my-remote"), call("my-remote", tags=True, force=True)] + ) + mock_git.checkout.assert_not_called() + mock_git.tag.assert_called_once_with("2.0.0-rc1", "my-remote/release/2.0") + mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc1") + mock_git.get_commit_sha.assert_called_once_with("my-remote/release/2.0") + + updated_body = mock_gh.get_issue_body(123) + assert "tag=2.0.0-rc1" in updated_body + + assert 123 in mock_gh.issue_comments + comment_text = mock_gh.issue_comments[123][0] + assert "**New Release Candidate Tagged!** 🐍🌿" in comment_text + + +def test_create_rc_gating_on_backports(mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=None, dry_run=False + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [ ] Tag RC0 | status=pending + +## Backports +- [ ] #124 | status=pending +""", + "labels": ["type: release"], + } + # Act + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + mock_git.tag.assert_not_called() + mock_git.push.assert_not_called() + + +def test_create_rc_not_blocked_by_ignored_backports(mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=None, dry_run=False + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [ ] Tag RC0 | status=pending + +## Backports +- [ ] #124 | status=ignore +""", + "labels": ["type: release"], + } + mock_git.get_remote_tags.return_value = [] + mock_git.get_commit_sha.return_value = "1234567890" + + # Act + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.tag.assert_called_once_with("2.0.0-rc0", "my-remote/release/2.0") + mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc0") + + +def test_create_rc_with_finished_backports(mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=None, dry_run=False + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [ ] Tag RC0 | status=pending + +## Backports +- [x] #124 | status=done rc=rc0 commit=abcdef12 +""", + "labels": ["type: release"], + } + mock_git.get_remote_tags.return_value = [] + mock_git.get_commit_sha.return_value = "1234567890" + + # Act + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.tag.assert_called_once_with("2.0.0-rc0", "my-remote/release/2.0") + mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc0") + + +def test_create_rc_auto_add_task(mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=None, dry_run=False + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [x] Tag RC0 | status=done tag=2.0.0-rc0 commit=abcdef12 +- [ ] Tag Final +""", + "labels": ["type: release"], + } + mock_git.get_remote_tags.return_value = ["2.0.0-rc0"] + mock_git.get_commit_sha.return_value = "1234567890" + + # Act + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.tag.assert_called_once_with("2.0.0-rc1", "my-remote/release/2.0") + mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc1") + + updated_body = mock_gh.get_issue_body(123) + assert "- [x] Tag RC1 | status=done tag=2.0.0-rc1 commit= 12345678" in updated_body + + +def test_create_rc_calls_process_backports(mocker, mock_git, mock_gh): + # Arrange + mock_pb_class = mocker.patch("tools.private.release.create_rc.ProcessBackports") + mock_pb = mock_pb_class.return_value + mock_pb.run.return_value = 0 + + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=None, dry_run=False + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [ ] Tag RC0 | status=pending +""", + "labels": ["type: release"], + } + mock_git.get_remote_tags.return_value = [] + mock_git.get_commit_sha.return_value = "1234567890" + + # Act + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_pb_class.assert_called_once() + called_args = mock_pb_class.call_args[0][0] + assert called_args.issue == 123 + assert called_args.remote == "my-remote" + assert not called_args.dry_run + assert called_args.add is None + assert called_args.triggering_comment is None + mock_pb.run.assert_called_once() + + +def test_create_rc_aborts_on_process_backports_failure(mocker, mock_git, mock_gh): + # Arrange + mock_pb_class = mocker.patch("tools.private.release.create_rc.ProcessBackports") + mock_pb = mock_pb_class.return_value + mock_pb.run.return_value = 1 + + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=None, dry_run=False + ) + + # Act + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + mock_pb_class.assert_called_once() + mock_pb.run.assert_called_once() + mock_git.tag.assert_not_called() + + +def test_create_rc_failure_reacts_to_comment(mocker, mock_git, mock_gh): + # Arrange + mock_pb_class = mocker.patch("tools.private.release.create_rc.ProcessBackports") + mock_pb = mock_pb_class.return_value + mock_pb.run.return_value = 1 # Simulate failure + + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=456, dry_run=False + ) + + # Act + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + assert mock_gh.reactions.get(456) == ["-1"] + + +def test_create_rc_failure_no_comment_no_reaction(mocker, mock_git, mock_gh): + # Arrange + mock_pb_class = mocker.patch("tools.private.release.create_rc.ProcessBackports") + mock_pb = mock_pb_class.return_value + mock_pb.run.return_value = 1 # Simulate failure + + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=None, dry_run=False + ) + + # Act + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + assert 456 not in mock_gh.reactions + + +def test_create_rc_success_with_comment_no_reaction(mocker, mock_git, mock_gh): + # Arrange + mock_pb_class = mocker.patch("tools.private.release.create_rc.ProcessBackports") + mock_pb = mock_pb_class.return_value + mock_pb.run.return_value = 0 + + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=456, dry_run=False + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [ ] Tag RC0 | status=pending +""", + "labels": ["type: release"], + } + mock_git.get_remote_tags.return_value = [] + mock_git.get_commit_sha.return_value = "1234567890" + + # Act + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + assert 456 not in mock_gh.reactions + + +def test_create_rc_precondition_failure_reacts_to_comment(mocker, mock_git, mock_gh): + # Arrange + mock_pb_class = mocker.patch("tools.private.release.create_rc.ProcessBackports") + mock_pb = mock_pb_class.return_value + mock_pb.run.return_value = 0 # Backports succeed + + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=456, dry_run=False + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [ ] Prepare Release | status=pending +- [ ] Create Release branch | status=pending +- [ ] Tag RC0 | status=pending +""", + "labels": ["type: release"], + } + + # Act + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + assert mock_gh.reactions.get(456) == ["-1"] diff --git a/tests/tools/private/release/create_release_branch_test.py b/tests/tools/private/release/create_release_branch_test.py new file mode 100644 index 0000000000..332ffec851 --- /dev/null +++ b/tests/tools/private/release/create_release_branch_test.py @@ -0,0 +1,157 @@ +import argparse + +from tools.private.release.create_release_branch import CreateReleaseBranch + +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +def test_create_release_branch_success(mock_git, mock_gh): + # Arrange + args = argparse.Namespace(issue=123, remote="my-remote") + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [ ] Create Release branch | status=pending +""", + "labels": ["type: release"], + } + mock_git.branch_exists.return_value = False + mock_git.remote_branch_exists.return_value = False + + # Act + result = CreateReleaseBranch(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.fetch.assert_called_once_with("my-remote") + mock_git.checkout.assert_not_called() + mock_git.push.assert_called_once_with( + "my-remote", "abcdef12:refs/heads/release/2.0" + ) + + updated_body = mock_gh.get_issue_body(123) + assert ( + "branch_url=https://github.com/bazel-contrib/rules_python/tree/release/2.0" + in updated_body + ) + assert "commit= abcdef12" in updated_body + + +def test_create_release_branch_prepare_not_done(mock_git, mock_gh): + # Arrange + args = argparse.Namespace(issue=123, remote="my-remote") + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [ ] Prepare Release | status=pending +- [ ] Create Release branch | status=pending +""", + "labels": ["type: release"], + } + # Act + result = CreateReleaseBranch(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + mock_git.fetch.assert_not_called() + mock_git.push.assert_not_called() + + +def test_create_release_branch_already_checked(mock_git, mock_gh): + # Arrange + args = argparse.Namespace(issue=123, remote="my-remote") + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +""", + "labels": ["type: release"], + } + # Act + result = CreateReleaseBranch(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.fetch.assert_not_called() + mock_git.push.assert_not_called() + + +def test_create_release_branch_already_exists_same_commit(mock_git, mock_gh): + # Arrange + args = argparse.Namespace(issue=123, remote="my-remote") + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [ ] Create Release branch | status=pending +""", + "labels": ["type: release"], + } + mock_git.remote_branch_exists.return_value = True + mock_git.get_commit_sha.return_value = "abcdef12" + + # Act + result = CreateReleaseBranch(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.fetch.assert_called_once_with("my-remote") + mock_git.push.assert_not_called() + + +def test_create_release_branch_already_exists_fast_forward(mock_git, mock_gh): + # Arrange + args = argparse.Namespace(issue=123, remote="my-remote") + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [ ] Create Release branch | status=pending +""", + "labels": ["type: release"], + } + mock_git.remote_branch_exists.return_value = True + mock_git.get_commit_sha.return_value = "oldcommit" + mock_git.is_ancestor.return_value = True + + # Act + result = CreateReleaseBranch(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.fetch.assert_called_once_with("my-remote") + mock_git.push.assert_called_once_with( + "my-remote", "abcdef12:refs/heads/release/2.0" + ) + + +def test_create_release_branch_already_exists_non_ff(mock_git, mock_gh): + # Arrange + args = argparse.Namespace(issue=123, remote="my-remote") + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [ ] Create Release branch | status=pending +""", + "labels": ["type: release"], + } + mock_git.remote_branch_exists.return_value = True + mock_git.get_commit_sha.return_value = "othercommit" + mock_git.is_ancestor.return_value = False + + # Act + result = CreateReleaseBranch(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + mock_git.fetch.assert_called_once_with("my-remote") + mock_git.push.assert_not_called() diff --git a/tests/tools/private/release/gh_test.py b/tests/tools/private/release/gh_test.py new file mode 100644 index 0000000000..7e19b2232c --- /dev/null +++ b/tests/tools/private/release/gh_test.py @@ -0,0 +1,96 @@ +import pytest + +from tools.private.release import shell +from tools.private.release.gh import GitHub +from tools.private.release.git import Git + +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +@pytest.fixture(name="gh") +def fixture_gh(): + return GitHub("my-owner/my-repo") + + +def test_resolve_pr_number_digit(mocker, gh): + mock_run_cmd = mocker.patch("tools.private.release.gh.run_cmd") + # 124 and #125 should resolve immediately without running command + assert gh.resolve_pr_number("124") == 124 + assert gh.resolve_pr_number("#125") == 125 + mock_run_cmd.assert_not_called() + + +def test_resolve_pr_number_url_simple(mocker, gh): + mock_run_cmd = mocker.patch("tools.private.release.gh.run_cmd") + url = "https://github.com/my-owner/my-repo/pull/126" + # Should resolve via regex without calling gh + result = gh.resolve_pr_number(url) + assert result == 126 + mock_run_cmd.assert_not_called() + + +def test_resolve_pr_number_url_with_subpath(mocker, gh): + mock_run_cmd = mocker.patch("tools.private.release.gh.run_cmd") + url = "https://github.com/my-owner/my-repo/pull/126/files" + # Should resolve via regex without calling gh + result = gh.resolve_pr_number(url) + assert result == 126 + mock_run_cmd.assert_not_called() + + +def test_resolve_pr_number_url_with_query(mocker, gh): + mock_run_cmd = mocker.patch("tools.private.release.gh.run_cmd") + url = "https://github.com/my-owner/my-repo/pull/126/files?w=1" + # Should resolve via regex without calling gh + result = gh.resolve_pr_number(url) + assert result == 126 + mock_run_cmd.assert_not_called() + + +def test_resolve_pr_number_url_other_repo(mocker, gh): + mock_run_cmd = mocker.patch("tools.private.release.gh.run_cmd") + # URL for a different repo should fail immediately without calling gh + url = "https://github.com/other-owner/other-repo/pull/126" + with pytest.raises(ValueError, match="URL is not for the configured repository"): + gh.resolve_pr_number(url) + mock_run_cmd.assert_not_called() + + +def test_resolve_pr_number_invalid(mocker, gh): + mock_run_cmd = mocker.patch("tools.private.release.gh.run_cmd") + with pytest.raises(ValueError, match="Could not resolve PR reference"): + gh.resolve_pr_number("invalid-ref") + mock_run_cmd.assert_not_called() + + +def test_auto_patched_helpers_prevent_real_execution(auto_patch_cmd_helpers): + # Calling run_cmd directly hits the mock + shell.run_cmd("echo", "test") + auto_patch_cmd_helpers.run_cmd.assert_called_with("echo", "test") + + # Git._run_git hits the mock + git = Git(".") + git._run_git("status") + auto_patch_cmd_helpers.run_git.assert_called_with("status") + + # GitHub._run_gh hits the mock + gh_obj = GitHub("foo/bar") + gh_obj._run_gh("issue", "list") + auto_patch_cmd_helpers.run_gh.assert_called_with("issue", "list") + + +def test_update_issue_body(gh, auto_patch_cmd_helpers): + captured_body = {} + + def mock_run(*args, **kwargs): + for arg in args: + if isinstance(arg, str) and arg.startswith("--body-file="): + path = arg.split("=", 1)[1] + with open(path, encoding="utf-8") as f: + captured_body["content"] = f.read() + return None + + auto_patch_cmd_helpers.run_gh.side_effect = mock_run + gh.update_issue_body(123, "new body content") + auto_patch_cmd_helpers.run_gh.assert_called_once() + assert captured_body["content"] == "new body content" diff --git a/tests/tools/private/release/git_test.py b/tests/tools/private/release/git_test.py new file mode 100644 index 0000000000..4a8cada4cc --- /dev/null +++ b/tests/tools/private/release/git_test.py @@ -0,0 +1,154 @@ +import subprocess + +import pytest + +from tools.private.release.git import Git + +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +@pytest.fixture(name="git_obj") +def fixture_git_obj(mocker): + git = Git(".") + git.mock_run_git = mocker.patch.object(git, "_run_git") + return git + + +def test_checkout_simple(git_obj): + git_obj.checkout("my-branch") + git_obj.mock_run_git.assert_called_once_with( + "checkout", "my-branch", capture_output=False + ) + + +def test_checkout_track_remote_new_branch(mocker, git_obj): + mock_branch_exists = mocker.patch( + "tools.private.release.git.Git.branch_exists", return_value=False + ) + + git_obj.checkout("my-branch", track_remote="origin") + + mock_branch_exists.assert_called_once_with("my-branch") + git_obj.mock_run_git.assert_called_once_with( + "checkout", "--track", "origin/my-branch", capture_output=False + ) + + +def test_checkout_track_remote_existing_branch(mocker, git_obj): + mock_branch_exists = mocker.patch( + "tools.private.release.git.Git.branch_exists", return_value=True + ) + mock_reset_hard = mocker.patch("tools.private.release.git.Git.reset_hard") + + git_obj.checkout("my-branch", track_remote="origin") + + mock_branch_exists.assert_called_once_with("my-branch") + git_obj.mock_run_git.assert_called_once_with( + "checkout", "my-branch", capture_output=False + ) + mock_reset_hard.assert_called_once_with(reset_to="origin/my-branch") + + +def test_fetch_default(git_obj): + git_obj.fetch() + git_obj.mock_run_git.assert_called_once_with( + "fetch", "origin", capture_output=False + ) + + +def test_fetch_custom_remote(git_obj): + git_obj.fetch("upstream") + git_obj.mock_run_git.assert_called_once_with( + "fetch", "upstream", capture_output=False + ) + + +def test_fetch_with_refspec(git_obj): + git_obj.fetch("origin", refspec="my-branch") + git_obj.mock_run_git.assert_called_once_with( + "fetch", "origin", "my-branch", capture_output=False + ) + + +def test_fetch_with_tags_and_force(git_obj): + git_obj.fetch("origin", tags=True, force=True) + git_obj.mock_run_git.assert_called_once_with( + "fetch", "origin", "--tags", "--force", capture_output=False + ) + + +def test_fetch_all_options(git_obj): + git_obj.fetch("origin", refspec="my-branch", tags=True, force=True) + git_obj.mock_run_git.assert_called_once_with( + "fetch", "origin", "my-branch", "--tags", "--force", capture_output=False + ) + + +def test_get_modified_files(git_obj): + git_obj.mock_run_git.return_value = "file1.txt\nfile2.py\n\n" + files = git_obj.get_modified_files("HEAD") + git_obj.mock_run_git.assert_called_once_with( + "show", "--name-only", "--format=", "HEAD" + ) + assert files == ["file1.txt", "file2.py"] + + +def test_get_modified_files_empty(git_obj): + git_obj.mock_run_git.return_value = "" + files = git_obj.get_modified_files("HEAD") + assert files == [] + + +def test_diff_has_changes(git_obj): + git_obj.mock_run_git.return_value = "some diff output" + output = git_obj.diff() + git_obj.mock_run_git.assert_called_once_with("diff") + assert output == "some diff output" + + +def test_diff_empty(git_obj): + git_obj.mock_run_git.return_value = "" + output = git_obj.diff() + git_obj.mock_run_git.assert_called_once_with("diff") + assert output == "" + + +def test_apply(git_obj): + git_obj.apply("patch.patch") + git_obj.mock_run_git.assert_called_once_with( + "apply", "patch.patch", capture_output=False + ) + + +def test_apply_check_clean(git_obj): + git_obj.mock_run_git.return_value = "" + result = git_obj.apply_check("patch.patch") + git_obj.mock_run_git.assert_called_once_with( + "apply", "--check", "patch.patch", capture_output=False + ) + assert result is True + + +def test_apply_check_conflict(git_obj): + git_obj.mock_run_git.side_effect = subprocess.CalledProcessError( + 1, ["git", "apply", "--check", "patch.patch"] + ) + result = git_obj.apply_check("patch.patch") + git_obj.mock_run_git.assert_called_once_with( + "apply", "--check", "patch.patch", capture_output=False + ) + assert result is False + + +def test_reset_hard_default(git_obj): + git_obj.reset_hard() + git_obj.mock_run_git.assert_called_once_with( + "reset", "--hard", "HEAD", capture_output=False + ) + + +def test_reset_hard_custom(git_obj): + git_obj.reset_hard(reset_to="my-commit") + git_obj.mock_run_git.assert_called_once_with( + "reset", "--hard", "my-commit", capture_output=False + ) diff --git a/tests/tools/private/release/on_pr_merged_test.py b/tests/tools/private/release/on_pr_merged_test.py new file mode 100644 index 0000000000..df3698d1dd --- /dev/null +++ b/tests/tools/private/release/on_pr_merged_test.py @@ -0,0 +1,92 @@ +import argparse +from unittest.mock import MagicMock + +from tools.private.release.on_pr_merged import OnPrMerged + +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +def test_on_pr_merged_no_comment(mocker, mock_git, mock_gh): + args = argparse.Namespace(pr=124, remote="origin", dry_run=True) + mock_gh.pr_comments = { + 124: [ + {"body": "Some comment"}, + {"body": "Another comment /backport_wrong"}, + ] + } + + mock_pb = mocker.patch("tools.private.release.on_pr_merged.ProcessBackports") + result = OnPrMerged(args, mock_git, mock_gh).run() + + assert result == 1 + mock_pb.assert_not_called() + + +def test_on_pr_merged_has_comment_no_active_release(mocker, mock_git, mock_gh): + args = argparse.Namespace(pr=124, remote="origin", dry_run=True) + mock_gh.pr_comments = {124: [{"body": "/backport"}]} + + mock_pb = mocker.patch("tools.private.release.on_pr_merged.ProcessBackports") + result = OnPrMerged(args, mock_git, mock_gh).run() + + assert result == 1 + mock_pb.assert_not_called() + + +def test_on_pr_merged_has_comment_not_in_backports(mocker, mock_git, mock_gh): + args = argparse.Namespace(pr=124, remote="origin", dry_run=True) + mock_gh.pr_comments = {124: [{"body": " /backport "}]} + mock_gh.create_issue( + title="Release 2.1.0", + body=""" +## Checklist +- [ ] Prepare Release + +## Backports +- [ ] #125 | status=pending +""", + labels=["type: release"], + ) + + mock_pb = mocker.patch("tools.private.release.on_pr_merged.ProcessBackports") + result = OnPrMerged(args, mock_git, mock_gh).run() + + assert result == 1 + mock_pb.assert_not_called() + + +def test_on_pr_merged_success(mocker, mock_git, mock_gh): + args = argparse.Namespace(pr=124, remote="origin", dry_run=True) + mock_gh.pr_comments = {124: [{"body": "/backport"}]} + issue_num = mock_gh.create_issue( + title="Release 2.1.0", + body=""" +## Checklist +- [ ] Prepare Release + +## Backports +- [ ] #124 | status=pending +""", + labels=["type: release"], + ) + + mock_pb_class = mocker.patch("tools.private.release.on_pr_merged.ProcessBackports") + mock_pb_instance = MagicMock() + mock_pb_instance.run.return_value = 0 + mock_pb_class.return_value = mock_pb_instance + + result = OnPrMerged(args, mock_git, mock_gh).run() + + assert result == 0 + + # Verify ProcessBackports was instantiated with correct args + mock_pb_class.assert_called_once() + called_args = mock_pb_class.call_args[0][0] + assert called_args.issue == issue_num + assert called_args.remote == "origin" + assert called_args.dry_run is True + assert called_args.add is None + assert called_args.triggering_comment is None + + # Verify ProcessBackports.run() was called + mock_pb_instance.run.assert_called_once() diff --git a/tests/tools/private/release/prepare_test.py b/tests/tools/private/release/prepare_test.py new file mode 100644 index 0000000000..64275df96f --- /dev/null +++ b/tests/tools/private/release/prepare_test.py @@ -0,0 +1,211 @@ +import argparse + +from tools.private.release.gh import RELEASE_PREPARED_LABEL +from tools.private.release.prepare import Prepare + +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +def test_prepare_success_existing_issue(mocker, release_tool_env, mock_git, mock_gh): + mocker.patch("tools.private.release.prepare.replace_version_next") + mocker.patch("tools.private.release.prepare.changelog_news") + + # Arrange + args = argparse.Namespace(version="2.0.0", issue=None, dry_run=False) + mock_gh.create_issue( + title="Release 2.0.0", + body="- [ ] Prepare Release", + labels=["type: release"], + ) # Assigns issue 1001 + mock_git.status.side_effect = ["", "M foo"] + mock_git.branch_exists.return_value = False + + # Act + result = Prepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + assert 1002 in mock_gh.prs + assert mock_gh.prs[1002]["title"] == "Prepare release v2.0.0" + assert mock_gh.prs[1002]["body"] == "Work towards #1001" + assert mock_gh.prs[1002]["labels"] == [RELEASE_PREPARED_LABEL] + mock_git.add_modified_and_deleted.assert_called_once() + + +def test_prepare_success_create_issue(mocker, release_tool_env, mock_git, mock_gh): + mocker.patch("tools.private.release.prepare.replace_version_next") + mocker.patch("tools.private.release.prepare.changelog_news") + + # Arrange: release_tool_env sets up template_file automatically + args = argparse.Namespace(version="2.0.0", issue=None, dry_run=False) + mock_git.status.side_effect = ["", "M foo"] + mock_git.branch_exists.return_value = False + + # Act + result = Prepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + assert 1001 in mock_gh.issues + assert mock_gh.issues[1001]["title"] == "Release 2.0.0" + assert 1002 in mock_gh.prs + assert mock_gh.prs[1002]["title"] == "Prepare release v2.0.0" + assert mock_gh.prs[1002]["body"] == "Work towards #1001" + assert mock_gh.prs[1002]["labels"] == [RELEASE_PREPARED_LABEL] + mock_git.add_modified_and_deleted.assert_called_once() + + +def test_prepare_ambiguous_issue(mocker, release_tool_env, mock_git, mock_gh): + mocker.patch("tools.private.release.prepare.replace_version_next") + mocker.patch("tools.private.release.prepare.changelog_news") + + # Arrange + args = argparse.Namespace(version="2.0.0", issue=None, dry_run=False) + mock_gh.create_issue( + title="Release 2.0.0", body="issue 1", labels=["type: release"] + ) + mock_gh.create_issue( + title="Release 2.0.0", body="issue 2", labels=["type: release"] + ) + mock_git.status.side_effect = ["", "M foo"] + mock_git.branch_exists.return_value = False + + # Act + result = Prepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + mock_git.add_modified_and_deleted.assert_not_called() + + +def test_prepare_dry_run(mocker, release_tool_env, mock_git, mock_gh): + mocker.patch("tools.private.release.prepare.replace_version_next") + mocker.patch("tools.private.release.prepare.changelog_news") + + # Arrange + args = argparse.Namespace(version="2.0.0", issue=None, dry_run=True) + mock_gh.create_issue(title="Release 2.0.0", body="body", labels=["type: release"]) + mock_git.status.side_effect = [""] + + # Act + result = Prepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.checkout.assert_not_called() + mock_git.commit.assert_not_called() + mock_git.push.assert_not_called() + mock_git.fetch.assert_called_once() + mock_git.add_modified_and_deleted.assert_not_called() + + +def test_prepare_use_associated_pr_from_tracking_issue( + mocker, release_tool_env, mock_git, mock_gh +): + mocker.patch("tools.private.release.prepare.replace_version_next") + mocker.patch("tools.private.release.prepare.changelog_news") + + # Arrange + args = argparse.Namespace(version="2.0.0", issue=None, dry_run=False) + mock_gh.create_issue( + title="Release 2.0.0", + body="- [ ] Prepare Release | status=pending pr=#456", + labels=["type: release"], + ) + mock_git.status.side_effect = ["", ""] + mock_git.branch_exists.return_value = True + + # Act + result = Prepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.checkout.assert_called_once_with("prepare-2.0.0") + mock_git.commit.assert_not_called() + mock_git.push.assert_called_once_with( + "origin", "prepare-2.0.0", set_upstream=True, force=True + ) + updated_body = mock_gh.get_issue_body(1001) + assert "pr=#456" in updated_body + + +def test_prepare_create_pr_when_none_associated( + mocker, release_tool_env, mock_git, mock_gh +): + mocker.patch("tools.private.release.prepare.replace_version_next") + mocker.patch("tools.private.release.prepare.changelog_news") + + # Arrange + args = argparse.Namespace(version="2.0.0", issue=None, dry_run=False) + mock_gh.create_issue( + title="Release 2.0.0", + body="- [ ] Prepare Release", + labels=["type: release"], + ) + mock_git.status.side_effect = ["", ""] + mock_git.branch_exists.return_value = True + + # Act + result = Prepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.checkout.assert_called_once_with("prepare-2.0.0") + mock_git.commit.assert_not_called() + mock_git.push.assert_called_once_with( + "origin", "prepare-2.0.0", set_upstream=True, force=True + ) + updated_body = mock_gh.get_issue_body(1001) + assert "pr=#1002" in updated_body + assert mock_gh.prs[1002]["labels"] == [RELEASE_PREPARED_LABEL] + + +def test_prepare_reuse_existing_pr(mocker, release_tool_env, mock_git, mock_gh): + mocker.patch("tools.private.release.prepare.replace_version_next") + mocker.patch("tools.private.release.prepare.changelog_news") + + # Arrange + args = argparse.Namespace(version="2.0.0", issue=None, dry_run=False) + mock_gh.create_issue( + title="Release 2.0.0", + body="- [ ] Prepare Release", + labels=["type: release"], + ) + mock_gh.prs[456] = { + "number": 456, + "head": "prepare-2.0.0", + "state": "OPEN", + "url": "https://github.com/foo/bar/pull/456", + } + mock_git.status.side_effect = ["", ""] + mock_git.branch_exists.return_value = True + + # Act + result = Prepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.checkout.assert_called_once_with("prepare-2.0.0") + mock_git.commit.assert_not_called() + mock_git.push.assert_called_once_with( + "origin", "prepare-2.0.0", set_upstream=True, force=True + ) + updated_body = mock_gh.get_issue_body(1001) + assert "pr=#456" in updated_body + + +def test_prepare_dry_run_no_issue(mocker, release_tool_env, mock_git, mock_gh): + mocker.patch("tools.private.release.prepare.replace_version_next") + mocker.patch("tools.private.release.prepare.changelog_news") + + # Arrange + args = argparse.Namespace(version="2.0.0", issue=None, dry_run=True) + mock_git.status.side_effect = [""] + + # Act + result = Prepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.checkout.assert_not_called() + mock_git.add_modified_and_deleted.assert_not_called() diff --git a/tests/tools/private/release/process_backports_test.py b/tests/tools/private/release/process_backports_test.py new file mode 100644 index 0000000000..5c37e63a97 --- /dev/null +++ b/tests/tools/private/release/process_backports_test.py @@ -0,0 +1,493 @@ +import argparse +import datetime +from unittest.mock import ANY, call + +from tools.private.release.process_backports import ProcessBackports + +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +def test_process_backports_no_pending(mock_git, mock_gh): + args = argparse.Namespace( + issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": "No backports here", + "labels": ["type: release"], + } + + result = ProcessBackports(args, mock_git, mock_gh).run() + + assert result == 0 + mock_git.fetch.assert_not_called() + + +def test_process_backports_success(mocker, mock_git, mock_gh): + mock_changelog = mocker.patch( + "tools.private.release.process_backports.changelog_news" + ) + mock_replace = mocker.patch( + "tools.private.release.process_backports.replace_version_next" + ) + mock_datetime = mocker.patch("tools.private.release.process_backports.datetime") + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + + args = argparse.Namespace( + issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Sync Changelog #124 +- [ ] Tag Final + +## Backports +- [ ] #124 | status=pending +""", + "labels": ["type: release"], + } + mock_gh.prs[124] = { + "state": "MERGED", + "mergeCommit": {"oid": "abcdef12"}, + } + mock_git.get_remote_tags.return_value = [] + mock_git.sort_commits_chronologically.return_value = ["abcdef12"] + mock_git.get_commit_sha.side_effect = ["12345678", "12345678", "main_sha"] + mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' + mock_git.get_modified_files.return_value = ["news/124.fixed.md"] + mock_git.diff.return_value = "version diff for 124" + mock_git.apply_check.return_value = True + + result = ProcessBackports(args, mock_git, mock_gh).run() + + assert result == 0 + mock_git.fetch.assert_has_calls( + [ + call("origin", tags=True, force=True), + call("origin"), + call("origin", refspec="main"), + ] + ) + mock_git.checkout.assert_has_calls( + [ + call("release/2.0", track_remote="origin"), + call("main", track_remote="origin"), + call("prepare-2.0.0-backports-6affdae", create_branch=True), + call("release/2.0"), + ] + ) + mock_git.cherry_pick.assert_called_once_with("abcdef12") + mock_git.diff.assert_called_once() + mock_git.apply_check.assert_called_once_with(ANY) + mock_git.apply.assert_called_once_with(ANY) + mock_changelog.update_changelog.assert_has_calls( + [ + call("2.0.0", "2026-07-01"), + call( + "2.0.0", + "2026-07-01", + news_files=["news/124.fixed.md"], + delete_news=True, + ), + ] + ) + assert mock_git.add_modified_and_deleted.call_count == 2 + mock_replace.assert_called_once_with("2.0.0") + mock_git.commit.assert_has_calls( + [ + call('Cherry-pick "fix bug"\n\nWork towards #123', amend=True), + call("chore(release): sync changelog for v2.0.0 backports"), + ] + ) + + updated_body = mock_gh.get_issue_body(123) + assert "- [x] #124 | status=done rc=rc0 commit= 12345678" in updated_body + assert "- [ ] Sync Changelog #124 | status=pending pr=#1001" in updated_body + + +def test_process_backports_sync_branch_exists(mocker, mock_git, mock_gh): + mocker.patch("tools.private.release.process_backports.changelog_news") + mocker.patch("tools.private.release.process_backports.replace_version_next") + mock_datetime = mocker.patch("tools.private.release.process_backports.datetime") + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + + args = argparse.Namespace( + issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Sync Changelog #124 +- [ ] Tag Final + +## Backports +- [ ] #124 | status=pending +""", + "labels": ["type: release"], + } + mock_gh.prs[124] = { + "state": "MERGED", + "mergeCommit": {"oid": "abcdef12"}, + } + mock_git.get_remote_tags.return_value = [] + mock_git.sort_commits_chronologically.return_value = ["abcdef12"] + mock_git.get_commit_sha.side_effect = ["12345678", "12345678", "main_sha"] + mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' + mock_git.get_modified_files.return_value = ["news/124.fixed.md"] + mock_git.diff.return_value = "version diff for 124" + mock_git.apply_check.return_value = True + mock_git.branch_exists.return_value = True + + result = ProcessBackports(args, mock_git, mock_gh).run() + + assert result == 0 + mock_git.checkout.assert_has_calls( + [ + call("release/2.0", track_remote="origin"), + call("main", track_remote="origin"), + call("prepare-2.0.0-backports-6affdae"), + call("release/2.0"), + ] + ) + mock_git.reset_hard.assert_has_calls([call(reset_to="main")]) + + +def test_process_backports_dry_run(mocker, mock_git, mock_gh): + mocker.patch("tools.private.release.process_backports.changelog_news") + mocker.patch("tools.private.release.process_backports.replace_version_next") + mock_datetime = mocker.patch("tools.private.release.process_backports.datetime") + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + + args = argparse.Namespace( + issue=123, remote="origin", dry_run=True, add=None, triggering_comment=None + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Sync Changelog #124 +- [ ] Tag Final + +## Backports +- [ ] #124 | status=pending +""", + "labels": ["type: release"], + } + mock_gh.prs[124] = { + "state": "MERGED", + "mergeCommit": {"oid": "abcdef12"}, + } + mock_git.get_remote_tags.return_value = [] + mock_git.sort_commits_chronologically.return_value = ["abcdef12"] + mock_git.get_commit_sha.side_effect = ["12345678", "main_sha"] + mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' + mock_git.get_modified_files.return_value = ["news/124.fixed.md"] + mock_git.diff.return_value = "version diff for 124" + mock_git.apply_check.return_value = True + + result = ProcessBackports(args, mock_git, mock_gh).run() + + assert result == 0 + mock_git.apply.assert_not_called() + mock_git.push.assert_not_called() + + +def test_process_backports_ignored_and_failed_states(mock_git, mock_gh): + args = argparse.Namespace( + issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch + +## Backports +- [ ] #124 | status=pending +- [ ] #125 | status=pending +- [ ] #126 | status=pending +""", + "labels": ["type: release"], + } + mock_git.get_remote_tags.return_value = [] + + mock_gh.prs[124] = {"state": "OPEN"} + mock_gh.prs[125] = {"state": "OPEN", "isDraft": True} + mock_gh.prs[126] = {"state": "CLOSED"} + + result = ProcessBackports(args, mock_git, mock_gh).run() + + assert result == 1 + updated_body = mock_gh.get_issue_body(123) + assert "- [ ] #126 | status=error-closed-pr" in updated_body + mock_git.checkout.assert_not_called() + mock_git.cherry_pick.assert_not_called() + + +def test_process_backports_ignored_error_status(mock_git, mock_gh): + args = argparse.Namespace( + issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch + +## Backports +- [ ] #124 | status=error-merge-conflict +- [ ] #125 | status=error-some-other-error +""", + "labels": ["type: release"], + } + mock_git.get_remote_tags.return_value = [] + + result = ProcessBackports(args, mock_git, mock_gh).run() + + assert result == 0 + mock_git.checkout.assert_not_called() + + +def test_process_backports_cherry_pick_failed(mocker, mock_git, mock_gh): + mock_datetime = mocker.patch("tools.private.release.process_backports.datetime") + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + args = argparse.Namespace( + issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch + +## Backports +- [ ] #124 | status=pending +""", + "labels": ["type: release"], + } + mock_gh.prs[124] = { + "state": "MERGED", + "mergeCommit": {"oid": "abcdef12"}, + } + mock_git.get_remote_tags.return_value = [] + mock_git.sort_commits_chronologically.return_value = ["abcdef12"] + mock_git.cherry_pick.side_effect = Exception("Cherry-pick conflict") + + result = ProcessBackports(args, mock_git, mock_gh).run() + + assert result == 1 + mock_git.checkout.assert_called_once_with("release/2.0", track_remote="origin") + mock_git.cherry_pick.assert_called_once_with("abcdef12") + mock_git.cherry_pick_abort.assert_called_once() + + updated_body = mock_gh.get_issue_body(123) + assert "- [ ] #124 | status=error-merge-conflict" in updated_body + mock_git.commit.assert_not_called() + mock_git.push.assert_not_called() + + +def test_process_backports_add_backports_and_auto_add_rc_task( + mocker, mock_git, mock_gh +): + mocker.patch("tools.private.release.process_backports.changelog_news") + mocker.patch("tools.private.release.process_backports.replace_version_next") + mock_datetime = mocker.patch("tools.private.release.process_backports.datetime") + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + args = argparse.Namespace( + issue=123, + remote="origin", + dry_run=False, + add=["https://github.com/bazel-contrib/rules_python/pull/124"], + triggering_comment=None, + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [x] Tag RC0 | status=done tag=2.0.0-rc0 commit=abcdef12 +- [ ] Tag Final + +## Backports +""", + "labels": ["type: release"], + } + mock_gh.prs[124] = { + "state": "MERGED", + "mergeCommit": {"oid": "abcdef12"}, + } + mock_git.get_remote_tags.return_value = ["2.0.0-rc0"] + mock_git.get_commit_sha.return_value = "12345678" + mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' + mock_git.sort_commits_chronologically.return_value = ["abcdef12"] + mock_git.diff.return_value = "version diff" + mock_git.apply_check.return_value = True + + result = ProcessBackports(args, mock_git, mock_gh).run() + + assert result == 0 + updated_body = mock_gh.get_issue_body(123) + assert "- [x] #124 | status=done rc=rc1 commit= 12345678" in updated_body + assert "- [ ] Sync Changelog #124 | status=pending pr=#1001" in updated_body + + +def test_process_backports_add_backports_marks_invalid(mocker, mock_git, mock_gh): + mocker.patch("tools.private.release.process_backports.changelog_news") + mocker.patch("tools.private.release.process_backports.replace_version_next") + mock_datetime = mocker.patch("tools.private.release.process_backports.datetime") + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + args = argparse.Namespace( + issue=123, + remote="origin", + dry_run=False, + add=["124", "invalid", "125"], + triggering_comment=None, + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [x] Tag RC0 | status=done tag=2.0.0-rc0 commit=abcdef12 +- [ ] Tag Final + +## Backports +""", + "labels": ["type: release"], + } + mock_gh.prs[124] = { + "state": "MERGED", + "mergeCommit": {"oid": "sha_124"}, + } + mock_gh.prs[125] = { + "state": "MERGED", + "mergeCommit": {"oid": "sha_125"}, + } + mock_git.get_remote_tags.return_value = ["2.0.0-rc0"] + mock_git.get_commit_sha.return_value = "1234567890" + mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' + mock_git.sort_commits_chronologically.return_value = ["sha_124", "sha_125"] + mock_git.diff.return_value = "version diff" + mock_git.apply_check.return_value = True + + result = ProcessBackports(args, mock_git, mock_gh).run() + + assert result == 0 + updated_body = mock_gh.get_issue_body(123) + assert "- [ ] invalid | status=error-invalid-pr" in updated_body + assert "- [ ] Sync Changelog #124 | status=pending pr=#1001" in updated_body + assert "- [ ] Sync Changelog #125 | status=pending pr=#1001" in updated_body + + +def test_process_backports_version_sync_failure(mocker, mock_git, mock_gh): + mock_changelog = mocker.patch( + "tools.private.release.process_backports.changelog_news" + ) + mock_replace = mocker.patch( + "tools.private.release.process_backports.replace_version_next" + ) + mock_datetime = mocker.patch("tools.private.release.process_backports.datetime") + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + args = argparse.Namespace( + issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Sync Changelog #124 +- [ ] Sync Changelog #125 +- [ ] Tag Final + +## Backports +- [ ] #124 | status=pending +- [ ] #125 | status=pending +""", + "labels": ["type: release"], + } + mock_gh.prs[124] = { + "state": "MERGED", + "mergeCommit": {"oid": "sha_124"}, + } + mock_gh.prs[125] = { + "state": "MERGED", + "mergeCommit": {"oid": "sha_125"}, + } + mock_git.get_remote_tags.return_value = [] + mock_git.sort_commits_chronologically.return_value = ["sha_124", "sha_125"] + mock_git.get_commit_sha.side_effect = [ + "12345678", + "sha_124_amended", + "sha_125_amended", + "main_sha", + ] + mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' + mock_git.get_modified_files.side_effect = [ + ["news/124.fixed.md"], + ["news/125.fixed.md"], + ] + mock_git.diff.return_value = "diff content" + mock_git.apply_check.side_effect = [False, True] + + result = ProcessBackports(args, mock_git, mock_gh).run() + + assert result == 0 + mock_git.checkout.assert_has_calls( + [ + call("release/2.0", track_remote="origin"), + call("main", track_remote="origin"), + call("prepare-2.0.0-backports-b552a96", create_branch=True), + call("release/2.0"), + ] + ) + mock_git.cherry_pick.assert_has_calls( + [ + call("sha_124"), + call("sha_125"), + ] + ) + assert mock_git.diff.call_count == 2 + assert mock_git.apply_check.call_count == 2 + mock_git.apply.assert_called_once_with(ANY) + + mock_changelog.update_changelog.assert_has_calls( + [ + call("2.0.0", "2026-07-01"), + call("2.0.0", "2026-07-01"), + call( + "2.0.0", + "2026-07-01", + news_files=["news/124.fixed.md", "news/125.fixed.md"], + delete_news=True, + ), + ] + ) + assert mock_git.add_modified_and_deleted.call_count == 3 + assert mock_replace.call_count == 2 + + assert 1001 in mock_gh.prs + pr = mock_gh.prs[1001] + assert ( + "Warning: These PRs failed to update their version markers:\n- #124" + in pr["body"] + ) + + updated_body = mock_gh.get_issue_body(123) + assert "- [ ] Sync Changelog #124 | status=pending pr=#1001" in updated_body + assert "- [ ] Sync Changelog #125 | status=pending pr=#1001" in updated_body diff --git a/tests/tools/private/release/promote_test.py b/tests/tools/private/release/promote_test.py new file mode 100644 index 0000000000..b7fe9c9848 --- /dev/null +++ b/tests/tools/private/release/promote_test.py @@ -0,0 +1,281 @@ +import argparse +from pathlib import Path +from unittest.mock import call, patch + +from tools.private.release.promote import Promote + +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +def test_promote_rc_success(mock_git, mock_gh): + # Arrange + issue_num = mock_gh.create_issue( + title="Release 2.0.0", + body="- [ ] Tag Final", + labels=["type: release"], + ) + args = argparse.Namespace( + version="2.0.0", issue=issue_num, dry_run=False, remote="my-remote" + ) + mock_git.get_remote_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] + mock_git.get_commit_sha.return_value = "abcdef123456" + mock_git.tag_exists.return_value = False + + # Act + result = Promote(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.fetch.assert_has_calls( + [ + call("my-remote", tags=True, force=True), + call("my-remote", refspec="release/2.0"), + ] + ) + mock_git.get_commit_sha.assert_has_calls( + [call("2.0.0-rc1"), call("my-remote/release/2.0")] + ) + mock_git.checkout.assert_not_called() + mock_git.tag_exists.assert_called_once_with("2.0.0") + mock_git.tag.assert_called_once_with("2.0.0", "abcdef123456") + mock_git.push.assert_called_once_with("my-remote", "2.0.0") + + # Verify issue update + expected_updated_body = "- [x] Tag Final | status=done tag=2.0.0 commit= abcdef12" + assert mock_gh.get_issue_body(issue_num) == expected_updated_body + + expected_comment = ( + "**New Release Tagged!** 🐍🌿\n\n" + "Version **2.0.0** has been successfully generated and tagged on branch [`release/2.0`](https://github.com/bazel-contrib/rules_python/tree/release/2.0).\n\n" + "- [Github Release 2.0.0](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0)\n" + "- [BCR Entry 2.0.0](https://registry.bazel.build/modules/rules_python/2.0.0)\n" + "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.0%22%20in%3Atitle%29)\n" + "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_promote.yaml)" + ) + assert mock_gh.issue_comments[issue_num] == [expected_comment] + + +def test_promote_rc_writes_github_output(tmp_path, monkeypatch, mock_git, mock_gh): + # Arrange + github_output_path = str(tmp_path / "github_output") + monkeypatch.setenv("GITHUB_OUTPUT", github_output_path) + issue_num = mock_gh.create_issue( + title="Release 2.0.0", + body="- [ ] Tag Final", + labels=["type: release"], + ) + args = argparse.Namespace( + version="2.0.0", issue=issue_num, dry_run=False, remote="my-remote" + ) + mock_git.get_remote_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] + mock_git.get_commit_sha.return_value = "abcdef123456" + mock_git.tag_exists.return_value = False + + # Act + result = Promote(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + assert Path(github_output_path).exists() + content = Path(github_output_path).read_text(encoding="utf-8") + assert content == "version=2.0.0\n" + + +def test_promote_rc_resolve_issue_success(mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + version="2.0.0", issue=None, dry_run=False, remote="my-remote" + ) + issue_num = mock_gh.create_issue( + title="Release 2.0.0", + body="- [ ] Tag Final", + labels=["type: release"], + ) + mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] + mock_git.tag_exists.return_value = False + mock_git.get_commit_sha.return_value = "abcdef123456" + + # Act + result = Promote(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.fetch.assert_has_calls( + [ + call("my-remote", tags=True, force=True), + call("my-remote", refspec="release/2.0"), + ] + ) + mock_git.get_commit_sha.assert_has_calls( + [call("2.0.0-rc1"), call("my-remote/release/2.0")] + ) + mock_git.checkout.assert_not_called() + mock_git.tag.assert_called_once_with("2.0.0", "abcdef123456") + mock_git.push.assert_called_once_with("my-remote", "2.0.0") + + expected_updated_body = "- [x] Tag Final | status=done tag=2.0.0 commit= abcdef12" + assert mock_gh.get_issue_body(issue_num) == expected_updated_body + + +def test_promote_patch_success(mock_git, mock_gh): + # Arrange + issue_num = mock_gh.create_issue( + title="Release 2.0.1", + body="- [ ] Tag Final", + labels=["type: release"], + ) + args = argparse.Namespace( + version="2.0.1", issue=issue_num, dry_run=False, remote="my-remote" + ) + mock_git.tag_exists.return_value = False + mock_git.get_commit_sha.return_value = "12345678" + + # Act + result = Promote(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.fetch.assert_has_calls( + [ + call("my-remote", tags=True, force=True), + call("my-remote", refspec="release/2.0"), + ] + ) + mock_git.get_current_branch.assert_not_called() + mock_git.get_tags.assert_not_called() + mock_git.get_remote_tags.assert_not_called() + + mock_git.checkout.assert_not_called() + mock_git.get_commit_sha.assert_called_once_with("my-remote/release/2.0") + mock_git.tag.assert_called_once_with("2.0.1", "12345678") + mock_git.push.assert_called_once_with("my-remote", "2.0.1") + + expected_updated_body = "- [x] Tag Final | status=done tag=2.0.1 commit= 12345678" + assert mock_gh.get_issue_body(issue_num) == expected_updated_body + + +@patch("builtins.print") +def test_promote_rc_dry_run_success(mock_print, mock_git, mock_gh): + # Arrange + issue_num = mock_gh.create_issue( + title="Release 2.0.0", + body="- [ ] Tag Final", + labels=["type: release"], + ) + args = argparse.Namespace( + version="2.0.0", issue=issue_num, dry_run=True, remote="my-remote" + ) + mock_git.get_remote_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] + mock_git.get_commit_sha.return_value = "abcdef123456" + mock_git.tag_exists.return_value = False + + # Act + result = Promote(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.fetch.assert_has_calls( + [ + call("my-remote", tags=True, force=True), + call("my-remote", refspec="release/2.0"), + ] + ) + mock_git.get_commit_sha.assert_has_calls( + [call("2.0.0-rc1"), call("my-remote/release/2.0")] + ) + mock_git.tag_exists.assert_called_once_with("2.0.0") + + # Core dry-run assertions: NO modifications + mock_git.tag.assert_not_called() + mock_git.push.assert_not_called() + + mock_print.assert_has_calls( + [ + call(f"Verifying tracking issue #{issue_num} format..."), + call("Fetching remote branch my-remote/release/2.0..."), + call( + "[DRY RUN] Pre-conditions passed successfully for promoting" + " 2.0.0-rc1 to 2.0.0." + ), + call("[DRY RUN] Would tag commit abcdef12 as 2.0.0"), + call("[DRY RUN] Would push tag 2.0.0 to my-remote"), + call(f"[DRY RUN] Would update tracking issue #{issue_num} checklist"), + call(f"[DRY RUN] Would post comment to tracking issue #{issue_num}"), + ] + ) + + +def test_promote_rc_tag_already_exists(mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + version="2.0.0", issue=123, dry_run=False, remote="my-remote" + ) + mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] + mock_git.tag_exists.return_value = True + + # Act + result = Promote(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + mock_git.checkout.assert_not_called() + mock_git.tag.assert_not_called() + mock_git.push.assert_not_called() + + +def test_promote_rc_issue_not_found(mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + version="2.0.0", issue=None, dry_run=False, remote="my-remote" + ) + mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] + mock_git.tag_exists.return_value = False + + # Act + result = Promote(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + mock_git.checkout.assert_not_called() + mock_git.tag.assert_not_called() + mock_git.push.assert_not_called() + + +def test_promote_rc_issue_malformed(mock_git, mock_gh): + # Arrange + issue_num = mock_gh.create_issue( + title="Release 2.0.0", + body="malformed body", + labels=["type: release"], + ) + args = argparse.Namespace( + version="2.0.0", issue=issue_num, dry_run=False, remote="my-remote" + ) + mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] + mock_git.tag_exists.return_value = False + mock_git.get_commit_sha.return_value = "abcdef123456" + + # Act + result = Promote(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + mock_git.checkout.assert_not_called() + mock_git.tag.assert_not_called() + mock_git.push.assert_not_called() + + +def test_promote_rc_no_rc_found(mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + version="2.0.0", issue=123, dry_run=False, remote="my-remote" + ) + mock_git.get_remote_tags.return_value = [] + + # Act + result = Promote(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + mock_git.checkout.assert_not_called() + mock_git.tag.assert_not_called() diff --git a/tests/tools/private/release/release_issue_test.py b/tests/tools/private/release/release_issue_test.py new file mode 100644 index 0000000000..869a303ebf --- /dev/null +++ b/tests/tools/private/release/release_issue_test.py @@ -0,0 +1,157 @@ +from tools.private.release.release_issue import ( + add_backports_to_body, + add_sync_changelog_task_to_body, + format_metadata_line, + parse_checklist_state, + parse_metadata_line, +) + + +def test_parse_metadata_line_spaces(): + # Test with spaces around '=' + line = "- [ ] Tag Final | tag = 2.0.0 commit = abcdef12" + expected = { + "checked": False, + "name": "Tag Final", + "metadata": {"tag": "2.0.0", "commit": "abcdef12"}, + "original_line": line, + } + assert parse_metadata_line(line) == expected + + # Test with spaces after '=' + line = "- [ ] Tag Final | tag= 2.0.0 commit= abcdef12" + expected = { + "checked": False, + "name": "Tag Final", + "metadata": {"tag": "2.0.0", "commit": "abcdef12"}, + "original_line": line, + } + assert parse_metadata_line(line) == expected + + # Test with standard format (no spaces) + line = "- [ ] Tag Final | tag=2.0.0 commit=abcdef12" + expected = { + "checked": False, + "name": "Tag Final", + "metadata": {"tag": "2.0.0", "commit": "abcdef12"}, + "original_line": line, + } + assert parse_metadata_line(line) == expected + + # Test with no metadata + line = "- [ ] Tag Final" + expected = { + "checked": False, + "name": "Tag Final", + "metadata": {}, + "original_line": line, + } + assert parse_metadata_line(line) == expected + + +def test_format_metadata_line(): + # Test with commit metadata (should have space) + metadata = {"status": "done", "tag": "2.0.0", "commit": "abcdef12"} + expected = "- [x] Tag Final | status=done tag=2.0.0 commit= abcdef12" + assert format_metadata_line(True, "Tag Final", metadata) == expected + + # Test with other metadata (should not have space) + metadata = {"status": "done", "pr": "#122"} + expected = "- [x] Prepare Release | status=done pr=#122" + assert format_metadata_line(True, "Prepare Release", metadata) == expected + + # Test with no metadata + expected = "- [ ] Tag Final" + assert format_metadata_line(False, "Tag Final", {}) == expected + + +def test_add_backports_to_body(): + body = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Tag Final + +## Backports +- [ ] #123 | status=done +""" + items = [ + {"ref": "124"}, + {"ref": "#124"}, + {"ref": "125"}, + {"ref": "#123"}, + ] + updated_body = add_backports_to_body(body, items) + expected_body = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Tag Final + +## Backports +- [ ] #123 | status=done +- [ ] #124 +- [ ] #125 +""" + assert updated_body.strip() == expected_body.strip() + + +def test_add_sync_changelog_task_to_body(): + body = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Tag Final +""" + # Insert first task (should go before Tag Final) + body = add_sync_changelog_task_to_body(body, 124) + expected = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Sync Changelog #124 +- [ ] Tag Final +""" + assert body.strip() == expected.strip() + + # Insert second task (should go after the last Sync Changelog task) + body = add_sync_changelog_task_to_body(body, 125) + expected = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Sync Changelog #124 +- [ ] Sync Changelog #125 +- [ ] Tag Final +""" + assert body.strip() == expected.strip() + + # Insert duplicate (should be ignored) + body = add_sync_changelog_task_to_body(body, 124) + assert body.strip() == expected.strip() + + +def test_parse_checklist_state_with_sync_changelogs(): + body = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 +- [ ] Sync Changelog #124 | status=pending pr=#125 +- [ ] Sync Changelog #126 +- [ ] Tag Final +""" + state = parse_checklist_state(body) + assert 124 in state["sync_changelogs"] + assert 126 in state["sync_changelogs"] + + task_124 = state["sync_changelogs"][124] + assert task_124.name == "Sync Changelog #124" + assert not task_124.checked + assert task_124.status == "pending" + assert task_124.pr == "#125" + + task_126 = state["sync_changelogs"][126] + assert task_126.name == "Sync Changelog #126" + assert not task_126.checked + assert task_126.status is None + assert task_126.pr is None diff --git a/tests/tools/private/release/release_test.py b/tests/tools/private/release/release_test.py new file mode 100644 index 0000000000..651af24c2f --- /dev/null +++ b/tests/tools/private/release/release_test.py @@ -0,0 +1,19 @@ +import pytest + +from tools.private.release import release as releaser + + +def test_valid_version(): + # These should not raise an exception + releaser.create_parser().parse_args(["prepare", "0.28.0"]) + releaser.create_parser().parse_args(["promote", "1.0.0", "--remote", "origin"]) + releaser.create_parser().parse_args( + ["create-release-issue", "--version", "1.2.3rc4"] + ) + + +def test_invalid_version(): + with pytest.raises(SystemExit): + releaser.create_parser().parse_args(["prepare", "0.28"]) + with pytest.raises(SystemExit): + releaser.create_parser().parse_args(["prepare", "a.b.c"]) diff --git a/tests/tools/private/release/release_test_helper.py b/tests/tools/private/release/release_test_helper.py new file mode 100644 index 0000000000..0b1c4e0351 --- /dev/null +++ b/tests/tools/private/release/release_test_helper.py @@ -0,0 +1,62 @@ +import dataclasses +from pathlib import Path +from unittest.mock import MagicMock, patch + +import pytest + +from tools.private.release.mock_gh import MockGitHub + + +@dataclasses.dataclass +class ReleaseToolEnv: + """Environment setup for testing release tools. + + Attributes: + git_root: The root path of the temporary Git repository workspace. + github_output_file: Path to the mocked GITHUB_OUTPUT file. + """ + + git_root: Path + github_output_file: Path + + +DEFAULT_RELEASE_TEMPLATE_CONTENT = ( + "template content\n" + "- [ ] Prepare Release\n" + "- [ ] Tag RC0\n" + "- [ ] Tag Final\n" + "\n" + "## Backports\n" +) + + +@pytest.fixture(name="mock_git") +def fixture_mock_git(): + mock_git_inst = MagicMock() + mock_git_inst.get_current_branch.return_value = None + mock_git_inst.get_tags.return_value = [] + mock_git_inst.get_remote_tags.return_value = [] + mock_git_inst.status.return_value = "" + mock_git_inst.branch_exists.return_value = False + mock_git_inst.tag_exists.return_value = False + + with patch("tools.private.release.utils.Git", return_value=mock_git_inst): + yield mock_git_inst + + +@pytest.fixture(name="mock_gh") +def fixture_mock_gh(): + return MockGitHub() + + +@pytest.fixture(name="release_tool_env") +def fixture_release_tool_env(tmp_path, monkeypatch): + """Fixture providing a temp cwd with release template set up.""" + monkeypatch.chdir(tmp_path) + template_dir = tmp_path / ".github" / "ISSUE_TEMPLATE" + template_dir.mkdir(parents=True, exist_ok=True) + template_file = template_dir / "release_tracking_template.md" + template_file.write_text(DEFAULT_RELEASE_TEMPLATE_CONTENT, encoding="utf-8") + github_output_file = tmp_path / "github_output" + monkeypatch.setenv("GITHUB_OUTPUT", str(github_output_file)) + yield ReleaseToolEnv(git_root=tmp_path, github_output_file=github_output_file) diff --git a/tests/tools/private/release/utils_test.py b/tests/tools/private/release/utils_test.py new file mode 100644 index 0000000000..d39b5e108e --- /dev/null +++ b/tests/tools/private/release/utils_test.py @@ -0,0 +1,313 @@ +import pytest + +from tools.private.release import utils + +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +def test_get_latest_version_success(mocker): + mocker.patch( + "tools.private.release.git.Git.get_tags", + return_value=["0.1.0", "1.0.0", "0.2.0"], + ) + assert utils.get_latest_version() == "1.0.0" + + +def test_get_latest_version_rc_is_latest(mocker): + mocker.patch( + "tools.private.release.git.Git.get_tags", + return_value=["0.1.0", "1.0.0", "1.1.0rc0"], + ) + with pytest.raises( + ValueError, match="The latest version is a pre-release version: 1.1.0rc0" + ): + utils.get_latest_version() + + +def test_get_latest_version_no_tags(mocker): + mocker.patch("tools.private.release.git.Git.get_tags", return_value=[]) + with pytest.raises( + RuntimeError, match="No git tags found matching X.Y.Z or X.Y.ZrcN format." + ): + utils.get_latest_version() + + +def test_get_latest_version_no_matching_tags(mocker): + mocker.patch( + "tools.private.release.git.Git.get_tags", return_value=["v1.0", "latest"] + ) + with pytest.raises( + RuntimeError, match="No git tags found matching X.Y.Z or X.Y.ZrcN format." + ): + utils.get_latest_version() + + +def test_get_latest_version_only_rc_tags(mocker): + mocker.patch( + "tools.private.release.git.Git.get_tags", return_value=["1.0.0rc0", "1.1.0rc0"] + ) + with pytest.raises( + ValueError, match="The latest version is a pre-release version: 1.1.0rc0" + ): + utils.get_latest_version() + + +def test_get_latest_rc_tag_no_tags(mocker): + mocker.patch("tools.private.release.git.Git.get_tags", return_value=[]) + assert utils.get_latest_rc_tag("2.0.0") is None + + +def test_get_latest_rc_tag_no_matching_tags(mocker): + mocker.patch( + "tools.private.release.git.Git.get_tags", + return_value=[ + "1.0.0", + "2.0.0", + "v2.0.0-rc0", + "2.1.0-rc0", + ], + ) + assert utils.get_latest_rc_tag("2.0.0") is None + + +def test_get_latest_rc_tag_success(mocker): + mocker.patch( + "tools.private.release.git.Git.get_tags", + return_value=[ + "2.0.0-rc0", + "2.0.0-rc2", + "2.0.0-rc1", + "2.1.0-rc0", + ], + ) + assert utils.get_latest_rc_tag("2.0.0") == "2.0.0-rc2" + + +def test_get_latest_rc_tag_ignores_v_prefix(mocker): + mocker.patch( + "tools.private.release.git.Git.get_tags", + return_value=["v2.0.0-rc0", "2.0.0-rc1"], + ) + assert utils.get_latest_rc_tag("2.0.0") == "2.0.0-rc1" + + +def test_get_latest_rc_tag_remote_success(mocker): + mock_get_remote_tags = mocker.patch( + "tools.private.release.git.Git.get_remote_tags", + return_value=[ + "2.0.0-rc0", + "2.0.0-rc2", + "2.0.0-rc1", + "2.1.0-rc0", + ], + ) + assert utils.get_latest_rc_tag("2.0.0", remote="origin") == "2.0.0-rc2" + mock_get_remote_tags.assert_called_once_with("origin") + + +def test_determine_next_version_no_markers(mocker, release_tool_env): + mocker.patch( + "tools.private.release.git.Git.get_current_branch", return_value="main" + ) + mocker.patch("tools.private.release.utils.get_latest_version", return_value="1.2.3") + (release_tool_env.git_root / "mock_file.bzl").write_text("no markers here") + + next_version = utils.determine_next_version() + + assert next_version == "1.2.4" + + +def test_determine_next_version_only_patch(mocker, release_tool_env): + mocker.patch( + "tools.private.release.git.Git.get_current_branch", return_value="main" + ) + mocker.patch("tools.private.release.utils.get_latest_version", return_value="1.2.3") + (release_tool_env.git_root / "mock_file.bzl").write_text( + ":::{versionchanged} VERSION_NEXT_PATCH" + ) + + next_version = utils.determine_next_version() + + assert next_version == "1.2.4" + + +def test_determine_next_version_only_feature(mocker, release_tool_env): + mocker.patch( + "tools.private.release.git.Git.get_current_branch", return_value="main" + ) + mocker.patch("tools.private.release.utils.get_latest_version", return_value="1.2.3") + (release_tool_env.git_root / "mock_file.bzl").write_text( + ":::{versionadded} VERSION_NEXT_FEATURE" + ) + + next_version = utils.determine_next_version() + + assert next_version == "1.3.0" + + +def test_determine_next_version_both_markers(mocker, release_tool_env): + mocker.patch( + "tools.private.release.git.Git.get_current_branch", return_value="main" + ) + mocker.patch("tools.private.release.utils.get_latest_version", return_value="1.2.3") + (release_tool_env.git_root / "mock_file_patch.bzl").write_text( + ":::{versionchanged} VERSION_NEXT_PATCH" + ) + (release_tool_env.git_root / "mock_file_feature.bzl").write_text( + ":::{versionadded} VERSION_NEXT_FEATURE" + ) + + next_version = utils.determine_next_version() + + assert next_version == "1.3.0" + + +def test_determine_next_version_on_release_branch_with_existing_tags(mocker): + mocker.patch( + "tools.private.release.git.Git.get_current_branch", return_value="release/0.37" + ) + mocker.patch( + "tools.private.release.git.Git.get_tags", + return_value=["0.37.0", "0.37.1", "0.36.0"], + ) + + next_version = utils.determine_next_version() + + assert next_version == "0.37.2" + + +def test_determine_next_version_on_release_branch_no_tags(mocker): + mocker.patch( + "tools.private.release.git.Git.get_current_branch", return_value="release/0.38" + ) + mocker.patch( + "tools.private.release.git.Git.get_tags", return_value=["0.37.0"] + ) # No 0.38.x tags + + next_version = utils.determine_next_version() + + assert next_version == "0.38.0" + + +def test_determine_next_version_on_release_branch_with_active_rc(mocker): + mocker.patch( + "tools.private.release.git.Git.get_current_branch", return_value="release/0.37" + ) + # 0.37.0-rc0 and rc1 exist, but no stable 0.37.0 yet + mocker.patch( + "tools.private.release.git.Git.get_tags", + return_value=["0.37.0-rc0", "0.37.0-rc1", "0.36.0"], + ) + + next_version = utils.determine_next_version() + + # Should target 0.37.0, not 0.37.1 + assert next_version == "0.37.0" + + +def test_determine_next_version_on_release_branch_with_stable_and_active_patch_rc( + mocker, +): + mocker.patch( + "tools.private.release.git.Git.get_current_branch", return_value="release/0.37" + ) + # 0.37.0 stable exists, and 0.37.1-rc0 exists (but no stable 0.37.1 yet) + mocker.patch( + "tools.private.release.git.Git.get_tags", + return_value=["0.37.0", "0.37.1-rc0", "0.36.0"], + ) + + next_version = utils.determine_next_version() + + # Should target 0.37.1, not 0.37.2 + assert next_version == "0.37.1" + + +def test_determine_next_version_on_main_branch_fallback(mocker, release_tool_env): + mocker.patch( + "tools.private.release.git.Git.get_current_branch", return_value="main" + ) + mocker.patch("tools.private.release.utils.get_latest_version", return_value="1.2.3") + (release_tool_env.git_root / "mock_file.bzl").write_text("no markers here") + + next_version = utils.determine_next_version() + + assert next_version == "1.2.4" + + +def test_replace_version_next(release_tool_env): + # Arrange + mock_file_content = """ +:::{versionadded} VERSION_NEXT_FEATURE +blabla +::: + +:::{versionchanged} VERSION_NEXT_PATCH +blabla +::: +""" + (release_tool_env.git_root / "mock_file.bzl").write_text(mock_file_content) + + utils.replace_version_next("0.28.0") + + new_content = (release_tool_env.git_root / "mock_file.bzl").read_text() + + assert ":::{versionadded} 0.28.0" in new_content + assert "VERSION_NEXT_FEATURE" not in new_content + assert "VERSION_NEXT_PATCH" not in new_content + + +def test_replace_version_next_excludes_bazel_dirs(release_tool_env): + # Arrange + mock_file_content = """ +:::{versionadded} VERSION_NEXT_FEATURE +blabla +::: +""" + agents_dir = release_tool_env.git_root / ".agents" + agents_dir.mkdir() + (agents_dir / "mock_file.md").write_text(mock_file_content) + + bazel_dir = release_tool_env.git_root / "bazel-rules_python" + bazel_dir.mkdir() + (bazel_dir / "mock_file.bzl").write_text(mock_file_content) + + tools_dir = release_tool_env.git_root / "tools" / "private" / "release" + tools_dir.mkdir(parents=True) + (tools_dir / "mock_file.bzl").write_text(mock_file_content) + + tests_dir = release_tool_env.git_root / "tests" / "tools" / "private" / "release" + tests_dir.mkdir(parents=True) + (tests_dir / "mock_file.bzl").write_text(mock_file_content) + + version = "0.28.0" + + # Act + utils.replace_version_next(version) + + # Assert + new_content = (agents_dir / "mock_file.md").read_text() + assert "VERSION_NEXT_FEATURE" in new_content + + new_content = (bazel_dir / "mock_file.bzl").read_text() + assert "VERSION_NEXT_FEATURE" in new_content + + new_content = (tools_dir / "mock_file.bzl").read_text() + assert "VERSION_NEXT_FEATURE" in new_content + + new_content = (tests_dir / "mock_file.bzl").read_text() + assert "VERSION_NEXT_FEATURE" in new_content + + +def test_determine_next_version_ignores_agents_markers(mocker, release_tool_env): + mocker.patch( + "tools.private.release.git.Git.get_current_branch", return_value="main" + ) + mocker.patch("tools.private.release.utils.get_latest_version", return_value="1.2.3") + agents_dir = release_tool_env.git_root / ".agents" + agents_dir.mkdir() + (agents_dir / "mock_file.md").write_text(":::{versionadded} VERSION_NEXT_FEATURE") + + next_version = utils.determine_next_version() + + assert next_version == "1.2.4" diff --git a/tests/tools/wheelmaker_test.py b/tests/tools/wheelmaker_test.py new file mode 100644 index 0000000000..cc160869df --- /dev/null +++ b/tests/tools/wheelmaker_test.py @@ -0,0 +1,175 @@ +import io +import unittest +from dataclasses import dataclass, field + +import tools.wheelmaker as wheelmaker + + +class QuoteAllFilenamesTest(unittest.TestCase): + """Tests for quote_all_filenames behavior in _WhlFile. + + Some wheels (like torch) have all filenames quoted in their RECORD file. + When repacking, we preserve this style to minimize diffs. + """ + + def _make_whl_file(self, quote_all: bool) -> wheelmaker._WhlFile: + """Create a _WhlFile instance for testing.""" + buf = io.BytesIO() + return wheelmaker._WhlFile( + buf, + mode="w", + distribution_prefix="test-1.0.0", + quote_all_filenames=quote_all, + ) + + def test_quote_all_quotes_simple_filenames(self) -> None: + """When quote_all_filenames=True, all filenames are quoted.""" + whl = self._make_whl_file(quote_all=True) + self.assertEqual(whl._quote_filename("foo/bar.py"), '"foo/bar.py"') + + def test_quote_all_false_leaves_simple_filenames_unquoted(self) -> None: + """When quote_all_filenames=False, simple filenames stay unquoted.""" + whl = self._make_whl_file(quote_all=False) + self.assertEqual(whl._quote_filename("foo/bar.py"), "foo/bar.py") + + def test_quote_all_quotes_filenames_with_commas(self) -> None: + """Filenames with commas are always quoted, regardless of quote_all_filenames.""" + whl = self._make_whl_file(quote_all=True) + self.assertEqual(whl._quote_filename("foo,bar/baz.py"), '"foo,bar/baz.py"') + + whl = self._make_whl_file(quote_all=False) + self.assertEqual(whl._quote_filename("foo,bar/baz.py"), '"foo,bar/baz.py"') + + +@dataclass +class ArcNameTestCase: + name: str + expected: str + distribution_prefix: str = "" + strip_path_prefixes: list[str] = field(default_factory=list) + add_path_prefix: str = "" + + +class ArcNameFromTest(unittest.TestCase): + def test_arcname_from(self) -> None: + test_cases = [ + ArcNameTestCase(name="a/b/c/file.py", expected="a/b/c/file.py"), + ArcNameTestCase( + name="a/b/c/file.py", + strip_path_prefixes=["a"], + expected="/b/c/file.py", + ), + ArcNameTestCase( + name="a/b/c/file.py", + strip_path_prefixes=["a/b/"], + expected="c/file.py", + ), + # only first found is used and it's not cumulative. + ArcNameTestCase( + name="a/b/c/file.py", + strip_path_prefixes=["a/", "b/"], + expected="b/c/file.py", + ), + # Examples from docs + ArcNameTestCase( + name="foo/bar/baz/file.py", + strip_path_prefixes=["foo", "foo/bar/baz"], + expected="/bar/baz/file.py", + ), + ArcNameTestCase( + name="foo/bar/baz/file.py", + strip_path_prefixes=["foo/bar/baz", "foo"], + expected="/file.py", + ), + ArcNameTestCase( + name="foo/file2.py", + strip_path_prefixes=["foo/bar/baz", "foo"], + expected="/file2.py", + ), + # Files under the distribution prefix (eg mylib-1.0.0-dist-info) + # are unmodified + ArcNameTestCase( + name="mylib-0.0.1-dist-info/WHEEL", + distribution_prefix="mylib", + expected="mylib-0.0.1-dist-info/WHEEL", + ), + ArcNameTestCase( + name="mylib/a/b/c/WHEEL", + distribution_prefix="mylib", + strip_path_prefixes=["mylib"], + expected="mylib/a/b/c/WHEEL", + ), + # Check that prefixes are added + ArcNameTestCase( + name="a/b/c/file.py", + add_path_prefix="namespace/", + expected="namespace/a/b/c/file.py", + ), + ArcNameTestCase( + name="a/b/c/file.py", + strip_path_prefixes=["a"], + add_path_prefix="namespace", + expected="namespace/b/c/file.py", + ), + ArcNameTestCase( + name="a/b/c/file.py", + strip_path_prefixes=["a/b/"], + add_path_prefix="namespace_", + expected="namespace_c/file.py", + ), + ] + for test_case in test_cases: + with self.subTest( + name=test_case.name, + distribution_prefix=test_case.distribution_prefix, + strip_path_prefixes=test_case.strip_path_prefixes, + add_path_prefix=test_case.add_path_prefix, + want=test_case.expected, + ): + got = wheelmaker.arcname_from( + name=test_case.name, + distribution_prefix=test_case.distribution_prefix, + strip_path_prefixes=test_case.strip_path_prefixes, + add_path_prefix=test_case.add_path_prefix, + ) + self.assertEqual(got, test_case.expected) + + +class GetNewRequirementLineTest(unittest.TestCase): + def test_requirement(self): + result = wheelmaker.get_new_requirement_line("requests>=2.0", "") + self.assertEqual(result, "Requires-Dist: requests>=2.0") + + def test_requirement_and_extra(self): + result = wheelmaker.get_new_requirement_line("requests>=2.0", "extra=='dev'") + self.assertEqual(result, "Requires-Dist: requests>=2.0; extra=='dev'") + + def test_requirement_with_url(self): + result = wheelmaker.get_new_requirement_line( + "requests @ git+https://github.com/psf/requests.git@3aa6386c3", "" + ) + self.assertEqual( + result, + "Requires-Dist: requests @ git+https://github.com/psf/requests.git@3aa6386c3", + ) + + def test_requirement_with_marker(self): + result = wheelmaker.get_new_requirement_line( + "requests>=2.0; python_version>='3.6'", "" + ) + self.assertEqual( + result, 'Requires-Dist: requests>=2.0; python_version >= "3.6"' + ) + + def test_requirement_with_marker_and_extra(self): + result = wheelmaker.get_new_requirement_line( + "requests>=2.0; python_version>='3.6'", "extra=='dev'" + ) + self.assertEqual( + result, + "Requires-Dist: requests>=2.0; (python_version >= \"3.6\") and extra=='dev'", + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/tools/zipapp/BUILD.bazel b/tests/tools/zipapp/BUILD.bazel new file mode 100644 index 0000000000..97c8096ce9 --- /dev/null +++ b/tests/tools/zipapp/BUILD.bazel @@ -0,0 +1,23 @@ +load("//tests/support:support.bzl", "SUPPORTS_BZLMOD") +load("//tests/support/pytest_test:pytest_test.bzl", "pytest_test") + +pytest_test( + name = "zipper_test", + srcs = ["zipper_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, + deps = ["//tools/private/zipapp:zipper_lib"], +) + +pytest_test( + name = "exe_zip_maker_test", + srcs = ["exe_zip_maker_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, + deps = ["//tools/private/zipapp:exe_zip_maker_lib"], +) + +pytest_test( + name = "zip_main_maker_test", + srcs = ["zip_main_maker_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, + deps = ["//tools/private/zipapp:zip_main_maker_lib"], +) diff --git a/tests/tools/zipapp/exe_zip_maker_test.py b/tests/tools/zipapp/exe_zip_maker_test.py new file mode 100644 index 0000000000..97df258e92 --- /dev/null +++ b/tests/tools/zipapp/exe_zip_maker_test.py @@ -0,0 +1,46 @@ +import hashlib +import stat + +from tools.private.zipapp import exe_zip_maker + + +def test_create_exe_zip(tmp_path): + preamble_path = tmp_path / "preamble.txt" + zip_path = tmp_path / "data.zip" + output_path = tmp_path / "output.exe" + + # Create dummy zip file + zip_content = b"PK\x03\x04dummyzipcontent" + zip_path.write_bytes(zip_content) + + # Calculate expected hash + expected_hash = hashlib.sha256(zip_content).hexdigest().encode("utf-8") + + # Create preamble with placeholder + preamble_text = b"#!/bin/bash\nEXPECTED_HASH='%ZIP_HASH%'\n# ... logic ...\n" + preamble_path.write_bytes(preamble_text) + + # Call create_exe_zip directly + exe_zip_maker.create_exe_zip(str(preamble_path), str(zip_path), str(output_path)) + + # Verify output exists + assert output_path.exists(), f"Output path '{output_path}' should exist" + + # Verify executable bit + st = output_path.stat() + assert st.st_mode & stat.S_IEXEC, ( + f"Output path '{output_path}' should be executable" + ) + + # Verify content + content = output_path.read_bytes() + + # Split content back into preamble and zip + # We know the preamble text length after substitution. + expected_preamble = preamble_text.replace(b"%ZIP_HASH%", expected_hash) + + assert content.startswith(expected_preamble) + assert content.endswith(zip_content), ( + "Output content should end with the zip content" + ) + assert len(content) == len(expected_preamble) + len(zip_content) diff --git a/tests/tools/zipapp/zip_main_maker_test.py b/tests/tools/zipapp/zip_main_maker_test.py new file mode 100644 index 0000000000..5c7f57c590 --- /dev/null +++ b/tests/tools/zipapp/zip_main_maker_test.py @@ -0,0 +1,90 @@ +import hashlib +import os + +from tools.private.zipapp import zip_main_maker + + +def test_creates_zip_main(tmp_path, monkeypatch): + temp_dir = str(tmp_path) + template_path = os.path.join(temp_dir, "template.py") + with open(template_path, "w", encoding="utf-8") as f: + f.write("hash=%APP_HASH%\nfoo=%FOO%\n") + + output_path = os.path.join(temp_dir, "output.py") + + file1_path = os.path.join(temp_dir, "file1.txt") + with open(file1_path, "wb") as f: + f.write(b"content1") + + file2_path = os.path.join(temp_dir, "file2.txt") + with open(file2_path, "wb") as f: + f.write(b"content2") + + # Add a symlink to test symlink hashing + symlink_path = os.path.join(temp_dir, "symlink.txt") + os.symlink(file1_path, symlink_path) + + manifest_path = os.path.join(temp_dir, "manifest.txt") + with open(manifest_path, "w", encoding="utf-8") as f: + f.write(f"rf-file|0|file1.txt|{file1_path}\n") + f.write(f"rf-file|0|file2.txt|{file2_path}\n") + f.write(f"rf-symlink|1|symlink.txt|{symlink_path}\n") + f.write("rf-empty|empty_file.txt\n") + + argv = [ + "zip_main_maker.py", + "--template", + template_path, + "--output", + output_path, + "--substitution", + "%FOO%=bar", + "--hash_files_manifest", + manifest_path, + ] + + monkeypatch.setattr("sys.argv", argv) + zip_main_maker.main() + + # Calculate expected hash + h = hashlib.sha256() + line1 = f"rf-file|0|file1.txt|{file1_path}" + line2 = f"rf-file|0|file2.txt|{file2_path}" + line3 = f"rf-symlink|1|symlink.txt|{symlink_path}" + line4 = "rf-empty|empty_file.txt" + + # Sort lines like the program does + lines = sorted([line1, line2, line3, line4]) + for line in lines: + parts = line.split("|") + if len(parts) > 1: + _, rest = line.split("|", 1) + h.update(rest.encode("utf-8")) + else: + h.update(line.encode("utf-8")) + + type_ = parts[0] + if type_ == "rf-empty": + continue + if len(parts) >= 4: + is_symlink_str = parts[1] + path = parts[-1] + if not path: + continue + if is_symlink_str == "-1": + is_symlink = not os.path.exists(path) + else: + is_symlink = is_symlink_str == "1" + + if is_symlink: + h.update(os.readlink(path).encode("utf-8")) + else: + with open(path, "rb") as f: + h.update(f.read()) + + expected_hash = h.hexdigest() + + with open(output_path, "r", encoding="utf-8") as f: + content = f.read() + + assert content == f"hash={expected_hash}\nfoo=bar\n" diff --git a/tests/tools/zipapp/zipper_test.py b/tests/tools/zipapp/zipper_test.py new file mode 100644 index 0000000000..bbd85b6536 --- /dev/null +++ b/tests/tools/zipapp/zipper_test.py @@ -0,0 +1,302 @@ +import os +import shutil +import zipfile + +from tools.private.zipapp import zipper + + +def symlink_target_path(p): + return p.replace("/", os.sep) + + +def is_symlink(zip_info): + # Check upper 4 bits of external_attr for S_IFLNK + # S_IFLNK is 0o120000 = 0xA000 + attr = zip_info.external_attr >> 16 + return (attr & 0xF000) == 0xA000 + + +def assert_zip_file_content(zf, path, content=None, is_symlink_file=False, target=None): + info = zf.getinfo(path) + if is_symlink_file: + assert is_symlink(info), f"{path} should be a symlink but is not" + assert zf.read(path).decode() == target + else: + assert not is_symlink(info), f"{path} should NOT be a symlink but is" + assert zf.read(path).decode() == content + + +def create_zip(manifest_path, output_zip, **kwargs): + defaults = { + "manifest_path": manifest_path, + "output_zip": output_zip, + "compress_level": 0, + "workspace_name": "my_ws", + "legacy_external_runfiles": False, + "runfiles_dir": "runfiles", + "platform_pathsep": os.sep, + } + defaults.update(kwargs) + zipper.create_zip(**defaults) + + +def extract_zip(zip_path, extract_dir): + # Manually extract to preserve symlinks + with zipfile.ZipFile(zip_path, "r") as zf: + for info in zf.infolist(): + extract_path = extract_dir / info.filename + extract_path.parent.mkdir(parents=True, exist_ok=True) + if is_symlink(info): + target = zf.read(info).decode() + os.symlink(target, extract_path) + else: + with zf.open(info) as src, open(extract_path, "wb") as dst: + shutil.copyfileobj(src, dst) + + +def test_create_zip_with_files_and_symlinks(tmp_path): + manifest_path = tmp_path / "manifest.txt" + output_zip = tmp_path / "output.zip" + + file1_path = tmp_path / "file1.txt" + file1_path.write_text("content1") + + link_target_path = "target.txt" + symlink_path = tmp_path / "symlink_source" + symlink_path.symlink_to(link_target_path) + + manifest_content = [ + f"regular|0|file1.txt|{file1_path}", + f"rf-file|0|foo/bar.txt|{file1_path}", + f"rf-symlink|1|link1|{symlink_path}", + f"rf-root-symlink|0|root_file|{file1_path}", + "rf-empty|empty_file", + ] + manifest_path.write_text("\n".join(manifest_content)) + + create_zip(manifest_path, output_zip) + + assert output_zip.exists() + + with zipfile.ZipFile(output_zip, "r") as zf: + assert set(zf.namelist()) == { + "file1.txt", + "runfiles/my_ws/foo/bar.txt", + "runfiles/my_ws/link1", + "runfiles/root_file", + "runfiles/my_ws/empty_file", + } + + assert_zip_file_content(zf, "file1.txt", content="content1") + assert_zip_file_content(zf, "runfiles/my_ws/foo/bar.txt", content="content1") + assert_zip_file_content( + zf, "runfiles/my_ws/link1", is_symlink_file=True, target="target.txt" + ) + assert_zip_file_content(zf, "runfiles/root_file", content="content1") + assert_zip_file_content(zf, "runfiles/my_ws/empty_file", content="") + + +def test_create_zip_with_direct_symlink(tmp_path): + manifest_path = tmp_path / "manifest.txt" + output_zip = tmp_path / "output.zip" + + manifest_content = ["symlink|path/to/link|target/path"] + manifest_path.write_text("\n".join(manifest_content)) + + create_zip(manifest_path, output_zip) + + with zipfile.ZipFile(output_zip, "r") as zf: + assert zf.namelist() == ["runfiles/path/to/link"] + assert_zip_file_content( + zf, + "runfiles/path/to/link", + is_symlink_file=True, + target=symlink_target_path("../../target/path"), + ) + + +def test_pathsep_normalization(tmp_path): + manifest_path = tmp_path / "manifest.txt" + output_zip = tmp_path / "output.zip" + + file1_path = tmp_path / "file1.txt" + file1_path.write_text("content1") + + manifest_content = [ + f"regular|0|dir/file.txt|{file1_path}", + "symlink|link/path|target/path", + ] + manifest_path.write_text("\n".join(manifest_content)) + + create_zip(manifest_path, output_zip, platform_pathsep="\\") + + with zipfile.ZipFile(output_zip, "r") as zf: + assert set(zf.namelist()) == {"dir/file.txt", "runfiles/link/path"} + assert_zip_file_content( + zf, + "runfiles/link/path", + is_symlink_file=True, + target="..\\target\\path", + ) + + +def test_symlink_precedence(tmp_path): + manifest_path = tmp_path / "manifest.txt" + output_zip = tmp_path / "output.zip" + + file1_path = tmp_path / "file1.txt" + file1_path.write_text("content1") + + manifest_content = [ + f"rf-file|0|path/to/file|{file1_path}", + "symlink|my_ws/path/to/file|symlink/target", + ] + manifest_path.write_text("\n".join(manifest_content)) + + create_zip(manifest_path, output_zip) + + with zipfile.ZipFile(output_zip, "r") as zf: + assert zf.namelist() == ["runfiles/my_ws/path/to/file"] + assert_zip_file_content( + zf, + "runfiles/my_ws/path/to/file", + is_symlink_file=True, + target=symlink_target_path("../../../symlink/target"), + ) + + +def test_timestamps_are_deterministic(tmp_path): + manifest_path = tmp_path / "manifest.txt" + output_zip = tmp_path / "output.zip" + + file1_path = tmp_path / "file1.txt" + file1_path.write_text("content1") + os.utime(file1_path, None) + + manifest_content = [f"regular|0|file1.txt|{file1_path}"] + manifest_path.write_text("\n".join(manifest_content)) + + create_zip(manifest_path, output_zip) + + with zipfile.ZipFile(output_zip, "r") as zf: + info = zf.getinfo("file1.txt") + expected_date_time = (1980, 1, 1, 0, 0, 0) + assert info.date_time == expected_date_time + + +def test_runfiles_mapping_with_cross_repo_paths(tmp_path): + manifest_path = tmp_path / "manifest.txt" + output_zip = tmp_path / "output.zip" + + file1_path = tmp_path / "file1.txt" + file1_path.write_text("content1") + + manifest_content = [ + f"rf-file|0|../other_repo/foo.txt|{file1_path}", + "rf-empty|../other_repo/empty_file", + ] + manifest_path.write_text("\n".join(manifest_content)) + + create_zip(manifest_path, output_zip, workspace_name="my_ws") + + with zipfile.ZipFile(output_zip, "r") as zf: + assert set(zf.namelist()) == { + "runfiles/other_repo/foo.txt", + "runfiles/other_repo/empty_file", + } + assert_zip_file_content(zf, "runfiles/other_repo/foo.txt", content="content1") + assert_zip_file_content(zf, "runfiles/other_repo/empty_file", content="") + + +def test_runfiles_mapping_with_legacy_external_paths(tmp_path): + manifest_path = tmp_path / "manifest.txt" + output_zip = tmp_path / "output.zip" + + file1_path = tmp_path / "file1.txt" + file1_path.write_text("content1") + + manifest_content = [ + f"rf-file|0|external/other_repo/foo.txt|{file1_path}", + "rf-empty|external/other_repo/empty_file", + ] + manifest_path.write_text("\n".join(manifest_content)) + + create_zip( + manifest_path, output_zip, workspace_name="my_ws", legacy_external_runfiles=True + ) + + with zipfile.ZipFile(output_zip, "r") as zf: + assert set(zf.namelist()) == { + "runfiles/other_repo/foo.txt", + "runfiles/other_repo/empty_file", + } + assert_zip_file_content(zf, "runfiles/other_repo/foo.txt", content="content1") + assert_zip_file_content(zf, "runfiles/other_repo/empty_file", content="") + + +def test_output_deterministic(tmp_path): + manifest_path = tmp_path / "manifest.txt" + output_zip = tmp_path / "output.zip" + + file1 = tmp_path / "file1" + file1.write_text("1") + file2 = tmp_path / "file2" + file2.write_text("2") + file3 = tmp_path / "file3" + file3.write_text("3") + + manifest_content = [ + f"regular|0|z/regular|{file1}", + f"rf-file|0|b_rf_file|{file2}", + f"rf-root-symlink|0|a_root_link|{file3}", + f"regular|0|a/regular|{file3}", + "rf-empty|d_rf_empty", + f"rf-symlink|0|c_rf_link|{file3}", + ] + + manifest_path.write_text("\n".join(manifest_content)) + + create_zip(manifest_path, output_zip, workspace_name="my_ws") + + with zipfile.ZipFile(output_zip, "r") as zf: + assert zf.namelist() == [ + "a/regular", + "runfiles/a_root_link", + "runfiles/my_ws/b_rf_file", + "runfiles/my_ws/c_rf_link", + "runfiles/my_ws/d_rf_empty", + "z/regular", + ] + + +def test_symlink_extraction(tmp_path): + manifest_path = tmp_path / "manifest.txt" + output_zip = tmp_path / "output.zip" + + target_file = tmp_path / "target_file.txt" + target_file.write_text("target content") + + manifest_content = [ + f"rf-file|0|target/path|{target_file}", + "symlink|my_ws/path/to/link|my_ws/target/path", + f"rf-file|0|same_dir_target|{target_file}", + "symlink|my_ws/same_dir_link|my_ws/same_dir_target", + ] + manifest_path.write_text("\n".join(manifest_content)) + + create_zip(manifest_path, output_zip, workspace_name="my_ws") + + extract_dir = tmp_path / "extract" + extract_dir.mkdir() + + extract_zip(output_zip, extract_dir) + + link_path = extract_dir / "runfiles/my_ws/path/to/link" + assert link_path.is_symlink(), f"{link_path} should be a symlink" + assert os.readlink(link_path) == "../../target/path".replace("/", os.path.sep) + assert link_path.read_text() == "target content" + + link2_path = extract_dir / "runfiles/my_ws/same_dir_link" + assert link2_path.is_symlink(), f"{link2_path} should be a symlink" + assert os.readlink(link2_path) == "same_dir_target" + assert link2_path.read_text() == "target content" diff --git a/tests/uv/BUILD.bazel b/tests/uv/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/uv/lock/BUILD.bazel b/tests/uv/lock/BUILD.bazel new file mode 100644 index 0000000000..60d680bde8 --- /dev/null +++ b/tests/uv/lock/BUILD.bazel @@ -0,0 +1,15 @@ +load(":lock_tests.bzl", "lock_test_suite") +load(":uv_lock_to_requirements_tests.bzl", "uv_lock_to_requirements_test_suite") + +exports_files( + glob(["testdata/*"]), + visibility = ["//tests:__subpackages__"], +) + +lock_test_suite( + name = "lock_tests", +) + +uv_lock_to_requirements_test_suite( + name = "uv_lock_to_requirements_tests", +) diff --git a/tests/uv/lock/lock_run_test.py b/tests/uv/lock/lock_run_test.py new file mode 100644 index 0000000000..6de5a96378 --- /dev/null +++ b/tests/uv/lock/lock_run_test.py @@ -0,0 +1,289 @@ +import os +import subprocess +import tempfile +import unittest +from pathlib import Path + +from python import runfiles + +rfiles = runfiles.Create() + + +def _relative_rpath(path: str) -> Path: + """Find file in runfiles, handling Windows .bat/.exe wrappers.""" + # On Windows, try executable extensions first to avoid matching symlink + # entries in the runfiles manifest that point to non-executable files + # (e.g. a Python source file instead of the .exe launcher). + exts = (".exe", ".bat", "") if os.name == "nt" else ("", ".exe", ".bat") + for ext in exts: + p = (Path("_main") / "tests" / "uv" / "lock" / (path + ext)).as_posix() + rpath = rfiles.Rlocation(p) + if rpath: + rp = Path(rpath) + if rp.exists(): + return rp + + # Fallback: look in runfiles directory directly (handles .bat wrappers on + # Windows where Rlocation may return a runfiles link that doesn't exist) + runfiles_dir = os.environ.get("RUNFILES_DIR") + if runfiles_dir: + exts = (".exe", ".bat", "") if os.name == "nt" else ("", ".bat", ".exe") + for ext in exts: + rp = Path(runfiles_dir, "_main", "tests", "uv", "lock", path + ext) + if rp.exists(): + return rp + + raise ValueError(f"Could not find file in runfiles: {path}") + + +def _run_binary(path: Path, **kwargs): + """Run a binary, handling Windows .bat files.""" + if os.name == "nt": + return subprocess.run( + ["cmd.exe", "/c", str(path)], + **kwargs, + ) + return subprocess.run(path, **kwargs) + + +class LockTests(unittest.TestCase): + def _subprocess_env(self, workspace_dir: Path) -> dict[str, str]: + env = { + "BUILD_WORKSPACE_DIRECTORY": str(workspace_dir), + } + # Inherit specific env vars needed for finding runfiles on Windows + for key in ( + "PATH", + "RUNFILES_DIR", + "RUNFILES_MANIFEST_FILE", + "SYSTEMROOT", + "PATHEXT", + ): + if key in os.environ: + env[key] = os.environ[key] + return env + + def test_requirements_run_script_for_new_file(self): + """Verify the requirements_new_file.run script has expected args.""" + run_script_path = _relative_rpath("requirements_new_file.run") + content = run_script_path.read_text() + + if os.name == "nt": + self.assertIn("@echo off", content) + else: + self.assertIn("#!/usr/bin/env bash", content) + self.assertIn("BUILD_WORKSPACE_DIRECTORY", content) + self.assertIn("--no-progress", content) + self.assertIn("--quiet", content) + self.assertIn("does_not_exist.txt", content) + + def test_uv_lock_run_script(self): + """Verify the uv_lock_test.run script has expected args.""" + run_script_path = _relative_rpath("uv_lock_test.run") + content = run_script_path.read_text() + + if os.name == "nt": + self.assertIn("@echo off", content) + else: + self.assertIn("#!/usr/bin/env bash", content) + self.assertIn("--no-progress", content) + self.assertIn("--quiet", content) + + def test_run_script_has_no_output_file_arg(self): + """Verify the uv lock .run script does NOT have --output-file (uv lock doesn't use it).""" + run_script_path = _relative_rpath("uv_lock_test.run") + content = run_script_path.read_text() + + self.assertNotIn("--output-file", content) + + def test_debug_requirements_diff(self): + """Temporary test to print diff of generated vs expected requirements on Windows.""" + expected_path = _relative_rpath("testdata/requirements.txt") + try: + # Pass "requirements.out" directly. We need to handle the case where + # _relative_rpath might append extensions on Windows. + # Actually, _relative_rpath has: + # exts = (".exe", ".bat", "") if os.name == "nt" else ... + # So on Windows it will try .exe, .bat, then empty. + # If "requirements.out" exists, the empty extension will match it. + generated_path = _relative_rpath("requirements.out") + except ValueError as e: + print(f"Could not find requirements.out: {e}") + # Dump runfiles directory to help debug if needed + runfiles_dir = os.environ.get("RUNFILES_DIR") + if runfiles_dir: + print(f"RUNFILES_DIR: {runfiles_dir}") + for root, _, files in os.walk(runfiles_dir): + for f in files: + if "requirements" in f: + print(os.path.join(root, f)) + raise + + expected = expected_path.read_text() + generated = generated_path.read_text() + + if expected != generated: + import difflib + + diff = list( + difflib.unified_diff( + expected.splitlines(keepends=True), + generated.splitlines(keepends=True), + fromfile="expected (testdata/requirements.txt)", + tofile="generated (requirements.out)", + ) + ) + print("\n=== DIFF START ===") + print("".join(diff)) + print("=== DIFF END ===\n") + + # Also print representation to see line endings + print(f"Expected line endings: {repr(expected[:100])}") + print(f"Generated line endings: {repr(generated[:100])}") + + self.assertEqual(expected, generated, "Files differ! See diff above.") + + def test_requirements_updating_for_the_first_time(self): + # Given + copier_path = _relative_rpath("requirements_new_file.update") + + # When + with tempfile.TemporaryDirectory() as dir: + workspace_dir = Path(dir) + want_path = workspace_dir / "tests" / "uv" / "lock" / "does_not_exist.txt" + + self.assertFalse( + want_path.exists(), "The path should not exist after the test" + ) + output = _run_binary( + copier_path, + capture_output=True, + env=self._subprocess_env(workspace_dir), + ) + + # Then + self.assertEqual(0, output.returncode, output.stderr) + stdout = output.stdout.decode("utf-8").replace("\\", "/") + self.assertIn( + "cp /tests/uv/lock/requirements_new_file", + stdout, + ) + self.assertTrue(want_path.exists(), "The path should exist after the test") + self.assertNotEqual(want_path.read_text(), "") + + def test_requirements_updating(self): + # Given + copier_path = _relative_rpath("requirements.update") + + # When + with tempfile.TemporaryDirectory() as dir: + workspace_dir = Path(dir) + want_path = ( + workspace_dir + / "tests" + / "uv" + / "lock" + / "testdata" + / "requirements.txt" + ) + want_path.parent.mkdir(parents=True) + + output = _run_binary( + copier_path, + capture_output=True, + env=self._subprocess_env(workspace_dir), + ) + + # Then + self.assertEqual(0, output.returncode) + stdout = output.stdout.decode("utf-8").replace("\\", "/") + self.assertIn( + "cp /tests/uv/lock/requirements", + stdout, + ) + self.assertTrue(want_path.exists(), "The path should exist after the test") + self.assertNotEqual(want_path.read_text(), "") + + def test_requirements_run_on_the_first_time(self): + # Given + copier_path = _relative_rpath("requirements_new_file.run") + + # When + with tempfile.TemporaryDirectory() as dir: + workspace_dir = Path(dir) + want_path = workspace_dir / "tests" / "uv" / "lock" / "does_not_exist.txt" + # NOTE @aignas 2025-03-18: right now we require users to have the folder + # there already + want_path.parent.mkdir(parents=True) + + self.assertFalse( + want_path.exists(), "The path should not exist before the test" + ) + output = _run_binary( + copier_path, + capture_output=True, + env=self._subprocess_env(workspace_dir), + ) + + # Then + self.assertEqual(0, output.returncode, output.stderr) + self.assertTrue(want_path.exists(), "The path should exist after the test") + got_contents = want_path.read_text() + self.assertNotEqual(got_contents, "") + # NOTE: stdout is typically empty because uv runs with --quiet --no-progress + + def test_requirements_run_script_has_expected_args(self): + """Verify the .run script template has expected args embedded.""" + run_script_path = _relative_rpath("requirements.run") + content = run_script_path.read_text() + + if os.name == "nt": + self.assertIn("@echo off", content) + self.assertIn("%*", content) + else: + self.assertIn("#!/usr/bin/env bash", content) + self.assertIn('"$@"', content) + self.assertIn("BUILD_WORKSPACE_DIRECTORY", content) + self.assertIn("--custom-compile-command", content) + self.assertIn("--generate-hashes", content) + self.assertIn("--no-strip-extras", content) + self.assertIn("--no-python-downloads", content) + self.assertIn("--no-cache", content) + self.assertIn("--no-progress", content) + self.assertIn("--quiet", content) + self.assertIn("--output-file", content) + self.assertIn("requirements.txt", content) + + def test_requirements_run(self): + # Given + copier_path = _relative_rpath("requirements.run") + + # When + with tempfile.TemporaryDirectory() as dir: + workspace_dir = Path(dir) + want_path = ( + workspace_dir + / "tests" + / "uv" + / "lock" + / "testdata" + / "requirements.txt" + ) + want_path.parent.mkdir(parents=True) + + output = _run_binary( + copier_path, + capture_output=True, + env=self._subprocess_env(workspace_dir), + ) + + # Then + self.assertEqual(0, output.returncode, output.stderr) + self.assertTrue(want_path.exists(), "The path should exist after the test") + got_contents = want_path.read_text() + self.assertNotEqual(got_contents, "") + # NOTE: stdout is typically empty because uv runs with --quiet --no-progress + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/uv/lock/lock_tests.bzl b/tests/uv/lock/lock_tests.bzl new file mode 100644 index 0000000000..e3e035717f --- /dev/null +++ b/tests/uv/lock/lock_tests.bzl @@ -0,0 +1,117 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"" + +load("@bazel_skylib//rules:diff_test.bzl", "diff_test") +load("@bazel_skylib//rules:native_binary.bzl", "native_test") +load("//python/uv:lock.bzl", "lock") +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") + +def lock_test_suite(name): + """The test suite with various lock-related integration tests + + Args: + name: {type}`str` the name of the test suite + """ + lock( + name = "requirements", + srcs = ["testdata/requirements.in"], + constraints = [ + "testdata/constraints.txt", + "testdata/constraints2.txt", + ], + build_constraints = [ + "testdata/build_constraints.txt", + "testdata/build_constraints2.txt", + ], + # It seems that the CI remote executors for the RBE do not have network + # connectivity due to current CI setup. + tags = ["no-remote-exec"], + out = "testdata/requirements.txt", + ) + + lock( + name = "requirements_new_file", + srcs = ["testdata/requirements.in"], + out = "does_not_exist.txt", + # It seems that the CI remote executors for the RBE do not have network + # connectivity due to current CI setup. + tags = ["no-remote-exec"], + ) + + py_reconfig_test( + name = "requirements_run_tests", + env = { + "BUILD_WORKSPACE_DIRECTORY": "foo", + }, + srcs = ["lock_run_test.py"], + deps = [ + "//python/runfiles", + ], + data = [ + "requirements_new_file.update", + "requirements_new_file.run", + "requirements.update", + "requirements.run", + "testdata/requirements.txt", + "uv_lock_test.run", + ":requirements", + ], + main = "lock_run_test.py", + tags = [ + "requires-network", + # FIXME @aignas 2025-03-19: it seems that the RBE tests are failing + # to execute the `requirements.run` targets that require network. + # + # We could potentially dump the required `.html` files and somehow + # provide it to the `uv`, but may rely on internal uv handling of + # `--index-url`. + "no-remote-exec", + ], + ) + + # Document and check that the action output matches the in-source file. + diff_test( + name = "requirements_test", + timeout = "short", + file1 = ":requirements", + file2 = "testdata/requirements.txt", + ) + + lock( + name = "uv_lock_test", + srcs = ["testdata/pyproject.toml"], + out = "testdata/uv_lock_expected.lock", + tags = ["no-remote-exec"], + ) + + native_test( + name = "uv_lock_test_check", + src = ":uv_lock_test.update", + target_compatible_with = select({ + "@platforms//os:windows": ["@platforms//:incompatible"], + "//conditions:default": [], + }), + ) + + native.test_suite( + name = name, + tests = [ + ":requirements_test", + "//tests/uv/lock/pyproject_toml:requirements_test", + ":requirements_run_tests", + ":uv_lock_test_check", + ], + ) diff --git a/tests/uv/lock/pyproject_toml/BUILD.bazel b/tests/uv/lock/pyproject_toml/BUILD.bazel new file mode 100644 index 0000000000..d73d206293 --- /dev/null +++ b/tests/uv/lock/pyproject_toml/BUILD.bazel @@ -0,0 +1,31 @@ +load("@bazel_skylib//rules:diff_test.bzl", "diff_test") +load("//python/uv:lock.bzl", "lock") + +# This test verifies that the `lock` rule automatically passes `--project` to `uv pip compile` based +# on the package directory, so that `[tool.uv]` settings from a `pyproject.toml` in the same +# directory are applied. It will exclude a particular dependency from the lock-file, so it will be +# easy to see if we have any issues. +lock( + name = "requirements", + srcs = ["pyproject.toml"], + out = "requirements.txt", + build_constraints = [ + "//tests/uv/lock:testdata/build_constraints.txt", + "//tests/uv/lock:testdata/build_constraints2.txt", + ], + constraints = [ + "//tests/uv/lock:testdata/constraints.txt", + "//tests/uv/lock:testdata/constraints2.txt", + ], + # It seems that the CI remote executors for the RBE do not have network + # connectivity due to current CI setup. + tags = ["no-remote-exec"], +) + +diff_test( + name = "requirements_test", + timeout = "short", + file1 = ":requirements", + file2 = "requirements.txt", + visibility = ["//tests/uv/lock:__pkg__"], +) diff --git a/tests/uv/lock/pyproject_toml/pyproject.toml b/tests/uv/lock/pyproject_toml/pyproject.toml new file mode 100644 index 0000000000..06b96309da --- /dev/null +++ b/tests/uv/lock/pyproject_toml/pyproject.toml @@ -0,0 +1,8 @@ +[project] +name = "test" +version = "0.0.0" +dependencies = ["requests"] + +[tool.uv] +no-build-isolation = true +exclude-dependencies = ["charset-normalizer"] diff --git a/tests/uv/lock/pyproject_toml/requirements.txt b/tests/uv/lock/pyproject_toml/requirements.txt new file mode 100644 index 0000000000..2f4330e1ae --- /dev/null +++ b/tests/uv/lock/pyproject_toml/requirements.txt @@ -0,0 +1,26 @@ +# This file was autogenerated by uv via the following command: +# bazel run //tests/uv/lock/pyproject_toml:requirements.update +certifi==2025.1.31 \ + --hash=sha256:3d5da6925056f6f18f119200434a4780a94263f10d1c21d032a6f6b2baa20651 \ + --hash=sha256:ca78db4565a652026a4db2bcdf68f2fb589ea80d0be70e03929ed730746b84fe + # via + # -c tests/uv/lock/testdata/constraints.txt + # requests +idna==3.10 \ + --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ + --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3 + # via + # -c tests/uv/lock/testdata/constraints.txt + # requests +requests==2.32.3 \ + --hash=sha256:55365417734eb18255590a9ff9eb97e9e1da868d4ccd6402399eaf68af20a760 \ + --hash=sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6 + # via + # -c tests/uv/lock/testdata/constraints.txt + # test (tests/uv/lock/pyproject_toml/pyproject.toml) +urllib3==2.3.0 \ + --hash=sha256:1cee9ad369867bfdbbb48b7dd50374c0967a0bb7710050facf0dd6911440e3df \ + --hash=sha256:f8c5449b3cf0861679ce7e0503c7b44b5ec981bec0d1d3795a07f1ba96f0204d + # via + # -c tests/uv/lock/testdata/constraints.txt + # requests diff --git a/tests/uv/lock/testdata/build_constraints.txt b/tests/uv/lock/testdata/build_constraints.txt new file mode 100644 index 0000000000..34c3ebe3de --- /dev/null +++ b/tests/uv/lock/testdata/build_constraints.txt @@ -0,0 +1 @@ +certifi==2025.1.31 diff --git a/tests/uv/lock/testdata/build_constraints2.txt b/tests/uv/lock/testdata/build_constraints2.txt new file mode 100644 index 0000000000..34c3ebe3de --- /dev/null +++ b/tests/uv/lock/testdata/build_constraints2.txt @@ -0,0 +1 @@ +certifi==2025.1.31 diff --git a/tests/uv/lock/testdata/constraints.txt b/tests/uv/lock/testdata/constraints.txt new file mode 100644 index 0000000000..1d4f29173d --- /dev/null +++ b/tests/uv/lock/testdata/constraints.txt @@ -0,0 +1,5 @@ +charset-normalizer==3.4.0 +certifi==2025.1.31 +requests==2.32.3 +idna==3.10 +urllib3==2.3.0 diff --git a/tests/uv/lock/testdata/constraints2.txt b/tests/uv/lock/testdata/constraints2.txt new file mode 100644 index 0000000000..18ade2c5b9 --- /dev/null +++ b/tests/uv/lock/testdata/constraints2.txt @@ -0,0 +1 @@ +charset-normalizer==3.4.0 diff --git a/tests/uv/lock/testdata/pyproject.toml b/tests/uv/lock/testdata/pyproject.toml new file mode 100644 index 0000000000..d72efcf7c0 --- /dev/null +++ b/tests/uv/lock/testdata/pyproject.toml @@ -0,0 +1,5 @@ +[project] +name = "test-project" +version = "0.0.1" +dependencies = ["requests"] +requires-python = ">=3.9" diff --git a/tests/uv/lock/testdata/requirements.in b/tests/uv/lock/testdata/requirements.in new file mode 100644 index 0000000000..f2293605cf --- /dev/null +++ b/tests/uv/lock/testdata/requirements.in @@ -0,0 +1 @@ +requests diff --git a/tests/uv/lock/testdata/requirements.txt b/tests/uv/lock/testdata/requirements.txt new file mode 100644 index 0000000000..8b718fb145 --- /dev/null +++ b/tests/uv/lock/testdata/requirements.txt @@ -0,0 +1,136 @@ +# This file was autogenerated by uv via the following command: +# bazel run //tests/uv/lock:requirements.update +certifi==2025.1.31 \ + --hash=sha256:3d5da6925056f6f18f119200434a4780a94263f10d1c21d032a6f6b2baa20651 \ + --hash=sha256:ca78db4565a652026a4db2bcdf68f2fb589ea80d0be70e03929ed730746b84fe + # via + # -c tests/uv/lock/testdata/constraints.txt + # requests +charset-normalizer==3.4.0 \ + --hash=sha256:0099d79bdfcf5c1f0c2c72f91516702ebf8b0b8ddd8905f97a8aecf49712c621 \ + --hash=sha256:0713f3adb9d03d49d365b70b84775d0a0d18e4ab08d12bc46baa6132ba78aaf6 \ + --hash=sha256:07afec21bbbbf8a5cc3651aa96b980afe2526e7f048fdfb7f1014d84acc8b6d8 \ + --hash=sha256:0b309d1747110feb25d7ed6b01afdec269c647d382c857ef4663bbe6ad95a912 \ + --hash=sha256:0d99dd8ff461990f12d6e42c7347fd9ab2532fb70e9621ba520f9e8637161d7c \ + --hash=sha256:0de7b687289d3c1b3e8660d0741874abe7888100efe14bd0f9fd7141bcbda92b \ + --hash=sha256:1110e22af8ca26b90bd6364fe4c763329b0ebf1ee213ba32b68c73de5752323d \ + --hash=sha256:130272c698667a982a5d0e626851ceff662565379baf0ff2cc58067b81d4f11d \ + --hash=sha256:136815f06a3ae311fae551c3df1f998a1ebd01ddd424aa5603a4336997629e95 \ + --hash=sha256:14215b71a762336254351b00ec720a8e85cada43b987da5a042e4ce3e82bd68e \ + --hash=sha256:1db4e7fefefd0f548d73e2e2e041f9df5c59e178b4c72fbac4cc6f535cfb1565 \ + --hash=sha256:1ffd9493de4c922f2a38c2bf62b831dcec90ac673ed1ca182fe11b4d8e9f2a64 \ + --hash=sha256:2006769bd1640bdf4d5641c69a3d63b71b81445473cac5ded39740a226fa88ab \ + --hash=sha256:20587d20f557fe189b7947d8e7ec5afa110ccf72a3128d61a2a387c3313f46be \ + --hash=sha256:223217c3d4f82c3ac5e29032b3f1c2eb0fb591b72161f86d93f5719079dae93e \ + --hash=sha256:27623ba66c183eca01bf9ff833875b459cad267aeeb044477fedac35e19ba907 \ + --hash=sha256:285e96d9d53422efc0d7a17c60e59f37fbf3dfa942073f666db4ac71e8d726d0 \ + --hash=sha256:2de62e8801ddfff069cd5c504ce3bc9672b23266597d4e4f50eda28846c322f2 \ + --hash=sha256:2f6c34da58ea9c1a9515621f4d9ac379871a8f21168ba1b5e09d74250de5ad62 \ + --hash=sha256:309a7de0a0ff3040acaebb35ec45d18db4b28232f21998851cfa709eeff49d62 \ + --hash=sha256:35c404d74c2926d0287fbd63ed5d27eb911eb9e4a3bb2c6d294f3cfd4a9e0c23 \ + --hash=sha256:3710a9751938947e6327ea9f3ea6332a09bf0ba0c09cae9cb1f250bd1f1549bc \ + --hash=sha256:3d59d125ffbd6d552765510e3f31ed75ebac2c7470c7274195b9161a32350284 \ + --hash=sha256:40d3ff7fc90b98c637bda91c89d51264a3dcf210cade3a2c6f838c7268d7a4ca \ + --hash=sha256:425c5f215d0eecee9a56cdb703203dda90423247421bf0d67125add85d0c4455 \ + --hash=sha256:43193c5cda5d612f247172016c4bb71251c784d7a4d9314677186a838ad34858 \ + --hash=sha256:44aeb140295a2f0659e113b31cfe92c9061622cadbc9e2a2f7b8ef6b1e29ef4b \ + --hash=sha256:47334db71978b23ebcf3c0f9f5ee98b8d65992b65c9c4f2d34c2eaf5bcaf0594 \ + --hash=sha256:4796efc4faf6b53a18e3d46343535caed491776a22af773f366534056c4e1fbc \ + --hash=sha256:4a51b48f42d9358460b78725283f04bddaf44a9358197b889657deba38f329db \ + --hash=sha256:4b67fdab07fdd3c10bb21edab3cbfe8cf5696f453afce75d815d9d7223fbe88b \ + --hash=sha256:4ec9dd88a5b71abfc74e9df5ebe7921c35cbb3b641181a531ca65cdb5e8e4dea \ + --hash=sha256:4f9fc98dad6c2eaa32fc3af1417d95b5e3d08aff968df0cd320066def971f9a6 \ + --hash=sha256:54b6a92d009cbe2fb11054ba694bc9e284dad30a26757b1e372a1fdddaf21920 \ + --hash=sha256:55f56e2ebd4e3bc50442fbc0888c9d8c94e4e06a933804e2af3e89e2f9c1c749 \ + --hash=sha256:5726cf76c982532c1863fb64d8c6dd0e4c90b6ece9feb06c9f202417a31f7dd7 \ + --hash=sha256:5d447056e2ca60382d460a604b6302d8db69476fd2015c81e7c35417cfabe4cd \ + --hash=sha256:5ed2e36c3e9b4f21dd9422f6893dec0abf2cca553af509b10cd630f878d3eb99 \ + --hash=sha256:5ff2ed8194587faf56555927b3aa10e6fb69d931e33953943bc4f837dfee2242 \ + --hash=sha256:62f60aebecfc7f4b82e3f639a7d1433a20ec32824db2199a11ad4f5e146ef5ee \ + --hash=sha256:63bc5c4ae26e4bc6be6469943b8253c0fd4e4186c43ad46e713ea61a0ba49129 \ + --hash=sha256:6b40e8d38afe634559e398cc32b1472f376a4099c75fe6299ae607e404c033b2 \ + --hash=sha256:6b493a043635eb376e50eedf7818f2f322eabbaa974e948bd8bdd29eb7ef2a51 \ + --hash=sha256:6dba5d19c4dfab08e58d5b36304b3f92f3bd5d42c1a3fa37b5ba5cdf6dfcbcee \ + --hash=sha256:6fd30dc99682dc2c603c2b315bded2799019cea829f8bf57dc6b61efde6611c8 \ + --hash=sha256:707b82d19e65c9bd28b81dde95249b07bf9f5b90ebe1ef17d9b57473f8a64b7b \ + --hash=sha256:7706f5850360ac01d80c89bcef1640683cc12ed87f42579dab6c5d3ed6888613 \ + --hash=sha256:7782afc9b6b42200f7362858f9e73b1f8316afb276d316336c0ec3bd73312742 \ + --hash=sha256:79983512b108e4a164b9c8d34de3992f76d48cadc9554c9e60b43f308988aabe \ + --hash=sha256:7f683ddc7eedd742e2889d2bfb96d69573fde1d92fcb811979cdb7165bb9c7d3 \ + --hash=sha256:82357d85de703176b5587dbe6ade8ff67f9f69a41c0733cf2425378b49954de5 \ + --hash=sha256:84450ba661fb96e9fd67629b93d2941c871ca86fc38d835d19d4225ff946a631 \ + --hash=sha256:86f4e8cca779080f66ff4f191a685ced73d2f72d50216f7112185dc02b90b9b7 \ + --hash=sha256:8cda06946eac330cbe6598f77bb54e690b4ca93f593dee1568ad22b04f347c15 \ + --hash=sha256:8ce7fd6767a1cc5a92a639b391891bf1c268b03ec7e021c7d6d902285259685c \ + --hash=sha256:8ff4e7cdfdb1ab5698e675ca622e72d58a6fa2a8aa58195de0c0061288e6e3ea \ + --hash=sha256:9289fd5dddcf57bab41d044f1756550f9e7cf0c8e373b8cdf0ce8773dc4bd417 \ + --hash=sha256:92a7e36b000bf022ef3dbb9c46bfe2d52c047d5e3f3343f43204263c5addc250 \ + --hash=sha256:92db3c28b5b2a273346bebb24857fda45601aef6ae1c011c0a997106581e8a88 \ + --hash=sha256:95c3c157765b031331dd4db3c775e58deaee050a3042fcad72cbc4189d7c8dca \ + --hash=sha256:980b4f289d1d90ca5efcf07958d3eb38ed9c0b7676bf2831a54d4f66f9c27dfa \ + --hash=sha256:9ae4ef0b3f6b41bad6366fb0ea4fc1d7ed051528e113a60fa2a65a9abb5b1d99 \ + --hash=sha256:9c98230f5042f4945f957d006edccc2af1e03ed5e37ce7c373f00a5a4daa6149 \ + --hash=sha256:9fa2566ca27d67c86569e8c85297aaf413ffab85a8960500f12ea34ff98e4c41 \ + --hash=sha256:a14969b8691f7998e74663b77b4c36c0337cb1df552da83d5c9004a93afdb574 \ + --hash=sha256:a8aacce6e2e1edcb6ac625fb0f8c3a9570ccc7bfba1f63419b3769ccf6a00ed0 \ + --hash=sha256:a8e538f46104c815be19c975572d74afb53f29650ea2025bbfaef359d2de2f7f \ + --hash=sha256:aa41e526a5d4a9dfcfbab0716c7e8a1b215abd3f3df5a45cf18a12721d31cb5d \ + --hash=sha256:aa693779a8b50cd97570e5a0f343538a8dbd3e496fa5dcb87e29406ad0299654 \ + --hash=sha256:ab22fbd9765e6954bc0bcff24c25ff71dcbfdb185fcdaca49e81bac68fe724d3 \ + --hash=sha256:ab2e5bef076f5a235c3774b4f4028a680432cded7cad37bba0fd90d64b187d19 \ + --hash=sha256:ab973df98fc99ab39080bfb0eb3a925181454d7c3ac8a1e695fddfae696d9e90 \ + --hash=sha256:af73657b7a68211996527dbfeffbb0864e043d270580c5aef06dc4b659a4b578 \ + --hash=sha256:b197e7094f232959f8f20541ead1d9862ac5ebea1d58e9849c1bf979255dfac9 \ + --hash=sha256:b295729485b06c1a0683af02a9e42d2caa9db04a373dc38a6a58cdd1e8abddf1 \ + --hash=sha256:b8831399554b92b72af5932cdbbd4ddc55c55f631bb13ff8fe4e6536a06c5c51 \ + --hash=sha256:b8dcd239c743aa2f9c22ce674a145e0a25cb1566c495928440a181ca1ccf6719 \ + --hash=sha256:bcb4f8ea87d03bc51ad04add8ceaf9b0f085ac045ab4d74e73bbc2dc033f0236 \ + --hash=sha256:bd7af3717683bea4c87acd8c0d3d5b44d56120b26fd3f8a692bdd2d5260c620a \ + --hash=sha256:bf4475b82be41b07cc5e5ff94810e6a01f276e37c2d55571e3fe175e467a1a1c \ + --hash=sha256:c3e446d253bd88f6377260d07c895816ebf33ffffd56c1c792b13bff9c3e1ade \ + --hash=sha256:c57516e58fd17d03ebe67e181a4e4e2ccab1168f8c2976c6a334d4f819fe5944 \ + --hash=sha256:c94057af19bc953643a33581844649a7fdab902624d2eb739738a30e2b3e60fc \ + --hash=sha256:cab5d0b79d987c67f3b9e9c53f54a61360422a5a0bc075f43cab5621d530c3b6 \ + --hash=sha256:ce031db0408e487fd2775d745ce30a7cd2923667cf3b69d48d219f1d8f5ddeb6 \ + --hash=sha256:cee4373f4d3ad28f1ab6290684d8e2ebdb9e7a1b74fdc39e4c211995f77bec27 \ + --hash=sha256:d5b054862739d276e09928de37c79ddeec42a6e1bfc55863be96a36ba22926f6 \ + --hash=sha256:dbe03226baf438ac4fda9e2d0715022fd579cb641c4cf639fa40d53b2fe6f3e2 \ + --hash=sha256:dc15e99b2d8a656f8e666854404f1ba54765871104e50c8e9813af8a7db07f12 \ + --hash=sha256:dcaf7c1524c0542ee2fc82cc8ec337f7a9f7edee2532421ab200d2b920fc97cf \ + --hash=sha256:dd4eda173a9fcccb5f2e2bd2a9f423d180194b1bf17cf59e3269899235b2a114 \ + --hash=sha256:dd9a8bd8900e65504a305bf8ae6fa9fbc66de94178c420791d0293702fce2df7 \ + --hash=sha256:de7376c29d95d6719048c194a9cf1a1b0393fbe8488a22008610b0361d834ecf \ + --hash=sha256:e7fdd52961feb4c96507aa649550ec2a0d527c086d284749b2f582f2d40a2e0d \ + --hash=sha256:e91f541a85298cf35433bf66f3fab2a4a2cff05c127eeca4af174f6d497f0d4b \ + --hash=sha256:e9e3c4c9e1ed40ea53acf11e2a386383c3304212c965773704e4603d589343ed \ + --hash=sha256:ee803480535c44e7f5ad00788526da7d85525cfefaf8acf8ab9a310000be4b03 \ + --hash=sha256:f09cb5a7bbe1ecae6e87901a2eb23e0256bb524a79ccc53eb0b7629fbe7677c4 \ + --hash=sha256:f19c1585933c82098c2a520f8ec1227f20e339e33aca8fa6f956f6691b784e67 \ + --hash=sha256:f1a2f519ae173b5b6a2c9d5fa3116ce16e48b3462c8b96dfdded11055e3d6365 \ + --hash=sha256:f28f891ccd15c514a0981f3b9db9aa23d62fe1a99997512b0491d2ed323d229a \ + --hash=sha256:f3e73a4255342d4eb26ef6df01e3962e73aa29baa3124a8e824c5d3364a65748 \ + --hash=sha256:f606a1881d2663630ea5b8ce2efe2111740df4b687bd78b34a8131baa007f79b \ + --hash=sha256:fe9f97feb71aa9896b81973a7bbada8c49501dc73e58a10fcef6663af95e5079 \ + --hash=sha256:ffc519621dce0c767e96b9c53f09c5d215578e10b02c285809f76509a3931482 + # via + # -c tests/uv/lock/testdata/constraints.txt + # -c tests/uv/lock/testdata/constraints2.txt + # requests +idna==3.10 \ + --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ + --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3 + # via + # -c tests/uv/lock/testdata/constraints.txt + # requests +requests==2.32.3 \ + --hash=sha256:55365417734eb18255590a9ff9eb97e9e1da868d4ccd6402399eaf68af20a760 \ + --hash=sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6 + # via + # -c tests/uv/lock/testdata/constraints.txt + # -r tests/uv/lock/testdata/requirements.in +urllib3==2.3.0 \ + --hash=sha256:1cee9ad369867bfdbbb48b7dd50374c0967a0bb7710050facf0dd6911440e3df \ + --hash=sha256:f8c5449b3cf0861679ce7e0503c7b44b5ec981bec0d1d3795a07f1ba96f0204d + # via + # -c tests/uv/lock/testdata/constraints.txt + # requests diff --git a/tests/uv/lock/testdata/uv_lock_expected.lock b/tests/uv/lock/testdata/uv_lock_expected.lock new file mode 100644 index 0000000000..fc027674ef --- /dev/null +++ b/tests/uv/lock/testdata/uv_lock_expected.lock @@ -0,0 +1,218 @@ +version = 1 +revision = 3 +requires-python = ">=3.9" +resolution-markers = [ + "python_full_version >= '3.10'", + "python_full_version < '3.10'", +] + +[[package]] +name = "certifi" +version = "2026.4.22" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/25/ee/6caf7a40c36a1220410afe15a1cc64993a1f864871f698c0f93acb72842a/certifi-2026.4.22.tar.gz", hash = "sha256:8d455352a37b71bf76a79caa83a3d6c25afee4a385d632127b6afb3963f1c580", size = 137077, upload-time = "2026-04-22T11:26:11.191Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/22/30/7cd8fdcdfbc5b869528b079bfb76dcdf6056b1a2097a662e5e8c04f42965/certifi-2026.4.22-py3-none-any.whl", hash = "sha256:3cb2210c8f88ba2318d29b0388d1023c8492ff72ecdde4ebdaddbb13a31b1c4a", size = 135707, upload-time = "2026-04-22T11:26:09.372Z" }, +] + +[[package]] +name = "charset-normalizer" +version = "3.4.7" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/e7/a1/67fe25fac3c7642725500a3f6cfe5821ad557c3abb11c9d20d12c7008d3e/charset_normalizer-3.4.7.tar.gz", hash = "sha256:ae89db9e5f98a11a4bf50407d4363e7b09b31e55bc117b4f7d80aab97ba009e5", size = 144271, upload-time = "2026-04-02T09:28:39.342Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/26/08/0f303cb0b529e456bb116f2d50565a482694fbb94340bf56d44677e7ed03/charset_normalizer-3.4.7-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:cdd68a1fb318e290a2077696b7eb7a21a49163c455979c639bf5a5dcdc46617d", size = 315182, upload-time = "2026-04-02T09:25:40.673Z" }, + { url = "https://files.pythonhosted.org/packages/24/47/b192933e94b546f1b1fe4df9cc1f84fcdbf2359f8d1081d46dd029b50207/charset_normalizer-3.4.7-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e17b8d5d6a8c47c85e68ca8379def1303fd360c3e22093a807cd34a71cd082b8", size = 209329, upload-time = "2026-04-02T09:25:42.354Z" }, + { url = "https://files.pythonhosted.org/packages/c2/b4/01fa81c5ca6141024d89a8fc15968002b71da7f825dd14113207113fabbd/charset_normalizer-3.4.7-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:511ef87c8aec0783e08ac18565a16d435372bc1ac25a91e6ac7f5ef2b0bff790", size = 231230, upload-time = "2026-04-02T09:25:44.281Z" }, + { url = "https://files.pythonhosted.org/packages/20/f7/7b991776844dfa058017e600e6e55ff01984a063290ca5622c0b63162f68/charset_normalizer-3.4.7-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:007d05ec7321d12a40227aae9e2bc6dca73f3cb21058999a1df9e193555a9dcc", size = 225890, upload-time = "2026-04-02T09:25:45.475Z" }, + { url = "https://files.pythonhosted.org/packages/20/e7/bed0024a0f4ab0c8a9c64d4445f39b30c99bd1acd228291959e3de664247/charset_normalizer-3.4.7-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:cf29836da5119f3c8a8a70667b0ef5fdca3bb12f80fd06487cfa575b3909b393", size = 216930, upload-time = "2026-04-02T09:25:46.58Z" }, + { url = "https://files.pythonhosted.org/packages/e2/ab/b18f0ab31cdd7b3ddb8bb76c4a414aeb8160c9810fdf1bc62f269a539d87/charset_normalizer-3.4.7-cp310-cp310-manylinux_2_31_armv7l.whl", hash = "sha256:12d8baf840cc7889b37c7c770f478adea7adce3dcb3944d02ec87508e2dcf153", size = 202109, upload-time = "2026-04-02T09:25:48.031Z" }, + { url = "https://files.pythonhosted.org/packages/82/e5/7e9440768a06dfb3075936490cb82dbf0ee20a133bf0dd8551fa096914ec/charset_normalizer-3.4.7-cp310-cp310-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:d560742f3c0d62afaccf9f41fe485ed69bd7661a241f86a3ef0f0fb8b1a397af", size = 214684, upload-time = "2026-04-02T09:25:49.245Z" }, + { url = "https://files.pythonhosted.org/packages/71/94/8c61d8da9f062fdf457c80acfa25060ec22bf1d34bbeaca4350f13bcfd07/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:b14b2d9dac08e28bb8046a1a0434b1750eb221c8f5b87a68f4fa11a6f97b5e34", size = 212785, upload-time = "2026-04-02T09:25:50.671Z" }, + { url = "https://files.pythonhosted.org/packages/66/cd/6e9889c648e72c0ab2e5967528bb83508f354d706637bc7097190c874e13/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_armv7l.whl", hash = "sha256:bc17a677b21b3502a21f66a8cc64f5bfad4df8a0b8434d661666f8ce90ac3af1", size = 203055, upload-time = "2026-04-02T09:25:51.802Z" }, + { url = "https://files.pythonhosted.org/packages/92/2e/7a951d6a08aefb7eb8e1b54cdfb580b1365afdd9dd484dc4bee9e5d8f258/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_ppc64le.whl", hash = "sha256:750e02e074872a3fad7f233b47734166440af3cdea0add3e95163110816d6752", size = 232502, upload-time = "2026-04-02T09:25:53.388Z" }, + { url = "https://files.pythonhosted.org/packages/58/d5/abcf2d83bf8e0a1286df55cd0dc1d49af0da4282aa77e986df343e7de124/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_riscv64.whl", hash = "sha256:4e5163c14bffd570ef2affbfdd77bba66383890797df43dc8b4cc7d6f500bf53", size = 214295, upload-time = "2026-04-02T09:25:54.765Z" }, + { url = "https://files.pythonhosted.org/packages/47/3a/7d4cd7ed54be99973a0dc176032cba5cb1f258082c31fa6df35cff46acfc/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_s390x.whl", hash = "sha256:6ed74185b2db44f41ef35fd1617c5888e59792da9bbc9190d6c7300617182616", size = 227145, upload-time = "2026-04-02T09:25:55.904Z" }, + { url = "https://files.pythonhosted.org/packages/1d/98/3a45bf8247889cf28262ebd3d0872edff11565b2a1e3064ccb132db3fbb0/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:94e1885b270625a9a828c9793b4d52a64445299baa1fea5a173bf1d3dd9a1a5a", size = 218884, upload-time = "2026-04-02T09:25:57.074Z" }, + { url = "https://files.pythonhosted.org/packages/ad/80/2e8b7f8915ed5c9ef13aa828d82738e33888c485b65ebf744d615040c7ea/charset_normalizer-3.4.7-cp310-cp310-win32.whl", hash = "sha256:6785f414ae0f3c733c437e0f3929197934f526d19dfaa75e18fdb4f94c6fb374", size = 148343, upload-time = "2026-04-02T09:25:58.199Z" }, + { url = "https://files.pythonhosted.org/packages/35/1b/3b8c8c77184af465ee9ad88b5aea46ea6b2e1f7b9dc9502891e37af21e30/charset_normalizer-3.4.7-cp310-cp310-win_amd64.whl", hash = "sha256:6696b7688f54f5af4462118f0bfa7c1621eeb87154f77fa04b9295ce7a8f2943", size = 159174, upload-time = "2026-04-02T09:25:59.322Z" }, + { url = "https://files.pythonhosted.org/packages/be/c1/feb40dca40dbb21e0a908801782d9288c64fc8d8e562c2098e9994c8c21b/charset_normalizer-3.4.7-cp310-cp310-win_arm64.whl", hash = "sha256:66671f93accb62ed07da56613636f3641f1a12c13046ce91ffc923721f23c008", size = 147805, upload-time = "2026-04-02T09:26:00.756Z" }, + { url = "https://files.pythonhosted.org/packages/c2/d7/b5b7020a0565c2e9fa8c09f4b5fa6232feb326b8c20081ccded47ea368fd/charset_normalizer-3.4.7-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:7641bb8895e77f921102f72833904dcd9901df5d6d72a2ab8f31d04b7e51e4e7", size = 309705, upload-time = "2026-04-02T09:26:02.191Z" }, + { url = "https://files.pythonhosted.org/packages/5a/53/58c29116c340e5456724ecd2fff4196d236b98f3da97b404bc5e51ac3493/charset_normalizer-3.4.7-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:202389074300232baeb53ae2569a60901f7efadd4245cf3a3bf0617d60b439d7", size = 206419, upload-time = "2026-04-02T09:26:03.583Z" }, + { url = "https://files.pythonhosted.org/packages/b2/02/e8146dc6591a37a00e5144c63f29fb7c97a734ea8a111190783c0e60ab63/charset_normalizer-3.4.7-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:30b8d1d8c52a48c2c5690e152c169b673487a2a58de1ec7393196753063fcd5e", size = 227901, upload-time = "2026-04-02T09:26:04.738Z" }, + { url = "https://files.pythonhosted.org/packages/fb/73/77486c4cd58f1267bf17db420e930c9afa1b3be3fe8c8b8ebbebc9624359/charset_normalizer-3.4.7-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:532bc9bf33a68613fd7d65e4b1c71a6a38d7d42604ecf239c77392e9b4e8998c", size = 222742, upload-time = "2026-04-02T09:26:06.36Z" }, + { url = "https://files.pythonhosted.org/packages/a1/fa/f74eb381a7d94ded44739e9d94de18dc5edc9c17fb8c11f0a6890696c0a9/charset_normalizer-3.4.7-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2fe249cb4651fd12605b7288b24751d8bfd46d35f12a20b1ba33dea122e690df", size = 214061, upload-time = "2026-04-02T09:26:08.347Z" }, + { url = "https://files.pythonhosted.org/packages/dc/92/42bd3cefcf7687253fb86694b45f37b733c97f59af3724f356fa92b8c344/charset_normalizer-3.4.7-cp311-cp311-manylinux_2_31_armv7l.whl", hash = "sha256:65bcd23054beab4d166035cabbc868a09c1a49d1efe458fe8e4361215df40265", size = 199239, upload-time = "2026-04-02T09:26:09.823Z" }, + { url = "https://files.pythonhosted.org/packages/4c/3d/069e7184e2aa3b3cddc700e3dd267413dc259854adc3380421c805c6a17d/charset_normalizer-3.4.7-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:08e721811161356f97b4059a9ba7bafb23ea5ee2255402c42881c214e173c6b4", size = 210173, upload-time = "2026-04-02T09:26:10.953Z" }, + { url = "https://files.pythonhosted.org/packages/62/51/9d56feb5f2e7074c46f93e0ebdbe61f0848ee246e2f0d89f8e20b89ebb8f/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:e060d01aec0a910bdccb8be71faf34e7799ce36950f8294c8bf612cba65a2c9e", size = 209841, upload-time = "2026-04-02T09:26:12.142Z" }, + { url = "https://files.pythonhosted.org/packages/d2/59/893d8f99cc4c837dda1fe2f1139079703deb9f321aabcb032355de13b6c7/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:38c0109396c4cfc574d502df99742a45c72c08eff0a36158b6f04000043dbf38", size = 200304, upload-time = "2026-04-02T09:26:13.711Z" }, + { url = "https://files.pythonhosted.org/packages/7d/1d/ee6f3be3464247578d1ed5c46de545ccc3d3ff933695395c402c21fa6b77/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:1c2a768fdd44ee4a9339a9b0b130049139b8ce3c01d2ce09f67f5a68048d477c", size = 229455, upload-time = "2026-04-02T09:26:14.941Z" }, + { url = "https://files.pythonhosted.org/packages/54/bb/8fb0a946296ea96a488928bdce8ef99023998c48e4713af533e9bb98ef07/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:1a87ca9d5df6fe460483d9a5bbf2b18f620cbed41b432e2bddb686228282d10b", size = 210036, upload-time = "2026-04-02T09:26:16.478Z" }, + { url = "https://files.pythonhosted.org/packages/9a/bc/015b2387f913749f82afd4fcba07846d05b6d784dd16123cb66860e0237d/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_s390x.whl", hash = "sha256:d635aab80466bc95771bb78d5370e74d36d1fe31467b6b29b8b57b2a3cd7d22c", size = 224739, upload-time = "2026-04-02T09:26:17.751Z" }, + { url = "https://files.pythonhosted.org/packages/17/ab/63133691f56baae417493cba6b7c641571a2130eb7bceba6773367ab9ec5/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:ae196f021b5e7c78e918242d217db021ed2a6ace2bc6ae94c0fc596221c7f58d", size = 216277, upload-time = "2026-04-02T09:26:18.981Z" }, + { url = "https://files.pythonhosted.org/packages/06/6d/3be70e827977f20db77c12a97e6a9f973631a45b8d186c084527e53e77a4/charset_normalizer-3.4.7-cp311-cp311-win32.whl", hash = "sha256:adb2597b428735679446b46c8badf467b4ca5f5056aae4d51a19f9570301b1ad", size = 147819, upload-time = "2026-04-02T09:26:20.295Z" }, + { url = "https://files.pythonhosted.org/packages/20/d9/5f67790f06b735d7c7637171bbfd89882ad67201891b7275e51116ed8207/charset_normalizer-3.4.7-cp311-cp311-win_amd64.whl", hash = "sha256:8e385e4267ab76874ae30db04c627faaaf0b509e1ccc11a95b3fc3e83f855c00", size = 159281, upload-time = "2026-04-02T09:26:21.74Z" }, + { url = "https://files.pythonhosted.org/packages/ca/83/6413f36c5a34afead88ce6f66684d943d91f233d76dd083798f9602b75ae/charset_normalizer-3.4.7-cp311-cp311-win_arm64.whl", hash = "sha256:d4a48e5b3c2a489fae013b7589308a40146ee081f6f509e047e0e096084ceca1", size = 147843, upload-time = "2026-04-02T09:26:22.901Z" }, + { url = "https://files.pythonhosted.org/packages/0c/eb/4fc8d0a7110eb5fc9cc161723a34a8a6c200ce3b4fbf681bc86feee22308/charset_normalizer-3.4.7-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:eca9705049ad3c7345d574e3510665cb2cf844c2f2dcfe675332677f081cbd46", size = 311328, upload-time = "2026-04-02T09:26:24.331Z" }, + { url = "https://files.pythonhosted.org/packages/f8/e3/0fadc706008ac9d7b9b5be6dc767c05f9d3e5df51744ce4cc9605de7b9f4/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6178f72c5508bfc5fd446a5905e698c6212932f25bcdd4b47a757a50605a90e2", size = 208061, upload-time = "2026-04-02T09:26:25.568Z" }, + { url = "https://files.pythonhosted.org/packages/42/f0/3dd1045c47f4a4604df85ec18ad093912ae1344ac706993aff91d38773a2/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e1421b502d83040e6d7fb2fb18dff63957f720da3d77b2fbd3187ceb63755d7b", size = 229031, upload-time = "2026-04-02T09:26:26.865Z" }, + { url = "https://files.pythonhosted.org/packages/dc/67/675a46eb016118a2fbde5a277a5d15f4f69d5f3f5f338e5ee2f8948fcf43/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:edac0f1ab77644605be2cbba52e6b7f630731fc42b34cb0f634be1a6eface56a", size = 225239, upload-time = "2026-04-02T09:26:28.044Z" }, + { url = "https://files.pythonhosted.org/packages/4b/f8/d0118a2f5f23b02cd166fa385c60f9b0d4f9194f574e2b31cef350ad7223/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5649fd1c7bade02f320a462fdefd0b4bd3ce036065836d4f42e0de958038e116", size = 216589, upload-time = "2026-04-02T09:26:29.239Z" }, + { url = "https://files.pythonhosted.org/packages/b1/f1/6d2b0b261b6c4ceef0fcb0d17a01cc5bc53586c2d4796fa04b5c540bc13d/charset_normalizer-3.4.7-cp312-cp312-manylinux_2_31_armv7l.whl", hash = "sha256:203104ed3e428044fd943bc4bf45fa73c0730391f9621e37fe39ecf477b128cb", size = 202733, upload-time = "2026-04-02T09:26:30.5Z" }, + { url = "https://files.pythonhosted.org/packages/6f/c0/7b1f943f7e87cc3db9626ba17807d042c38645f0a1d4415c7a14afb5591f/charset_normalizer-3.4.7-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:298930cec56029e05497a76988377cbd7457ba864beeea92ad7e844fe74cd1f1", size = 212652, upload-time = "2026-04-02T09:26:31.709Z" }, + { url = "https://files.pythonhosted.org/packages/38/dd/5a9ab159fe45c6e72079398f277b7d2b523e7f716acc489726115a910097/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:708838739abf24b2ceb208d0e22403dd018faeef86ddac04319a62ae884c4f15", size = 211229, upload-time = "2026-04-02T09:26:33.282Z" }, + { url = "https://files.pythonhosted.org/packages/d5/ff/531a1cad5ca855d1c1a8b69cb71abfd6d85c0291580146fda7c82857caa1/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:0f7eb884681e3938906ed0434f20c63046eacd0111c4ba96f27b76084cd679f5", size = 203552, upload-time = "2026-04-02T09:26:34.845Z" }, + { url = "https://files.pythonhosted.org/packages/c1/4c/a5fb52d528a8ca41f7598cb619409ece30a169fbdf9cdce592e53b46c3a6/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:4dc1e73c36828f982bfe79fadf5919923f8a6f4df2860804db9a98c48824ce8d", size = 230806, upload-time = "2026-04-02T09:26:36.152Z" }, + { url = "https://files.pythonhosted.org/packages/59/7a/071feed8124111a32b316b33ae4de83d36923039ef8cf48120266844285b/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:aed52fea0513bac0ccde438c188c8a471c4e0f457c2dd20cdbf6ea7a450046c7", size = 212316, upload-time = "2026-04-02T09:26:37.672Z" }, + { url = "https://files.pythonhosted.org/packages/fd/35/f7dba3994312d7ba508e041eaac39a36b120f32d4c8662b8814dab876431/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:fea24543955a6a729c45a73fe90e08c743f0b3334bbf3201e6c4bc1b0c7fa464", size = 227274, upload-time = "2026-04-02T09:26:38.93Z" }, + { url = "https://files.pythonhosted.org/packages/8a/2d/a572df5c9204ab7688ec1edc895a73ebded3b023bb07364710b05dd1c9be/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:bb6d88045545b26da47aa879dd4a89a71d1dce0f0e549b1abcb31dfe4a8eac49", size = 218468, upload-time = "2026-04-02T09:26:40.17Z" }, + { url = "https://files.pythonhosted.org/packages/86/eb/890922a8b03a568ca2f336c36585a4713c55d4d67bf0f0c78924be6315ca/charset_normalizer-3.4.7-cp312-cp312-win32.whl", hash = "sha256:2257141f39fe65a3fdf38aeccae4b953e5f3b3324f4ff0daf9f15b8518666a2c", size = 148460, upload-time = "2026-04-02T09:26:41.416Z" }, + { url = "https://files.pythonhosted.org/packages/35/d9/0e7dffa06c5ab081f75b1b786f0aefc88365825dfcd0ac544bdb7b2b6853/charset_normalizer-3.4.7-cp312-cp312-win_amd64.whl", hash = "sha256:5ed6ab538499c8644b8a3e18debabcd7ce684f3fa91cf867521a7a0279cab2d6", size = 159330, upload-time = "2026-04-02T09:26:42.554Z" }, + { url = "https://files.pythonhosted.org/packages/9e/5d/481bcc2a7c88ea6b0878c299547843b2521ccbc40980cb406267088bc701/charset_normalizer-3.4.7-cp312-cp312-win_arm64.whl", hash = "sha256:56be790f86bfb2c98fb742ce566dfb4816e5a83384616ab59c49e0604d49c51d", size = 147828, upload-time = "2026-04-02T09:26:44.075Z" }, + { url = "https://files.pythonhosted.org/packages/c1/3b/66777e39d3ae1ddc77ee606be4ec6d8cbd4c801f65e5a1b6f2b11b8346dd/charset_normalizer-3.4.7-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:f496c9c3cc02230093d8330875c4c3cdfc3b73612a5fd921c65d39cbcef08063", size = 309627, upload-time = "2026-04-02T09:26:45.198Z" }, + { url = "https://files.pythonhosted.org/packages/2e/4e/b7f84e617b4854ade48a1b7915c8ccfadeba444d2a18c291f696e37f0d3b/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0ea948db76d31190bf08bd371623927ee1339d5f2a0b4b1b4a4439a65298703c", size = 207008, upload-time = "2026-04-02T09:26:46.824Z" }, + { url = "https://files.pythonhosted.org/packages/c4/bb/ec73c0257c9e11b268f018f068f5d00aa0ef8c8b09f7753ebd5f2880e248/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a277ab8928b9f299723bc1a2dabb1265911b1a76341f90a510368ca44ad9ab66", size = 228303, upload-time = "2026-04-02T09:26:48.397Z" }, + { url = "https://files.pythonhosted.org/packages/85/fb/32d1f5033484494619f701e719429c69b766bfc4dbc61aa9e9c8c166528b/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:3bec022aec2c514d9cf199522a802bd007cd588ab17ab2525f20f9c34d067c18", size = 224282, upload-time = "2026-04-02T09:26:49.684Z" }, + { url = "https://files.pythonhosted.org/packages/fa/07/330e3a0dda4c404d6da83b327270906e9654a24f6c546dc886a0eb0ffb23/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e044c39e41b92c845bc815e5ae4230804e8e7bc29e399b0437d64222d92809dd", size = 215595, upload-time = "2026-04-02T09:26:50.915Z" }, + { url = "https://files.pythonhosted.org/packages/e3/7c/fc890655786e423f02556e0216d4b8c6bcb6bdfa890160dc66bf52dee468/charset_normalizer-3.4.7-cp313-cp313-manylinux_2_31_armv7l.whl", hash = "sha256:f495a1652cf3fbab2eb0639776dad966c2fb874d79d87ca07f9d5f059b8bd215", size = 201986, upload-time = "2026-04-02T09:26:52.197Z" }, + { url = "https://files.pythonhosted.org/packages/d8/97/bfb18b3db2aed3b90cf54dc292ad79fdd5ad65c4eae454099475cbeadd0d/charset_normalizer-3.4.7-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e712b419df8ba5e42b226c510472b37bd57b38e897d3eca5e8cfd410a29fa859", size = 211711, upload-time = "2026-04-02T09:26:53.49Z" }, + { url = "https://files.pythonhosted.org/packages/6f/a5/a581c13798546a7fd557c82614a5c65a13df2157e9ad6373166d2a3e645d/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:7804338df6fcc08105c7745f1502ba68d900f45fd770d5bdd5288ddccb8a42d8", size = 210036, upload-time = "2026-04-02T09:26:54.975Z" }, + { url = "https://files.pythonhosted.org/packages/8c/bf/b3ab5bcb478e4193d517644b0fb2bf5497fbceeaa7a1bc0f4d5b50953861/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:481551899c856c704d58119b5025793fa6730adda3571971af568f66d2424bb5", size = 202998, upload-time = "2026-04-02T09:26:56.303Z" }, + { url = "https://files.pythonhosted.org/packages/e7/4e/23efd79b65d314fa320ec6017b4b5834d5c12a58ba4610aa353af2e2f577/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:f59099f9b66f0d7145115e6f80dd8b1d847176df89b234a5a6b3f00437aa0832", size = 230056, upload-time = "2026-04-02T09:26:57.554Z" }, + { url = "https://files.pythonhosted.org/packages/b9/9f/1e1941bc3f0e01df116e68dc37a55c4d249df5e6fa77f008841aef68264f/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:f59ad4c0e8f6bba240a9bb85504faa1ab438237199d4cce5f622761507b8f6a6", size = 211537, upload-time = "2026-04-02T09:26:58.843Z" }, + { url = "https://files.pythonhosted.org/packages/80/0f/088cbb3020d44428964a6c97fe1edfb1b9550396bf6d278330281e8b709c/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:3dedcc22d73ec993f42055eff4fcfed9318d1eeb9a6606c55892a26964964e48", size = 226176, upload-time = "2026-04-02T09:27:00.437Z" }, + { url = "https://files.pythonhosted.org/packages/6a/9f/130394f9bbe06f4f63e22641d32fc9b202b7e251c9aef4db044324dac493/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:64f02c6841d7d83f832cd97ccf8eb8a906d06eb95d5276069175c696b024b60a", size = 217723, upload-time = "2026-04-02T09:27:02.021Z" }, + { url = "https://files.pythonhosted.org/packages/73/55/c469897448a06e49f8fa03f6caae97074fde823f432a98f979cc42b90e69/charset_normalizer-3.4.7-cp313-cp313-win32.whl", hash = "sha256:4042d5c8f957e15221d423ba781e85d553722fc4113f523f2feb7b188cc34c5e", size = 148085, upload-time = "2026-04-02T09:27:03.192Z" }, + { url = "https://files.pythonhosted.org/packages/5d/78/1b74c5bbb3f99b77a1715c91b3e0b5bdb6fe302d95ace4f5b1bec37b0167/charset_normalizer-3.4.7-cp313-cp313-win_amd64.whl", hash = "sha256:3946fa46a0cf3e4c8cb1cc52f56bb536310d34f25f01ca9b6c16afa767dab110", size = 158819, upload-time = "2026-04-02T09:27:04.454Z" }, + { url = "https://files.pythonhosted.org/packages/68/86/46bd42279d323deb8687c4a5a811fd548cb7d1de10cf6535d099877a9a9f/charset_normalizer-3.4.7-cp313-cp313-win_arm64.whl", hash = "sha256:80d04837f55fc81da168b98de4f4b797ef007fc8a79ab71c6ec9bc4dd662b15b", size = 147915, upload-time = "2026-04-02T09:27:05.971Z" }, + { url = "https://files.pythonhosted.org/packages/97/c8/c67cb8c70e19ef1960b97b22ed2a1567711de46c4ddf19799923adc836c2/charset_normalizer-3.4.7-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:c36c333c39be2dbca264d7803333c896ab8fa7d4d6f0ab7edb7dfd7aea6e98c0", size = 309234, upload-time = "2026-04-02T09:27:07.194Z" }, + { url = "https://files.pythonhosted.org/packages/99/85/c091fdee33f20de70d6c8b522743b6f831a2f1cd3ff86de4c6a827c48a76/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1c2aed2e5e41f24ea8ef1590b8e848a79b56f3a5564a65ceec43c9d692dc7d8a", size = 208042, upload-time = "2026-04-02T09:27:08.749Z" }, + { url = "https://files.pythonhosted.org/packages/87/1c/ab2ce611b984d2fd5d86a5a8a19c1ae26acac6bad967da4967562c75114d/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:54523e136b8948060c0fa0bc7b1b50c32c186f2fceee897a495406bb6e311d2b", size = 228706, upload-time = "2026-04-02T09:27:09.951Z" }, + { url = "https://files.pythonhosted.org/packages/a8/29/2b1d2cb00bf085f59d29eb773ce58ec2d325430f8c216804a0a5cd83cbca/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:715479b9a2802ecac752a3b0efa2b0b60285cf962ee38414211abdfccc233b41", size = 224727, upload-time = "2026-04-02T09:27:11.175Z" }, + { url = "https://files.pythonhosted.org/packages/47/5c/032c2d5a07fe4d4855fea851209cca2b6f03ebeb6d4e3afdb3358386a684/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:bd6c2a1c7573c64738d716488d2cdd3c00e340e4835707d8fdb8dc1a66ef164e", size = 215882, upload-time = "2026-04-02T09:27:12.446Z" }, + { url = "https://files.pythonhosted.org/packages/2c/c2/356065d5a8b78ed04499cae5f339f091946a6a74f91e03476c33f0ab7100/charset_normalizer-3.4.7-cp314-cp314-manylinux_2_31_armv7l.whl", hash = "sha256:c45e9440fb78f8ddabcf714b68f936737a121355bf59f3907f4e17721b9d1aae", size = 200860, upload-time = "2026-04-02T09:27:13.721Z" }, + { url = "https://files.pythonhosted.org/packages/0c/cd/a32a84217ced5039f53b29f460962abb2d4420def55afabe45b1c3c7483d/charset_normalizer-3.4.7-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:3534e7dcbdcf757da6b85a0bbf5b6868786d5982dd959b065e65481644817a18", size = 211564, upload-time = "2026-04-02T09:27:15.272Z" }, + { url = "https://files.pythonhosted.org/packages/44/86/58e6f13ce26cc3b8f4a36b94a0f22ae2f00a72534520f4ae6857c4b81f89/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:e8ac484bf18ce6975760921bb6148041faa8fef0547200386ea0b52b5d27bf7b", size = 211276, upload-time = "2026-04-02T09:27:16.834Z" }, + { url = "https://files.pythonhosted.org/packages/8f/fe/d17c32dc72e17e155e06883efa84514ca375f8a528ba2546bee73fc4df81/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:a5fe03b42827c13cdccd08e6c0247b6a6d4b5e3cdc53fd1749f5896adcdc2356", size = 201238, upload-time = "2026-04-02T09:27:18.229Z" }, + { url = "https://files.pythonhosted.org/packages/6a/29/f33daa50b06525a237451cdb6c69da366c381a3dadcd833fa5676bc468b3/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:2d6eb928e13016cea4f1f21d1e10c1cebd5a421bc57ddf5b1142ae3f86824fab", size = 230189, upload-time = "2026-04-02T09:27:19.445Z" }, + { url = "https://files.pythonhosted.org/packages/b6/6e/52c84015394a6a0bdcd435210a7e944c5f94ea1055f5cc5d56c5fe368e7b/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:e74327fb75de8986940def6e8dee4f127cc9752bee7355bb323cc5b2659b6d46", size = 211352, upload-time = "2026-04-02T09:27:20.79Z" }, + { url = "https://files.pythonhosted.org/packages/8c/d7/4353be581b373033fb9198bf1da3cf8f09c1082561e8e922aa7b39bf9fe8/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:d6038d37043bced98a66e68d3aa2b6a35505dc01328cd65217cefe82f25def44", size = 227024, upload-time = "2026-04-02T09:27:22.063Z" }, + { url = "https://files.pythonhosted.org/packages/30/45/99d18aa925bd1740098ccd3060e238e21115fffbfdcb8f3ece837d0ace6c/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:7579e913a5339fb8fa133f6bbcfd8e6749696206cf05acdbdca71a1b436d8e72", size = 217869, upload-time = "2026-04-02T09:27:23.486Z" }, + { url = "https://files.pythonhosted.org/packages/5c/05/5ee478aa53f4bb7996482153d4bfe1b89e0f087f0ab6b294fcf92d595873/charset_normalizer-3.4.7-cp314-cp314-win32.whl", hash = "sha256:5b77459df20e08151cd6f8b9ef8ef1f961ef73d85c21a555c7eed5b79410ec10", size = 148541, upload-time = "2026-04-02T09:27:25.146Z" }, + { url = "https://files.pythonhosted.org/packages/48/77/72dcb0921b2ce86420b2d79d454c7022bf5be40202a2a07906b9f2a35c97/charset_normalizer-3.4.7-cp314-cp314-win_amd64.whl", hash = "sha256:92a0a01ead5e668468e952e4238cccd7c537364eb7d851ab144ab6627dbbe12f", size = 159634, upload-time = "2026-04-02T09:27:26.642Z" }, + { url = "https://files.pythonhosted.org/packages/c6/a3/c2369911cd72f02386e4e340770f6e158c7980267da16af8f668217abaa0/charset_normalizer-3.4.7-cp314-cp314-win_arm64.whl", hash = "sha256:67f6279d125ca0046a7fd386d01b311c6363844deac3e5b069b514ba3e63c246", size = 148384, upload-time = "2026-04-02T09:27:28.271Z" }, + { url = "https://files.pythonhosted.org/packages/94/09/7e8a7f73d24dba1f0035fbbf014d2c36828fc1bf9c88f84093e57d315935/charset_normalizer-3.4.7-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:effc3f449787117233702311a1b7d8f59cba9ced946ba727bdc329ec69028e24", size = 330133, upload-time = "2026-04-02T09:27:29.474Z" }, + { url = "https://files.pythonhosted.org/packages/8d/da/96975ddb11f8e977f706f45cddd8540fd8242f71ecdb5d18a80723dcf62c/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:fbccdc05410c9ee21bbf16a35f4c1d16123dcdeb8a1d38f33654fa21d0234f79", size = 216257, upload-time = "2026-04-02T09:27:30.793Z" }, + { url = "https://files.pythonhosted.org/packages/e5/e8/1d63bf8ef2d388e95c64b2098f45f84758f6d102a087552da1485912637b/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:733784b6d6def852c814bce5f318d25da2ee65dd4839a0718641c696e09a2960", size = 234851, upload-time = "2026-04-02T09:27:32.44Z" }, + { url = "https://files.pythonhosted.org/packages/9b/40/e5ff04233e70da2681fa43969ad6f66ca5611d7e669be0246c4c7aaf6dc8/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a89c23ef8d2c6b27fd200a42aa4ac72786e7c60d40efdc76e6011260b6e949c4", size = 233393, upload-time = "2026-04-02T09:27:34.03Z" }, + { url = "https://files.pythonhosted.org/packages/be/c1/06c6c49d5a5450f76899992f1ee40b41d076aee9279b49cf9974d2f313d5/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6c114670c45346afedc0d947faf3c7f701051d2518b943679c8ff88befe14f8e", size = 223251, upload-time = "2026-04-02T09:27:35.369Z" }, + { url = "https://files.pythonhosted.org/packages/2b/9f/f2ff16fb050946169e3e1f82134d107e5d4ae72647ec8a1b1446c148480f/charset_normalizer-3.4.7-cp314-cp314t-manylinux_2_31_armv7l.whl", hash = "sha256:a180c5e59792af262bf263b21a3c49353f25945d8d9f70628e73de370d55e1e1", size = 206609, upload-time = "2026-04-02T09:27:36.661Z" }, + { url = "https://files.pythonhosted.org/packages/69/d5/a527c0cd8d64d2eab7459784fb4169a0ac76e5a6fc5237337982fd61347e/charset_normalizer-3.4.7-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:3c9a494bc5ec77d43cea229c4f6db1e4d8fe7e1bbffa8b6f0f0032430ff8ab44", size = 220014, upload-time = "2026-04-02T09:27:38.019Z" }, + { url = "https://files.pythonhosted.org/packages/7e/80/8a7b8104a3e203074dc9aa2c613d4b726c0e136bad1cc734594b02867972/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:8d828b6667a32a728a1ad1d93957cdf37489c57b97ae6c4de2860fa749b8fc1e", size = 218979, upload-time = "2026-04-02T09:27:39.37Z" }, + { url = "https://files.pythonhosted.org/packages/02/9a/b759b503d507f375b2b5c153e4d2ee0a75aa215b7f2489cf314f4541f2c0/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:cf1493cd8607bec4d8a7b9b004e699fcf8f9103a9284cc94962cb73d20f9d4a3", size = 209238, upload-time = "2026-04-02T09:27:40.722Z" }, + { url = "https://files.pythonhosted.org/packages/c2/4e/0f3f5d47b86bdb79256e7290b26ac847a2832d9a4033f7eb2cd4bcf4bb5b/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:0c96c3b819b5c3e9e165495db84d41914d6894d55181d2d108cc1a69bfc9cce0", size = 236110, upload-time = "2026-04-02T09:27:42.33Z" }, + { url = "https://files.pythonhosted.org/packages/96/23/bce28734eb3ed2c91dcf93abeb8a5cf393a7b2749725030bb630e554fdd8/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:752a45dc4a6934060b3b0dab47e04edc3326575f82be64bc4fc293914566503e", size = 219824, upload-time = "2026-04-02T09:27:43.924Z" }, + { url = "https://files.pythonhosted.org/packages/2c/6f/6e897c6984cc4d41af319b077f2f600fc8214eb2fe2d6bcb79141b882400/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:8778f0c7a52e56f75d12dae53ae320fae900a8b9b4164b981b9c5ce059cd1fcb", size = 233103, upload-time = "2026-04-02T09:27:45.348Z" }, + { url = "https://files.pythonhosted.org/packages/76/22/ef7bd0fe480a0ae9b656189ec00744b60933f68b4f42a7bb06589f6f576a/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:ce3412fbe1e31eb81ea42f4169ed94861c56e643189e1e75f0041f3fe7020abe", size = 225194, upload-time = "2026-04-02T09:27:46.706Z" }, + { url = "https://files.pythonhosted.org/packages/c5/a7/0e0ab3e0b5bc1219bd80a6a0d4d72ca74d9250cb2382b7c699c147e06017/charset_normalizer-3.4.7-cp314-cp314t-win32.whl", hash = "sha256:c03a41a8784091e67a39648f70c5f97b5b6a37f216896d44d2cdcb82615339a0", size = 159827, upload-time = "2026-04-02T09:27:48.053Z" }, + { url = "https://files.pythonhosted.org/packages/7a/1d/29d32e0fb40864b1f878c7f5a0b343ae676c6e2b271a2d55cc3a152391da/charset_normalizer-3.4.7-cp314-cp314t-win_amd64.whl", hash = "sha256:03853ed82eeebbce3c2abfdbc98c96dc205f32a79627688ac9a27370ea61a49c", size = 174168, upload-time = "2026-04-02T09:27:49.795Z" }, + { url = "https://files.pythonhosted.org/packages/de/32/d92444ad05c7a6e41fb2036749777c163baf7a0301a040cb672d6b2b1ae9/charset_normalizer-3.4.7-cp314-cp314t-win_arm64.whl", hash = "sha256:c35abb8bfff0185efac5878da64c45dafd2b37fb0383add1be155a763c1f083d", size = 153018, upload-time = "2026-04-02T09:27:51.116Z" }, + { url = "https://files.pythonhosted.org/packages/01/1b/ef725f8eb19b5a261b30f78efa9252ef9d017985cb499102f6f49834cd12/charset_normalizer-3.4.7-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:177a0ba5f0211d488e295aaf82707237e331c24788d8d76c96c5a41594723217", size = 299121, upload-time = "2026-04-02T09:28:14.372Z" }, + { url = "https://files.pythonhosted.org/packages/a3/22/2f12878fbc680fbbb52386cd39a379801f62eaca74fc8b323381325f0f04/charset_normalizer-3.4.7-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6e0d51f618228538a3e8f46bd246f87a6cd030565e015803691603f55e12afb5", size = 200612, upload-time = "2026-04-02T09:28:16.162Z" }, + { url = "https://files.pythonhosted.org/packages/bc/b6/10c84e789126ca97d4a7228863a30481e786980a8b8cfcbf4f30658ca63c/charset_normalizer-3.4.7-cp39-cp39-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:14265bfe1f09498b9d8ec91e9ec9fa52775edf90fcbde092b25f4a33d444fea9", size = 221041, upload-time = "2026-04-02T09:28:17.554Z" }, + { url = "https://files.pythonhosted.org/packages/21/7b/c414866a138400b2e81973d006da7f694cfeaf895ef07d2cba9a8743841a/charset_normalizer-3.4.7-cp39-cp39-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:87fad7d9ba98c86bcb41b2dc8dbb326619be2562af1f8ff50776a39e55721c5a", size = 216323, upload-time = "2026-04-02T09:28:18.863Z" }, + { url = "https://files.pythonhosted.org/packages/2e/92/bdcf94997e06b223d826df3abed45a5ad6e17f609b7df9d25cd23b5bde30/charset_normalizer-3.4.7-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f22dec1690b584cea26fade98b2435c132c1b5f68e39f5a0b7627cd7ae31f1dc", size = 208419, upload-time = "2026-04-02T09:28:20.332Z" }, + { url = "https://files.pythonhosted.org/packages/1a/64/3f9142293c88b1b10e199649ed1330f070c2a68e305335a5819fa7f25fa7/charset_normalizer-3.4.7-cp39-cp39-manylinux_2_31_armv7l.whl", hash = "sha256:d61f00a0869d77422d9b2aba989e2d24afa6ffd552af442e0e58de4f35ea6d00", size = 195016, upload-time = "2026-04-02T09:28:21.657Z" }, + { url = "https://files.pythonhosted.org/packages/c1/d1/d8a6b7dd5c5636b76ce0d080bc57d8e56c7bbd6bc2ac941529a35e41d84a/charset_normalizer-3.4.7-cp39-cp39-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:6370e8686f662e6a3941ee48ed4742317cafbe5707e36406e9df792cdb535776", size = 206115, upload-time = "2026-04-02T09:28:23.259Z" }, + { url = "https://files.pythonhosted.org/packages/dd/8c/60ebe912379627d023eb96995b40bc50308729f210f43d66109ca0a7bbd2/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:a6c5863edfbe888d9eff9c8b8087354e27618d9da76425c119293f11712a6319", size = 204022, upload-time = "2026-04-02T09:28:24.779Z" }, + { url = "https://files.pythonhosted.org/packages/d5/2a/41816ceda78a551cbfdfbeab6f3891152b0e3f758ce6580c2c18c829f774/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_armv7l.whl", hash = "sha256:ed065083d0898c9d5b4bbec7b026fd755ff7454e6e8b73a67f8c744b13986e24", size = 195914, upload-time = "2026-04-02T09:28:26.181Z" }, + { url = "https://files.pythonhosted.org/packages/8f/9b/7c7f4b7f11525fcbdfba752455314ac60646bae91cdd671d531c1f7a97c6/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_ppc64le.whl", hash = "sha256:2cd4a60d0e2fb04537162c62bbbb4182f53541fe0ede35cdf270a1c1e723cc42", size = 222159, upload-time = "2026-04-02T09:28:27.504Z" }, + { url = "https://files.pythonhosted.org/packages/9f/57/301682e7469bdbfa2ce219a804f0668b2266ab8520570d85d3b3ef483ea3/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_riscv64.whl", hash = "sha256:813c0e0132266c08eb87469a642cb30aaff57c5f426255419572aaeceeaa7bf4", size = 206154, upload-time = "2026-04-02T09:28:28.848Z" }, + { url = "https://files.pythonhosted.org/packages/20/ec/90339ff5cdc598b265748c1f231c7d7fbd9123a92cee10f757e0b1448de4/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_s390x.whl", hash = "sha256:07d9e39b01743c3717745f4c530a6349eadbfa043c7577eef86c502c15df2c67", size = 217423, upload-time = "2026-04-02T09:28:30.248Z" }, + { url = "https://files.pythonhosted.org/packages/2e/e7/a7a6147f8e3375676309cf584b25c72a3bab784ea4085b0011fa07b23aeb/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:c0f081d69a6e58272819b70288d3221a6ee64b98df852631c80f293514d3b274", size = 210604, upload-time = "2026-04-02T09:28:31.736Z" }, + { url = "https://files.pythonhosted.org/packages/1a/62/d9340c7a79c393e57807d7fb6c57e82060687891f81b74d3201958b919c1/charset_normalizer-3.4.7-cp39-cp39-win32.whl", hash = "sha256:8751d2787c9131302398b11e6c8068053dcb55d5a8964e114b6e196cf16cb366", size = 144631, upload-time = "2026-04-02T09:28:33.158Z" }, + { url = "https://files.pythonhosted.org/packages/21/e7/92901117e2ddc8facfe8235a3ecd4eb482185b2ad5d5b6606b37c1afea06/charset_normalizer-3.4.7-cp39-cp39-win_amd64.whl", hash = "sha256:12a6fff75f6bc66711b73a2f0addfc4c8c15a20e805146a02d147a318962c444", size = 154710, upload-time = "2026-04-02T09:28:34.557Z" }, + { url = "https://files.pythonhosted.org/packages/cc/4f/e1fb138201ad9a32499dd9a98aa4a5a5441fbf7f56b52b619a54b7ee8777/charset_normalizer-3.4.7-cp39-cp39-win_arm64.whl", hash = "sha256:bb8cc7534f51d9a017b93e3e85b260924f909601c3df002bcdb58ddb4dc41a5c", size = 143716, upload-time = "2026-04-02T09:28:35.908Z" }, + { url = "https://files.pythonhosted.org/packages/db/8f/61959034484a4a7c527811f4721e75d02d653a35afb0b6054474d8185d4c/charset_normalizer-3.4.7-py3-none-any.whl", hash = "sha256:3dce51d0f5e7951f8bb4900c257dad282f49190fdbebecd4ba99bcc41fef404d", size = 61958, upload-time = "2026-04-02T09:28:37.794Z" }, +] + +[[package]] +name = "idna" +version = "3.15" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/82/77/7b3966d0b9d1d31a36ddf1746926a11dface89a83409bf1483f0237aa758/idna-3.15.tar.gz", hash = "sha256:ca962446ea538f7092a95e057da437618e886f4d349216d2b1e294abfdb65fdc", size = 199245, upload-time = "2026-05-12T22:45:57.011Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d2/23/408243171aa9aaba178d3e2559159c24c1171a641aa83b67bdd3394ead8e/idna-3.15-py3-none-any.whl", hash = "sha256:048adeaf8c2d788c40fee287673ccaa74c24ffd8dcf09ffa555a2fbb59f10ac8", size = 72340, upload-time = "2026-05-12T22:45:55.733Z" }, +] + +[[package]] +name = "requests" +version = "2.32.5" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +dependencies = [ + { name = "certifi", marker = "python_full_version < '3.10'" }, + { name = "charset-normalizer", marker = "python_full_version < '3.10'" }, + { name = "idna", marker = "python_full_version < '3.10'" }, + { name = "urllib3", version = "2.6.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/c9/74/b3ff8e6c8446842c3f5c837e9c3dfcfe2018ea6ecef224c710c85ef728f4/requests-2.32.5.tar.gz", hash = "sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf", size = 134517, upload-time = "2025-08-18T20:46:02.573Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/1e/db/4254e3eabe8020b458f1a747140d32277ec7a271daf1d235b70dc0b4e6e3/requests-2.32.5-py3-none-any.whl", hash = "sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6", size = 64738, upload-time = "2025-08-18T20:46:00.542Z" }, +] + +[[package]] +name = "requests" +version = "2.34.2" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.10'", +] +dependencies = [ + { name = "certifi", marker = "python_full_version >= '3.10'" }, + { name = "charset-normalizer", marker = "python_full_version >= '3.10'" }, + { name = "idna", marker = "python_full_version >= '3.10'" }, + { name = "urllib3", version = "2.7.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/ac/c3/e2a2b89f2d3e2179abd6d00ebd70bff6273f37fb3e0cc209f48b39d00cbf/requests-2.34.2.tar.gz", hash = "sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed", size = 142856, upload-time = "2026-05-14T19:25:27.735Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a0/f4/c67b0b3f1b9245e8d266f0f112c500d50e5b4e83cb6f3b71b6528104182a/requests-2.34.2-py3-none-any.whl", hash = "sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0", size = 73075, upload-time = "2026-05-14T19:25:26.443Z" }, +] + +[[package]] +name = "test-project" +version = "0.0.1" +source = { virtual = "." } +dependencies = [ + { name = "requests", version = "2.32.5", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "requests", version = "2.34.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, +] + +[package.metadata] +requires-dist = [{ name = "requests" }] + +[[package]] +name = "urllib3" +version = "2.6.3" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +sdist = { url = "https://files.pythonhosted.org/packages/c7/24/5f1b3bdffd70275f6661c76461e25f024d5a38a46f04aaca912426a2b1d3/urllib3-2.6.3.tar.gz", hash = "sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed", size = 435556, upload-time = "2026-01-07T16:24:43.925Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/39/08/aaaad47bc4e9dc8c725e68f9d04865dbcb2052843ff09c97b08904852d84/urllib3-2.6.3-py3-none-any.whl", hash = "sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4", size = 131584, upload-time = "2026-01-07T16:24:42.685Z" }, +] + +[[package]] +name = "urllib3" +version = "2.7.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.10'", +] +sdist = { url = "https://files.pythonhosted.org/packages/53/0c/06f8b233b8fd13b9e5ee11424ef85419ba0d8ba0b3138bf360be2ff56953/urllib3-2.7.0.tar.gz", hash = "sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c", size = 433602, upload-time = "2026-05-07T16:13:18.596Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7f/3e/5db95bcf282c52709639744ca2a8b149baccf648e39c8cc87553df9eae0c/urllib3-2.7.0-py3-none-any.whl", hash = "sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897", size = 131087, upload-time = "2026-05-07T16:13:17.151Z" }, +] diff --git a/tests/uv/lock/uv_lock_to_requirements_tests.bzl b/tests/uv/lock/uv_lock_to_requirements_tests.bzl new file mode 100644 index 0000000000..1b947877c9 --- /dev/null +++ b/tests/uv/lock/uv_lock_to_requirements_tests.bzl @@ -0,0 +1,289 @@ +"""Tests for uv_lock_to_requirements.""" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/uv/private:uv_lock_to_requirements.bzl", "uv_lock_to_requirements") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_empty(env): + got = uv_lock_to_requirements(json.decode("""{"package":[]}""")) + env.expect.that_str(got).equals("") + +_tests.append(_test_empty) + +def _test_simple_package(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/foo.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +foo==0.0.1 \\ + --hash=sha256:deadbeef +""") + +_tests.append(_test_simple_package) + +def _test_package_with_markers(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"resolution-markers":["python_full_version < '3.10'"],"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/foo.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +foo==0.0.1 ; python_full_version < '3.10' \\ + --hash=sha256:deadbeef +""") + +_tests.append(_test_package_with_markers) + +def _test_package_with_multiple_markers(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"resolution-markers":["python_full_version == '3.10.*'","python_full_version >= '3.14'"],"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/foo.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +foo==0.0.1 ; python_full_version == '3.10.*' or python_full_version >= '3.14' \\ + --hash=sha256:deadbeef +""") + +_tests.append(_test_package_with_multiple_markers) + +def _test_package_with_deps(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"bar","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:baadbeef","url":"https://example.org/bar.whl"}]}, + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"dependencies":[{"name":"bar"}],"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/foo.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +bar==0.0.1 \\ + --hash=sha256:baadbeef + # via foo + +foo==0.0.1 \\ + --hash=sha256:deadbeef +""") + +_tests.append(_test_package_with_deps) + +def _test_package_with_optional_deps(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"bar","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:baadbeef","url":"https://example.org/bar.whl"}]}, + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"optional-dependencies":{"extra1":[{"name":"bar"}]},"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/foo.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +bar==0.0.1 \\ + --hash=sha256:baadbeef + # via foo + +foo[extra1]==0.0.1 \\ + --hash=sha256:deadbeef +""") + +_tests.append(_test_package_with_optional_deps) + +def _test_self_edge_excluded(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"pydantic","version":"2.0.0","source":{"registry":"https://pypi.org/simple"},"dependencies":[{"name":"pydantic","extra":["email"]}],"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/pydantic.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +pydantic==2.0.0 \\ + --hash=sha256:deadbeef +""") + +_tests.append(_test_self_edge_excluded) + +def _test_multiple_dependents(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"common","version":"1.0.0","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:aaaa","url":"https://example.org/common.whl"}]}, + {"name":"pkg_a","version":"0.1.0","source":{"registry":"https://pypi.org/simple"},"dependencies":[{"name":"common"}],"wheels":[{"hash":"sha256:bbbb","url":"https://example.org/a.whl"}]}, + {"name":"pkg_b","version":"0.2.0","source":{"registry":"https://pypi.org/simple"},"dependencies":[{"name":"common"}],"wheels":[{"hash":"sha256:cccc","url":"https://example.org/b.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +common==1.0.0 \\ + --hash=sha256:aaaa + # via + # pkg_a + # pkg_b + +pkg_a==0.1.0 \\ + --hash=sha256:bbbb + +pkg_b==0.2.0 \\ + --hash=sha256:cccc +""") + +_tests.append(_test_multiple_dependents) + +def _test_git_source_skipped(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"foo","version":"0.1.0","source":{"git":"https://github.com/org/foo.git"}}, + {"name":"bar","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/bar.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +bar==0.0.1 \\ + --hash=sha256:deadbeef +""") + +_tests.append(_test_git_source_skipped) + +def _test_virtual_source_skipped(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"virtual-pkg","version":"0.0.0","source":{"virtual":true}}, + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/foo.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +foo==0.0.1 \\ + --hash=sha256:deadbeef +""") + +_tests.append(_test_virtual_source_skipped) + +def _test_sdist_hash(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"bar","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"sdist":{"hash":"sha256:deadb00f","url":"https://example.org/bar.tar.gz"}} + ]}""")) + env.expect.that_str(got).equals("""\ +bar==0.0.1 \\ + --hash=sha256:deadb00f +""") + +_tests.append(_test_sdist_hash) + +def _test_wheel_and_sdist_hashes(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"sdist":{"hash":"sha256:feedcafe","url":"https://example.org/foo.tar.gz"},"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/foo.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +foo==0.0.1 \\ + --hash=sha256:deadbeef \\ + --hash=sha256:feedcafe +""") + +_tests.append(_test_wheel_and_sdist_hashes) + +def _test_multiple_versions_same_package(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"resolution-markers":["python_full_version < '3.10'"],"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/foo-0.0.1.whl"}]}, + {"name":"foo","version":"0.0.2","source":{"registry":"https://pypi.org/simple"},"resolution-markers":["python_full_version >= '3.10'"],"wheels":[{"hash":"sha256:deadb11f","url":"https://example.org/foo-0.0.2.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +foo==0.0.1 ; python_full_version < '3.10' \\ + --hash=sha256:deadbeef + +foo==0.0.2 ; python_full_version >= '3.10' \\ + --hash=sha256:deadb11f +""") + +_tests.append(_test_multiple_versions_same_package) + +def _test_dep_with_extras(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/foo.whl"}]}, + {"name":"bar","version":"0.0.2","source":{"registry":"https://pypi.org/simple"},"dependencies":[{"name":"foo","extra":["extra1"]}],"wheels":[{"hash":"sha256:baadbeef","url":"https://example.org/bar.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +foo[extra1]==0.0.1 \\ + --hash=sha256:deadbeef + # via bar + +bar==0.0.2 \\ + --hash=sha256:baadbeef +""") + +_tests.append(_test_dep_with_extras) + +def _test_multiple_hashes_from_wheels(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[ + {"hash":"sha256:aaaa","url":"https://example.org/foo-0.0.1-cp39.whl"}, + {"hash":"sha256:bbbb","url":"https://example.org/foo-0.0.1-py3-none-any.whl"} + ]} + ]}""")) + env.expect.that_str(got).equals("""\ +foo==0.0.1 \\ + --hash=sha256:aaaa \\ + --hash=sha256:bbbb +""") + +_tests.append(_test_multiple_hashes_from_wheels) + +def _test_package_no_hashes_no_deps(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"}} + ]}""")) + env.expect.that_str(got).equals("""\ +foo==0.0.1 +""") + +_tests.append(_test_package_no_hashes_no_deps) + +def _test_package_with_multiple_optional_deps(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"bar","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:baadbeef","url":"https://example.org/bar.whl"}]}, + {"name":"baz","version":"0.0.2","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:deadc0de","url":"https://example.org/baz.whl"}]}, + {"name":"foo","version":"0.0.3","source":{"registry":"https://pypi.org/simple"},"optional-dependencies":{"extra1":[{"name":"bar"}],"extra2":[{"name":"baz"}]},"wheels":[{"hash":"sha256:feedcafe","url":"https://example.org/foo.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +bar==0.0.1 \\ + --hash=sha256:baadbeef + # via foo + +baz==0.0.2 \\ + --hash=sha256:deadc0de + # via foo + +foo[extra1,extra2]==0.0.3 \\ + --hash=sha256:feedcafe +""") + +_tests.append(_test_package_with_multiple_optional_deps) + +def _test_dep_with_multiple_extras(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/foo.whl"}]}, + {"name":"bar","version":"0.0.2","source":{"registry":"https://pypi.org/simple"},"dependencies":[{"name":"foo","extra":["extra1","extra2"]}],"wheels":[{"hash":"sha256:baadbeef","url":"https://example.org/bar.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +foo[extra1,extra2]==0.0.1 \\ + --hash=sha256:deadbeef + # via bar + +bar==0.0.2 \\ + --hash=sha256:baadbeef +""") + +_tests.append(_test_dep_with_multiple_extras) + +def _test_extras_from_multiple_dependents(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"common","version":"1.0.0","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:aaaa","url":"https://example.org/common.whl"}]}, + {"name":"pkg_a","version":"0.1.0","source":{"registry":"https://pypi.org/simple"},"dependencies":[{"name":"common","extra":["extra1"]}],"wheels":[{"hash":"sha256:bbbb","url":"https://example.org/a.whl"}]}, + {"name":"pkg_b","version":"0.2.0","source":{"registry":"https://pypi.org/simple"},"dependencies":[{"name":"common","extra":["extra2"]}],"wheels":[{"hash":"sha256:cccc","url":"https://example.org/b.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +common[extra1,extra2]==1.0.0 \\ + --hash=sha256:aaaa + # via + # pkg_a + # pkg_b + +pkg_a==0.1.0 \\ + --hash=sha256:bbbb + +pkg_b==0.2.0 \\ + --hash=sha256:cccc +""") + +_tests.append(_test_extras_from_multiple_dependents) + +def _test_requires_dist_extras(env): + """Test that extras from metadata.requires-dist are included.""" + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"pytest-bazel","version":"0.1.6","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:a29e80e1d67c3db801bdd4d0b6b742f2bfb48cd6841caa33401458e5c4e29c21","url":"https://example.org/pytest_bazel-0.1.6.whl"}]}, + {"name":"root-pkg","version":"0.0.0","source":{"virtual":"."},"dependencies":[{"name":"pytest-bazel"}],"metadata":{"requires-dist":[{"name":"pytest-bazel","extras":["all"]}]}} + ]}""")) + env.expect.that_str(got).equals("""\ +pytest-bazel[all]==0.1.6 \\ + --hash=sha256:a29e80e1d67c3db801bdd4d0b6b742f2bfb48cd6841caa33401458e5c4e29c21 + # via root-pkg +""") + +_tests.append(_test_requires_dist_extras) + +def uv_lock_to_requirements_test_suite(name): + test_suite(name = name, basic_tests = _tests) diff --git a/tests/uv/lock/workspaces/BUILD.bazel b/tests/uv/lock/workspaces/BUILD.bazel new file mode 100644 index 0000000000..aa1fabe2e3 --- /dev/null +++ b/tests/uv/lock/workspaces/BUILD.bazel @@ -0,0 +1,27 @@ +load("@bazel_skylib//rules:diff_test.bzl", "diff_test") +load("//python/uv:lock.bzl", "lock") +load("//tests/support:support.bzl", "NOT_WINDOWS") + +# This test verifies that the `lock` rule automatically passes `--project` to `uv pip compile` based +# on the package directory, so that `[tool.uv]` settings from a `pyproject.toml` in the same +# directory are applied. It will exclude a particular dependency from the lock-file, so it will be +# easy to see if we have any issues. +lock( + name = "requirements", + srcs = [ + "pyproject.toml", + "//tests/uv/lock/workspaces/packages", + ], + out = "requirements.txt", + # It seems that the CI remote executors for the RBE do not have network + # connectivity due to current CI setup. + tags = ["no-remote-exec"], +) + +diff_test( + name = "requirements_test", + timeout = "short", + file1 = ":requirements", + file2 = "requirements.txt", + target_compatible_with = NOT_WINDOWS, +) diff --git a/tests/uv/lock/workspaces/packages/BUILD.bazel b/tests/uv/lock/workspaces/packages/BUILD.bazel new file mode 100644 index 0000000000..5461287d9f --- /dev/null +++ b/tests/uv/lock/workspaces/packages/BUILD.bazel @@ -0,0 +1,5 @@ +filegroup( + name = "packages", + srcs = ["//tests/uv/lock/workspaces/packages/foo:pyproject.toml"], + visibility = ["//tests/uv/lock/workspaces:__pkg__"], +) diff --git a/tests/uv/lock/workspaces/packages/foo/BUILD.bazel b/tests/uv/lock/workspaces/packages/foo/BUILD.bazel new file mode 100644 index 0000000000..d89a7b784e --- /dev/null +++ b/tests/uv/lock/workspaces/packages/foo/BUILD.bazel @@ -0,0 +1 @@ +exports_files(["pyproject.toml"]) diff --git a/tests/uv/lock/workspaces/packages/foo/pyproject.toml b/tests/uv/lock/workspaces/packages/foo/pyproject.toml new file mode 100644 index 0000000000..f26ea85b51 --- /dev/null +++ b/tests/uv/lock/workspaces/packages/foo/pyproject.toml @@ -0,0 +1,7 @@ +[project] +name = "foo" +version = "0.0.0" +dependencies = ["black"] + +[tool.uv] +no-build-isolation = true diff --git a/tests/uv/lock/workspaces/pyproject.toml b/tests/uv/lock/workspaces/pyproject.toml new file mode 100644 index 0000000000..01e1e2e076 --- /dev/null +++ b/tests/uv/lock/workspaces/pyproject.toml @@ -0,0 +1,10 @@ +[project] +name = "test" +version = "0.0.0" +dependencies = ["requests"] + +[tool.uv] +no-build-isolation = true + +[tool.uv.workspace] +members = ["packages/*"] diff --git a/tests/uv/lock/workspaces/requirements.txt b/tests/uv/lock/workspaces/requirements.txt new file mode 100644 index 0000000000..5ce35d7571 --- /dev/null +++ b/tests/uv/lock/workspaces/requirements.txt @@ -0,0 +1,242 @@ +# This file was autogenerated by uv via the following command: +# bazel run //tests/uv/lock/workspaces:requirements.update +black==26.5.1 \ + --hash=sha256:0e48b87e03bf109288e55cfceadcfa15ff5470aca2851a851950ed2926f450d7 \ + --hash=sha256:1037d5ac7b7b310b2632ad867ec8d0e4c4819dcdb0b820f63135da746a24e418 \ + --hash=sha256:1ef92b76f7733f282fd096ea406200b5a286c42947412b0eaff3a74e3616cefe \ + --hash=sha256:1f7ea64ebfa01b50f693508fc39f875e264446d3b097088f84f203b9d09618a0 \ + --hash=sha256:22f2cd76d069cc54c71f10360744ba8983fbb616903b4304a85b734915c8e1b4 \ + --hash=sha256:2b36cf2ddf5566e205f6535f782a62194a184d33e175b64ae8c40b1737522be3 \ + --hash=sha256:30d3c14661f2792e9142cce3eeeb1cbc175b3eb5f733be0c8eeb99651e52b0c3 \ + --hash=sha256:32d5ea7f6c8bdfa6e648326ebca1f02b0764e2a029edc6f8dce2627e19d468c3 \ + --hash=sha256:3915f256e75a2d7cf88d8953d37f780455dc586cc72dee059c528fe77f581217 \ + --hash=sha256:4ad6fa01f941920f54f2bbb35f3df7673428a0ef98a0b0840c2eaef3b110efa8 \ + --hash=sha256:4ed7f7da04046d2e488437170797d3b4a4ad83906683bcb7dfc68b673bbce5e2 \ + --hash=sha256:5119fa92ae61f786e8c3662fd60aece1d0a2dd5cca5d0c79417a95e7a4272a59 \ + --hash=sha256:577f21094ea469ef92ec1adaf2c9441a226d2144d01a5be2fa823cecf6543e50 \ + --hash=sha256:58b4bd92cf88aacf83d88479c8f9caee044b1ec55f2451a337354a7ea2590a22 \ + --hash=sha256:5c34b25da232ead53a6f335b76dbea124f4d152ad568b9080d6f944bc2b34b52 \ + --hash=sha256:87ed5c6f450580a2f6790bc7cbfb016dfc73bc750249762268a3695361315eef \ + --hash=sha256:89c93167a74d3a75dfaa38a5c7cca015537d5820dd7f17d63267d674a61cae90 \ + --hash=sha256:96ae2c733b2aabdd9986e2c5df628ff3473676cd1c5faded1ff496cf6d74083c \ + --hash=sha256:9942db8888e06943c5dde66ca0037dcff82a2a4ec1ad0ada9e0d2ee9d9823893 \ + --hash=sha256:9d98d4137277c75dfb898ec8d846c4fd68ba1e9cf77f95e2865c203dc18f4c3d \ + --hash=sha256:a1dca32d9f1784af512a13410ec204c6f7f0aa9797a111c42e1c03449821c264 \ + --hash=sha256:dd321f668053961824bcc1be1cc1df748b2d7e4fa28086b08331e577b0100a73 \ + --hash=sha256:e1a26503279b6b310669fb0b219c39e4820b77e8189fe80f522bb511f247db0a \ + --hash=sha256:e88976690a64b0af98312ca958415849cb42423423c5f2ee74af4b49a97a2168 \ + --hash=sha256:ea8d16dc41655aa113cd64665e7219446cd7e4ff2248d7178eaa905190c86b18 \ + --hash=sha256:ecb3e624844c798144e9bd986954e0adc81d8911a1f30f375e1252fe26e8c294 \ + --hash=sha256:ed1a20af114c301a0269bf01163d51dbef72737fd65f850001e7cbe7f3c7abae + # via foo (tests/uv/lock/workspaces/packages/foo/pyproject.toml) +certifi==2025.1.31 \ + --hash=sha256:3d5da6925056f6f18f119200434a4780a94263f10d1c21d032a6f6b2baa20651 \ + --hash=sha256:ca78db4565a652026a4db2bcdf68f2fb589ea80d0be70e03929ed730746b84fe + # via requests +charset-normalizer==3.4.7 \ + --hash=sha256:007d05ec7321d12a40227aae9e2bc6dca73f3cb21058999a1df9e193555a9dcc \ + --hash=sha256:03853ed82eeebbce3c2abfdbc98c96dc205f32a79627688ac9a27370ea61a49c \ + --hash=sha256:07d9e39b01743c3717745f4c530a6349eadbfa043c7577eef86c502c15df2c67 \ + --hash=sha256:08e721811161356f97b4059a9ba7bafb23ea5ee2255402c42881c214e173c6b4 \ + --hash=sha256:0c96c3b819b5c3e9e165495db84d41914d6894d55181d2d108cc1a69bfc9cce0 \ + --hash=sha256:0ea948db76d31190bf08bd371623927ee1339d5f2a0b4b1b4a4439a65298703c \ + --hash=sha256:0f7eb884681e3938906ed0434f20c63046eacd0111c4ba96f27b76084cd679f5 \ + --hash=sha256:12a6fff75f6bc66711b73a2f0addfc4c8c15a20e805146a02d147a318962c444 \ + --hash=sha256:12d8baf840cc7889b37c7c770f478adea7adce3dcb3944d02ec87508e2dcf153 \ + --hash=sha256:14265bfe1f09498b9d8ec91e9ec9fa52775edf90fcbde092b25f4a33d444fea9 \ + --hash=sha256:16d971e29578a5e97d7117866d15889a4a07befe0e87e703ed63cd90cb348c01 \ + --hash=sha256:177a0ba5f0211d488e295aaf82707237e331c24788d8d76c96c5a41594723217 \ + --hash=sha256:1a87ca9d5df6fe460483d9a5bbf2b18f620cbed41b432e2bddb686228282d10b \ + --hash=sha256:1c2a768fdd44ee4a9339a9b0b130049139b8ce3c01d2ce09f67f5a68048d477c \ + --hash=sha256:1c2aed2e5e41f24ea8ef1590b8e848a79b56f3a5564a65ceec43c9d692dc7d8a \ + --hash=sha256:1dc8b0ea451d6e69735094606991f32867807881400f808a106ee1d963c46a83 \ + --hash=sha256:1efde3cae86c8c273f1eb3b287be7d8499420cf2fe7585c41d370d3e790054a5 \ + --hash=sha256:202389074300232baeb53ae2569a60901f7efadd4245cf3a3bf0617d60b439d7 \ + --hash=sha256:203104ed3e428044fd943bc4bf45fa73c0730391f9621e37fe39ecf477b128cb \ + --hash=sha256:2257141f39fe65a3fdf38aeccae4b953e5f3b3324f4ff0daf9f15b8518666a2c \ + --hash=sha256:298930cec56029e05497a76988377cbd7457ba864beeea92ad7e844fe74cd1f1 \ + --hash=sha256:2cd4a60d0e2fb04537162c62bbbb4182f53541fe0ede35cdf270a1c1e723cc42 \ + --hash=sha256:2d6eb928e13016cea4f1f21d1e10c1cebd5a421bc57ddf5b1142ae3f86824fab \ + --hash=sha256:2fe249cb4651fd12605b7288b24751d8bfd46d35f12a20b1ba33dea122e690df \ + --hash=sha256:30b8d1d8c52a48c2c5690e152c169b673487a2a58de1ec7393196753063fcd5e \ + --hash=sha256:320ade88cfb846b8cd6b4ddf5ee9e80ee0c1f52401f2456b84ae1ae6a1a5f207 \ + --hash=sha256:3534e7dcbdcf757da6b85a0bbf5b6868786d5982dd959b065e65481644817a18 \ + --hash=sha256:36836d6ff945a00b88ba1e4572d721e60b5b8c98c155d465f56ad19d68f23734 \ + --hash=sha256:38c0109396c4cfc574d502df99742a45c72c08eff0a36158b6f04000043dbf38 \ + --hash=sha256:3946fa46a0cf3e4c8cb1cc52f56bb536310d34f25f01ca9b6c16afa767dab110 \ + --hash=sha256:3bec022aec2c514d9cf199522a802bd007cd588ab17ab2525f20f9c34d067c18 \ + --hash=sha256:3c9a494bc5ec77d43cea229c4f6db1e4d8fe7e1bbffa8b6f0f0032430ff8ab44 \ + --hash=sha256:3dce51d0f5e7951f8bb4900c257dad282f49190fdbebecd4ba99bcc41fef404d \ + --hash=sha256:3dedcc22d73ec993f42055eff4fcfed9318d1eeb9a6606c55892a26964964e48 \ + --hash=sha256:4042d5c8f957e15221d423ba781e85d553722fc4113f523f2feb7b188cc34c5e \ + --hash=sha256:481551899c856c704d58119b5025793fa6730adda3571971af568f66d2424bb5 \ + --hash=sha256:4dc1e73c36828f982bfe79fadf5919923f8a6f4df2860804db9a98c48824ce8d \ + --hash=sha256:4e5163c14bffd570ef2affbfdd77bba66383890797df43dc8b4cc7d6f500bf53 \ + --hash=sha256:511ef87c8aec0783e08ac18565a16d435372bc1ac25a91e6ac7f5ef2b0bff790 \ + --hash=sha256:532bc9bf33a68613fd7d65e4b1c71a6a38d7d42604ecf239c77392e9b4e8998c \ + --hash=sha256:54523e136b8948060c0fa0bc7b1b50c32c186f2fceee897a495406bb6e311d2b \ + --hash=sha256:5649fd1c7bade02f320a462fdefd0b4bd3ce036065836d4f42e0de958038e116 \ + --hash=sha256:56be790f86bfb2c98fb742ce566dfb4816e5a83384616ab59c49e0604d49c51d \ + --hash=sha256:5b77459df20e08151cd6f8b9ef8ef1f961ef73d85c21a555c7eed5b79410ec10 \ + --hash=sha256:5ed6ab538499c8644b8a3e18debabcd7ce684f3fa91cf867521a7a0279cab2d6 \ + --hash=sha256:6178f72c5508bfc5fd446a5905e698c6212932f25bcdd4b47a757a50605a90e2 \ + --hash=sha256:6370e8686f662e6a3941ee48ed4742317cafbe5707e36406e9df792cdb535776 \ + --hash=sha256:64f02c6841d7d83f832cd97ccf8eb8a906d06eb95d5276069175c696b024b60a \ + --hash=sha256:65bcd23054beab4d166035cabbc868a09c1a49d1efe458fe8e4361215df40265 \ + --hash=sha256:66671f93accb62ed07da56613636f3641f1a12c13046ce91ffc923721f23c008 \ + --hash=sha256:6696b7688f54f5af4462118f0bfa7c1621eeb87154f77fa04b9295ce7a8f2943 \ + --hash=sha256:6785f414ae0f3c733c437e0f3929197934f526d19dfaa75e18fdb4f94c6fb374 \ + --hash=sha256:67f6279d125ca0046a7fd386d01b311c6363844deac3e5b069b514ba3e63c246 \ + --hash=sha256:6c114670c45346afedc0d947faf3c7f701051d2518b943679c8ff88befe14f8e \ + --hash=sha256:6e0d51f618228538a3e8f46bd246f87a6cd030565e015803691603f55e12afb5 \ + --hash=sha256:6ed74185b2db44f41ef35fd1617c5888e59792da9bbc9190d6c7300617182616 \ + --hash=sha256:708838739abf24b2ceb208d0e22403dd018faeef86ddac04319a62ae884c4f15 \ + --hash=sha256:715479b9a2802ecac752a3b0efa2b0b60285cf962ee38414211abdfccc233b41 \ + --hash=sha256:733784b6d6def852c814bce5f318d25da2ee65dd4839a0718641c696e09a2960 \ + --hash=sha256:750e02e074872a3fad7f233b47734166440af3cdea0add3e95163110816d6752 \ + --hash=sha256:752a45dc4a6934060b3b0dab47e04edc3326575f82be64bc4fc293914566503e \ + --hash=sha256:7579e913a5339fb8fa133f6bbcfd8e6749696206cf05acdbdca71a1b436d8e72 \ + --hash=sha256:7641bb8895e77f921102f72833904dcd9901df5d6d72a2ab8f31d04b7e51e4e7 \ + --hash=sha256:7804338df6fcc08105c7745f1502ba68d900f45fd770d5bdd5288ddccb8a42d8 \ + --hash=sha256:80d04837f55fc81da168b98de4f4b797ef007fc8a79ab71c6ec9bc4dd662b15b \ + --hash=sha256:813c0e0132266c08eb87469a642cb30aaff57c5f426255419572aaeceeaa7bf4 \ + --hash=sha256:82b271f5137d07749f7bf32f70b17ab6eaabedd297e75dce75081a24f76eb545 \ + --hash=sha256:84c018e49c3bf790f9c2771c45e9313a08c2c2a6342b162cd650258b57817706 \ + --hash=sha256:8751d2787c9131302398b11e6c8068053dcb55d5a8964e114b6e196cf16cb366 \ + --hash=sha256:8778f0c7a52e56f75d12dae53ae320fae900a8b9b4164b981b9c5ce059cd1fcb \ + --hash=sha256:87fad7d9ba98c86bcb41b2dc8dbb326619be2562af1f8ff50776a39e55721c5a \ + --hash=sha256:8d828b6667a32a728a1ad1d93957cdf37489c57b97ae6c4de2860fa749b8fc1e \ + --hash=sha256:8e385e4267ab76874ae30db04c627faaaf0b509e1ccc11a95b3fc3e83f855c00 \ + --hash=sha256:92a0a01ead5e668468e952e4238cccd7c537364eb7d851ab144ab6627dbbe12f \ + --hash=sha256:94e1885b270625a9a828c9793b4d52a64445299baa1fea5a173bf1d3dd9a1a5a \ + --hash=sha256:a180c5e59792af262bf263b21a3c49353f25945d8d9f70628e73de370d55e1e1 \ + --hash=sha256:a277ab8928b9f299723bc1a2dabb1265911b1a76341f90a510368ca44ad9ab66 \ + --hash=sha256:a5fe03b42827c13cdccd08e6c0247b6a6d4b5e3cdc53fd1749f5896adcdc2356 \ + --hash=sha256:a6c5863edfbe888d9eff9c8b8087354e27618d9da76425c119293f11712a6319 \ + --hash=sha256:a89c23ef8d2c6b27fd200a42aa4ac72786e7c60d40efdc76e6011260b6e949c4 \ + --hash=sha256:adb2597b428735679446b46c8badf467b4ca5f5056aae4d51a19f9570301b1ad \ + --hash=sha256:ae196f021b5e7c78e918242d217db021ed2a6ace2bc6ae94c0fc596221c7f58d \ + --hash=sha256:ae89db9e5f98a11a4bf50407d4363e7b09b31e55bc117b4f7d80aab97ba009e5 \ + --hash=sha256:aed52fea0513bac0ccde438c188c8a471c4e0f457c2dd20cdbf6ea7a450046c7 \ + --hash=sha256:aef65cd602a6d0e0ff6f9930fcb1c8fec60dd2cfcb6facaf4bdb0e5873042db0 \ + --hash=sha256:af21eb4409a119e365397b2adbaca4c9ccab56543a65d5dbd9f920d6ac29f686 \ + --hash=sha256:b14b2d9dac08e28bb8046a1a0434b1750eb221c8f5b87a68f4fa11a6f97b5e34 \ + --hash=sha256:bb6d88045545b26da47aa879dd4a89a71d1dce0f0e549b1abcb31dfe4a8eac49 \ + --hash=sha256:bb8cc7534f51d9a017b93e3e85b260924f909601c3df002bcdb58ddb4dc41a5c \ + --hash=sha256:bc17a677b21b3502a21f66a8cc64f5bfad4df8a0b8434d661666f8ce90ac3af1 \ + --hash=sha256:bd6c2a1c7573c64738d716488d2cdd3c00e340e4835707d8fdb8dc1a66ef164e \ + --hash=sha256:bd9b23791fe793e4968dba0c447e12f78e425c59fc0e3b97f6450f4781f3ee60 \ + --hash=sha256:c03a41a8784091e67a39648f70c5f97b5b6a37f216896d44d2cdcb82615339a0 \ + --hash=sha256:c0f081d69a6e58272819b70288d3221a6ee64b98df852631c80f293514d3b274 \ + --hash=sha256:c35abb8bfff0185efac5878da64c45dafd2b37fb0383add1be155a763c1f083d \ + --hash=sha256:c36c333c39be2dbca264d7803333c896ab8fa7d4d6f0ab7edb7dfd7aea6e98c0 \ + --hash=sha256:c45e9440fb78f8ddabcf714b68f936737a121355bf59f3907f4e17721b9d1aae \ + --hash=sha256:c593052c465475e64bbfe5dbd81680f64a67fdc752c56d7a0ae205dc8aeefe0f \ + --hash=sha256:cdd68a1fb318e290a2077696b7eb7a21a49163c455979c639bf5a5dcdc46617d \ + --hash=sha256:ce3412fbe1e31eb81ea42f4169ed94861c56e643189e1e75f0041f3fe7020abe \ + --hash=sha256:cf1493cd8607bec4d8a7b9b004e699fcf8f9103a9284cc94962cb73d20f9d4a3 \ + --hash=sha256:cf29836da5119f3c8a8a70667b0ef5fdca3bb12f80fd06487cfa575b3909b393 \ + --hash=sha256:d4a48e5b3c2a489fae013b7589308a40146ee081f6f509e047e0e096084ceca1 \ + --hash=sha256:d560742f3c0d62afaccf9f41fe485ed69bd7661a241f86a3ef0f0fb8b1a397af \ + --hash=sha256:d6038d37043bced98a66e68d3aa2b6a35505dc01328cd65217cefe82f25def44 \ + --hash=sha256:d61f00a0869d77422d9b2aba989e2d24afa6ffd552af442e0e58de4f35ea6d00 \ + --hash=sha256:d635aab80466bc95771bb78d5370e74d36d1fe31467b6b29b8b57b2a3cd7d22c \ + --hash=sha256:dca4bbc466a95ba9c0234ef56d7dd9509f63da22274589ebd4ed7f1f4d4c54e3 \ + --hash=sha256:dd915403e231e6b1809fe9b6d9fc55cf8fb5e02765ac625d9cd623342a7905d7 \ + --hash=sha256:e044c39e41b92c845bc815e5ae4230804e8e7bc29e399b0437d64222d92809dd \ + --hash=sha256:e060d01aec0a910bdccb8be71faf34e7799ce36950f8294c8bf612cba65a2c9e \ + --hash=sha256:e1421b502d83040e6d7fb2fb18dff63957f720da3d77b2fbd3187ceb63755d7b \ + --hash=sha256:e17b8d5d6a8c47c85e68ca8379def1303fd360c3e22093a807cd34a71cd082b8 \ + --hash=sha256:e5f4d355f0a2b1a31bc3edec6795b46324349c9cb25eed068049e4f472fb4259 \ + --hash=sha256:e712b419df8ba5e42b226c510472b37bd57b38e897d3eca5e8cfd410a29fa859 \ + --hash=sha256:e74327fb75de8986940def6e8dee4f127cc9752bee7355bb323cc5b2659b6d46 \ + --hash=sha256:e80c8378d8f3d83cd3164da1ad2df9e37a666cdde7b1cb2298ed0b558064be30 \ + --hash=sha256:e8ac484bf18ce6975760921bb6148041faa8fef0547200386ea0b52b5d27bf7b \ + --hash=sha256:eca9705049ad3c7345d574e3510665cb2cf844c2f2dcfe675332677f081cbd46 \ + --hash=sha256:ed065083d0898c9d5b4bbec7b026fd755ff7454e6e8b73a67f8c744b13986e24 \ + --hash=sha256:edac0f1ab77644605be2cbba52e6b7f630731fc42b34cb0f634be1a6eface56a \ + --hash=sha256:effc3f449787117233702311a1b7d8f59cba9ced946ba727bdc329ec69028e24 \ + --hash=sha256:f22dec1690b584cea26fade98b2435c132c1b5f68e39f5a0b7627cd7ae31f1dc \ + --hash=sha256:f495a1652cf3fbab2eb0639776dad966c2fb874d79d87ca07f9d5f059b8bd215 \ + --hash=sha256:f496c9c3cc02230093d8330875c4c3cdfc3b73612a5fd921c65d39cbcef08063 \ + --hash=sha256:f59099f9b66f0d7145115e6f80dd8b1d847176df89b234a5a6b3f00437aa0832 \ + --hash=sha256:f59ad4c0e8f6bba240a9bb85504faa1ab438237199d4cce5f622761507b8f6a6 \ + --hash=sha256:fbccdc05410c9ee21bbf16a35f4c1d16123dcdeb8a1d38f33654fa21d0234f79 \ + --hash=sha256:fea24543955a6a729c45a73fe90e08c743f0b3334bbf3201e6c4bc1b0c7fa464 + # via requests +click==8.4.1 \ + --hash=sha256:482be17c6991b8c19c5429a1e995d9b0efdbb63172824c41f99965dc0ade8ec2 \ + --hash=sha256:918b5633eddf6b41c32d4f454bf0de810065c74e3f7dbf8ee5452f8be88d3e96 + # via black +idna==3.10 \ + --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ + --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3 + # via requests +mypy-extensions==1.1.0 \ + --hash=sha256:1be4cccdb0f2482337c4743e60421de3a356cd97508abadd57d47403e94f5505 \ + --hash=sha256:52e68efc3284861e772bbcd66823fde5ae21fd2fdb51c62a211403730b916558 + # via black +packaging==26.2 \ + --hash=sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e \ + --hash=sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661 + # via black +pathspec==1.1.1 \ + --hash=sha256:17db5ecd524104a120e173814c90367a96a98d07c45b2e10c2f3919fff91bf5a \ + --hash=sha256:a00ce642f577bf7f473932318056212bc4f8bfdf53128c78bbd5af0b9b20b189 + # via black +platformdirs==4.10.0 \ + --hash=sha256:31e761a6a0ca04faf7353ea759bdba55652be214725111e5aac52dfa29d4bef7 \ + --hash=sha256:fb516cdb12eb0d857d0cd85a7c57cea4d060bee4578d6cf5a14dfdf8cbf8784a + # via black +pytokens==0.4.1 \ + --hash=sha256:0fc71786e629cef478cbf29d7ea1923299181d0699dbe7c3c0f4a583811d9fc1 \ + --hash=sha256:11edda0942da80ff58c4408407616a310adecae1ddd22eef8c692fe266fa5009 \ + --hash=sha256:140709331e846b728475786df8aeb27d24f48cbcf7bcd449f8de75cae7a45083 \ + --hash=sha256:24afde1f53d95348b5a0eb19488661147285ca4dd7ed752bbc3e1c6242a304d1 \ + --hash=sha256:26cef14744a8385f35d0e095dc8b3a7583f6c953c2e3d269c7f82484bf5ad2de \ + --hash=sha256:27b83ad28825978742beef057bfe406ad6ed524b2d28c252c5de7b4a6dd48fa2 \ + --hash=sha256:292052fe80923aae2260c073f822ceba21f3872ced9a68bb7953b348e561179a \ + --hash=sha256:29d1d8fb1030af4d231789959f21821ab6325e463f0503a61d204343c9b355d1 \ + --hash=sha256:2a44ed93ea23415c54f3face3b65ef2b844d96aeb3455b8a69b3df6beab6acc5 \ + --hash=sha256:30f51edd9bb7f85c748979384165601d028b84f7bd13fe14d3e065304093916a \ + --hash=sha256:34bcc734bd2f2d5fe3b34e7b3c0116bfb2397f2d9666139988e7a3eb5f7400e3 \ + --hash=sha256:3ad72b851e781478366288743198101e5eb34a414f1d5627cdd585ca3b25f1db \ + --hash=sha256:3f901fe783e06e48e8cbdc82d631fca8f118333798193e026a50ce1b3757ea68 \ + --hash=sha256:42f144f3aafa5d92bad964d471a581651e28b24434d184871bd02e3a0d956037 \ + --hash=sha256:4a14d5f5fc78ce85e426aa159489e2d5961acf0e47575e08f35584009178e321 \ + --hash=sha256:4a58d057208cb9075c144950d789511220b07636dd2e4708d5645d24de666bdc \ + --hash=sha256:4e691d7f5186bd2842c14813f79f8884bb03f5995f0575272009982c5ac6c0f7 \ + --hash=sha256:5502408cab1cb18e128570f8d598981c68a50d0cbd7c61312a90507cd3a1276f \ + --hash=sha256:584c80c24b078eec1e227079d56dc22ff755e0ba8654d8383b2c549107528918 \ + --hash=sha256:5ad948d085ed6c16413eb5fec6b3e02fa00dc29a2534f088d3302c47eb59adf9 \ + --hash=sha256:670d286910b531c7b7e3c0b453fd8156f250adb140146d234a82219459b9640c \ + --hash=sha256:682fa37ff4d8e95f7df6fe6fe6a431e8ed8e788023c6bcc0f0880a12eab80ad1 \ + --hash=sha256:6d6c4268598f762bc8e91f5dbf2ab2f61f7b95bdc07953b602db879b3c8c18e1 \ + --hash=sha256:79fc6b8699564e1f9b521582c35435f1bd32dd06822322ec44afdeba666d8cb3 \ + --hash=sha256:8bdb9d0ce90cbf99c525e75a2fa415144fd570a1ba987380190e8b786bc6ef9b \ + --hash=sha256:8fcb9ba3709ff77e77f1c7022ff11d13553f3c30299a9fe246a166903e9091eb \ + --hash=sha256:941d4343bf27b605e9213b26bfa1c4bf197c9c599a9627eb7305b0defcfe40c1 \ + --hash=sha256:967cf6e3fd4adf7de8fc73cd3043754ae79c36475c1c11d514fc72cf5490094a \ + --hash=sha256:970b08dd6b86058b6dc07efe9e98414f5102974716232d10f32ff39701e841c4 \ + --hash=sha256:97f50fd18543be72da51dd505e2ed20d2228c74e0464e4262e4899797803d7fa \ + --hash=sha256:9bd7d7f544d362576be74f9d5901a22f317efc20046efe2034dced238cbbfe78 \ + --hash=sha256:add8bf86b71a5d9fb5b89f023a80b791e04fba57960aa790cc6125f7f1d39dfe \ + --hash=sha256:b35d7e5ad269804f6697727702da3c517bb8a5228afa450ab0fa787732055fc9 \ + --hash=sha256:b49750419d300e2b5a3813cf229d4e5a4c728dae470bcc89867a9ad6f25a722d \ + --hash=sha256:d31b97b3de0f61571a124a00ffe9a81fb9939146c122c11060725bd5aea79975 \ + --hash=sha256:d70e77c55ae8380c91c0c18dea05951482e263982911fc7410b1ffd1dadd3440 \ + --hash=sha256:d9907d61f15bf7261d7e775bd5d7ee4d2930e04424bab1972591918497623a16 \ + --hash=sha256:da5baeaf7116dced9c6bb76dc31ba04a2dc3695f3d9f74741d7910122b456edc \ + --hash=sha256:dc74c035f9bfca0255c1af77ddd2d6ae8419012805453e4b0e7513e17904545d \ + --hash=sha256:dcafc12c30dbaf1e2af0490978352e0c4041a7cde31f4f81435c2a5e8b9cabb6 \ + --hash=sha256:ee44d0f85b803321710f9239f335aafe16553b39106384cef8e6de40cb4ef2f6 \ + --hash=sha256:f66a6bbe741bd431f6d741e617e0f39ec7257ca1f89089593479347cc4d13324 + # via black +requests==2.32.3 \ + --hash=sha256:55365417734eb18255590a9ff9eb97e9e1da868d4ccd6402399eaf68af20a760 \ + --hash=sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6 + # via test (tests/uv/lock/workspaces/pyproject.toml) +urllib3==2.3.0 \ + --hash=sha256:1cee9ad369867bfdbbb48b7dd50374c0967a0bb7710050facf0dd6911440e3df \ + --hash=sha256:f8c5449b3cf0861679ce7e0503c7b44b5ec981bec0d1d3795a07f1ba96f0204d + # via requests diff --git a/tests/uv/toolchain/uv_help_test.py b/tests/uv/toolchain/uv_help_test.py index be5e755d91..c4515277f7 100755 --- a/tests/uv/toolchain/uv_help_test.py +++ b/tests/uv/toolchain/uv_help_test.py @@ -14,9 +14,9 @@ def test_uv_help(self): data_rpath = os.environ["DATA"] uv_help_path = rfiles.Rlocation(data_rpath) - assert ( - uv_help_path is not None - ), f"the rlocation path was not found: {data_rpath}" + assert uv_help_path is not None, ( + f"the rlocation path was not found: {data_rpath}" + ) uv_help = Path(uv_help_path).read_text() diff --git a/tests/semver/BUILD.bazel b/tests/uv/uv/BUILD.bazel similarity index 87% rename from tests/semver/BUILD.bazel rename to tests/uv/uv/BUILD.bazel index e12b1e5300..e1535ab5d8 100644 --- a/tests/semver/BUILD.bazel +++ b/tests/uv/uv/BUILD.bazel @@ -12,6 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. -load(":semver_test.bzl", "semver_test_suite") +load(":uv_tests.bzl", "uv_test_suite") -semver_test_suite(name = "semver_tests") +uv_test_suite(name = "uv_tests") diff --git a/tests/uv/uv/uv_tests.bzl b/tests/uv/uv/uv_tests.bzl new file mode 100644 index 0000000000..d11bf065d8 --- /dev/null +++ b/tests/uv/uv/uv_tests.bzl @@ -0,0 +1,600 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"" + +load("@rules_testing//lib:analysis_test.bzl", "analysis_test") +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("@rules_testing//lib:truth.bzl", "subjects") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility +load("//python/uv:uv_toolchain_info.bzl", "UvToolchainInfo") +load("//python/uv/private:uv.bzl", "process_modules") # buildifier: disable=bzl-visibility +load("//python/uv/private:uv_toolchain.bzl", "uv_toolchain") # buildifier: disable=bzl-visibility +load("//tests/support/mocks:mocks.bzl", "mocks") +load("//tests/support/platforms:platforms.bzl", "platform_targets") + +_tests = [] + +def _uv_mock_mctx(*modules, download = None): + # Here we construct a fake minimal manifest file that we use to mock what would + # be otherwise read from GH files + manifest_files = { + "different.json": { + x: { + "checksum": x + ".sha256", + "kind": "executable-zip", + "target_triples": [x], + } + for x in ["linux", "osx"] + } | { + x + ".sha256": { + "name": x + ".sha256", + "target_triples": [x], + } + for x in ["linux", "osx"] + }, + "manifest.json": { + x: { + "checksum": x + ".sha256", + "kind": "executable-zip", + "target_triples": [x], + } + for x in ["linux", "os", "osx", "something_extra"] + } | { + x + ".sha256": { + "name": x + ".sha256", + "target_triples": [x], + } + for x in ["linux", "os", "osx", "something_extra"] + }, + } + + fake_fs = { + "linux.sha256": "deadbeef linux", + "os.sha256": "deadbeef os", + "osx.sha256": "deadb00f osx", + } | { + fname: json.encode({"artifacts": contents}) + for fname, contents in manifest_files.items() + } + + return mocks.mctx( + modules = list(modules), + mock_downloads = { + "*": download, + } if download else {}, + mock_files = fake_fs, + ) + +def _mod(*, name = None, default = [], configure = [], is_root = True): + return mocks.module( + name, + is_root = is_root, + default = default, + configure = configure, + ) + +def _process_modules(env, **kwargs): + result = process_modules(hub_repo = struct, get_auth = lambda *_, **__: None, **kwargs) + + return env.expect.that_struct( + struct( + names = result.toolchain_names, + implementations = result.toolchain_implementations, + compatible_with = result.toolchain_compatible_with, + target_settings = result.toolchain_target_settings, + ), + attrs = dict( + names = subjects.collection, + implementations = subjects.dict, + compatible_with = subjects.dict, + target_settings = subjects.dict, + ), + ) + +def _default( + base_url = None, + compatible_with = None, + manifest_filename = None, + platform = None, + target_settings = None, + version = None, + netrc = None, + auth_patterns = None, + **kwargs): + return struct( + base_url = base_url, + compatible_with = [] + (compatible_with or []), # ensure that the type is correct + manifest_filename = manifest_filename, + platform = platform, + target_settings = [] + (target_settings or []), # ensure that the type is correct + version = version, + netrc = netrc, + auth_patterns = {} | (auth_patterns or {}), # ensure that the type is correct + **kwargs + ) + +def _configure(urls = None, sha256 = None, **kwargs): + # We have the same attributes + return _default(sha256 = sha256, urls = urls, **kwargs) + +def _test_only_defaults(env): + uv = _process_modules( + env, + module_ctx = _uv_mock_mctx( + _mod( + default = [ + _default( + base_url = "https://example.org", + manifest_filename = "manifest.json", + version = "1.0.0", + platform = "some_name", + compatible_with = ["@platforms//:incompatible"], + ), + ], + ), + ), + ) + + # No defined platform means nothing gets registered + uv.names().contains_exactly([ + "none", + ]) + uv.implementations().contains_exactly({ + "none": labels.NONE, + }) + uv.compatible_with().contains_exactly({ + "none": ["@platforms//:incompatible"], + }) + uv.target_settings().contains_exactly({}) + +_tests.append(_test_only_defaults) + +def _test_manual_url_spec(env): + calls = [] + uv = _process_modules( + env, + module_ctx = _uv_mock_mctx( + _mod( + default = [ + _default( + manifest_filename = "manifest.json", + version = "1.0.0", + ), + _default( + platform = "linux", + compatible_with = ["@platforms//os:linux"], + ), + # This will be ignored because urls are passed for some of + # the binaries. + _default( + platform = "osx", + compatible_with = ["@platforms//os:osx"], + ), + ], + configure = [ + _configure( + platform = "linux", + urls = ["https://example.org/1.0.0/manifest.json.zip"], + sha256 = "deadbeef", + ), + ], + ), + ), + uv_repository = lambda **kwargs: calls.append(kwargs), + ) + + uv.names().contains_exactly([ + "1_0_0_linux", + ]) + uv.implementations().contains_exactly({ + "1_0_0_linux": "@uv_1_0_0_linux//:uv_toolchain", + }) + uv.compatible_with().contains_exactly({ + "1_0_0_linux": ["@platforms//os:linux"], + }) + uv.target_settings().contains_exactly({}) + env.expect.that_collection(calls).contains_exactly([ + { + "name": "uv_1_0_0_linux", + "platform": "linux", + "sha256": "deadbeef", + "urls": ["https://example.org/1.0.0/manifest.json.zip"], + "version": "1.0.0", + }, + ]) + +_tests.append(_test_manual_url_spec) + +def _test_defaults(env): + calls = [] + uv = _process_modules( + env, + module_ctx = _uv_mock_mctx( + _mod( + default = [ + _default( + base_url = "https://example.org", + manifest_filename = "manifest.json", + version = "1.0.0", + platform = "linux", + compatible_with = ["@platforms//os:linux"], + target_settings = ["//:my_flag"], + ), + ], + configure = [ + _configure(), # use defaults + ], + ), + ), + uv_repository = lambda **kwargs: calls.append(kwargs), + ) + + uv.names().contains_exactly([ + "1_0_0_linux", + ]) + uv.implementations().contains_exactly({ + "1_0_0_linux": "@uv_1_0_0_linux//:uv_toolchain", + }) + uv.compatible_with().contains_exactly({ + "1_0_0_linux": ["@platforms//os:linux"], + }) + uv.target_settings().contains_exactly({ + "1_0_0_linux": ["//:my_flag"], + }) + env.expect.that_collection(calls).contains_exactly([ + { + "name": "uv_1_0_0_linux", + "platform": "linux", + "sha256": "deadbeef", + "urls": ["https://example.org/1.0.0/linux"], + "version": "1.0.0", + }, + ]) + +_tests.append(_test_defaults) + +def _test_default_building(env): + calls = [] + uv = _process_modules( + env, + module_ctx = _uv_mock_mctx( + _mod( + default = [ + _default( + base_url = "https://example.org", + manifest_filename = "manifest.json", + version = "1.0.0", + ), + _default( + platform = "linux", + compatible_with = ["@platforms//os:linux"], + target_settings = ["//:my_flag"], + ), + _default( + platform = "osx", + compatible_with = ["@platforms//os:osx"], + ), + ], + configure = [ + _configure(), # use defaults + ], + ), + ), + uv_repository = lambda **kwargs: calls.append(kwargs), + ) + + uv.names().contains_exactly([ + "1_0_0_linux", + "1_0_0_osx", + ]) + uv.implementations().contains_exactly({ + "1_0_0_linux": "@uv_1_0_0_linux//:uv_toolchain", + "1_0_0_osx": "@uv_1_0_0_osx//:uv_toolchain", + }) + uv.compatible_with().contains_exactly({ + "1_0_0_linux": ["@platforms//os:linux"], + "1_0_0_osx": ["@platforms//os:osx"], + }) + uv.target_settings().contains_exactly({ + "1_0_0_linux": ["//:my_flag"], + }) + env.expect.that_collection(calls).contains_exactly([ + { + "name": "uv_1_0_0_linux", + "platform": "linux", + "sha256": "deadbeef", + "urls": ["https://example.org/1.0.0/linux"], + "version": "1.0.0", + }, + { + "name": "uv_1_0_0_osx", + "platform": "osx", + "sha256": "deadb00f", + "urls": ["https://example.org/1.0.0/osx"], + "version": "1.0.0", + }, + ]) + +_tests.append(_test_default_building) + +def _test_complex_configuring(env): + calls = [] + uv = _process_modules( + env, + module_ctx = _uv_mock_mctx( + _mod( + default = [ + _default( + base_url = "https://example.org", + manifest_filename = "manifest.json", + version = "1.0.0", + platform = "osx", + compatible_with = ["@platforms//os:os"], + ), + ], + configure = [ + _configure(), # use defaults + _configure( + version = "1.0.1", + ), # use defaults + _configure( + version = "1.0.2", + base_url = "something_different", + manifest_filename = "different.json", + ), # use defaults + _configure( + platform = "osx", + compatible_with = ["@platforms//os:different"], + ), + _configure( + version = "1.0.3", + ), + _configure(platform = "osx"), # remove the default + _configure( + platform = "linux", + compatible_with = ["@platforms//os:linux"], + ), + _configure( + version = "1.0.4", + netrc = "~/.my_netrc", + auth_patterns = {"foo": "bar"}, + ), # use auth + ], + ), + ), + uv_repository = lambda **kwargs: calls.append(kwargs), + ) + + uv.names().contains_exactly([ + "1_0_0_osx", + "1_0_1_osx", + "1_0_2_osx", + "1_0_3_linux", + "1_0_4_osx", + ]) + uv.implementations().contains_exactly({ + "1_0_0_osx": "@uv_1_0_0_osx//:uv_toolchain", + "1_0_1_osx": "@uv_1_0_1_osx//:uv_toolchain", + "1_0_2_osx": "@uv_1_0_2_osx//:uv_toolchain", + "1_0_3_linux": "@uv_1_0_3_linux//:uv_toolchain", + "1_0_4_osx": "@uv_1_0_4_osx//:uv_toolchain", + }) + uv.compatible_with().contains_exactly({ + "1_0_0_osx": ["@platforms//os:os"], + "1_0_1_osx": ["@platforms//os:os"], + "1_0_2_osx": ["@platforms//os:different"], + "1_0_3_linux": ["@platforms//os:linux"], + "1_0_4_osx": ["@platforms//os:os"], + }) + uv.target_settings().contains_exactly({}) + env.expect.that_collection(calls).contains_exactly([ + { + "name": "uv_1_0_0_osx", + "platform": "osx", + "sha256": "deadb00f", + "urls": ["https://example.org/1.0.0/osx"], + "version": "1.0.0", + }, + { + "name": "uv_1_0_1_osx", + "platform": "osx", + "sha256": "deadb00f", + "urls": ["https://example.org/1.0.1/osx"], + "version": "1.0.1", + }, + { + "name": "uv_1_0_2_osx", + "platform": "osx", + "sha256": "deadb00f", + "urls": ["something_different/1.0.2/osx"], + "version": "1.0.2", + }, + { + "name": "uv_1_0_3_linux", + "platform": "linux", + "sha256": "deadbeef", + "urls": ["https://example.org/1.0.3/linux"], + "version": "1.0.3", + }, + { + "auth_patterns": {"foo": "bar"}, + "name": "uv_1_0_4_osx", + "netrc": "~/.my_netrc", + "platform": "osx", + "sha256": "deadb00f", + "urls": ["https://example.org/1.0.4/osx"], + "version": "1.0.4", + }, + ]) + +_tests.append(_test_complex_configuring) + +def _test_non_rules_python_non_root_is_ignored(env): + calls = [] + uv = _process_modules( + env, + module_ctx = _uv_mock_mctx( + _mod( + default = [ + _default( + base_url = "https://example.org", + manifest_filename = "manifest.json", + version = "1.0.0", + platform = "osx", + compatible_with = ["@platforms//os:os"], + ), + ], + configure = [ + _configure(), # use defaults + ], + ), + _mod( + name = "something", + configure = [ + _configure(version = "6.6.6"), # use defaults whatever they are + ], + is_root = False, + ), + ), + uv_repository = lambda **kwargs: calls.append(kwargs), + ) + + uv.names().contains_exactly([ + "1_0_0_osx", + ]) + uv.implementations().contains_exactly({ + "1_0_0_osx": "@uv_1_0_0_osx//:uv_toolchain", + }) + uv.compatible_with().contains_exactly({ + "1_0_0_osx": ["@platforms//os:os"], + }) + uv.target_settings().contains_exactly({}) + env.expect.that_collection(calls).contains_exactly([ + { + "name": "uv_1_0_0_osx", + "platform": "osx", + "sha256": "deadb00f", + "urls": ["https://example.org/1.0.0/osx"], + "version": "1.0.0", + }, + ]) + +_tests.append(_test_non_rules_python_non_root_is_ignored) + +def _test_rules_python_does_not_take_precedence(env): + calls = [] + uv = _process_modules( + env, + module_ctx = _uv_mock_mctx( + _mod( + default = [ + _default( + base_url = "https://example.org", + manifest_filename = "manifest.json", + version = "1.0.0", + platform = "osx", + compatible_with = ["@platforms//os:os"], + ), + ], + configure = [ + _configure(), # use defaults + ], + ), + _mod( + name = "rules_python", + configure = [ + _configure( + version = "1.0.0", + base_url = "https://foobar.org", + platform = "osx", + compatible_with = ["@platforms//os:osx"], + ), + ], + is_root = False, + ), + ), + uv_repository = lambda **kwargs: calls.append(kwargs), + ) + + uv.names().contains_exactly([ + "1_0_0_osx", + ]) + uv.implementations().contains_exactly({ + "1_0_0_osx": "@uv_1_0_0_osx//:uv_toolchain", + }) + uv.compatible_with().contains_exactly({ + "1_0_0_osx": ["@platforms//os:os"], + }) + uv.target_settings().contains_exactly({}) + env.expect.that_collection(calls).contains_exactly([ + { + "name": "uv_1_0_0_osx", + "platform": "osx", + "sha256": "deadb00f", + "urls": ["https://example.org/1.0.0/osx"], + "version": "1.0.0", + }, + ]) + +_tests.append(_test_rules_python_does_not_take_precedence) + +_analysis_tests = [] + +def _test_toolchain_precedence(name): + analysis_test( + name = name, + impl = _test_toolchain_precedence_impl, + target = "//python/uv:current_toolchain", + config_settings = { + "//command_line_option:extra_toolchains": [ + str(Label("//tests/uv/uv_toolchains:all")), + ], + "//command_line_option:platforms": str(platform_targets.LINUX_AARCH64), + }, + ) + +def _test_toolchain_precedence_impl(env, target): + # Check that the forwarded UvToolchainInfo looks vaguely correct. + uv_info = env.expect.that_target(target).provider( + UvToolchainInfo, + factory = lambda v, meta: v, + ) + env.expect.that_str(str(uv_info.label)).contains("//tests/uv/uv:fake_foof") + +_analysis_tests.append(_test_toolchain_precedence) + +def uv_test_suite(name): + """Create the test suite. + + Args: + name: the name of the test suite + """ + test_suite( + name = name, + basic_tests = _tests, + tests = _analysis_tests, + ) + + uv_toolchain( + name = "fake_bar", + uv = ":BUILD.bazel", + version = "0.0.1", + ) + + uv_toolchain( + name = "fake_foof", + uv = ":BUILD.bazel", + version = "0.0.1", + ) diff --git a/tests/uv/uv_toolchains/BUILD.bazel b/tests/uv/uv_toolchains/BUILD.bazel new file mode 100644 index 0000000000..4e2a12dcae --- /dev/null +++ b/tests/uv/uv_toolchains/BUILD.bazel @@ -0,0 +1,25 @@ +load("//python/uv/private:toolchains_hub.bzl", "toolchains_hub") # buildifier: disable=bzl-visibility + +toolchains_hub( + name = "uv_unit_test", + implementations = { + "bar": "//tests/uv/uv:fake_bar", + "foo": "//tests/uv/uv:fake_foof", + }, + target_compatible_with = { + "bar": [ + "@platforms//os:linux", + "@platforms//cpu:aarch64", + ], + "foo": [ + "@platforms//os:linux", + "@platforms//cpu:aarch64", + ], + }, + target_settings = {}, + # We expect foo to take precedence over bar + toolchains = [ + "foo", + "bar", + ], +) diff --git a/tests/validate_test_main/BUILD.bazel b/tests/validate_test_main/BUILD.bazel new file mode 100644 index 0000000000..94163a2fde --- /dev/null +++ b/tests/validate_test_main/BUILD.bazel @@ -0,0 +1,10 @@ +load("//python:py_test.bzl", "py_test") + +py_test( + name = "validate_test_main_test", + srcs = ["validate_test_main_test.py"], + main = "validate_test_main_test.py", + deps = [ + "//python/private:py_test_main_validator_lib", + ], +) diff --git a/tests/validate_test_main/validate_test_main_test.py b/tests/validate_test_main/validate_test_main_test.py new file mode 100644 index 0000000000..4cbc55c937 --- /dev/null +++ b/tests/validate_test_main/validate_test_main_test.py @@ -0,0 +1,261 @@ +#!/usr/bin/env python3 +import ast +import textwrap +import unittest + +from python.private.py_test_main_validator import module_runs_tests + + +def _runs_tests(source: str) -> bool: + tree = ast.parse(textwrap.dedent(source)) + return module_runs_tests(tree) + + +class ModuleRunsTestsTest(unittest.TestCase): + def test_only_definitions_is_rejected(self): + self.assertFalse( + _runs_tests( + """ + import unittest + + class MyTest(unittest.TestCase): + def test_foo(self): + self.assertTrue(True) + """ + ) + ) + + def test_definitions_with_assignments_is_rejected(self): + self.assertFalse( + _runs_tests( + """ + import unittest + + CONSTANT = 5 + + class MyTest(unittest.TestCase): + def test_foo(self): + pass + + def helper(): pass + """ + ) + ) + + def test_global_statement_with_definition_is_rejected(self): + self.assertFalse( + _runs_tests( + """ + global x + + class MyTest: + def test_foo(self): + pass + """ + ) + ) + + def test_assert_statement_with_definition_is_rejected(self): + # A bare `assert` with an inert condition runs no tests. + self.assertFalse( + _runs_tests( + """ + assert True + + class MyTest: + def test_foo(self): + pass + """ + ) + ) + + def test_assert_statement_with_active_expression_runs_tests(self): + # An assert whose condition runs a call actually runs the tests. + self.assertTrue( + _runs_tests( + """ + import unittest + + class MyTest(unittest.TestCase): + def test_foo(self): + pass + + assert unittest.main() + """ + ) + ) + + @unittest.skipUnless( + hasattr(ast, "TypeAlias"), "PEP 695 type aliases require Python 3.12+" + ) + def test_type_alias_with_definition_is_rejected(self): + self.assertFalse( + _runs_tests( + """ + type Alias = int + + class MyTest: + def test_foo(self): + pass + """ + ) + ) + + def test_type_checking_block_with_definition_is_rejected(self): + # `if TYPE_CHECKING:` guarding typing-only imports is inert; with a test + # definition and no runner, the module is still rejected. + self.assertFalse( + _runs_tests( + """ + from typing import TYPE_CHECKING + import unittest + + if TYPE_CHECKING: + from typing import Any + + class MyTest(unittest.TestCase): + def test_foo(self): + pass + """ + ) + ) + + def test_try_except_import_error_with_definition_is_rejected(self): + # `try: import foo except ImportError:` optional imports are inert. + self.assertFalse( + _runs_tests( + """ + try: + import foo + except ImportError: + foo = None + + import unittest + + class MyTest(unittest.TestCase): + def test_foo(self): + pass + """ + ) + ) + + def test_definitions_with_active_assignment_runs_tests(self): + # An assignment whose value runs a call actually runs the tests. + self.assertTrue( + _runs_tests( + """ + import unittest + + class MyTest(unittest.TestCase): + def test_foo(self): + pass + + exit_code = unittest.main() + """ + ) + ) + + def test_definitions_with_active_expression_runs_tests(self): + self.assertTrue( + _runs_tests( + """ + import sys + import unittest + + class MyTest(unittest.TestCase): + def test_foo(self): + pass + + sys.exit(unittest.main()) + """ + ) + ) + + def test_if_block_invoking_runner_runs_tests(self): + # A runner call inside an `if` body must still count as active. + self.assertTrue( + _runs_tests( + """ + import sys + import unittest + + class MyTest(unittest.TestCase): + def test_foo(self): + pass + + if "--run" in sys.argv: + unittest.main() + """ + ) + ) + + def test_import_only_module_is_allowed(self): + # A module that defines nothing and only imports other modules is not + # the "defined but never run" case, so it is allowed. + self.assertTrue( + _runs_tests( + """ + import my_tests + from my_pkg.tests import suite + """ + ) + ) + + def test_imports_and_assignments_without_definitions_is_allowed(self): + self.assertTrue( + _runs_tests( + """ + import my_tests + + CONSTANT = 5 + """ + ) + ) + + def test_empty_module_is_allowed(self): + self.assertTrue(_runs_tests("")) + + def test_docstring_only_is_allowed(self): + self.assertTrue(_runs_tests('"""A module docstring."""')) + + def test_if_name_main_guard_runs_tests(self): + self.assertTrue( + _runs_tests( + """ + import unittest + + class MyTest(unittest.TestCase): + def test_foo(self): + pass + + if __name__ == "__main__": + unittest.main() + """ + ) + ) + + def test_bare_call_runs_tests(self): + self.assertTrue( + _runs_tests( + """ + import pytest + pytest.main() + """ + ) + ) + + def test_top_level_loop_runs_tests(self): + self.assertTrue( + _runs_tests( + """ + def f(): pass + + for _ in range(1): + f() + """ + ) + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/venv_site_packages_libs/BUILD.bazel b/tests/venv_site_packages_libs/BUILD.bazel new file mode 100644 index 0000000000..6a7b3b9e12 --- /dev/null +++ b/tests/venv_site_packages_libs/BUILD.bazel @@ -0,0 +1,101 @@ +load("@rules_shell//shell:sh_test.bzl", "sh_test") +load("//python:py_library.bzl", "py_library") +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") + +py_library( + name = "user_lib", + deps = ["@other//simple_v1"], +) + +py_library( + name = "closer_lib", + deps = [ + ":user_lib", + "@other//simple_v2", + ], +) + +sh_test( + name = "py_binary_other_module_test", + srcs = [ + "py_binary_other_module_test.sh", + ], + data = [ + "@other//:venv_bin", + ], + env = { + "VENV_BIN": "$(rootpath @other//:venv_bin)", + }, +) + +py_reconfig_test( + name = "venvs_site_packages_libs_test", + srcs = ["bin.py"], + bootstrap_impl = "script", + main = "bin.py", + venvs_site_packages = "yes", + deps = [ + ":closer_lib", + "//tests/venv_site_packages_libs/nested_with_pth", + "//tests/venv_site_packages_libs/nspkg_alpha", + "//tests/venv_site_packages_libs/nspkg_beta", + "//tests/venv_site_packages_libs/pkgutil_top", + "//tests/venv_site_packages_libs/pkgutil_top_sub", + "@other//nspkg_delta", + "@other//nspkg_gamma", + "@other//nspkg_single", + "@other//with_external_data", + "@whl_with_data1//:pkg", + "@whl_with_data2//:pkg", + ], +) + +py_reconfig_test( + name = "shared_lib_loading_test", + srcs = ["shared_lib_loading_test.py"], + bootstrap_impl = select({ + "@platforms//os:windows": "system_python", + "//conditions:default": "script", + }), + main = "shared_lib_loading_test.py", + venvs_site_packages = "yes", + deps = select({ + "@platforms//os:windows": [ + "@dev_pip//markupsafe", + ], + "//conditions:default": [ + "//tests/venv_site_packages_libs/ext_with_libs", + "@dev_pip//macholib", + "@dev_pip//pyelftools", + ], + }), +) + +py_reconfig_test( + name = "whl_scripts_runnable_test", + srcs = ["whl_scripts_runnable_test.py"], + bootstrap_impl = select({ + "@platforms//os:windows": "system_python", + "//conditions:default": "script", + }), + main = "whl_scripts_runnable_test.py", + venvs_site_packages = "yes", + deps = [ + "@whl_with_data1//:pkg", + "@whl_with_data2//:pkg", + ], +) + +py_reconfig_test( + name = "importlib_metadata_test", + srcs = ["importlib_metadata_test.py"], + bootstrap_impl = select({ + "@platforms//os:windows": "system_python", + "//conditions:default": "script", + }), + main = "importlib_metadata_test.py", + venvs_site_packages = "yes", + deps = [ + "@whl_with_data1//:pkg", + ], +) diff --git a/tests/venv_site_packages_libs/app_files_building/BUILD.bazel b/tests/venv_site_packages_libs/app_files_building/BUILD.bazel new file mode 100644 index 0000000000..60afd34c38 --- /dev/null +++ b/tests/venv_site_packages_libs/app_files_building/BUILD.bazel @@ -0,0 +1,5 @@ +load(":app_files_building_tests.bzl", "app_files_building_test_suite") + +app_files_building_test_suite( + name = "app_files_building_tests", +) diff --git a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl new file mode 100644 index 0000000000..663ab6963e --- /dev/null +++ b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl @@ -0,0 +1,763 @@ +"" + +load("@rules_testing//lib:analysis_test.bzl", "analysis_test") +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python:py_info.bzl", "PyInfo", "VenvSymlinkEntry", "VenvSymlinkKind") +load("//python:py_library.bzl", "py_library") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility +load("//python/private:venv_runfiles.bzl", "build_link_map", "get_venv_symlinks") # buildifier: disable=bzl-visibility +load("//tests/support:support.bzl", "SUPPORTS_BZLMOD") + +def _empty_files_impl(ctx): + files = [] + for p in ctx.attr.paths: + f = ctx.actions.declare_file(p) + ctx.actions.write(output = f, content = "") + files.append(f) + return [DefaultInfo(files = depset(files))] + +empty_files = rule( + implementation = _empty_files_impl, + attrs = { + "paths": attr.string_list( + doc = "A list of paths to create as files.", + mandatory = True, + ), + }, +) + +_tests = [] + +# NOTE: In bzlmod, the workspace name is always "_main". +# Under workspace, the workspace name is the name configured in WORKSPACE, +# or "__main__" if was unspecified. +# NOTE: ctx.workspace_name is always the root workspace, not the workspace +# of the target being processed (ctx.label). +def _ctx(workspace_name = "_main"): + return struct( + workspace_name = workspace_name, + label = Label("@@FAKE-CTX//:fake_ctx"), + ) + +def _file(short_path): + return struct( + short_path = short_path, + ) + +def _venv_symlink(venv_path, *, link_to_path = None, files = []): + return struct( + link_to_path = link_to_path, + venv_path = venv_path, + files = files, + ) + +def _venv_symlinks_from_entries(entries): + result = [] + for symlink_entry in entries: + result.append(struct( + venv_path = symlink_entry.venv_path, + link_to_path = symlink_entry.link_to_path, + files = [f.short_path for f in symlink_entry.files.to_list()], + )) + return sorted(result, key = lambda e: (e.link_to_path, e.venv_path)) + +def _entry(venv_path, link_to_path, files, **kwargs): + kwargs.setdefault("kind", VenvSymlinkKind.LIB) + kwargs.setdefault("package", None) + kwargs.setdefault("version", None) + kwargs.setdefault("link_to_file", None) + + return VenvSymlinkEntry( + venv_path = venv_path, + link_to_path = link_to_path, + files = depset(files), + **kwargs + ) + +def _test_conflict_merging(name): + analysis_test( + name = name, + impl = _test_conflict_merging_impl, + target = "//python:none", + ) + +_tests.append(_test_conflict_merging) + +def _test_conflict_merging_impl(env, _): + entries = [ + _entry("a", "+pypi_a/site-packages/a", [ + _file("../+pypi_a/site-packages/a/a.txt"), + ]), + _entry("a-1.0.dist-info", "+pypi_a/site-packages/a-1.0.dist-info", [ + _file("../+pypi_a/site-packages/a-1.0.dist-info/METADATA"), + ]), + _entry("a/b", "+pypi_a_b/site-packages/a/b", [ + _file("../+pypi_a_b/site-packages/a/b/b.txt"), + ]), + _entry("x", "_main/src/x", [ + _file("src/x/x.txt"), + ]), + _entry("x/p", "_main/src-dev/x/p", [ + _file("src-dev/x/p/p.txt"), + ]), + _entry("duplicate", "+dupe_a/site-packages/duplicate", [ + _file("../+dupe_a/site-packages/duplicate/d.py"), + ]), + _entry("duplicate", "+dupe_b/site-packages/duplicate", [ + _file("../+dupe_b/site-packages/duplicate/d.py"), + ]), + # Case: two distributions provide the same file (instead of directory) + _entry("ff/fmod.py", "+ff_a/site-packages/ff/fmod.py", [ + _file("../+ff_a/site-packages/ff/fmod.py"), + ]), + _entry("ff/fmod.py", "+ff_b/site-packages/ff/fmod.py", [ + _file("../+ff_b/site-packages/ff/fmod.py"), + ]), + ] + + actual, conflicts = build_link_map(_ctx(), entries, return_conflicts = True) + expected_libs = { + "a-1.0.dist-info": "+pypi_a/site-packages/a-1.0.dist-info", + "a/a.txt": _file("../+pypi_a/site-packages/a/a.txt"), + "a/b/b.txt": _file("../+pypi_a_b/site-packages/a/b/b.txt"), + "duplicate/d.py": _file("../+dupe_a/site-packages/duplicate/d.py"), + "ff/fmod.py": _file("../+ff_a/site-packages/ff/fmod.py"), + "x/p/p.txt": _file("src-dev/x/p/p.txt"), + "x/x.txt": _file("src/x/x.txt"), + } + env.expect.that_dict(actual[VenvSymlinkKind.LIB]).contains_exactly(expected_libs) + env.expect.that_dict(actual).keys().contains_exactly([VenvSymlinkKind.LIB]) + + env.expect.that_int(len(conflicts)).is_greater_than(0) + +def _test_optimized_grouping_complex(name): + empty_files( + name = name + "_files", + paths = [ + "site-packages/pkg1/a.txt", + "site-packages/pkg1/b/b_mod_so", + "site-packages/pkg1/c/c1.txt", + "site-packages/pkg1/c/c2.txt", + "site-packages/pkg1/d/d1.txt", + "site-packages/pkg1/dd/dd1.txt", + "site-packages/pkg1/q1/q1.txt", + "site-packages/pkg1/q1/q2a/libq.so", + "site-packages/pkg1/q1/q2a/q2.txt", + "site-packages/pkg1/q1/q2a/q3/q3a.txt", + "site-packages/pkg1/q1/q2a/q3/q3b.txt", + "site-packages/pkg1/q1/q2b/q2b.txt", + "site-packages/pkg1/q1/q2c/c_mod.so", + "site-packages/pkg1/q1/q2c/q2.txt", + "site-packages/pkg1/q1/q2c/q3/q3a.txt", + "site-packages/pkg1/q1/q2c/q3/q3b.txt", + ], + ) + analysis_test( + name = name, + impl = _test_optimized_grouping_complex_impl, + target = name + "_files", + ) + +_tests.append(_test_optimized_grouping_complex) + +def _test_optimized_grouping_complex_impl(env, target): + test_ctx = _ctx(workspace_name = env.ctx.workspace_name) + entries = get_venv_symlinks( + test_ctx, + target.files.to_list(), + package = "pkg1", + version_str = "1.0", + site_packages_root = env.ctx.label.package + "/site-packages", + ) + actual = _venv_symlinks_from_entries(entries) + + rr = "{}/{}/site-packages/".format(test_ctx.workspace_name, env.ctx.label.package) + expected = [ + _venv_symlink( + "pkg1/a.txt", + link_to_path = rr + "pkg1/a.txt", + files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/a.txt", + ], + ), + _venv_symlink( + "pkg1/b", + link_to_path = rr + "pkg1/b", + files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/b/b_mod_so", + ], + ), + _venv_symlink("pkg1/c", link_to_path = rr + "pkg1/c", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/c/c1.txt", + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/c/c2.txt", + ]), + _venv_symlink("pkg1/d", link_to_path = rr + "pkg1/d", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/d/d1.txt", + ]), + _venv_symlink("pkg1/dd", link_to_path = rr + "pkg1/dd", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/dd/dd1.txt", + ]), + _venv_symlink("pkg1/q1/q1.txt", link_to_path = rr + "pkg1/q1/q1.txt", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/q1/q1.txt", + ]), + _venv_symlink("pkg1/q1/q2a/libq.so", link_to_path = rr + "pkg1/q1/q2a/libq.so", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/q1/q2a/libq.so", + ]), + _venv_symlink("pkg1/q1/q2a/q2.txt", link_to_path = rr + "pkg1/q1/q2a/q2.txt", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/q1/q2a/q2.txt", + ]), + _venv_symlink("pkg1/q1/q2a/q3", link_to_path = rr + "pkg1/q1/q2a/q3", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/q1/q2a/q3/q3a.txt", + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/q1/q2a/q3/q3b.txt", + ]), + _venv_symlink("pkg1/q1/q2b", link_to_path = rr + "pkg1/q1/q2b", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/q1/q2b/q2b.txt", + ]), + _venv_symlink("pkg1/q1/q2c/c_mod.so", link_to_path = rr + "pkg1/q1/q2c/c_mod.so", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/q1/q2c/c_mod.so", + ]), + _venv_symlink("pkg1/q1/q2c/q2.txt", link_to_path = rr + "pkg1/q1/q2c/q2.txt", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/q1/q2c/q2.txt", + ]), + _venv_symlink("pkg1/q1/q2c/q3", link_to_path = rr + "pkg1/q1/q2c/q3", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/q1/q2c/q3/q3a.txt", + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/q1/q2c/q3/q3b.txt", + ]), + ] + expected = sorted(expected, key = lambda e: (e.link_to_path, e.venv_path)) + env.expect.that_collection( + actual, + ).contains_exactly(expected) + _, conflicts = build_link_map(test_ctx, entries, return_conflicts = True) + + # The point of the optimization is to avoid having to merge conflicts. + env.expect.that_collection(conflicts).contains_exactly([]) + +def _test_optimized_grouping_single_toplevel(name): + empty_files( + name = name + "_files", + paths = [ + "site-packages/pkg2/__init__.py", + "site-packages/pkg2/a.txt", + "site-packages/pkg2/b_mod_so", + ], + ) + analysis_test( + name = name, + impl = _test_optimized_grouping_single_toplevel_impl, + target = name + "_files", + ) + +_tests.append(_test_optimized_grouping_single_toplevel) + +def _test_optimized_grouping_single_toplevel_impl(env, target): + test_ctx = _ctx(workspace_name = env.ctx.workspace_name) + entries = get_venv_symlinks( + test_ctx, + target.files.to_list(), + package = "pkg2", + version_str = "1.0", + site_packages_root = env.ctx.label.package + "/site-packages", + ) + actual = _venv_symlinks_from_entries(entries) + + rr = "{}/{}/site-packages/".format(test_ctx.workspace_name, env.ctx.label.package) + expected = [ + _venv_symlink( + "pkg2", + link_to_path = rr + "pkg2", + files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg2/__init__.py", + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg2/a.txt", + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg2/b_mod_so", + ], + ), + ] + expected = sorted(expected, key = lambda e: (e.link_to_path, e.venv_path)) + + env.expect.that_collection( + actual, + ).contains_exactly(expected) + + _, conflicts = build_link_map(test_ctx, entries, return_conflicts = True) + + # The point of the optimization is to avoid having to merge conflicts. + env.expect.that_collection(conflicts).contains_exactly([]) + +def _test_optimized_grouping_implicit_namespace_packages(name): + empty_files( + name = name + "_files", + paths = [ + # NOTE: An alphanumeric name with underscores is used to verify + # name matching is correct. + "site-packages/name_space9/part1/foo.py", + "site-packages/name_space9/part2/bar.py", + "site-packages/name_space9-1.0.dist-info/METADATA", + ], + ) + analysis_test( + name = name, + impl = _test_optimized_grouping_implicit_namespace_packages_impl, + target = name + "_files", + ) + +_tests.append(_test_optimized_grouping_implicit_namespace_packages) + +def _test_optimized_grouping_implicit_namespace_packages_impl(env, target): + test_ctx = _ctx(workspace_name = env.ctx.workspace_name) + entries = get_venv_symlinks( + test_ctx, + target.files.to_list(), + package = "pkg3", + version_str = "1.0", + site_packages_root = env.ctx.label.package + "/site-packages", + ) + actual = _venv_symlinks_from_entries(entries) + + rr = "{}/{}/site-packages/".format(test_ctx.workspace_name, env.ctx.label.package) + expected = [ + _venv_symlink( + "name_space9/part1", + link_to_path = rr + "name_space9/part1", + files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/name_space9/part1/foo.py", + ], + ), + _venv_symlink( + "name_space9/part2", + link_to_path = rr + "name_space9/part2", + files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/name_space9/part2/bar.py", + ], + ), + _venv_symlink( + "name_space9-1.0.dist-info", + link_to_path = rr + "name_space9-1.0.dist-info", + files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/name_space9-1.0.dist-info/METADATA", + ], + ), + ] + expected = sorted(expected, key = lambda e: (e.link_to_path, e.venv_path)) + env.expect.that_collection( + actual, + ).contains_exactly(expected) + + _, conflicts = build_link_map(test_ctx, entries, return_conflicts = True) + + # The point of the optimization is to avoid having to merge conflicts. + env.expect.that_collection(conflicts).contains_exactly([]) + +def _test_optimized_grouping_pkgutil_namespace_packages(name): + empty_files( + name = name + "_files", + paths = [ + "site-packages/pkgutilns/__init__.py", + "site-packages/pkgutilns/foo.py", + # Special cases: These dirnames under site-packages are always + # treated as namespace packages + "site-packages/nvidia/whatever/w.py", + ], + ) + analysis_test( + name = name, + impl = _test_optimized_grouping_pkgutil_namespace_packages_impl, + target = name + "_files", + ) + +_tests.append(_test_optimized_grouping_pkgutil_namespace_packages) + +def _test_optimized_grouping_pkgutil_namespace_packages_impl(env, target): + test_ctx = _ctx(workspace_name = env.ctx.workspace_name) + files = target.files.to_list() + ns_inits = [f for f in files if f.basename == "__init__.py"] + + entries = get_venv_symlinks( + test_ctx, + files, + package = "pkgutilns", + version_str = "1.0", + site_packages_root = env.ctx.label.package + "/site-packages", + namespace_package_files = ns_inits, + ) + actual = _venv_symlinks_from_entries(entries) + + rr = "{}/{}/site-packages/".format(test_ctx.workspace_name, env.ctx.label.package) + expected = [ + _venv_symlink( + "pkgutilns/__init__.py", + link_to_path = rr + "pkgutilns/__init__.py", + files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkgutilns/__init__.py", + ], + ), + _venv_symlink( + "pkgutilns/foo.py", + link_to_path = rr + "pkgutilns/foo.py", + files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkgutilns/foo.py", + ], + ), + _venv_symlink( + "nvidia/whatever", + link_to_path = rr + "nvidia/whatever", + files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/nvidia/whatever/w.py", + ], + ), + ] + expected = sorted(expected, key = lambda e: (e.link_to_path, e.venv_path)) + env.expect.that_collection( + actual, + ).contains_exactly(expected) + + _, conflicts = build_link_map(test_ctx, entries, return_conflicts = True) + + # The point of the optimization is to avoid having to merge conflicts. + env.expect.that_collection(conflicts).contains_exactly([]) + +def _test_optimized_grouping_pkgutil_whls(name): + """Verify that the whl_library pkgutli style detection logic works.""" + py_library( + name = name + "_lib", + deps = [ + "@pkgutil_nspkg1//:pkg", + "@pkgutil_nspkg2//:pkg", + ], + target_compatible_with = SUPPORTS_BZLMOD, + ) + analysis_test( + name = name, + impl = _test_optimized_grouping_pkgutil_whls_impl, + target = name + "_lib", + config_settings = { + labels.VENVS_SITE_PACKAGES: "yes", + }, + attr_values = dict( + target_compatible_with = SUPPORTS_BZLMOD, + ), + ) + +_tests.append(_test_optimized_grouping_pkgutil_whls) + +def _test_optimized_grouping_pkgutil_whls_impl(env, target): + test_ctx = _ctx(workspace_name = env.ctx.workspace_name) + actual_raw_entries = target[PyInfo].venv_symlinks.to_list() + + actual = _venv_symlinks_from_entries(actual_raw_entries) + + # The important condition is that the top-level 'nspkg' directory + # is NOT linked because it's a pkgutil namespace package. + env.expect.that_collection(actual).contains_exactly([ + # Entries from pkgutil_ns1 + _venv_symlink( + "nspkg/__init__.py", + link_to_path = "+internal_dev_deps+pkgutil_nspkg1/site-packages/nspkg/__init__.py", + files = [ + "../+internal_dev_deps+pkgutil_nspkg1/site-packages/nspkg/__init__.py", + ], + ), + _venv_symlink( + "nspkg/one", + link_to_path = "+internal_dev_deps+pkgutil_nspkg1/site-packages/nspkg/one", + files = [ + "../+internal_dev_deps+pkgutil_nspkg1/site-packages/nspkg/one/a.txt", + ], + ), + _venv_symlink( + "pkgutil_nspkg1-1.0.dist-info", + link_to_path = "+internal_dev_deps+pkgutil_nspkg1/site-packages/pkgutil_nspkg1-1.0.dist-info", + files = [ + "../+internal_dev_deps+pkgutil_nspkg1/site-packages/pkgutil_nspkg1-1.0.dist-info/INSTALLER", + "../+internal_dev_deps+pkgutil_nspkg1/site-packages/pkgutil_nspkg1-1.0.dist-info/METADATA", + "../+internal_dev_deps+pkgutil_nspkg1/site-packages/pkgutil_nspkg1-1.0.dist-info/RECORD", + "../+internal_dev_deps+pkgutil_nspkg1/site-packages/pkgutil_nspkg1-1.0.dist-info/WHEEL", + ], + ), + # Entries from pkgutil_ns2 + _venv_symlink( + "nspkg/__init__.py", + link_to_path = "+internal_dev_deps+pkgutil_nspkg2/site-packages/nspkg/__init__.py", + files = [ + "../+internal_dev_deps+pkgutil_nspkg2/site-packages/nspkg/__init__.py", + ], + ), + _venv_symlink( + "nspkg/two", + link_to_path = "+internal_dev_deps+pkgutil_nspkg2/site-packages/nspkg/two", + files = [ + "../+internal_dev_deps+pkgutil_nspkg2/site-packages/nspkg/two/b.txt", + ], + ), + _venv_symlink( + "pkgutil_nspkg2-1.0.dist-info", + link_to_path = "+internal_dev_deps+pkgutil_nspkg2/site-packages/pkgutil_nspkg2-1.0.dist-info", + files = [ + "../+internal_dev_deps+pkgutil_nspkg2/site-packages/pkgutil_nspkg2-1.0.dist-info/INSTALLER", + "../+internal_dev_deps+pkgutil_nspkg2/site-packages/pkgutil_nspkg2-1.0.dist-info/METADATA", + "../+internal_dev_deps+pkgutil_nspkg2/site-packages/pkgutil_nspkg2-1.0.dist-info/RECORD", + "../+internal_dev_deps+pkgutil_nspkg2/site-packages/pkgutil_nspkg2-1.0.dist-info/WHEEL", + ], + ), + ]) + + # Verifying that the expected VenvSymlink structure is processed with minimal number + # of conflicts (Just the single pkgutil style __init__.py file) + _, conflicts = build_link_map(test_ctx, actual_raw_entries, return_conflicts = True) + env.expect.that_collection(_venv_symlinks_from_entries(conflicts[0])).contains_exactly([ + _venv_symlink( + "nspkg/__init__.py", + link_to_path = "+internal_dev_deps+pkgutil_nspkg1/site-packages/nspkg/__init__.py", + files = [ + "../+internal_dev_deps+pkgutil_nspkg1/site-packages/nspkg/__init__.py", + ], + ), + _venv_symlink( + "nspkg/__init__.py", + link_to_path = "+internal_dev_deps+pkgutil_nspkg2/site-packages/nspkg/__init__.py", + files = [ + "../+internal_dev_deps+pkgutil_nspkg2/site-packages/nspkg/__init__.py", + ], + ), + ]) + env.expect.that_collection(conflicts).has_size(1) + +def _test_package_version_filtering(name): + analysis_test( + name = name, + impl = _test_package_version_filtering_impl, + target = "//python:none", + ) + +_tests.append(_test_package_version_filtering) + +def _test_package_version_filtering_impl(env, _): + entries = [ + _entry("foo", "+pypi_v1/site-packages/foo", [ + _file("../+pypi_v1/site-packages/foo/foo.txt"), + ], package = "foo", version = "1.0"), + _entry("foo", "+pypi_v2/site-packages/foo", [ + _file("../+pypi_v2/site-packages/foo/bar.txt"), + ], package = "foo", version = "2.0"), + ] + + actual = build_link_map(_ctx(), entries) + + expected_libs = { + "foo": "+pypi_v1/site-packages/foo", + } + env.expect.that_dict(actual[VenvSymlinkKind.LIB]).contains_exactly(expected_libs) + +def _test_malformed_entry(name): + analysis_test( + name = name, + impl = _test_malformed_entry_impl, + target = "//python:none", + ) + +_tests.append(_test_malformed_entry) + +def _test_malformed_entry_impl(env, _): + entries = [ + _entry( + "a", + "+pypi_a/site-packages/a", + # This file is outside the link_to_path, so it should be ignored. + [_file("../+pypi_a/site-packages/outside.txt")], + ), + # A second, conflicting, entry is added to force merging of the known + # files. Without this, there's no conflict, so files is never + # considered. + _entry( + "a", + "+pypi_b/site-packages/a", + [_file("../+pypi_b/site-packages/outside.txt")], + ), + ] + + actual = build_link_map(_ctx(), entries) + env.expect.that_dict(actual).contains_exactly({ + VenvSymlinkKind.LIB: {}, + }) + +def _test_complex_namespace_packages(name): + analysis_test( + name = name, + impl = _test_complex_namespace_packages_impl, + target = "//python:none", + ) + +_tests.append(_test_complex_namespace_packages) + +def _test_complex_namespace_packages_impl(env, _): + entries = [ + _entry("a/b", "+pypi_a_b/site-packages/a/b", [ + _file("../+pypi_a_b/site-packages/a/b/b.txt"), + ]), + _entry("a/c", "+pypi_a_c/site-packages/a/c", [ + _file("../+pypi_a_c/site-packages/a/cc.txt"), + ]), + _entry("x/y/z", "+pypi_x_y_z/site-packages/x/y/z", [ + _file("../+pypi_x_y_z/site-packages/x/y/z/z.txt"), + ]), + _entry("foo", "+pypi_foo/site-packages/foo", [ + _file("../+pypi_foo/site-packages/foo/foo.txt"), + ]), + _entry("foobar", "+pypi_foobar/site-packages/foobar", [ + _file("../+pypi_foobar/site-packages/foobar/foobar.txt"), + ]), + ] + + actual = build_link_map(_ctx(), entries) + expected_libs = { + "a/b": "+pypi_a_b/site-packages/a/b", + "a/c": "+pypi_a_c/site-packages/a/c", + "foo": "+pypi_foo/site-packages/foo", + "foobar": "+pypi_foobar/site-packages/foobar", + "x/y/z": "+pypi_x_y_z/site-packages/x/y/z", + } + env.expect.that_dict(actual[VenvSymlinkKind.LIB]).contains_exactly(expected_libs) + +def _test_empty_and_trivial_inputs(name): + analysis_test( + name = name, + impl = _test_empty_and_trivial_inputs_impl, + target = "//python:none", + ) + +_tests.append(_test_empty_and_trivial_inputs) + +def _test_empty_and_trivial_inputs_impl(env, _): + # Test with empty list of entries + actual = build_link_map(_ctx(), []) + env.expect.that_dict(actual).contains_exactly({}) + + # Test with an entry with no files + entries = [_entry("a", "+pypi_a/site-packages/a", [])] + actual = build_link_map(_ctx(), entries) + env.expect.that_dict(actual).contains_exactly({ + VenvSymlinkKind.LIB: {"a": "+pypi_a/site-packages/a"}, + }) + +def _test_multiple_venv_symlink_kinds(name): + analysis_test( + name = name, + impl = _test_multiple_venv_symlink_kinds_impl, + target = "//python:none", + ) + +_tests.append(_test_multiple_venv_symlink_kinds) + +def _test_multiple_venv_symlink_kinds_impl(env, _): + entries = [ + _entry( + "libfile", + "+pypi_lib/site-packages/libfile", + [_file("../+pypi_lib/site-packages/libfile/lib.txt")], + kind = VenvSymlinkKind.LIB, + ), + _entry( + "binfile", + "+pypi_bin/bin/binfile", + [_file("../+pypi_bin/bin/binfile/bin.txt")], + kind = VenvSymlinkKind.BIN, + ), + _entry( + "includefile", + "+pypi_include/include/includefile", + [_file("../+pypi_include/include/includefile/include.h")], + kind = + VenvSymlinkKind.INCLUDE, + ), + ] + + actual = build_link_map(_ctx(), entries) + + expected_libs = { + "libfile": "+pypi_lib/site-packages/libfile", + } + env.expect.that_dict(actual[VenvSymlinkKind.LIB]).contains_exactly(expected_libs) + + expected_bins = { + "binfile": "+pypi_bin/bin/binfile", + } + env.expect.that_dict(actual[VenvSymlinkKind.BIN]).contains_exactly(expected_bins) + + expected_includes = { + "includefile": "+pypi_include/include/includefile", + } + env.expect.that_dict(actual[VenvSymlinkKind.INCLUDE]).contains_exactly(expected_includes) + + env.expect.that_dict(actual).keys().contains_exactly([ + VenvSymlinkKind.LIB, + VenvSymlinkKind.BIN, + VenvSymlinkKind.INCLUDE, + ]) + +def _test_shared_library_symlinking(name): + empty_files( + name = name + "_files", + # NOTE: Test relies upon order + paths = [ + "site-packages/bar/libs/liby.so", + "site-packages/bar/x.py", + "site-packages/bar/y.so", + "site-packages/foo.libs/libx.so", + "site-packages/foo/a.py", + "site-packages/foo/b.so", + "site-packages/root.pth", + "site-packages/root.py", + "site-packages/root.so", + ], + ) + analysis_test( + name = name, + impl = _test_shared_library_symlinking_impl, + target = name + "_files", + ) + +_tests.append(_test_shared_library_symlinking) + +def _test_shared_library_symlinking_impl(env, target): + srcs = target.files.to_list() + actual_entries = get_venv_symlinks( + _ctx(), + srcs, + package = "foo", + version_str = "1.0", + site_packages_root = env.ctx.label.package + "/site-packages", + ) + + actual = _venv_symlinks_from_entries(actual_entries) + + env.expect.that_collection(actual).contains_at_least([ + _venv_symlink( + "bar/libs/liby.so", + link_to_path = "_main/tests/venv_site_packages_libs/app_files_building/site-packages/bar/libs/liby.so", + files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/bar/libs/liby.so", + ], + ), + _venv_symlink( + "foo.libs/libx.so", + link_to_path = "_main/tests/venv_site_packages_libs/app_files_building/site-packages/foo.libs/libx.so", + files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/foo.libs/libx.so", + ], + ), + ]) + + actual = build_link_map(_ctx(), actual_entries) + + # The important condition is that each lib*.so file is linked directly. + expected_libs = { + "bar/libs/liby.so": srcs[0], + "foo.libs/libx.so": srcs[3], + } + env.expect.that_dict(actual[VenvSymlinkKind.LIB]).contains_at_least(expected_libs) + +def app_files_building_test_suite(name): + test_suite( + name = name, + tests = _tests, + ) diff --git a/tests/venv_site_packages_libs/bin.py b/tests/venv_site_packages_libs/bin.py new file mode 100644 index 0000000000..b14ff54144 --- /dev/null +++ b/tests/venv_site_packages_libs/bin.py @@ -0,0 +1,156 @@ +import importlib +import sys +import sysconfig +import unittest +from pathlib import Path + + +class VenvSitePackagesLibraryTest(unittest.TestCase): + def setUp(self): + super().setUp() + if sys.prefix == sys.base_prefix: + raise AssertionError("Not running under a venv") + self.venv = Path(sys.prefix) + self.site_packages = Path(sysconfig.get_paths()["purelib"]) + + is_windows = sys.platform == "win32" + if is_windows: + self.bin_dir_name = Path("Scripts") + self.include_dir_name = Path("Include") + else: + self.bin_dir_name = Path("bin") + self.include_dir_name = Path("include") + + def assert_venv_path_exists(self, rel_path): + path = self.venv / rel_path + self.assertTrue( + path.exists(), + f"Expected {path} to exist. {path.parent.name} contents: {list(path.parent.iterdir()) if path.parent.exists() else 'N/A'}", + ) + + def assert_imported_from_venv(self, module_name): + module = importlib.import_module(module_name) + self.assertEqual(module.__name__, module_name) + self.assertIsNotNone( + module.__file__, + f"Expected module {module_name!r} to have" + + f"__file__ set, but got None. {module=}", + ) + self.assertTrue( + module.__file__.startswith(str(self.venv)), + f"\n{module_name} was imported, but not from the venv.\n" + + f" venv: {self.venv}\n" + + f"module file: {module.__file__}\n" + + "sys.path:\n" + + "\n".join(sys.path), + ) + return module + + def test_imported_from_venv(self): + m = self.assert_imported_from_venv("pkgutil_top") + self.assertEqual(m.WHOAMI, "pkgutil_top") + + m = self.assert_imported_from_venv("pkgutil_top.sub") + self.assertEqual(m.WHOAMI, "pkgutil_top.sub") + + self.assert_imported_from_venv("nspkg.subnspkg.alpha") + self.assert_imported_from_venv("nspkg.subnspkg.beta") + self.assert_imported_from_venv("nspkg.subnspkg.gamma") + self.assert_imported_from_venv("nspkg.subnspkg.delta") + self.assert_imported_from_venv("single_file") + self.assert_imported_from_venv("simple") + m = self.assert_imported_from_venv("nested_with_pth") + self.assertEqual(m.WHOAMI, "nested_with_pth") + + def test_data_is_included(self): + self.assert_imported_from_venv("simple") + module = importlib.import_module("simple") # noqa: F841 + # Ensure that packages from simple v1 are not present + files = [p.name for p in self.site_packages.glob("*")] + self.assertIn("simple_v1_extras", files) + + def test_override_pkg(self): + self.assert_imported_from_venv("simple") + module = importlib.import_module("simple") + self.assertEqual( + "1.0.0", + module.__version__, + ) + + def test_dirs_from_replaced_package_are_not_present(self): + self.assert_imported_from_venv("simple") + module = importlib.import_module("simple") # noqa: F841 + dist_info_dirs = [p.name for p in self.site_packages.glob("simple*.dist-info")] + self.assertEqual( + ["simple-1.0.0.dist-info"], + dist_info_dirs, + ) + + # Ensure that packages from simple v1 are not present + files = [p.name for p in self.site_packages.glob("*")] + self.assertNotIn("simple.libs", files) + + def test_data_from_another_pkg_is_included_via_copy_file(self): + self.assert_imported_from_venv("simple") + module = importlib.import_module("simple") # noqa: F841 + # Ensure that packages from simple v1 are not present + d = self.site_packages / "external_data" + files = [p.name for p in d.glob("*")] + self.assertIn("another_module_data.txt", files) + + def test_whl_with_data1_included(self): + module = self.assert_imported_from_venv("whl_with_data1") # noqa: F841 + site_packages_rel = self.site_packages.relative_to(self.venv) + # purelib + self.assert_venv_path_exists(site_packages_rel / "whl_with_data1/data_file.txt") + + # platlib + self.assert_venv_path_exists( + site_packages_rel / "whl_with_data1/platlib_file.txt" + ) + + venv_root = self.venv # noqa: F841 + + # data + self.assert_venv_path_exists("whl_with_data1/data_data_file.txt") + + # scripts + self.assert_venv_path_exists(self.bin_dir_name / "whl_script.sh") + + # headers + self.assert_venv_path_exists( + self.include_dir_name / "whl_with_data1/header_file.h" + ) + + def test_whl_with_data2_included(self): + module = self.assert_imported_from_venv("whl_with_data2") # noqa: F841 + + site_packages_rel = self.site_packages.relative_to(self.venv) + self.assert_venv_path_exists(site_packages_rel / "whl_with_data2/data_file.txt") + + self.assert_venv_path_exists(self.bin_dir_name / "whl_script.sh") + + # Ensure that `data` files are unpacked in `venv/root/` + # and then linked as `venv/whl_with_data1/data_data_file.txt`. + self.assert_venv_path_exists("whl_with_data2/data_data_file.txt") + + self.assert_venv_path_exists( + self.include_dir_name / "whl_with_data2/header_file.h" + ) + + def test_whl_with_data_overlap(self): + self.assert_venv_path_exists("overlap/both.txt") + self.assert_venv_path_exists("overlap/data1.txt") + self.assert_venv_path_exists("overlap/data2.txt") + + self.assert_venv_path_exists(self.bin_dir_name / "overlap/both.sh") + self.assert_venv_path_exists(self.bin_dir_name / "overlap/script1.sh") + self.assert_venv_path_exists(self.bin_dir_name / "overlap/script2.sh") + + self.assert_venv_path_exists(self.include_dir_name / "overlap/both.h") + self.assert_venv_path_exists(self.include_dir_name / "overlap/header1.h") + self.assert_venv_path_exists(self.include_dir_name / "overlap/header2.h") + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/venv_site_packages_libs/ext_with_libs/BUILD.bazel b/tests/venv_site_packages_libs/ext_with_libs/BUILD.bazel new file mode 100644 index 0000000000..a3a277dbb4 --- /dev/null +++ b/tests/venv_site_packages_libs/ext_with_libs/BUILD.bazel @@ -0,0 +1,94 @@ +load("@rules_cc//cc:cc_library.bzl", "cc_library") +load("@rules_cc//cc:cc_shared_library.bzl", "cc_shared_library") +load("//python:py_library.bzl", "py_library") +load("//tests/support:copy_file.bzl", "copy_file_to_dir") + +package( + default_visibility = ["//visibility:public"], +) + +cc_library( + name = "increment_impl", + srcs = ["increment.c"], + deps = [":increment_headers"], +) + +cc_library( + name = "increment_headers", + hdrs = ["increment.h"], +) + +cc_shared_library( + name = "increment", + user_link_flags = select({ + "@platforms//os:osx": [ + # Needed so that DT_NEEDED=libincrement.dylib can find + # this shared library + "-Wl,-install_name,@rpath/libincrement.dylib", + ], + "//conditions:default": [], + }), + deps = [":increment_impl"], +) + +cc_library( + name = "adder_impl", + srcs = ["adder.c"], + deps = [ + ":increment_headers", + "@rules_python//python/cc:current_py_cc_headers", + ], +) + +cc_shared_library( + name = "adder", + # Necessary for several reasons: + # 1. Ensures the output doesn't include increment itself (avoids ODRs) + # 2. Adds -lincrement (DT_NEEDED for libincrement.so) + # 3. Ensures libincrement.so is available at link time to satisfy (2) + dynamic_deps = [":increment"], + shared_lib_name = "adder.so", + tags = ["manual"], + # NOTE: cc_shared_library adds Bazelized rpath entries, too. + user_link_flags = [ + ] + select({ + "@platforms//os:osx": [ + "-Wl,-rpath,@loader_path/libs", + "-undefined", + "dynamic_lookup", + "-Wl,-exported_symbol", + "-Wl,_PyInit_adder", + ], + # Assume linux default + "//conditions:default": [ + "-Wl,-rpath,$ORIGIN/libs", + ], + }), + deps = [":adder_impl"], +) + +copy_file_to_dir( + name = "relocate_adder", + src = ":adder", + out_dir = "site-packages/ext_with_libs", + tags = ["manual"], +) + +copy_file_to_dir( + name = "relocate_increment", + src = ":increment", + out_dir = "site-packages/ext_with_libs/libs", + tags = ["manual"], +) + +py_library( + name = "ext_with_libs", + srcs = glob(["site-packages/**/*.py"]), + data = [ + ":relocate_adder", + ":relocate_increment", + ], + experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", + imports = ["site-packages"], + tags = ["manual"], +) diff --git a/tests/venv_site_packages_libs/ext_with_libs/adder.c b/tests/venv_site_packages_libs/ext_with_libs/adder.c new file mode 100644 index 0000000000..8b04b1721f --- /dev/null +++ b/tests/venv_site_packages_libs/ext_with_libs/adder.c @@ -0,0 +1,15 @@ +#include + +#include "increment.h" + +static PyObject *do_add(PyObject *self, PyObject *Py_UNUSED(args)) { + return PyLong_FromLong(increment(1)); +} + +static PyMethodDef AdderMethods[] = { + {"do_add", do_add, METH_NOARGS, "Add one"}, {NULL, NULL, 0, NULL}}; + +static struct PyModuleDef addermodule = {PyModuleDef_HEAD_INIT, "adder", NULL, + -1, AdderMethods}; + +PyMODINIT_FUNC PyInit_adder(void) { return PyModule_Create(&addermodule); } diff --git a/tests/venv_site_packages_libs/ext_with_libs/increment.c b/tests/venv_site_packages_libs/ext_with_libs/increment.c new file mode 100644 index 0000000000..b194325ac7 --- /dev/null +++ b/tests/venv_site_packages_libs/ext_with_libs/increment.c @@ -0,0 +1,3 @@ +#include "increment.h" + +int increment(int val) { return val + 1; } diff --git a/tests/venv_site_packages_libs/ext_with_libs/increment.h b/tests/venv_site_packages_libs/ext_with_libs/increment.h new file mode 100644 index 0000000000..8a13bf5621 --- /dev/null +++ b/tests/venv_site_packages_libs/ext_with_libs/increment.h @@ -0,0 +1,6 @@ +#ifndef TESTS_VENV_SITE_PACKAGES_LIBS_EXT_WITH_LIBS_INCREMENT_H_ +#define TESTS_VENV_SITE_PACKAGES_LIBS_EXT_WITH_LIBS_INCREMENT_H_ + +int increment(int); + +#endif // TESTS_VENV_SITE_PACKAGES_LIBS_EXT_WITH_LIBS_INCREMENT_H_ diff --git a/tests/venv_site_packages_libs/ext_with_libs/site-packages/ext_with_libs/__init__.py b/tests/venv_site_packages_libs/ext_with_libs/site-packages/ext_with_libs/__init__.py new file mode 100644 index 0000000000..ea0485cb1b --- /dev/null +++ b/tests/venv_site_packages_libs/ext_with_libs/site-packages/ext_with_libs/__init__.py @@ -0,0 +1,2 @@ +# This just marks the directory as a Pyton package. Python C extension modules +# and C libraries are populated in this directory at build time. diff --git a/tests/venv_site_packages_libs/importlib_metadata_test.py b/tests/venv_site_packages_libs/importlib_metadata_test.py new file mode 100644 index 0000000000..963d43b6e0 --- /dev/null +++ b/tests/venv_site_packages_libs/importlib_metadata_test.py @@ -0,0 +1,23 @@ +import importlib.metadata +import unittest + + +class ImportlibMetadataTest(unittest.TestCase): + def test_importlib_metadata_files(self): + files = importlib.metadata.files("whl-with-data1") + self.assertIsNotNone(files, "importlib.metadata.files returned None") + self.assertGreater( + len(files), 0, "importlib.metadata.files returned empty list" + ) + + # Verify it contains some expected files. + # The RECORD file lists paths relative to the installation root (site-packages). + # whl_with_data1-1.0.data/purelib/data_overlap.py should be installed as data_overlap.py + # whl_with_data1-1.0.data/platlib/whl_with_data1/platlib_file.txt should be whl_with_data1/platlib_file.txt + + file_names = [f.name for f in files] + self.assertIn("data_overlap.py", file_names) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/venv_site_packages_libs/nested_with_pth/BUILD.bazel b/tests/venv_site_packages_libs/nested_with_pth/BUILD.bazel new file mode 100644 index 0000000000..f0339270b4 --- /dev/null +++ b/tests/venv_site_packages_libs/nested_with_pth/BUILD.bazel @@ -0,0 +1,11 @@ +load("//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "nested_with_pth", + srcs = glob(["site-packages/**/*.py"]), + data = glob(["site-packages/*.pth"]), + experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", + imports = ["site-packages"], +) diff --git a/tests/venv_site_packages_libs/nested_with_pth/site-packages/nested.pth b/tests/venv_site_packages_libs/nested_with_pth/site-packages/nested.pth new file mode 100644 index 0000000000..924e0dccc8 --- /dev/null +++ b/tests/venv_site_packages_libs/nested_with_pth/site-packages/nested.pth @@ -0,0 +1 @@ +nested_sdk diff --git a/tests/venv_site_packages_libs/nested_with_pth/site-packages/nested_sdk/nested_with_pth/__init__.py b/tests/venv_site_packages_libs/nested_with_pth/site-packages/nested_sdk/nested_with_pth/__init__.py new file mode 100644 index 0000000000..adde64d3ef --- /dev/null +++ b/tests/venv_site_packages_libs/nested_with_pth/site-packages/nested_sdk/nested_with_pth/__init__.py @@ -0,0 +1 @@ +WHOAMI = "nested_with_pth" diff --git a/tests/venv_site_packages_libs/nspkg_alpha/BUILD.bazel b/tests/venv_site_packages_libs/nspkg_alpha/BUILD.bazel new file mode 100644 index 0000000000..9c0aa192a9 --- /dev/null +++ b/tests/venv_site_packages_libs/nspkg_alpha/BUILD.bazel @@ -0,0 +1,10 @@ +load("//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "nspkg_alpha", + srcs = glob(["site-packages/**/*.py"]), + experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", + imports = ["site-packages"], +) diff --git a/tests/venv_site_packages_libs/nspkg_alpha/site-packages/nspkg/subnspkg/alpha/__init__.py b/tests/venv_site_packages_libs/nspkg_alpha/site-packages/nspkg/subnspkg/alpha/__init__.py new file mode 100644 index 0000000000..b5ee093672 --- /dev/null +++ b/tests/venv_site_packages_libs/nspkg_alpha/site-packages/nspkg/subnspkg/alpha/__init__.py @@ -0,0 +1 @@ +whoami = "alpha" diff --git a/tests/venv_site_packages_libs/nspkg_beta/BUILD.bazel b/tests/venv_site_packages_libs/nspkg_beta/BUILD.bazel new file mode 100644 index 0000000000..d67ebf57d6 --- /dev/null +++ b/tests/venv_site_packages_libs/nspkg_beta/BUILD.bazel @@ -0,0 +1,10 @@ +load("@rules_python//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "nspkg_beta", + srcs = glob(["site-packages/**/*.py"]), + experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", + imports = ["site-packages"], +) diff --git a/tests/venv_site_packages_libs/nspkg_beta/site-packages/nspkg/subnspkg/beta/__init__.py b/tests/venv_site_packages_libs/nspkg_beta/site-packages/nspkg/subnspkg/beta/__init__.py new file mode 100644 index 0000000000..a2a65910c7 --- /dev/null +++ b/tests/venv_site_packages_libs/nspkg_beta/site-packages/nspkg/subnspkg/beta/__init__.py @@ -0,0 +1 @@ +whoami = "beta" diff --git a/tests/venv_site_packages_libs/pkgutil_top/BUILD.bazel b/tests/venv_site_packages_libs/pkgutil_top/BUILD.bazel new file mode 100644 index 0000000000..7a0d961ad8 --- /dev/null +++ b/tests/venv_site_packages_libs/pkgutil_top/BUILD.bazel @@ -0,0 +1,10 @@ +load("//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "pkgutil_top", + srcs = glob(["site-packages/**/*.py"]), + experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", + imports = ["site-packages"], +) diff --git a/tests/venv_site_packages_libs/pkgutil_top/site-packages/pkgutil_top/__init__.py b/tests/venv_site_packages_libs/pkgutil_top/site-packages/pkgutil_top/__init__.py new file mode 100644 index 0000000000..e1809a325a --- /dev/null +++ b/tests/venv_site_packages_libs/pkgutil_top/site-packages/pkgutil_top/__init__.py @@ -0,0 +1,2 @@ +WHOAMI = "pkgutil_top" +__path__ = __import__("pkgutil").extend_path(__path__, __name__) diff --git a/tests/venv_site_packages_libs/pkgutil_top/site-packages/pkgutil_top/top.py b/tests/venv_site_packages_libs/pkgutil_top/site-packages/pkgutil_top/top.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/venv_site_packages_libs/pkgutil_top_sub/BUILD.bazel b/tests/venv_site_packages_libs/pkgutil_top_sub/BUILD.bazel new file mode 100644 index 0000000000..52980fa790 --- /dev/null +++ b/tests/venv_site_packages_libs/pkgutil_top_sub/BUILD.bazel @@ -0,0 +1,10 @@ +load("//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "pkgutil_top_sub", + srcs = glob(["site-packages/**/*.py"]), + experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", + imports = ["site-packages"], +) diff --git a/tests/venv_site_packages_libs/pkgutil_top_sub/site-packages/pkgutil_top/sub/__init__.py b/tests/venv_site_packages_libs/pkgutil_top_sub/site-packages/pkgutil_top/sub/__init__.py new file mode 100644 index 0000000000..e7fb2340ea --- /dev/null +++ b/tests/venv_site_packages_libs/pkgutil_top_sub/site-packages/pkgutil_top/sub/__init__.py @@ -0,0 +1 @@ +WHOAMI = "pkgutil_top.sub" diff --git a/tests/venv_site_packages_libs/pkgutil_top_sub/site-packages/pkgutil_top/sub/suba.py b/tests/venv_site_packages_libs/pkgutil_top_sub/site-packages/pkgutil_top/sub/suba.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/venv_site_packages_libs/py_binary_other_module_test.sh b/tests/venv_site_packages_libs/py_binary_other_module_test.sh new file mode 100755 index 0000000000..9fb71d5bc7 --- /dev/null +++ b/tests/venv_site_packages_libs/py_binary_other_module_test.sh @@ -0,0 +1,6 @@ +# Test that for a py_binary from a dependency module, we place links created via +# runfiles(...) in the right place. This tests the fix made for issues/3503 + +set -eu +echo "[*] Testing running the binary" +"$VENV_BIN" diff --git a/tests/venv_site_packages_libs/shared_lib_loading_test.py b/tests/venv_site_packages_libs/shared_lib_loading_test.py new file mode 100644 index 0000000000..a3f7bfcd5a --- /dev/null +++ b/tests/venv_site_packages_libs/shared_lib_loading_test.py @@ -0,0 +1,170 @@ +import importlib.util +import os +import sys +import unittest +from pathlib import Path + +# Optional imports for ELF/Mach-O analysis +if os.name == "posix" and sys.platform != "darwin": + from elftools.elf.elffile import ELFFile +else: + ELFFile = None + +if sys.platform == "darwin": + from macholib import mach_o + from macholib.MachO import MachO +else: + mach_o = None + MachO = None + +ELF_MAGIC = b"\x7fELF" +MACHO_MAGICS = ( + b"\xce\xfa\xed\xfe", # 32-bit big-endian + b"\xcf\xfa\xed\xfe", # 64-bit big-endian + b"\xfe\xed\xfa\xce", # 32-bit little-endian + b"\xfe\xed\xfa\xcf", # 64-bit little-endian +) + + +class SharedLibLoadingTest(unittest.TestCase): + def setUp(self): + super().setUp() + if sys.prefix == sys.base_prefix: + raise AssertionError("Not running under a venv") + self.venv = Path(sys.prefix) + + @unittest.skipIf(os.name == "nt", "Tests Unix-specific extension loading") + def test_shared_library_linking_unix(self): + try: + import ext_with_libs.adder + except ImportError as e: + spec = importlib.util.find_spec("ext_with_libs.adder") + if not spec or not spec.origin: + self.fail(f"Import failed and could not find module spec: {e}") + + info = self._get_linking_info(spec.origin) + + # Give a useful error message for debugging. + self.fail( + f"Failed to import adder extension.\n" + f"Original error: {e}\n" + f"Linking info for {spec.origin}:\n" + f" RPATHs: {info.get('rpaths', 'N/A')}\n" + f" Needed libs: {info.get('needed', 'N/A')}" + ) + + # Check that the module was loaded from the venv. + self.assertIn(".venv/", ext_with_libs.adder.__file__) + + adder_path = os.path.realpath(ext_with_libs.adder.__file__) + + with open(adder_path, "rb") as f: + magic_bytes = f.read(4) + + if magic_bytes == ELF_MAGIC: + self._assert_elf_linking(adder_path) + elif magic_bytes in MACHO_MAGICS: + self._assert_macho_linking(adder_path) + else: + self.fail(f"Unsupported file format for adder: magic bytes {magic_bytes!r}") + + # Check the function works regardless of format. + self.assertEqual(ext_with_libs.adder.do_add(), 2) + + @unittest.skipUnless(os.name == "nt", "Tests Windows-specific extension loading") + def test_shared_library_loading_windows(self): + # We import markupsafe._speedups (a .cp311-win_amd64.pyd extension) + try: + import markupsafe._speedups + + module = markupsafe._speedups + except ImportError as e: + self.fail( + f"Failed to import markupsafe._speedups: {e}\n" + + "sys.path:\n" + + "\n".join(sys.path) + ) + + # Verify it's in the venv + # Normalize paths for Windows comparison. + # We DON'T use resolve() here because we want to see the path Python used, + # which should be within the venv's site-packages (even if it's a symlink). + actual_file = str(Path(module.__file__)).lower() + expected_prefix = str(self.venv).lower() + + self.assertTrue( + actual_file.startswith(expected_prefix), + f"Module {module.__name__} not loaded from venv.\n" + f"Venv: {expected_prefix}\n" + f"Module file: {actual_file}\n" + f"sys.path:\n" + "\n".join(sys.path), + ) + + # Verify it's a shared library (.pyd) + self.assertTrue( + actual_file.endswith(".pyd"), + f"Expected .pyd extension, got {module.__file__}", + ) + + def _get_linking_info(self, path): + """Parses a shared library and returns its rpaths and dependencies.""" + path = os.path.realpath(path) + with open(path, "rb") as f: + magic_bytes = f.read(4) + + if magic_bytes == ELF_MAGIC: + return self._get_elf_info(path) + elif magic_bytes in MACHO_MAGICS: + return self._get_macho_info(path) + return {} + + def _get_elf_info(self, path): + """Extracts linking information from an ELF file.""" + info = {"rpaths": [], "needed": [], "undefined_symbols": []} + with open(path, "rb") as f: + elf = ELFFile(f) + dynamic = elf.get_section_by_name(".dynamic") + if dynamic: + for tag in dynamic.iter_tags(): + if tag.entry.d_tag == "DT_NEEDED": + info["needed"].append(tag.needed) + elif tag.entry.d_tag == "DT_RPATH": + info["rpaths"].append(tag.rpath) + elif tag.entry.d_tag == "DT_RUNPATH": + info["rpaths"].append(tag.runpath) + + dynsym = elf.get_section_by_name(".dynsym") + if dynsym: + info["undefined_symbols"] = [ + s.name + for s in dynsym.iter_symbols() + if s.entry["st_shndx"] == "SHN_UNDEF" + ] + return info + + def _get_macho_info(self, path): + """Extracts linking information from a Mach-O file.""" + info = {"rpaths": [], "needed": []} + macho = MachO(path) + for header in macho.headers: + for cmd_load, cmd, data in header.commands: + if cmd_load.cmd == mach_o.LC_LOAD_DYLIB: + info["needed"].append(data.decode().strip("\x00")) + elif cmd_load.cmd == mach_o.LC_RPATH: + info["rpaths"].append(data.decode().strip("\x00")) + return info + + def _assert_elf_linking(self, path): + """Asserts dynamic linking properties for an ELF file.""" + info = self._get_elf_info(path) + self.assertIn("libincrement.so", info["needed"]) + self.assertIn("increment", info["undefined_symbols"]) + + def _assert_macho_linking(self, path): + """Asserts dynamic linking properties for a Mach-O file.""" + info = self._get_macho_info(path) + self.assertIn("@rpath/libincrement.dylib", info["needed"]) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/venv_site_packages_libs/whl_scripts_runnable_test.py b/tests/venv_site_packages_libs/whl_scripts_runnable_test.py new file mode 100644 index 0000000000..a0c4210f71 --- /dev/null +++ b/tests/venv_site_packages_libs/whl_scripts_runnable_test.py @@ -0,0 +1,116 @@ +import os +import subprocess +import sys +import tempfile +import unittest +from pathlib import Path + +BAZEL_8_OR_LATER = bool(int(os.environ.get("BAZEL_8_OR_LATER", "0"))) + + +class WhlScriptsRunnableTest(unittest.TestCase): + maxDiff = None + + def _get_script_path(self, name): + is_windows = sys.platform == "win32" + if is_windows: + bin_dir = Path(sys.prefix) / "Scripts" + pathexts = os.environ.get("PATHEXT", ".COM;.EXE;.BAT;.CMD").split(";") + for ext in [""] + [e.lower() for e in pathexts]: + script_path = bin_dir / f"{name}{ext}" + if script_path.exists(): + return script_path + return bin_dir / name + else: + bin_dir = Path(sys.prefix) / "bin" + script_path = bin_dir / name + return script_path + + def test_script_is_runnable(self): + script_path = self._get_script_path("whl_with_data1_script") + self.assertTrue(script_path.exists(), f"Script not found at {script_path}") + + result = subprocess.run( + [str(script_path)], + capture_output=True, + text=True, + check=True, + ) + + output = result.stdout.splitlines() + self.assertIn("hello from whl_with_data1_script", output) + + # The script prints sys.executable as its second line + # Depending on how it's invoked, it might have more output, + # but the user said it prints the hello message AND sys.executable. + script_executable = output[-1].strip() + self.assertEqual(script_executable, sys.executable) + + def test_entry_point_is_runnable(self): + script_path = self._get_script_path("whl_with_data2_bin") + self.assertTrue(script_path.exists(), f"Entry point not found at {script_path}") + + result = subprocess.run( + [str(script_path)], + capture_output=True, + text=True, + check=True, + ) + + output = result.stdout.splitlines() + self.assertIn("hello from whl_with_data2_bin", output) + + script_executable = output[-1].strip() + self.assertEqual(script_executable, sys.executable) + + def test_pythonw_script(self): + script_path = self._get_script_path("whl_with_data1_pythonw") + self.assertTrue(script_path.exists(), f"Script not found at {script_path}") + + with open(script_path, "r", encoding="utf-8") as f: + first_line = f.readline() + + is_windows = sys.platform == "win32" + if is_windows: + # On Windows, the shebang is replaced with a batch wrapper that + # invokes the interpreter. + self.assertIn("pythonw.exe", first_line) + self.assertTrue( + first_line.startswith("@setlocal") + or first_line.startswith("@echo off"), + f"Expected Windows batch wrapper, got {first_line}", + ) + else: + self.assertTrue( + first_line.startswith("#!/bin/sh"), + f"Expected #!/bin/sh, got {first_line}", + ) + + # For some reason, on Windows, the subprocess can't write + # to the temporary files unless mkstemp is used. + temp_fd, temp_str = tempfile.mkstemp() + try: + os.close(temp_fd) + out_path = Path(temp_str) + subprocess.run( + [str(script_path), str(out_path)], + capture_output=True, + text=True, + check=True, + ) + output = out_path.read_text().splitlines() + finally: + os.unlink(temp_str) + self.assertIn("hello from whl_with_data1_pythonw", output) + + script_executable = output[-1].strip() + + if is_windows: + self.assertTrue( + script_executable.endswith("pythonw.exe"), + f"Expected pythonw.exe, got {script_executable}", + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/version/BUILD.bazel b/tests/version/BUILD.bazel new file mode 100644 index 0000000000..d6fdecd4cf --- /dev/null +++ b/tests/version/BUILD.bazel @@ -0,0 +1,3 @@ +load(":version_test.bzl", "version_test_suite") + +version_test_suite(name = "version_tests") diff --git a/tests/version/version_test.bzl b/tests/version/version_test.bzl new file mode 100644 index 0000000000..7ddb6cc851 --- /dev/null +++ b/tests/version/version_test.bzl @@ -0,0 +1,165 @@ +"" + +load("@rules_testing//lib:analysis_test.bzl", "test_suite") +load("//python/private:version.bzl", "version") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_normalization(env): + prefixes = ["v", " v", " \t\r\nv"] + epochs = { + "": ["", "0!", "00!"], + "1!": ["1!", "001!"], + "200!": ["200!", "00200!"], + } + releases = { + "0.1": ["0.1", "0.01"], + "2023.7.19": ["2023.7.19", "2023.07.19"], + } + pres = { + "": [""], + "a0": ["a", ".a", "-ALPHA0", "_alpha0", ".a0"], + "a4": ["alpha4", ".a04"], + "b0": ["b", ".b", "-BETA0", "_beta0", ".b0"], + "b5": ["beta05", ".b5"], + "rc0": ["C", "_c0", "RC", "_rc0", "-preview_0"], + } + explicit_posts = { + "": [""], + ".post0": [], + ".post1": [".post1", "-r1", "_rev1"], + } + implicit_posts = [[".post1", "-1"], [".post2", "-2"]] + devs = { + "": [""], + ".dev0": ["dev", "-DEV", "_Dev-0"], + ".dev9": ["DEV9", ".dev09", ".dev9"], + ".dev{BUILD_TIMESTAMP}": [ + "-DEV{BUILD_TIMESTAMP}", + "_dev_{BUILD_TIMESTAMP}", + ], + } + locals = { + "": [""], + "+ubuntu.7": ["+Ubuntu_7", "+ubuntu-007"], + "+ubuntu.r007": ["+Ubuntu_R007"], + } + epochs = [ + [normalized_epoch, input_epoch] + for normalized_epoch, input_epochs in epochs.items() + for input_epoch in input_epochs + ] + releases = [ + [normalized_release, input_release] + for normalized_release, input_releases in releases.items() + for input_release in input_releases + ] + pres = [ + [normalized_pre, input_pre] + for normalized_pre, input_pres in pres.items() + for input_pre in input_pres + ] + explicit_posts = [ + [normalized_post, input_post] + for normalized_post, input_posts in explicit_posts.items() + for input_post in input_posts + ] + pres_and_posts = [ + [normalized_pre + normalized_post, input_pre + input_post] + for normalized_pre, input_pre in pres + for normalized_post, input_post in explicit_posts + ] + [ + [normalized_pre + normalized_post, input_pre + input_post] + for normalized_pre, input_pre in pres + for normalized_post, input_post in implicit_posts + if input_pre == "" or input_pre[-1].isdigit() + ] + devs = [ + [normalized_dev, input_dev] + for normalized_dev, input_devs in devs.items() + for input_dev in input_devs + ] + locals = [ + [normalized_local, input_local] + for normalized_local, input_locals in locals.items() + for input_local in input_locals + ] + postfixes = ["", " ", " \t\r\n"] + i = 0 + for nepoch, iepoch in epochs: + for nrelease, irelease in releases: + for nprepost, iprepost in pres_and_posts: + for ndev, idev in devs: + for nlocal, ilocal in locals: + prefix = prefixes[i % len(prefixes)] + postfix = postfixes[(i // len(prefixes)) % len(postfixes)] + env.expect.that_str( + version.normalize( + prefix + iepoch + irelease + iprepost + + idev + ilocal + postfix, + ), + ).equals( + nepoch + nrelease + nprepost + ndev + nlocal, + ) + i += 1 + +_tests.append(_test_normalization) + +def _test_normalize_local(env): + # Verify a local with a [digit][non-digit] sequence parses ok + in_str = "0.1.0+brt.9e" + actual = version.normalize(in_str) + env.expect.that_str(actual).equals(in_str) + +_tests.append(_test_normalize_local) + +def _test_ordering(env): + want = [ + # Taken from https://peps.python.org/pep-0440/#summary-of-permitted-suffixes-and-relative-ordering + "1.dev0", + "1.0.dev456", + "1.0a1", + "1.0a2.dev456", + "1.0a12.dev456", + "1.0a12", + "1.0b1.dev456", + "1.0b1.dev457", + "1.0b2", + "1.0b2.post345.dev456", + "1.0b2.post345.dev457", + "1.0b2.post345", + "1.0rc1.dev456", + "1.0rc1", + "1.0", + "1.0+abc.5", + "1.0+abc.7", + "1.0+5", + "1.0.post456.dev34", + "1.0.post456", + "1.0.15", + "1.1.dev1", + "1!0.1", + ] + + for lower, higher in zip(want[:-1], want[1:]): + lower = version.parse(lower, strict = True) + higher = version.parse(higher, strict = True) + + lower_key = version.key(lower) + higher_key = version.key(higher) + + if not lower_key < higher_key: + env.fail("Expected '{}'.key() to be smaller than '{}'.key(), but got otherwise: {} > {}".format( + lower.string, + higher.string, + lower_key, + higher_key, + )) + +_tests.append(_test_ordering) + +def version_test_suite(name): + test_suite( + name = name, + basic_tests = _tests, + ) diff --git a/tests/whl_filegroup/BUILD.bazel b/tests/whl_filegroup/BUILD.bazel index 2176e9e03a..61c1aa49ac 100644 --- a/tests/whl_filegroup/BUILD.bazel +++ b/tests/whl_filegroup/BUILD.bazel @@ -1,9 +1,10 @@ load("@bazel_skylib//rules:write_file.bzl", "write_file") load("@rules_cc//cc:cc_library.bzl", "cc_library") load("@rules_cc//cc:cc_test.bzl", "cc_test") -load("//python:defs.bzl", "py_library", "py_test") load("//python:packaging.bzl", "py_package", "py_wheel") load("//python:pip.bzl", "whl_filegroup") +load("//python:py_library.bzl", "py_library") +load("//python:py_test.bzl", "py_test") load(":whl_filegroup_tests.bzl", "whl_filegroup_test_suite") whl_filegroup_test_suite(name = "whl_filegroup_tests") diff --git a/tests/whl_filegroup/extract_wheel_files_test.py b/tests/whl_filegroup/extract_wheel_files_test.py index 434899d5cf..4bf1bf3f11 100644 --- a/tests/whl_filegroup/extract_wheel_files_test.py +++ b/tests/whl_filegroup/extract_wheel_files_test.py @@ -14,6 +14,8 @@ def test_get_wheel_record(self) -> None: "examples/wheel/lib/data,with,commas.txt", "examples/wheel/lib/data.txt", "examples/wheel/lib/module_with_data.py", + "examples/wheel/lib/module_with_type_annotations.py", + "examples/wheel/lib/module_with_type_annotations.pyi", "examples/wheel/lib/simple_module.py", "examples/wheel/main.py", "example_minimal_package-0.0.1.dist-info/WHEEL", @@ -24,7 +26,7 @@ def test_get_wheel_record(self) -> None: self.assertEqual(list(record), list(expected)) def test_get_files(self) -> None: - pattern = "(examples/wheel/lib/.*\.txt$|.*main)" + pattern = r"(examples/wheel/lib/.*\.txt$|.*main)" record = extract_wheel_files.get_record(_WHEEL) files = extract_wheel_files.get_files(record, pattern) expected = [ diff --git a/tests/whl_with_build_files/BUILD.bazel b/tests/whl_with_build_files/BUILD.bazel new file mode 100644 index 0000000000..1202876485 --- /dev/null +++ b/tests/whl_with_build_files/BUILD.bazel @@ -0,0 +1,9 @@ +load("//python:py_test.bzl", "py_test") +load("//tests/support:support.bzl", "SUPPORTS_BZLMOD") + +py_test( + name = "verify_files_test", + srcs = ["verify_files_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, + deps = ["@somepkg_with_build_files//:pkg"], +) diff --git a/tests/whl_with_build_files/testdata/BUILD b/tests/whl_with_build_files/testdata/BUILD new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/BUILD.bazel b/tests/whl_with_build_files/testdata/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/REPO.bazel b/tests/whl_with_build_files/testdata/REPO.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/BUILD b/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/BUILD new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/BUILD.bazel b/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/METADATA b/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/METADATA new file mode 100644 index 0000000000..0829d1f756 --- /dev/null +++ b/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/METADATA @@ -0,0 +1,2 @@ +Name: somepkg +Version: 1.0 diff --git a/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/RECORD b/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/RECORD new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/WHEEL b/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/WHEEL new file mode 100644 index 0000000000..a64521a1cc --- /dev/null +++ b/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/WHEEL @@ -0,0 +1 @@ +Wheel-Version: 1.0 diff --git a/tests/whl_with_build_files/testdata/somepkg/BUILD b/tests/whl_with_build_files/testdata/somepkg/BUILD new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg/BUILD.bazel b/tests/whl_with_build_files/testdata/somepkg/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg/__init__.py b/tests/whl_with_build_files/testdata/somepkg/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg/a.py b/tests/whl_with_build_files/testdata/somepkg/a.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg/subpkg/BUILD b/tests/whl_with_build_files/testdata/somepkg/subpkg/BUILD new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg/subpkg/BUILD.bazel b/tests/whl_with_build_files/testdata/somepkg/subpkg/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg/subpkg/__init__.py b/tests/whl_with_build_files/testdata/somepkg/subpkg/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg/subpkg/b.py b/tests/whl_with_build_files/testdata/somepkg/subpkg/b.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/verify_files_test.py b/tests/whl_with_build_files/verify_files_test.py new file mode 100644 index 0000000000..8d16b34348 --- /dev/null +++ b/tests/whl_with_build_files/verify_files_test.py @@ -0,0 +1,16 @@ +import unittest + + +class VerifyFilestest(unittest.TestCase): + def test_wheel_with_build_files_importable(self): + # If the BUILD files are present, then these imports should fail + # because globs won't pass package boundaries, and the necessary + # py files end up missing in runfiles. + import somepkg + import somepkg.a + import somepkg.subpkg + import somepkg.subpkg.b # noqa: F401 + + +if __name__ == "__main__": + unittest.main() diff --git a/third_party/rules_pycross/LICENSE b/third_party/rules_pycross/LICENSE deleted file mode 100644 index 261eeb9e9f..0000000000 --- a/third_party/rules_pycross/LICENSE +++ /dev/null @@ -1,201 +0,0 @@ - Apache License - Version 2.0, January 2004 - http://www.apache.org/licenses/ - - TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION - - 1. Definitions. - - "License" shall mean the terms and conditions for use, reproduction, - and distribution as defined by Sections 1 through 9 of this document. - - "Licensor" shall mean the copyright owner or entity authorized by - the copyright owner that is granting the License. - - "Legal Entity" shall mean the union of the acting entity and all - other entities that control, are controlled by, or are under common - control with that entity. For the purposes of this definition, - "control" means (i) the power, direct or indirect, to cause the - direction or management of such entity, whether by contract or - otherwise, or (ii) ownership of fifty percent (50%) or more of the - outstanding shares, or (iii) beneficial ownership of such entity. - - "You" (or "Your") shall mean an individual or Legal Entity - exercising permissions granted by this License. - - "Source" form shall mean the preferred form for making modifications, - including but not limited to software source code, documentation - source, and configuration files. - - "Object" form shall mean any form resulting from mechanical - transformation or translation of a Source form, including but - not limited to compiled object code, generated documentation, - and conversions to other media types. - - "Work" shall mean the work of authorship, whether in Source or - Object form, made available under the License, as indicated by a - copyright notice that is included in or attached to the work - (an example is provided in the Appendix below). - - "Derivative Works" shall mean any work, whether in Source or Object - form, that is based on (or derived from) the Work and for which the - editorial revisions, annotations, elaborations, or other modifications - represent, as a whole, an original work of authorship. For the purposes - of this License, Derivative Works shall not include works that remain - separable from, or merely link (or bind by name) to the interfaces of, - the Work and Derivative Works thereof. - - "Contribution" shall mean any work of authorship, including - the original version of the Work and any modifications or additions - to that Work or Derivative Works thereof, that is intentionally - submitted to Licensor for inclusion in the Work by the copyright owner - or by an individual or Legal Entity authorized to submit on behalf of - the copyright owner. For the purposes of this definition, "submitted" - means any form of electronic, verbal, or written communication sent - to the Licensor or its representatives, including but not limited to - communication on electronic mailing lists, source code control systems, - and issue tracking systems that are managed by, or on behalf of, the - Licensor for the purpose of discussing and improving the Work, but - excluding communication that is conspicuously marked or otherwise - designated in writing by the copyright owner as "Not a Contribution." - - "Contributor" shall mean Licensor and any individual or Legal Entity - on behalf of whom a Contribution has been received by Licensor and - subsequently incorporated within the Work. - - 2. Grant of Copyright License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - copyright license to reproduce, prepare Derivative Works of, - publicly display, publicly perform, sublicense, and distribute the - Work and such Derivative Works in Source or Object form. - - 3. Grant of Patent License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - (except as stated in this section) patent license to make, have made, - use, offer to sell, sell, import, and otherwise transfer the Work, - where such license applies only to those patent claims licensable - by such Contributor that are necessarily infringed by their - Contribution(s) alone or by combination of their Contribution(s) - with the Work to which such Contribution(s) was submitted. If You - institute patent litigation against any entity (including a - cross-claim or counterclaim in a lawsuit) alleging that the Work - or a Contribution incorporated within the Work constitutes direct - or contributory patent infringement, then any patent licenses - granted to You under this License for that Work shall terminate - as of the date such litigation is filed. - - 4. Redistribution. You may reproduce and distribute copies of the - Work or Derivative Works thereof in any medium, with or without - modifications, and in Source or Object form, provided that You - meet the following conditions: - - (a) You must give any other recipients of the Work or - Derivative Works a copy of this License; and - - (b) You must cause any modified files to carry prominent notices - stating that You changed the files; and - - (c) You must retain, in the Source form of any Derivative Works - that You distribute, all copyright, patent, trademark, and - attribution notices from the Source form of the Work, - excluding those notices that do not pertain to any part of - the Derivative Works; and - - (d) If the Work includes a "NOTICE" text file as part of its - distribution, then any Derivative Works that You distribute must - include a readable copy of the attribution notices contained - within such NOTICE file, excluding those notices that do not - pertain to any part of the Derivative Works, in at least one - of the following places: within a NOTICE text file distributed - as part of the Derivative Works; within the Source form or - documentation, if provided along with the Derivative Works; or, - within a display generated by the Derivative Works, if and - wherever such third-party notices normally appear. The contents - of the NOTICE file are for informational purposes only and - do not modify the License. You may add Your own attribution - notices within Derivative Works that You distribute, alongside - or as an addendum to the NOTICE text from the Work, provided - that such additional attribution notices cannot be construed - as modifying the License. - - You may add Your own copyright statement to Your modifications and - may provide additional or different license terms and conditions - for use, reproduction, or distribution of Your modifications, or - for any such Derivative Works as a whole, provided Your use, - reproduction, and distribution of the Work otherwise complies with - the conditions stated in this License. - - 5. Submission of Contributions. Unless You explicitly state otherwise, - any Contribution intentionally submitted for inclusion in the Work - by You to the Licensor shall be under the terms and conditions of - this License, without any additional terms or conditions. - Notwithstanding the above, nothing herein shall supersede or modify - the terms of any separate license agreement you may have executed - with Licensor regarding such Contributions. - - 6. Trademarks. This License does not grant permission to use the trade - names, trademarks, service marks, or product names of the Licensor, - except as required for reasonable and customary use in describing the - origin of the Work and reproducing the content of the NOTICE file. - - 7. Disclaimer of Warranty. Unless required by applicable law or - agreed to in writing, Licensor provides the Work (and each - Contributor provides its Contributions) on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or - implied, including, without limitation, any warranties or conditions - of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A - PARTICULAR PURPOSE. You are solely responsible for determining the - appropriateness of using or redistributing the Work and assume any - risks associated with Your exercise of permissions under this License. - - 8. Limitation of Liability. In no event and under no legal theory, - whether in tort (including negligence), contract, or otherwise, - unless required by applicable law (such as deliberate and grossly - negligent acts) or agreed to in writing, shall any Contributor be - liable to You for damages, including any direct, indirect, special, - incidental, or consequential damages of any character arising as a - result of this License or out of the use or inability to use the - Work (including but not limited to damages for loss of goodwill, - work stoppage, computer failure or malfunction, or any and all - other commercial damages or losses), even if such Contributor - has been advised of the possibility of such damages. - - 9. Accepting Warranty or Additional Liability. While redistributing - the Work or Derivative Works thereof, You may choose to offer, - and charge a fee for, acceptance of support, warranty, indemnity, - or other liability obligations and/or rights consistent with this - License. However, in accepting such obligations, You may act only - on Your own behalf and on Your sole responsibility, not on behalf - of any other Contributor, and only if You agree to indemnify, - defend, and hold each Contributor harmless for any liability - incurred by, or claims asserted against, such Contributor by reason - of your accepting any such warranty or additional liability. - - END OF TERMS AND CONDITIONS - - APPENDIX: How to apply the Apache License to your work. - - To apply the Apache License to your work, attach the following - boilerplate notice, with the fields enclosed by brackets "[]" - replaced with your own identifying information. (Don't include - the brackets!) The text should be enclosed in the appropriate - comment syntax for the file format. We also recommend that a - file or class name and description of purpose be included on the - same "printed page" as the copyright notice for easier - identification within third-party archives. - - Copyright [yyyy] [name of copyright owner] - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. diff --git a/third_party/rules_pycross/pycross/private/providers.bzl b/third_party/rules_pycross/pycross/private/providers.bzl deleted file mode 100644 index 47fc9f7271..0000000000 --- a/third_party/rules_pycross/pycross/private/providers.bzl +++ /dev/null @@ -1,32 +0,0 @@ -# Copyright 2023 Jeremy Volkman. All rights reserved. -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""Python providers.""" - -PyWheelInfo = provider( - doc = "Information about a Python wheel.", - fields = { - "name_file": "File: A file containing the canonical name of the wheel.", - "wheel_file": "File: The wheel file itself.", - }, -) - -PyTargetEnvironmentInfo = provider( - doc = "A target environment description.", - fields = { - "file": "The JSON file containing target environment information.", - "python_compatible_with": "A list of constraints used to select this platform.", - }, -) diff --git a/third_party/rules_pycross/pycross/private/tools/wheel_installer.py b/third_party/rules_pycross/pycross/private/tools/wheel_installer.py deleted file mode 100644 index c03c4c2523..0000000000 --- a/third_party/rules_pycross/pycross/private/tools/wheel_installer.py +++ /dev/null @@ -1,196 +0,0 @@ -# Copyright 2023 Jeremy Volkman. All rights reserved. -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -""" -A tool that invokes pypa/build to build the given sdist tarball. -""" - -import argparse -import os -import shutil -import subprocess -import sys -import tempfile -from pathlib import Path -from typing import Any - -from installer import install -from installer.destinations import SchemeDictionaryDestination -from installer.sources import WheelFile - -from python.private.pypi.whl_installer import namespace_pkgs - - -def setup_namespace_pkg_compatibility(wheel_dir: Path) -> None: - """Converts native namespace packages to pkgutil-style packages - - Namespace packages can be created in one of three ways. They are detailed here: - https://packaging.python.org/guides/packaging-namespace-packages/#creating-a-namespace-package - - 'pkgutil-style namespace packages' (2) and 'pkg_resources-style namespace packages' (3) works in Bazel, but - 'native namespace packages' (1) do not. - - We ensure compatibility with Bazel of method 1 by converting them into method 2. - - Args: - wheel_dir: the directory of the wheel to convert - """ - - namespace_pkg_dirs = namespace_pkgs.implicit_namespace_packages( - str(wheel_dir), - ignored_dirnames=["%s/bin" % wheel_dir], - ) - - for ns_pkg_dir in namespace_pkg_dirs: - namespace_pkgs.add_pkgutil_style_namespace_pkg_init(ns_pkg_dir) - - -def main(args: Any) -> None: - dest_dir = args.directory - lib_dir = dest_dir / "site-packages" - destination = SchemeDictionaryDestination( - scheme_dict={ - "platlib": str(lib_dir), - "purelib": str(lib_dir), - "headers": str(dest_dir / "include"), - "scripts": str(dest_dir / "bin"), - "data": str(dest_dir / "data"), - }, - interpreter="/usr/bin/env python3", # Generic; it's not feasible to run these scripts directly. - script_kind="posix", - bytecode_optimization_levels=[0, 1], - ) - - link_dir = Path(tempfile.mkdtemp()) - if args.wheel_name_file: - with open(args.wheel_name_file, "r") as f: - wheel_name = f.read().strip() - else: - wheel_name = os.path.basename(args.wheel) - - link_path = link_dir / wheel_name - os.symlink(os.path.join(os.getcwd(), args.wheel), link_path) - - try: - with WheelFile.open(link_path) as source: - install( - source=source, - destination=destination, - # Additional metadata that is generated by the installation tool. - additional_metadata={ - "INSTALLER": b"https://github.com/bazelbuild/rules_python/tree/main/third_party/rules_pycross", - }, - ) - finally: - shutil.rmtree(link_dir, ignore_errors=True) - - setup_namespace_pkg_compatibility(lib_dir) - - if args.patch: - if not args.patch_tool and not args.patch_tool_target: - raise ValueError("Specify one of 'patch_tool' or 'patch_tool_target'.") - - patch_args = [ - args.patch_tool or Path.cwd() / args.patch_tool_target - ] + args.patch_arg - for patch in args.patch: - with patch.open("r") as stdin: - try: - subprocess.run( - patch_args, - stdin=stdin, - check=True, - stdout=subprocess.PIPE, - stderr=subprocess.STDOUT, - cwd=args.directory, - ) - except subprocess.CalledProcessError as error: - print(f"Patch {patch} failed to apply:") - print(error.stdout.decode("utf-8")) - raise - - -def parse_flags(argv) -> Any: - parser = argparse.ArgumentParser(description="Extract a Python wheel.") - - parser.add_argument( - "--wheel", - type=Path, - required=True, - help="The wheel file path.", - ) - - parser.add_argument( - "--wheel-name-file", - type=Path, - required=False, - help="A file containing the canonical name of the wheel.", - ) - - parser.add_argument( - "--enable-implicit-namespace-pkgs", - action="store_true", - help="If true, disables conversion of implicit namespace packages and will unzip as-is.", - ) - - parser.add_argument( - "--directory", - type=Path, - help="The output path.", - ) - - parser.add_argument( - "--patch", - type=Path, - default=[], - action="append", - help="A patch file to apply.", - ) - - parser.add_argument( - "--patch-arg", - type=str, - default=[], - action="append", - help="An argument for the patch tool when applying the patches.", - ) - - parser.add_argument( - "--patch-tool", - type=str, - help=( - "The tool from PATH to invoke when applying patches. " - "If set, --patch-tool-target is ignored." - ), - ) - - parser.add_argument( - "--patch-tool-target", - type=Path, - help=( - "The path to the tool to invoke when applying patches. " - "Ignored when --patch-tool is set." - ), - ) - - return parser.parse_args(argv[1:]) - - -if __name__ == "__main__": - # When under `bazel run`, change to the actual working dir. - if "BUILD_WORKING_DIRECTORY" in os.environ: - os.chdir(os.environ["BUILD_WORKING_DIRECTORY"]) - - main(parse_flags(sys.argv)) diff --git a/third_party/rules_pycross/pycross/private/wheel_library.bzl b/third_party/rules_pycross/pycross/private/wheel_library.bzl deleted file mode 100644 index 166e1d06eb..0000000000 --- a/third_party/rules_pycross/pycross/private/wheel_library.bzl +++ /dev/null @@ -1,174 +0,0 @@ -# Copyright 2023 Jeremy Volkman. All rights reserved. -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""Implementation of the py_wheel_library rule.""" - -load("@bazel_skylib//lib:paths.bzl", "paths") -load("//python:defs.bzl", "PyInfo") -load(":providers.bzl", "PyWheelInfo") - -def _py_wheel_library_impl(ctx): - out = ctx.actions.declare_directory(ctx.attr.name) - - wheel_target = ctx.attr.wheel - if PyWheelInfo in wheel_target: - wheel_file = wheel_target[PyWheelInfo].wheel_file - name_file = wheel_target[PyWheelInfo].name_file - else: - wheel_file = ctx.file.wheel - name_file = None - - args = ctx.actions.args().use_param_file("--flagfile=%s") - args.add("--wheel", wheel_file) - args.add("--directory", out.path) - args.add_all(ctx.files.patches, format_each = "--patch=%s") - args.add_all(ctx.attr.patch_args, format_each = "--patch-arg=%s") - args.add("--patch-tool", ctx.attr.patch_tool) - - tools = [] - inputs = [wheel_file] + ctx.files.patches - if name_file: - inputs.append(name_file) - args.add("--wheel-name-file", name_file) - - if ctx.attr.patch_tool_target: - args.add("--patch-tool-target", ctx.attr.patch_tool_target.files_to_run.executable) - tools.append(ctx.executable.patch_tool_target) - - if ctx.attr.enable_implicit_namespace_pkgs: - args.add("--enable-implicit-namespace-pkgs") - - # We apply patches in the same action as the extraction to minimize the - # number of times we cache the wheel contents. If we were to split this - # into 2 actions, then the wheel contents would be cached twice. - ctx.actions.run( - inputs = inputs, - outputs = [out], - executable = ctx.executable._tool, - tools = tools, - arguments = [args], - # Set environment variables to make generated .pyc files reproducible. - env = { - "PYTHONHASHSEED": "0", - "SOURCE_DATE_EPOCH": "315532800", - }, - mnemonic = "WheelInstall", - progress_message = "Installing %s" % ctx.file.wheel.basename, - ) - - has_py2_only_sources = ctx.attr.python_version == "PY2" - has_py3_only_sources = ctx.attr.python_version == "PY3" - if not has_py2_only_sources: - for d in ctx.attr.deps: - if d[PyInfo].has_py2_only_sources: - has_py2_only_sources = True - break - if not has_py3_only_sources: - for d in ctx.attr.deps: - if d[PyInfo].has_py3_only_sources: - has_py3_only_sources = True - break - - # TODO: Is there a more correct way to get this runfiles-relative import path? - imp = paths.join( - ctx.label.workspace_name or ctx.workspace_name, # Default to the local workspace. - ctx.label.package, - ctx.label.name, - "site-packages", # we put lib files in this subdirectory. - ) - - imports = depset( - direct = [imp], - transitive = [d[PyInfo].imports for d in ctx.attr.deps], - ) - transitive_sources = depset( - direct = [out], - transitive = [dep[PyInfo].transitive_sources for dep in ctx.attr.deps if PyInfo in dep], - ) - runfiles = ctx.runfiles(files = [out]) - for d in ctx.attr.deps: - runfiles = runfiles.merge(d[DefaultInfo].default_runfiles) - - return [ - DefaultInfo( - files = depset(direct = [out]), - runfiles = runfiles, - ), - PyInfo( - has_py2_only_sources = has_py2_only_sources, - has_py3_only_sources = has_py3_only_sources, - imports = imports, - transitive_sources = transitive_sources, - uses_shared_libraries = True, # Docs say this is unused - ), - ] - -py_wheel_library = rule( - implementation = _py_wheel_library_impl, - attrs = { - "deps": attr.label_list( - doc = "A list of this wheel's Python library dependencies.", - providers = [DefaultInfo, PyInfo], - ), - "enable_implicit_namespace_pkgs": attr.bool( - default = True, - doc = """ -If true, disables conversion of native namespace packages into pkg-util style namespace packages. When set all py_binary -and py_test targets must specify either `legacy_create_init=False` or the global Bazel option -`--incompatible_default_to_explicit_init_py` to prevent `__init__.py` being automatically generated in every directory. -This option is required to support some packages which cannot handle the conversion to pkg-util style. - """, - ), - "patch_args": attr.string_list( - default = ["-p0"], - doc = - "The arguments given to the patch tool. Defaults to -p0, " + - "however -p1 will usually be needed for patches generated by " + - "git. If multiple -p arguments are specified, the last one will take effect.", - ), - "patch_tool": attr.string( - doc = "The patch(1) utility from the host to use. " + - "If set, overrides `patch_tool_target`. Please note that setting " + - "this means that builds are not completely hermetic.", - ), - "patch_tool_target": attr.label( - executable = True, - cfg = "exec", - doc = "The label of the patch(1) utility to use. " + - "Only used if `patch_tool` is not set.", - ), - "patches": attr.label_list( - allow_files = True, - default = [], - doc = - "A list of files that are to be applied as patches after " + - "extracting the archive. This will use the patch command line tool.", - ), - "python_version": attr.string( - doc = "The python version required for this wheel ('PY2' or 'PY3')", - values = ["PY2", "PY3", ""], - ), - "wheel": attr.label( - doc = "The wheel file.", - allow_single_file = [".whl"], - mandatory = True, - ), - "_tool": attr.label( - default = Label("//third_party/rules_pycross/pycross/private/tools:wheel_installer"), - cfg = "exec", - executable = True, - ), - }, -) diff --git a/tools/BUILD.bazel b/tools/BUILD.bazel index 4f42bcb02d..7829b33318 100644 --- a/tools/BUILD.bazel +++ b/tools/BUILD.bazel @@ -11,7 +11,7 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. -load("//python:defs.bzl", "py_binary") +load("//python:py_binary.bzl", "py_binary") package(default_visibility = ["//visibility:public"]) @@ -31,6 +31,7 @@ filegroup( "wheelmaker.py", "//tools/launcher:distribution", "//tools/precompiler:distribution", + "//tools/private:distribution", "//tools/publish:distribution", ], visibility = ["//:__pkg__"], diff --git a/tools/build_defs/python/private/BUILD.bazel b/tools/build_defs/python/private/BUILD.bazel index 0a7f308f02..e189b18880 100644 --- a/tools/build_defs/python/private/BUILD.bazel +++ b/tools/build_defs/python/private/BUILD.bazel @@ -20,8 +20,9 @@ filegroup( visibility = ["//python:__subpackages__"], ) +# keep bzl_library( - name = "py_internal_renamed_bzl", + name = "py_internal_renamed", srcs = ["py_internal_renamed.bzl"], - visibility = ["@rules_python_internal//:__subpackages__"], + visibility = ["//:__subpackages__"], ) diff --git a/tools/precompiler/BUILD.bazel b/tools/precompiler/BUILD.bazel index 268f41b032..e055980c72 100644 --- a/tools/precompiler/BUILD.bazel +++ b/tools/precompiler/BUILD.bazel @@ -14,6 +14,7 @@ load("@bazel_skylib//rules:common_settings.bzl", "string_list_flag") load("//python/private:py_interpreter_program.bzl", "py_interpreter_program") # buildifier: disable=bzl-visibility +load("//python/private:visibility.bzl", "NOT_ACTUALLY_PUBLIC") # buildifier: disable=bzl-visibility filegroup( name = "distribution", @@ -25,11 +26,9 @@ py_interpreter_program( name = "precompiler", execution_requirements = ":execution_requirements", main = "precompiler.py", - visibility = [ - # Not actually public. Only public so rules_python-generated toolchains - # are able to reference it. - "//visibility:public", - ], + # Not actually public. Only public so rules_python-generated toolchains + # are able to reference it. + visibility = NOT_ACTUALLY_PUBLIC, ) string_list_flag( diff --git a/tools/precompiler/precompiler.py b/tools/precompiler/precompiler.py index 310f2eb097..f83dd15951 100644 --- a/tools/precompiler/precompiler.py +++ b/tools/precompiler/precompiler.py @@ -68,18 +68,18 @@ def _compile(options: "argparse.Namespace") -> None: # A stub type alias for readability. # See the Bazel WorkRequest object definition: # https://github.com/bazelbuild/bazel/blob/master/src/main/protobuf/worker_protocol.proto -JsonWorkerRequest = object +JsonWorkRequest = object # A stub type alias for readability. # See the Bazel WorkResponse object definition: # https://github.com/bazelbuild/bazel/blob/master/src/main/protobuf/worker_protocol.proto -JsonWorkerResponse = object +JsonWorkResponse = object class _SerialPersistentWorker: """Simple, synchronous, serial persistent worker.""" - def __init__(self, instream: "typing.TextIO", outstream: "typing.TextIO"): + def __init__(self, instream: "typing.TextIO", outstream: "typing.TextIO"): # noqa: F821 self._instream = instream self._outstream = outstream self._parser = _create_parser() @@ -148,7 +148,7 @@ def _send_response(self, response: "JsonWorkResponse") -> None: class _AsyncPersistentWorker: """Asynchronous, concurrent, persistent worker.""" - def __init__(self, reader: "typing.TextIO", writer: "typing.TextIO"): + def __init__(self, reader: "typing.TextIO", writer: "typing.TextIO"): # noqa: F821 self._reader = reader self._writer = writer self._parser = _create_parser() @@ -156,13 +156,15 @@ def __init__(self, reader: "typing.TextIO", writer: "typing.TextIO"): self._task_to_request_id = {} @classmethod - async def main(cls, instream: "typing.TextIO", outstream: "typing.TextIO") -> None: + async def main(cls, instream: "typing.TextIO", outstream: "typing.TextIO") -> None: # noqa: F821 reader, writer = await cls._connect_streams(instream, outstream) await cls(reader, writer).run() @classmethod async def _connect_streams( - cls, instream: "typing.TextIO", outstream: "typing.TextIO" + cls, + instream: "typing.TextIO", # noqa: F821 + outstream: "typing.TextIO", # noqa: F821 ) -> "tuple[asyncio.StreamReader, asyncio.StreamWriter]": loop = asyncio.get_event_loop() reader = asyncio.StreamReader() @@ -214,7 +216,7 @@ async def _process_request(self, request: "JsonWorkRequest") -> None: # We don't send a response because we assume the request that # triggered cancelling sent the response raise - except: + except Exception: _logger.exception("Unhandled error: request=%s", request) self._send_response( { diff --git a/tools/private/BUILD.bazel b/tools/private/BUILD.bazel index e69de29bb2..ae6951c245 100644 --- a/tools/private/BUILD.bazel +++ b/tools/private/BUILD.bazel @@ -0,0 +1,18 @@ +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") + +package( + default_visibility = ["//:__subpackages__"], +) + +filegroup( + name = "distribution", + srcs = glob(["**"]) + [ + "//tools/private/zipapp:distribution", + ], +) + +bzl_library( + name = "publish_deps", + srcs = ["publish_deps.bzl"], + deps = ["//python/uv/private:lock"], +) diff --git a/tools/private/debug/print_defined_toolchains.sh b/tools/private/debug/print_defined_toolchains.sh new file mode 100755 index 0000000000..1694dc975c --- /dev/null +++ b/tools/private/debug/print_defined_toolchains.sh @@ -0,0 +1,14 @@ +#!/usr/bin/env bash +set -euo pipefail + +# Programmatically probe which repository target name is resolved successfully inside this workspace +if bazel query @pythons_hub//... >/dev/null 2>&1; then + HUB_REPO="@pythons_hub" +elif bazel query @@rules_python++python+pythons_hub//... >/dev/null 2>&1; then + HUB_REPO="@@rules_python++python+pythons_hub" +else + HUB_REPO="@@+python+pythons_hub" +fi + +# Query standard toolchains inside the resolved hub repository, excluding CC and Exec Tools toolchains. +bazel query "kind('toolchain', ${HUB_REPO}//...) - filter('_py_cc_toolchain$', ${HUB_REPO}//...) - filter('_py_exec_tools_toolchain$', ${HUB_REPO}//...)" "$@" diff --git a/tools/private/gazelle/BUILD.bazel b/tools/private/gazelle/BUILD.bazel new file mode 100644 index 0000000000..2a38e8f3a1 --- /dev/null +++ b/tools/private/gazelle/BUILD.bazel @@ -0,0 +1,21 @@ +load("@bazel_gazelle//:def.bzl", "gazelle", "gazelle_binary") + +package( + default_visibility = ["//:__subpackages__"], +) + +gazelle_binary( + name = "gazelle_bin", + languages = [ + "@bazel_skylib_gazelle_plugin//bzl", + ], + # Marked manual to avoid building in CI, which fails due to copt settings. + tags = ["manual"], +) + +gazelle( + name = "gazelle", + gazelle = ":gazelle_bin", + # Marked manual to avoid building in CI, which fails due to copt settings. + tags = ["manual"], +) diff --git a/tools/private/publish_deps.bzl b/tools/private/publish_deps.bzl index 538cc1d583..a9b0dbc562 100644 --- a/tools/private/publish_deps.bzl +++ b/tools/private/publish_deps.bzl @@ -17,13 +17,27 @@ load("//python/uv/private:lock.bzl", "lock") # buildifier: disable=bzl-visibility -def publish_deps(*, name, outs, **kwargs): - """Generate all of the requirements files for all platforms.""" +def publish_deps(*, name, args, outs, **kwargs): + """Generate all of the requirements files for all platforms. + + Args: + name: {type}`str`: the currently unused. + args: {type}`list[str]`: the common args to apply. + outs: {type}`dict[Label, str]`: the output files mapping to the platform + for each requirement file to be generated. + **kwargs: Extra args passed to the {rule}`lock` rule. + """ + all_args = args for out, platform in outs.items(): + args = [] + all_args + if platform: + args.append("--python-platform=" + platform) + else: + args.append("--universal") + lock( name = out.replace(".txt", ""), out = out, - universal = platform == "", - args = [] if not platform else ["--python-platform=" + platform], + args = args, **kwargs ) diff --git a/tools/private/release/BUILD.bazel b/tools/private/release/BUILD.bazel new file mode 100644 index 0000000000..5f80d1130a --- /dev/null +++ b/tools/private/release/BUILD.bazel @@ -0,0 +1,40 @@ +load("@rules_python//python:defs.bzl", "py_binary", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "changelog_news", + srcs = ["changelog_news.py"], +) + +py_library( + name = "release_lib", + srcs = glob( + ["*.py"], + exclude = [ + "changelog_news.py", + "mock_gh.py", + ], + ), + deps = [ + ":changelog_news", + "@dev_pip//packaging", + ], +) + +py_library( + name = "mock_gh", + srcs = ["mock_gh.py"], + deps = [ + ":release_lib", + ], +) + +py_binary( + name = "release", + srcs = ["release.py"], + main = "release.py", + deps = [ + ":release_lib", + ], +) diff --git a/tools/private/release/__init__.py b/tools/private/release/__init__.py new file mode 100644 index 0000000000..e68f0391f8 --- /dev/null +++ b/tools/private/release/__init__.py @@ -0,0 +1 @@ +"""Release tools package.""" diff --git a/tools/private/release/add_backports.py b/tools/private/release/add_backports.py new file mode 100644 index 0000000000..63c70af7d7 --- /dev/null +++ b/tools/private/release/add_backports.py @@ -0,0 +1,118 @@ +"""Subcommand to add PRs to the release tracking issue backports checklist.""" + +from tools.private.release.gh import GitHub +from tools.private.release.release_issue import ( + add_backports_to_body, + add_rc_task_to_body, + add_sync_changelog_task_to_body, + parse_checklist_state, +) + + +class AddBackports: + """Class to add PRs to the release tracking issue.""" + + def __init__(self, args, gh: GitHub): + self.args = args + self.gh = gh + + def run(self) -> int: + """Executes the add-backports subcommand.""" + args = self.args + + issue_num = args.issue + if not issue_num: + print( + "No issue specified. Trying to auto-discover active release" + " tracking issue..." + ) + try: + open_issues = self.gh.get_open_tracking_issues() + if not open_issues: + print("Error: No open release tracking issues found.") + return 1 + if len(open_issues) > 1: + print( + "Error: Multiple open release tracking issues found." + " Cannot determine active one:" + ) + for issue in open_issues: + print(f"- #{issue['number']}: {issue['title']}") + return 1 + issue_num = open_issues[0]["number"] + print(f"Auto-discovered active release tracking issue: #{issue_num}") + except Exception as e: + print(f"Error auto-discovering tracking issue: {e}") + return 1 + + resolved_prs = [] + for pr_ref in args.prs: + try: + pr_num = self.gh.resolve_pr_number(pr_ref) + resolved_prs.append(pr_num) + except Exception as e: + print(f"Error resolving PR ref '{pr_ref}': {e}") + return 1 + + print( + f"Adding backports {resolved_prs} (resolved from {args.prs}) to tracking issue #{issue_num}..." + ) + try: + body = self.gh.get_issue_body(issue_num) + items_to_add = [{"ref": f"#{pr}"} for pr in resolved_prs] + body = add_backports_to_body(body, items_to_add) + for pr in resolved_prs: + body = add_sync_changelog_task_to_body(body, pr) + state = parse_checklist_state(body) + rc_tags = state.get("rc_tags", {}) + has_pending_rc = any( + not task.checked and task.status != "done" for task in rc_tags.values() + ) + next_rc_num = max(rc_tags.keys()) + 1 if rc_tags else 0 + if not has_pending_rc: + print( + f"No pending RC task found. Adding 'Tag" + f" RC{next_rc_num}' to checklist..." + ) + body = add_rc_task_to_body(body, next_rc_num) + except ValueError as e: + print(f"Error: {e}") + return 1 + except Exception as e: + print(f"Failed to update tracking issue: {e}") + return 1 + + try: + self.gh.update_issue_body(issue_num, body) + print("Successfully updated tracking issue checklist.") + except Exception as e: + print(f"Failed to update tracking issue body: {e}") + return 1 + + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for add-backports subcommand.""" + parser = subparsers.add_parser( + "add-backports", + help="Add PRs to the release tracking issue backports checklist.", + ) + parser.add_argument( + "prs", + type=str, + nargs="+", + help="PR references (numbers, #numbers, or URLs) to add (positional, space-separated).", + ) + parser.add_argument( + "--issue", + type=int, + help="The tracking issue number. If omitted, will try to auto-discover the active release tracking issue.", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + gh = GitHub() + return cls(args, gh).run() diff --git a/tools/private/release/backport_create_releases.py b/tools/private/release/backport_create_releases.py new file mode 100644 index 0000000000..a6d2beabdf --- /dev/null +++ b/tools/private/release/backport_create_releases.py @@ -0,0 +1,237 @@ +"""Subcommand to initiate releases for verified backports.""" + +import argparse +import pathlib +import re +from dataclasses import dataclass + +from tools.private.release.backport_prepare import parse_backport_metadata +from tools.private.release.gh import GitHub +from tools.private.release.release_issue import ( + add_backports_to_body, + add_sync_changelog_task_to_body, + parse_metadata_line, + update_task_in_body, +) + + +@dataclass +class BackportTasksState: + # Map of minor version (e.g. "1.7") to verification status (e.g. "success", "failed-conflict") + verify: dict[str, str] + # Map of full version (e.g. "1.7.1") to tuple of (status, release_issue_number_or_None) + release: dict[str, tuple[str, str | None]] + + +def parse_backport_tasks(body) -> BackportTasksState: + """Parses tasks from backport issue body.""" + verify_tasks = {} + release_tasks = {} + + lines = body.splitlines() + for line in lines: + parsed = parse_metadata_line(line) + if not parsed: + continue + name = parsed["name"] + meta = parsed["metadata"] + + verify_match = re.match(r"Verify apply (\d+\.\d+)", name, re.IGNORECASE) + if verify_match: + minor = verify_match.group(1) + verify_tasks[minor] = meta.get("status", "pending") + continue + + release_match = re.match(r"Track Release (\d+\.\d+\.\d+)", name, re.IGNORECASE) + if release_match: + version = release_match.group(1) + status = meta.get("status", "pending") + issue_num = meta.get("release_issue") + release_tasks[version] = (status, issue_num) + + return BackportTasksState(verify=verify_tasks, release=release_tasks) + + +def is_release_eligible(version, target_minors, verify_statuses): + """Checks if a release version is eligible based on verification statuses.""" + version_minor = ".".join(version.split(".")[:2]) + v_minor_parsed = [int(x) for x in version_minor.split(".")] + + for minor in target_minors: + minor_parsed = [int(x) for x in minor.split(".")] + if minor_parsed >= v_minor_parsed: + status = verify_statuses.get(minor) + if status != "success": + return ( + False, + f"Blocked by verification failure on {minor} (status: {status})", + ) + + return True, "Eligible" + + +def _load_release_template() -> str: + """Loads the release tracking issue template.""" + template_path = pathlib.Path(".github/ISSUE_TEMPLATE/release_tracking_template.md") + if not template_path.exists(): + raise FileNotFoundError(f"Template file not found at {template_path}") + return template_path.read_text(encoding="utf-8") + + +class BackportCreateReleases: + """Class to initiate releases for verified backports.""" + + def __init__(self, args, gh: GitHub): + self._args = args + self._gh = gh + + def run(self) -> int: + """Executes the backport-create-releases subcommand.""" + args = self._args + issue_num = args.issue + if not issue_num: + raise ValueError("--issue is required.") + + print(f"Reading backport issue #{issue_num}...") + body = self._gh.get_issue_body(issue_num) + try: + metadata = parse_backport_metadata(body) + except ValueError as e: + e.add_note(f"Failed to parse backport metadata from issue #{issue_num}") + raise + pr_ref = metadata.pr + + pr_num = self._gh.resolve_pr_number(pr_ref) + + # Parse tasks + tasks_state = parse_backport_tasks(body) + verify_statuses = tasks_state.verify + release_tasks = tasks_state.release + + target_minors = sorted( + list(verify_statuses.keys()), key=lambda m: [int(x) for x in m.split(".")] + ) + + # We need the templates for release issues + template_content = _load_release_template() + + updated_body = body + changes_made = False + + for version, (status, release_issue) in release_tasks.items(): + # If status is success, we already created the issue. + # We check the actual checklist item status, but the helper returns it. + # The status in metadata is 'success' when done. + if status == "success": + print(f"Release for {version} already initiated: {release_issue}") + continue + + eligible, reason = is_release_eligible( + version, target_minors, verify_statuses + ) + + task_name = f"Track Release {version}" + + if eligible: + print(f"Initiating release for {version}...") + + # Check if release issue already exists + existing_issues = self._gh.get_open_tracking_issues(version) + if existing_issues: + new_issue_num = existing_issues[0]["number"] + print( + f"Release issue for {version} already exists: #{new_issue_num}. Reusing it." + ) + else: + # Create the issue + is_first_release = version.endswith(".0") + if is_first_release: + issue_template = template_content + else: + lines = template_content.splitlines() + lines = [ + line for line in lines if not re.search(r"Tag RC\d+", line) + ] + issue_template = "\n".join(lines) + if template_content.endswith("\n"): + issue_template += "\n" + + if args.dry_run: + print( + f"[DRY RUN] Would create release tracking issue for {version} (without RC tasks)" + ) + new_issue_num = "" + else: + new_issue_num = self._gh.create_release_tracking_issue( + version, issue_template + ) + print( + f"Created release tracking issue #{new_issue_num} for {version}" + ) + + # Add the backport PR to the new release issue + print( + f"Adding PR #{pr_num} to release issue #{new_issue_num} checklist..." + ) + rel_body = self._gh.get_issue_body(new_issue_num) + rel_body = add_backports_to_body( + rel_body, [{"ref": f"#{pr_num}"}] + ) + rel_body = add_sync_changelog_task_to_body(rel_body, pr_num) + self._gh.update_issue_body(new_issue_num, rel_body) + + # Update task in backport issue + metadata = {"status": "success", "release_issue": f"#{new_issue_num}"} + updated_body = update_task_in_body( + updated_body, task_name, checked=True, metadata=metadata + ) + changes_made = True + else: + print(f"Release for {version} is not eligible: {reason}") + metadata = {"status": "error-later-release-did-not-apply"} + + if status != "error-later-release-did-not-apply": + updated_body = update_task_in_body( + updated_body, task_name, checked=False, metadata=metadata + ) + changes_made = True + + if changes_made: + if args.dry_run: + print( + f"[DRY RUN] Would update backport issue #{issue_num} body:\n{updated_body}" + ) + else: + self._gh.update_issue_body(issue_num, updated_body) + print(f"Updated backport issue #{issue_num}") + else: + print("No changes needed for backport issue.") + + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for backport-create-releases subcommand.""" + parser = subparsers.add_parser( + "backport-create-releases", + help="Initiate releases for verified backports.", + ) + parser.add_argument( + "--issue", + type=int, + required=True, + help="The backport tracking issue number (required).", + ) + parser.add_argument( + "--dry-run", + action=argparse.BooleanOptionalAction, + default=True, + help="Perform a dry run (default: True). Use --no-dry-run to actually execute.", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + gh = GitHub() + return cls(args, gh).run() diff --git a/tools/private/release/backport_prepare.py b/tools/private/release/backport_prepare.py new file mode 100644 index 0000000000..459e91f703 --- /dev/null +++ b/tools/private/release/backport_prepare.py @@ -0,0 +1,305 @@ +"""Subcommand to prepare backport tracking issue and verify cherry-picks.""" + +import argparse +import datetime +import re +from dataclasses import dataclass + +from tools.private.release import changelog_news +from tools.private.release.gh import BACKPORT_LABEL, GitHub +from tools.private.release.git import Git +from tools.private.release.utils import determine_next_version + + +@dataclass +class BackportMetadata: + pr: str + from_minor: str + to_minor: str + + +def parse_backport_metadata(body) -> BackportMetadata: + """Parses backport metadata from issue body.""" + pr_match = re.search(r"^\s*\*\s*PR:\s*(\S+)", body, re.MULTILINE | re.IGNORECASE) + from_match = re.search( + r"^\s*\*\s*From version:\s*(\S+)", body, re.MULTILINE | re.IGNORECASE + ) + to_match = re.search( + r"^\s*\*\s*To version:\s*(\S+)", body, re.MULTILINE | re.IGNORECASE + ) + + if not pr_match or not from_match or not to_match: + raise ValueError( + "Missing metadata in issue body. Need PR, From version, and To version." + ) + + return BackportMetadata( + pr=pr_match.group(1), + from_minor=from_match.group(1), + to_minor=to_match.group(1), + ) + + +def get_target_branches(git, remote, from_minor, to_minor): + """Identifies target release branches in the given range.""" + branches = git.get_remote_branches(remote) + target_branches = [] + + # Parse version strings to compare (only major and minor) + from_v = [int(x) for x in from_minor.split(".")][:2] + to_v = [int(x) for x in to_minor.split(".")][:2] + + for branch in branches: + match = re.match(r"^release/(\d+)\.(\d+)$", branch) + if match: + major = int(match.group(1)) + minor = int(match.group(2)) + v = [major, minor] + if v >= from_v and v <= to_v: + target_branches.append(branch) + + # Sort branches by version + target_branches.sort(key=lambda b: [int(x) for x in b.split("/")[1].split(".")]) + return target_branches + + +def get_latest_release_branch(git, remote): + """Determines the latest release branch.""" + branches = git.get_remote_branches(remote) + release_branches = [] + for branch in branches: + match = re.match(r"^release/(\d+)\.(\d+)$", branch) + if match: + release_branches.append(branch) + if not release_branches: + return None + release_branches.sort(key=lambda b: [int(x) for x in b.split("/")[1].split(".")]) + return release_branches[-1] + + +class BackportPrepare: + """Class to prepare backport tracking issue and verify cherry-picks.""" + + def __init__(self, args, git: Git, gh: GitHub): + self.args = args + self.git = git + self.gh = gh + + def run(self) -> int: + """Executes the backport-prepare subcommand.""" + args = self.args + print("Fetching remote to verify release branches...") + self.git.fetch(args.remote, tags=True, force=True) + + issue_num = args.issue + + if issue_num: + # Triggered by comment, read from issue + print(f"Reading metadata from issue #{issue_num}...") + body = self.gh.get_issue_body(issue_num) + try: + metadata = parse_backport_metadata(body) + except ValueError as e: + e.add_note(f"Failed to parse backport metadata from issue #{issue_num}") + raise + pr_ref = metadata.pr + from_minor = metadata.from_minor + to_minor = metadata.to_minor + else: + # Triggered manually, use args + pr_ref = args.pr + from_minor = args.from_minor + to_minor = args.to_minor + + if not pr_ref or not from_minor: + raise ValueError( + "PR and From version are required if not running from an issue." + ) + + if not to_minor: + latest_branch = get_latest_release_branch(self.git, args.remote) + if not latest_branch: + raise RuntimeError( + "Could not determine latest release branch and --to-minor was not provided." + ) + to_minor = latest_branch.split("/")[1] + print(f"Auto-determined latest minor version: {to_minor}") + + # Resolve PR to merge commit + try: + pr_num = self.gh.resolve_pr_number(pr_ref) + print(f"Resolving PR #{pr_num} info...") + pr_info = self.gh.get_pr_info(pr_num) + except Exception as e: + e.add_note(f"Failed to resolve PR info for {pr_ref}") + raise + + if pr_info.get("state") != "MERGED": + raise ValueError( + f"PR #{pr_num} is not merged (state: {pr_info.get('state')})." + ) + merge_commit = pr_info.get("mergeCommit") + if not merge_commit or "oid" not in merge_commit: + raise ValueError(f"PR #{pr_num} has no merge commit SHA.") + pr_sha = merge_commit["oid"] + print(f"Resolved PR #{pr_num} to merge commit {pr_sha[:8]}") + + target_branches = get_target_branches( + self.git, args.remote, from_minor, to_minor + ) + if not target_branches: + raise ValueError( + f"No release branches found in range {from_minor} to {to_minor}" + ) + + print(f"Identified target branches: {target_branches}") + + # Verify workspace is clean + if self.git.status(): + raise RuntimeError("Workspace is dirty. Aborting.") + + current_branch = self.git.get_current_branch() + verify_results = {} # branch -> (success, reason) + version_map = {} # branch -> next_version + + try: + for branch in target_branches: + minor_ver = branch.split("/")[1] + print(f"Verifying application on {branch}...") + self.git.checkout(branch, track_remote=args.remote) + + # Determine next version + next_version = determine_next_version(branch) + version_map[branch] = next_version + + # Verify apply (cherry-pick + news) + try: + self.git.cherry_pick(pr_sha) + + # Try news update + try: + release_date = datetime.date.today().strftime("%Y-%m-%d") + changelog_news.update_changelog(next_version, release_date) + verify_results[branch] = (True, "Success") + print(f"Verification successful for {branch}") + except Exception as e: + verify_results[branch] = ( + False, + f"Changelog update failed: {e}", + ) + print( + f"Verification failed for {branch} (changelog update): {e}" + ) + except Exception as e: + verify_results[branch] = (False, f"Cherry-pick failed: {e}") + print(f"Verification failed for {branch} (cherry-pick): {e}") + finally: + # Always abort/reset + try: + self.git.cherry_pick_abort() + except Exception: + pass + self.git.reset_hard(reset_to=f"{args.remote}/{branch}") + finally: + # Restore original branch + if current_branch: + self.git.checkout(current_branch) + + # Generate issue content + body_lines = [ + f"* PR: #{pr_num}", + f"* From version: {from_minor}", + f"* To version: {to_minor}", + "", + "## Tasks", + "", + ] + + # Add Verify tasks + for branch in target_branches: + minor_ver = branch.split("/")[1] + success, reason = verify_results[branch] + if success: + body_lines.append(f"- [x] Verify apply {minor_ver} | status=success") + else: + status = "failed-conflict" + if "Changelog" in reason: + status = "failed-changelog" + body_lines.append(f"- [ ] Verify apply {minor_ver} | status={status}") + + # Add Release tasks + for branch in target_branches: + next_version = version_map[branch] + body_lines.append(f"- [ ] Track Release {next_version}") + + new_body = "\n".join(body_lines) + + if issue_num: + if args.dry_run: + print( + f"[DRY RUN] Would update issue #{issue_num} with body:\n{new_body}" + ) + else: + self.gh.update_issue_body(issue_num, new_body) + print(f"Successfully updated issue #{issue_num}") + else: + title = f"Backport: #{pr_num}" + if args.dry_run: + print( + f"[DRY RUN] Would create issue with title '{title}' and body:\n{new_body}" + ) + else: + new_issue_num = self.gh.create_issue( + title, new_body, labels=[BACKPORT_LABEL] + ) + print(f"Created backport tracking issue #{new_issue_num}") + + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for backport-prepare subcommand.""" + parser = subparsers.add_parser( + "backport-prepare", + help="Prepare backport tracking issue and verify cherry-picks.", + ) + parser.add_argument( + "--issue", + type=int, + help="The backport tracking issue number (if running from an existing issue).", + ) + parser.add_argument( + "--pr", + type=str, + help="PR reference to backport (required if not running from issue).", + ) + parser.add_argument( + "--from-minor", + type=str, + help="Oldest minor version to target (inclusive) (e.g. 1.7) (required if not running from issue).", + ) + parser.add_argument( + "--to-minor", + type=str, + help="Newest minor version to target (inclusive) (optional, defaults to latest).", + ) + parser.add_argument( + "--remote", + type=str, + default="origin", + help="The git remote (default: origin).", + ) + parser.add_argument( + "--dry-run", + action=argparse.BooleanOptionalAction, + default=True, + help="Perform a dry run (default: True). Use --no-dry-run to actually execute.", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + git = Git(".") + gh = GitHub() + return cls(args, git, gh).run() diff --git a/tools/private/release/changelog_news.py b/tools/private/release/changelog_news.py new file mode 100644 index 0000000000..6381aacb34 --- /dev/null +++ b/tools/private/release/changelog_news.py @@ -0,0 +1,368 @@ +"""Utility functions for handling news entries and merging them into CHANGELOG.md.""" + +import pathlib +import re + + +def _get_sub_category(content): + """Extracts the sub-category in parentheses from the entry content.""" + match = re.match(r"^(?:\*|-)\s*\(([^)]+)\)", content) + if match: + return match.group(1).lower() + return "" + + +def is_news_file(path): + """Checks if a file path is a valid news file.""" + path = pathlib.Path(path) + if not path.is_file(): + return False + if path.suffix != ".md": + return False + parts = path.name.split(".") + if len(parts) < 3: + return False + return True + + +def _get_news_files(news_dir): + """Returns a list of news files matching the ..md pattern.""" + news_path = pathlib.Path(news_dir) + if not news_path.exists(): + return [] + + return [p for p in news_path.iterdir() if is_news_file(p)] + + +def _format_entry(content): + """Formats news entry content as a markdown bullet list item. + + Continuation lines are indented with two spaces. + """ + lines = content.strip().splitlines() + if not lines: + return "" + + formatted_lines = [] + first_line = lines[0] + if not (first_line.startswith("* ") or first_line.startswith("- ")): + formatted_lines.append(f"* {first_line}") + else: + formatted_lines.append(first_line) + + for line in lines[1:]: + if not line: + formatted_lines.append("") + elif line.startswith("* ") or line.startswith("- "): + formatted_lines.append(line) + elif line.startswith(" "): + formatted_lines.append(line) + else: + formatted_lines.append(f" {line}") + + return "\n".join(formatted_lines) + + +def _parse_new_files(news_files): + """Parses news files and groups them by category.""" + entries = {} + for p in news_files: + if not is_news_file(p): + continue + parts = p.name.split(".") + category = parts[1].lower() + + content = p.read_text(encoding="utf-8").strip() + + if not content: + continue + + formatted_content = _format_entry(content) + if not formatted_content: + continue + + if category not in entries: + entries[category] = [] + entries[category].append(formatted_content) + + return entries + + +def generate_release_block(version, release_date, news_entries): + """Generates the markdown block for the release.""" + header_version = version.replace(".", "-") + lines = [ + f"{{#v{header_version}}}", + f"## [{version}] - {release_date}", + "", + f"[{version}]: https://github.com/bazel-contrib/rules_python/releases/tag/{version}", + "", + ] + + # Standard categories in preferred order + category_order = ["removed", "changed", "fixed", "added"] + # Add any other categories found + for cat in news_entries: + if cat not in category_order: + category_order.append(cat) + + has_entries = False + for cat in category_order: + if cat in news_entries and news_entries[cat]: + has_entries = True + lines.append(f"{{#v{header_version}-{cat}}}") + lines.append(f"### {cat.capitalize()}") + + # Sort entries by sub-category, then by content + sorted_entries = sorted( + news_entries[cat], key=lambda e: (_get_sub_category(e), e) + ) + + for entry in sorted_entries: + lines.append(entry) + lines.append("") + + if not has_entries: + lines.append("No notable changes.") + lines.append("") + + return "\n".join(lines) + + +def _parse_simple_version(ver_str): + """Parses a version string (X-Y-Z or X.Y.Z) into a tuple of ints.""" + normalized = ver_str.replace("-", ".") + return tuple(int(x) for x in normalized.split(".")) + + +def _find_insertion_point(changelog_content, new_version): + """Finds the character index to insert the new version block. + + It should be inserted before the first version in the changelog that is + smaller than new_version. + """ + # Find all version anchors: {#vX-Y-Z} + matches = list(re.finditer(r"\{#v(?P\d+-\d+-\d+)\}", changelog_content)) + + parsed_new_ver = _parse_simple_version(new_version) + + for m in matches: + ver_str = m.group("ver") + parsed_ver = _parse_simple_version(ver_str) + if parsed_ver < parsed_new_ver: + return m.start() + + raise ValueError( + f"Could not find a version in CHANGELOG.md smaller than {new_version} to insert before." + ) + + +def _add_news_to_changelog(input_path, output_path, version, entries, release_date): + """Adds or merges news entries into CHANGELOG.md.""" + input_path = pathlib.Path(input_path) + output_path = pathlib.Path(output_path) + changelog_content = input_path.read_text(encoding="utf-8") + + if version == "unreleased": + header_version = "unreleased" + version_anchor = "{#unreleased}" + category_anchor_fmt = "{{#unreleased-{cat}}}" + category_anchor_pattern = r"\{#unreleased-(?P[a-z]+)\}" + else: + header_version = version.replace(".", "-") + version_anchor = f"{{#v{header_version}}}" + category_anchor_fmt = "{{#v" + header_version + "-{cat}}}" + category_anchor_pattern = ( + r"\{#v" + re.escape(header_version) + r"-(?P[a-z]+)\}" + ) + + version_exists = version_anchor in changelog_content + + if version_exists: + if not entries and version != "unreleased": + print( + f"Version {version} already exists and no news entries found" + " to merge. Doing nothing." + ) + output_path.write_text(changelog_content, encoding="utf-8") + return + + print(f"Version {version} already exists in changelog. Merging news entries...") + # Extract the existing version block + pattern = ( + r"(?P" + + re.escape(version_anchor) + + r")(?P.*?)(?=\n\s*\{#v\d+-\d+-\d+\}|\Z)" + ) + match = re.search(pattern, changelog_content, re.DOTALL) + if not match: + raise RuntimeError( + f"Could not find content for existing version {version} in CHANGELOG.md" + ) + + content_block = match.group("content") + + # Strip the "Unreleased changes..." sentence for Unreleased preview + if version == "unreleased": + content_block = re.sub( + r"Unreleased\s+changes\s+are\s+tracked\s+as\s+individual\s+files\s+in\s+the\s+\[news/\]\(\./news\)\s+directory,\s+or\s+view\s+the\s+\[latest\s+generated\s+changelog\]\(https://rules-python\.readthedocs\.io/en/latest/changelog\.html\)\.\s*\n*", + "", + content_block, + ) + + # Parse existing categories + match_cat = re.search(category_anchor_pattern, content_block) + if match_cat: + header_end_idx = match_cat.start() + header_str = content_block[:header_end_idx] + categories_str = content_block[header_end_idx:] + else: + header_str = content_block + categories_str = "" + + existing_entries = {} + if categories_str: + cat_matches = list(re.finditer(category_anchor_pattern, categories_str)) + for i, m in enumerate(cat_matches): + cat = m.group("cat") + start_idx = m.end() + end_idx = ( + cat_matches[i + 1].start() + if i + 1 < len(cat_matches) + else len(categories_str) + ) + cat_content = categories_str[start_idx:end_idx].strip() + + lines = cat_content.splitlines() + cat_entries = [] + current_entry = [] + for line in lines: + if not line.strip() or line.strip().startswith("### "): + continue + if line.startswith("* ") or line.startswith("- "): + if current_entry: + cat_entries.append("\n".join(current_entry)) + current_entry = [line] + else: + if current_entry: + current_entry.append(line) + if current_entry: + cat_entries.append("\n".join(current_entry)) + existing_entries[cat] = cat_entries + + # Merge news entries + merged_entries = dict(existing_entries) + for cat, cat_entries in entries.items(): + if cat not in merged_entries: + merged_entries[cat] = [] + merged_entries[cat].extend(cat_entries) + + # Reconstruct categories + reconstructed_lines = [] + category_order = ["removed", "changed", "fixed", "added"] + for cat in merged_entries: + if cat not in category_order: + category_order.append(cat) + + for cat in category_order: + if cat in merged_entries and merged_entries[cat]: + reconstructed_lines.append(category_anchor_fmt.format(cat=cat)) + reconstructed_lines.append(f"### {cat.capitalize()}") + + sorted_entries = sorted( + merged_entries[cat], key=lambda e: (_get_sub_category(e), e) + ) + + for entry in sorted_entries: + reconstructed_lines.append(entry) + reconstructed_lines.append("") + + new_categories_str = "\n".join(reconstructed_lines) + new_release_block = ( + header_str.rstrip() + "\n\n" + new_categories_str.strip() + "\n" + ) + if version == "unreleased" and not new_categories_str.strip(): + new_release_block = ( + header_str.rstrip() + "\n\nNo notable unreleased changes.\n" + ) + + # Replace in changelog_content + new_content = re.sub( + pattern, + r"\g\n" + new_release_block.strip() + "\n", + changelog_content, + count=1, + flags=re.DOTALL, + ) + output_path.write_text(new_content, encoding="utf-8") + + else: + print( + f"Version {version} does not exist in changelog. Creating new" + " release section from news entries..." + ) + new_release_block = generate_release_block(version, release_date, entries) + + # Find insertion point + insertion_point = _find_insertion_point(changelog_content, version) + + new_content = ( + changelog_content[:insertion_point] + + new_release_block + + "\n\n" + + changelog_content[insertion_point:] + ) + output_path.write_text(new_content, encoding="utf-8") + + +def merge_new_into_changelog( + changelog_path, + output_path, + news_dir, + version, + release_date, + delete_news=False, + news_files=None, +): + """Merges news entries from news_dir into changelog_path and writes to output_path.""" + if news_files is None: + news_files = _get_news_files(news_dir) + else: + news_files = [pathlib.Path(f) for f in news_files] + entries = _parse_new_files(news_files) + _add_news_to_changelog( + input_path=changelog_path, + output_path=output_path, + version=version, + entries=entries, + release_date=release_date, + ) + if delete_news: + for p in news_files: + if p.exists(): + p.unlink() + if news_files: + print(f"Removed {len(news_files)} processed news files.") + + +def update_changelog( + version, + release_date, + changelog_path="CHANGELOG.md", + output_path=None, + news_dir="news", + delete_news=True, + news_files=None, +): + """Performs the version replacements in CHANGELOG.md.""" + if output_path is None: + output_path = changelog_path + merge_new_into_changelog( + changelog_path=changelog_path, + output_path=output_path, + news_dir=news_dir, + version=version, + release_date=release_date, + delete_news=delete_news, + news_files=news_files, + ) diff --git a/tools/private/release/complete_prepare.py b/tools/private/release/complete_prepare.py new file mode 100644 index 0000000000..e585a53e8c --- /dev/null +++ b/tools/private/release/complete_prepare.py @@ -0,0 +1,114 @@ +"""Subcommand to mark preparation task as complete.""" + +import re + +from tools.private.release.gh import GitHub +from tools.private.release.release_issue import ( + parse_checklist_state, + update_task_in_body, +) +from tools.private.release.utils import set_github_output + + +class CompletePrepare: + """Class to mark preparation task as complete.""" + + def __init__(self, args, gh: GitHub): + self.args = args + self.gh = gh + + def run(self) -> int: + """Executes the complete-prepare subcommand (Phase 2 PR merged).""" + args = self.args + pr_number = args.pr + issue_number = args.issue + + if not pr_number and not issue_number: + print("Error: Either --pr or --issue must be provided.") + return 1 + + if not pr_number and issue_number: + issue_body = self.gh.get_issue_body(issue_number) + state = parse_checklist_state(issue_body) + prep_task = state.get("prepare_release") + if not prep_task or not prep_task.pr: + print( + f"Error: Could not find PR reference for 'Prepare Release'" + f" in issue #{issue_number}." + ) + return 1 + pr_number = int(prep_task.pr.lstrip("#")) + + print(f"Completing preparation for PR #{pr_number}...") + + pr_info = self.gh.get_pr_info(pr_number) + if not pr_info or pr_info.get("state") != "MERGED": + state = pr_info.get("state", "UNKNOWN") if pr_info else "NOT_FOUND" + print(f"Error: PR #{pr_number} is not merged yet (state: {state}).") + return 1 + + if not issue_number: + # Resolve issue number from PR body + pr_body = pr_info.get("body") or "" + match = re.search(r"Work towards #(\d+)", pr_body) + if not match: + match = re.search(r"#(\d+)", pr_body) + if not match: + print( + f"Error: Could not determine tracking issue number from PR" + f" #{pr_number} body: {pr_body}" + ) + return 1 + + issue_number = int(match.group(1)) + print(f"Resolved tracking issue #{issue_number} from PR #{pr_number} body.") + + commit_sha = pr_info["mergeCommit"]["oid"] + short_commit = commit_sha[:8] + print( + f"PR #{pr_number} merged at commit {commit_sha}. Updating tracking issue..." + ) + + # Update checklist: mark Prepare Release as done (checked) and set SUCCESS + body = self.gh.get_issue_body(issue_number) + metadata = { + "status": "done", + "pr": f"#{pr_number}", + "commit": short_commit, + } + updated_body = update_task_in_body( + body, "Prepare Release", checked=True, metadata=metadata + ) + self.gh.update_issue_body(issue_number, updated_body) + print("Prepare Release task marked complete successfully!") + + set_github_output("issue", str(issue_number)) + + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for complete-prepare subcommand.""" + parser = subparsers.add_parser( + "complete-prepare", + help="Mark the Prepare Release task as complete in the tracking issue.", + ) + parser.add_argument( + "--pr", + type=int, + required=False, + help="The merged preparation PR number.", + ) + parser.add_argument( + "--issue", + type=int, + required=False, + help="The release tracking issue number.", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + gh = GitHub() + return cls(args, gh).run() diff --git a/tools/private/release/complete_sync_changelog.py b/tools/private/release/complete_sync_changelog.py new file mode 100644 index 0000000000..9c8d9028f0 --- /dev/null +++ b/tools/private/release/complete_sync_changelog.py @@ -0,0 +1,97 @@ +"""Subcommand to mark sync changelog tasks as complete.""" + +import re + +from tools.private.release.gh import GitHub +from tools.private.release.release_issue import ( + parse_checklist_state, + update_task_in_body, +) + + +class CompleteSyncChangelog: + """Class to mark sync changelog tasks as complete.""" + + def __init__(self, args, gh: GitHub): + self.args = args + self.gh = gh + + def run(self) -> int: + """Executes the complete-sync-changelog subcommand.""" + args = self.args + print(f"Completing sync changelog for PR #{args.pr}...") + + pr_info = self.gh.get_pr_info(args.pr) + if not pr_info or pr_info.get("state") != "MERGED": + state = pr_info.get("state", "UNKNOWN") + print(f"Error: PR #{args.pr} is not merged yet (state: {state}).") + return 1 + + # Resolve issue number from PR body using Release-Tracking-Issue: # + pr_body = pr_info.get("body") or "" + match = re.search(r"Release-Tracking-Issue:\s*#(\d+)", pr_body) + if not match: + print( + f"Error: Could not find 'Release-Tracking-Issue: #' in" + f" PR #{args.pr} body: {pr_body}" + ) + return 1 + + issue_num = int(match.group(1)) + print(f"Resolved tracking issue #{issue_num} from PR #{args.pr} body.") + + commit_sha = pr_info["mergeCommit"]["oid"] + short_commit = commit_sha[:8] + print( + f"PR #{args.pr} merged at commit {commit_sha}. Updating tracking issue..." + ) + + # Update checklist: mark all Sync Changelog tasks pointing to this PR as done + body = self.gh.get_issue_body(issue_num) + state = parse_checklist_state(body) + sync_changelogs = state.get("sync_changelogs", {}) + + updated_any = False + for pr_num, task in sync_changelogs.items(): + # Check if this task points to our merged PR + task_pr = task.metadata.get("pr") + if task_pr == f"#{args.pr}": + print(f"Marking task '{task.name}' as complete...") + metadata = { + "status": "done", + "pr": f"#{args.pr}", + "commit": short_commit, + } + body = update_task_in_body( + body, task.name, checked=True, metadata=metadata + ) + updated_any = True + + if not updated_any: + print(f"Warning: No 'Sync Changelog' tasks found pointing to PR #{args.pr}") + return 0 + + self.gh.update_issue_body(issue_num, body) + print("Sync changelog tasks marked complete successfully!") + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for complete-sync-changelog subcommand.""" + parser = subparsers.add_parser( + "complete-sync-changelog", + help="Mark the Sync Changelog tasks as complete in the tracking issue.", + ) + parser.add_argument( + "--pr", + type=int, + required=True, + help="The merged sync changelog PR number.", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + gh = GitHub() + return cls(args, gh).run() diff --git a/tools/private/release/create_rc.py b/tools/private/release/create_rc.py new file mode 100644 index 0000000000..0987bb3e43 --- /dev/null +++ b/tools/private/release/create_rc.py @@ -0,0 +1,211 @@ +"""Subcommand to tag and push the next release candidate.""" + +import os +import traceback +from argparse import Namespace + +from tools.private.release.gh import GH_REACTION_THUMBS_DOWN, GitHub +from tools.private.release.git import Git +from tools.private.release.process_backports import ProcessBackports +from tools.private.release.release_issue import ( + RELEASE_TITLE_RE, + add_rc_task_to_body, + parse_backports, + parse_checklist_state, + update_task_in_body, +) +from tools.private.release.utils import ( + REPO_URL, + get_latest_rc_tag, + set_github_output, +) + + +class CreateRc: + """Class to tag and push the next release candidate.""" + + def __init__(self, args, git: Git, gh: GitHub): + self.args = args + self.git = git + self.gh = gh + + def run(self) -> int: + """Executes the create-rc subcommand.""" + args = self.args + exit_code = 0 + try: + exit_code = self._run_internal() + except Exception as e: + print(f"Unexpected error: {e}") + traceback.print_exc() + exit_code = 1 + + if exit_code != 0 and args.triggering_comment: + print(f"Reacting with thumbs-down to comment {args.triggering_comment}...") + try: + self.gh.add_comment_reaction( + args.triggering_comment, GH_REACTION_THUMBS_DOWN + ) + except Exception as e: + print(f"Failed to add reaction to comment: {e}") + + return exit_code + + def _run_internal(self) -> int: + """Internal implementation of create-rc.""" + args = self.args + + # Try to process pending backports first + print("Processing pending backports before creating RC...") + backport_args = Namespace( + issue=args.issue, + remote=args.remote, + add=None, + triggering_comment=None, + dry_run=False, + ) + pb = ProcessBackports(backport_args, self.git, self.gh) + backports_exit_code = pb.run() + if backports_exit_code != 0: + print("Error: Processing backports failed. Aborting RC creation.") + return backports_exit_code + + body = self.gh.get_issue_body(args.issue) + state = parse_checklist_state(body) + + if ( + state["prepare_release"].status != "done" + or state["create_branch"].status != "done" + ): + print( + "Error: Preconditions not met (release must be prepared and" + " branch created)." + ) + return 1 + + # Gating: RC tagging is blocked if any backport is unchecked OR does not have status=done + backports = parse_backports(body) + conflicting_or_pending = [ + b + for b in backports + if (b.checked and b.status != "done") + or (not b.checked and b.status != "ignore") + ] + if conflicting_or_pending: + print( + f"Gating RC tagging: {len(conflicting_or_pending)} backports" + " are still unfinished, failed, or in conflict." + ) + return 1 + + # Resolve version and branch + issue_title = self.gh.get_issue_title(args.issue) + version_match = RELEASE_TITLE_RE.search(issue_title) + if not version_match: + print(f"Error: Could not parse version from issue title: {issue_title}") + return 1 + + version = version_match.group(1) + branch_version = ".".join(version.split(".")[:2]) + branch_name = f"release/{branch_version}" + + # Determine next RC tag + self.git.fetch(args.remote) + self.git.fetch(args.remote, tags=True, force=True) + latest_rc = get_latest_rc_tag(version, remote=args.remote) + + if not latest_rc: + next_rc_num = 0 + next_rc = f"{version}-rc0" + else: + rc_num = int(latest_rc.split("-rc")[-1]) + next_rc_num = rc_num + 1 + next_rc = f"{version}-rc{next_rc_num}" + + # Precheck: next RC number must exist and be unchecked in the checklist + rc_tags = state.get("rc_tags", {}) + if next_rc_num not in rc_tags: + print(f"Task 'Tag RC{next_rc_num}' not found in checklist. Adding it...") + body = add_rc_task_to_body(body, next_rc_num) + self.gh.update_issue_body(args.issue, body) + else: + target_rc_task = rc_tags[next_rc_num] + if target_rc_task.checked or target_rc_task.status == "done": + print( + f"Error: Task 'Tag RC{next_rc_num}' is already marked done in" + " the checklist." + ) + return 1 + + target_ref = f"{args.remote}/{branch_name}" + commit_sha = self.git.get_commit_sha(target_ref) + + print(f"Tagging and pushing next RC: {next_rc}...") + self.git.tag(next_rc, target_ref) + self.git.push(args.remote, next_rc) + + set_github_output("tag_name", next_rc) + + # Check off the appropriate "Tag RC{N}" task in the checklist + print(f"Checking off Tag RC{next_rc_num} task...") + metadata = {"status": "done", "tag": next_rc, "commit": commit_sha[:8]} + task_name = f"Tag RC{next_rc_num}" + updated_body = update_task_in_body( + body, task_name, checked=True, metadata=metadata + ) + self.gh.update_issue_body(args.issue, updated_body) + + tag_url = f"{REPO_URL}/releases/tag/{next_rc}" + bcr_entry_url = f"https://registry.bazel.build/modules/rules_python/{next_rc}" + bcr_search_url = f"https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+{next_rc}" + if run_id := os.environ.get("GITHUB_RUN_ID"): + release_workflow_url = f"{REPO_URL}/actions/runs/{run_id}" + else: + release_workflow_url = ( + f"{REPO_URL}/actions/workflows/release_create_rc.yaml" + ) + branch_url = f"{REPO_URL}/tree/{branch_name}" + comment_body = f"""**New Release Candidate Tagged!** 🐍🌿 + +Release Candidate **{next_rc}** has been successfully generated and tagged on branch [`{branch_name}`]({branch_url}). + +- [Github Release {next_rc}]({tag_url}) +- [BCR Entry {next_rc}]({bcr_entry_url}) +- [BCR PRs]({bcr_search_url}) +- [Release workflow status]({release_workflow_url})""" + self.gh.post_issue_comment(args.issue, comment_body) + print("RC creation completed successfully!") + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for create-rc subcommand.""" + parser = subparsers.add_parser( + "create-rc", + help="Tags the next RC on the release branch if no backports remain.", + ) + parser.add_argument( + "--issue", + type=int, + required=True, + help="The tracking issue number (required).", + ) + parser.add_argument( + "--remote", + type=str, + required=True, + help="The git remote to push the RC tag to (required).", + ) + parser.add_argument( + "--triggering-comment", + type=int, + help="The ID of the comment that triggered this run (optional).", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + git = Git(".") + gh = GitHub() + return cls(args, git, gh).run() diff --git a/tools/private/release/create_release_branch.py b/tools/private/release/create_release_branch.py new file mode 100644 index 0000000000..7a9d7eec37 --- /dev/null +++ b/tools/private/release/create_release_branch.py @@ -0,0 +1,131 @@ +"""Subcommand to create a release branch from a merged PR commit.""" + +from tools.private.release.gh import GitHub +from tools.private.release.git import Git +from tools.private.release.release_issue import ( + RELEASE_TITLE_RE, + parse_checklist_state, + update_task_in_body, +) +from tools.private.release.utils import REPO_URL + + +class CreateReleaseBranch: + """Class to create a release branch from a merged PR commit.""" + + def __init__(self, args, git: Git, gh: GitHub): + self.args = args + self.git = git + self.gh = gh + + def run(self) -> int: + """Executes the create-release-branch subcommand.""" + args = self.args + print(f"Evaluating branch creation for tracking issue #{args.issue}...") + body = self.gh.get_issue_body(args.issue) + state = parse_checklist_state(body) + + if ( + state["prepare_release"].status != "done" + or not state["prepare_release"].commit + ): + print( + "Error: Prepare Release task is not marked 'done' with a valid" + " commit SHA." + ) + return 1 + + if state["create_branch"].checked: + print("Release branch has already been created and checked. Skipping.") + return 0 + + # Extract version from issue title + issue_title = self.gh.get_issue_title(args.issue) + version_match = RELEASE_TITLE_RE.search(issue_title) + if not version_match: + print(f"Error: Could not parse version from issue title: {issue_title}") + return 1 + + version = version_match.group(1) + branch_version = ".".join(version.split(".")[:2]) + branch_name = f"release/{branch_version}" + + commit_sha = state["prepare_release"].commit + print(f"Cutting branch {branch_name} from commit {commit_sha}...") + + # Create and push branch without affecting local checkout + self.git.fetch(args.remote) + + if self.git.remote_branch_exists(args.remote, branch_name): + remote_ref = f"{args.remote}/{branch_name}" + remote_sha = self.git.get_commit_sha(remote_ref) + if remote_sha == commit_sha: + print( + f"Branch {branch_name} already exists on {args.remote} and" + f" points to {commit_sha}. Skipping push." + ) + elif self.git.is_ancestor(remote_ref, commit_sha): + print( + f"Branch {branch_name} exists on {args.remote} but can be" + f" fast-forwarded to {commit_sha}. Pushing..." + ) + ref_spec = f"{commit_sha}:refs/heads/{branch_name}" + self.git.push(args.remote, ref_spec) + else: + print( + f"Error: Branch {branch_name} already exists on" + f" {args.remote} at {remote_sha[:8]}, which is not an" + f" ancestor of {commit_sha[:8]}. Cannot fast-forward." + ) + return 1 + else: + print(f"Branch {branch_name} does not exist on {args.remote}. Pushing...") + ref_spec = f"{commit_sha}:refs/heads/{branch_name}" + self.git.push(args.remote, ref_spec) + print( + f"Successfully pushed branch {branch_name} pointing to" + f" {commit_sha} to {args.remote}" + ) + + # Update tracking issue checklist + print("Updating tracking issue checklist...") + branch_url = f"{REPO_URL}/tree/{branch_name}" + metadata = { + "status": "done", + "branch_url": branch_url, + "commit": commit_sha[:8], + } + updated_body = update_task_in_body( + body, "Create Release branch", checked=True, metadata=metadata + ) + self.gh.update_issue_body(args.issue, updated_body) + print("Create Release branch task marked complete successfully!") + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for create-release-branch subcommand.""" + parser = subparsers.add_parser( + "create-release-branch", + help="Create the release branch pointing to the merged PR commit.", + ) + parser.add_argument( + "--issue", + type=int, + required=True, + help="The tracking issue number (required).", + ) + parser.add_argument( + "--remote", + type=str, + required=True, + help="The git remote to create the branch on (required).", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + git = Git(".") + gh = GitHub() + return cls(args, git, gh).run() diff --git a/tools/private/release/create_release_issue.py b/tools/private/release/create_release_issue.py new file mode 100644 index 0000000000..72b71aa716 --- /dev/null +++ b/tools/private/release/create_release_issue.py @@ -0,0 +1,67 @@ +"""Subcommand to create a release tracking issue.""" + +import pathlib +import re + +from tools.private.release.gh import GitHub +from tools.private.release.utils import determine_next_version, semver_type + + +class CreateReleaseIssue: + """Class to create a release tracking issue.""" + + def __init__(self, args, gh: GitHub): + self.args = args + self.gh = gh + + def run(self) -> int: + """Executes the create-release-issue subcommand.""" + version = self.args.version + if version is None: + version = determine_next_version() + + # Concurrency check + open_issues = self.gh.get_open_tracking_issues() + if open_issues: + print("Error: A release is already in progress. Active tracking issues:") + for issue in open_issues: + print(f"- {issue['title']}: {issue['url']}") + return 1 + + template_path = pathlib.Path( + ".github/ISSUE_TEMPLATE/release_tracking_template.md" + ) + if not template_path.exists(): + raise FileNotFoundError(f"Template file not found at {template_path}") + template_content = template_path.read_text(encoding="utf-8") + + is_first_release = version.endswith(".0") + if not is_first_release: + # Patch release: remove RC tasks + lines = template_content.splitlines() + lines = [line for line in lines if not re.search(r"Tag RC\d+", line)] + template_content = "\n".join(lines) + + issue_num = self.gh.create_release_tracking_issue(version, template_content) + print(f"Created tracking issue #{issue_num} for v{version}") + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for create-release-issue subcommand.""" + parser = subparsers.add_parser( + "create-release-issue", + help="Search for open releases and create a new tracking issue.", + ) + parser.add_argument( + "--version", + type=semver_type, + help="The release version (e.g., 0.38.0). If not provided, determined automatically.", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + gh = GitHub() + return cls(args, gh).run() diff --git a/tools/private/release/determine_next_version.py b/tools/private/release/determine_next_version.py new file mode 100644 index 0000000000..541daddfe7 --- /dev/null +++ b/tools/private/release/determine_next_version.py @@ -0,0 +1,30 @@ +"""Subcommand to determine the next version.""" + +from tools.private.release.utils import determine_next_version + + +class DetermineNextVersion: + """Class to determine the next version.""" + + def __init__(self, args, git=None, gh=None): + self.args = args + + def run(self) -> int: + """Executes the determine-next-version subcommand.""" + version = determine_next_version() + print(version) + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for determine-next-version subcommand.""" + parser = subparsers.add_parser( + "determine-next-version", + help="Determine the next version and print it, without making any changes.", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + return cls(args).run() diff --git a/tools/private/release/gh.py b/tools/private/release/gh.py new file mode 100644 index 0000000000..c21041384e --- /dev/null +++ b/tools/private/release/gh.py @@ -0,0 +1,542 @@ +"""GitHub CLI helper functions for the release tool.""" + +import enum +import json +import os +import re +import tempfile +from typing import TypedDict + +from tools.private.release.release_issue import BackportTask +from tools.private.release.shell import run_cmd + +# GitHub label types +RELEASE_LABEL = "type: release" +BACKPORT_LABEL = "type: backport-pr" +RELEASE_PREPARED_LABEL = "release-prepared" + +# GitHub reaction types +# See: https://docs.github.com/en/rest/reactions/reactions?apiVersion=2022-11-28#about-reactions +GH_REACTION_THUMBS_UP = "+1" +GH_REACTION_THUMBS_DOWN = "-1" +GH_REACTION_LAUGH = "laugh" +GH_REACTION_CONFUSED = "confused" +GH_REACTION_HEART = "heart" +GH_REACTION_HOORAY = "hooray" +GH_REACTION_ROCKET = "rocket" +GH_REACTION_EYES = "eyes" + + +class BackportTaskStatus(str, enum.Enum): + """Status strings for backport tasks on a release tracking issue.""" + + PENDING = "pending" + DONE = "done" + RESOLVED = "resolved" + OPEN_PR = "open-pr" + DRAFT_PR = "draft-pr" + ERROR_NOT_FOUND = "error-not-found" + ERROR_CLOSED_PR = "error-closed-pr" + ERROR_NO_MERGE_COMMIT = "error-no-merge-commit" + ERROR_UNKNOWN = "error-unknown" + ERROR_RESOLUTION_FAILED = "error-resolution-failed" + ERROR_MERGE_CONFLICT = "error-merge-conflict" + ERROR_INVALID_PR = "error-invalid-pr" + IGNORE = "ignore" + + def __str__(self) -> str: + return self.value + + +class IssueDict(TypedDict, total=False): + """In-memory representation of a GitHub Issue object. + + See GitHub API docs: + https://docs.github.com/en/rest/issues/issues#get-an-issue + """ + + number: int + title: str + body: str + labels: list[str] + url: str + + +class AutoMergeDict(TypedDict, total=False): + """Representation of auto-merge status on a Pull Request. + + See GitHub API docs: + https://docs.github.com/en/rest/pulls/pulls#get-a-pull-request + """ + + merge_method: str + + +class PrDict(TypedDict, total=False): + """In-memory representation of a GitHub Pull Request object. + + See GitHub API docs: + https://docs.github.com/en/rest/pulls/pulls#get-a-pull-request + """ + + number: int + title: str + body: str + base: str + head: str + labels: list[str] + url: str + state: str + isDraft: bool + mergeCommit: dict[str, str] + auto_merge: AutoMergeDict | None + + +class MultipleTrackingIssuesError(ValueError): + """Raised when multiple open tracking issues are found for a version.""" + + pass + + +class NoTrackingIssueError(ValueError): + """Raised when no open tracking issue is found for a version.""" + + pass + + +class GitHub: + """GitHub CLI helper class for the release tool.""" + + def __init__(self, repo: str = "bazel-contrib/rules_python"): + """Initializes the GitHub helper. + + Args: + repo: The GitHub repository to operate on. + """ + self.repo = repo + + def _run_gh( + self, *args: str, check: bool = True, capture_output: bool = True + ) -> str | None: + """Runs a 'gh' command. + + Args: + *args: Arguments for 'gh' (excluding 'gh'). + check: If True, raises CalledProcessError on failure. + capture_output: If True, captures and returns stdout. + + Returns: + The stdout of the command, stripped, or None. + """ + return run_cmd("gh", *args, check=check, capture_output=capture_output) + + def _gh_issue( + self, *args: str, check: bool = True, capture_output: bool = True + ) -> str | None: + """Runs a 'gh issue' command.""" + return self._run_gh( + "issue", + *args, + f"--repo={self.repo}", + check=check, + capture_output=capture_output, + ) + + def _gh_pr( + self, *args: str, check: bool = True, capture_output: bool = True + ) -> str | None: + """Runs a 'gh pr' command.""" + return self._run_gh( + "pr", + *args, + f"--repo={self.repo}", + check=check, + capture_output=capture_output, + ) + + def list_issues( + self, + *, + fields: str, + label: str | None = None, + state: str | None = None, + search: str | None = None, + ) -> list[IssueDict]: + """Helper to list issues using gh CLI. + + Args: + fields: Comma-separated list of fields to return. + label: Filter by label. + state: Filter by state ('open', 'closed', 'all'). + search: Search query. + + Returns: + A list of issue dictionaries. + """ + cmd = ["list", f"--json={fields}"] + if label: + cmd.append(f"--label={label}") + if state: + cmd.append(f"--state={state}") + if search: + cmd.append(f"--search={search}") + + output = self._gh_issue(*cmd) + return json.loads(output) if output else [] + + def get_open_tracking_issues(self, version: str | None = None) -> list[IssueDict]: + """Finds open tracking issues for release. + + Args: + version: Optional specific version to match (e.g., "1.0.0"). + + Returns: + List of matching open release tracking issue dictionaries. + """ + search = f"Release {version}" if version else None + return self.list_issues( + fields="number,title,url", + label=RELEASE_LABEL, + state="open", + search=search, + ) + + def get_release_tracking_issue(self, version: str) -> int: + """Finds the single open tracking issue for a given version. + + Args: + version: Version string (e.g. "1.0.0"). + + Returns: + The issue number. + + Raises: + NoTrackingIssueError: If no open tracking issue is found. + MultipleTrackingIssuesError: If multiple open tracking issues are found. + """ + issues = self.get_open_tracking_issues(version) + matching = [i for i in issues if i["title"] == f"Release {version}"] + if not matching: + raise NoTrackingIssueError( + f"No open tracking issue found for Release {version}" + ) + if len(matching) > 1: + raise MultipleTrackingIssuesError( + f"Multiple open tracking issues found for Release {version}: " + + ", ".join(str(i["number"]) for i in matching) + ) + return matching[0]["number"] + + def create_issue( + self, title: str, body: str, labels: list[str] | None = None + ) -> int: + """Creates an issue using gh CLI. + + Args: + title: Title of the issue. + body: Body of the issue. + labels: List of labels to add. + + Returns: + The issue number. + """ + cmd = ["create", f"--title={title}", f"--body={body}"] + if labels: + for label in labels: + cmd.append(f"--label={label}") + + output = self._gh_issue(*cmd) + if not output: + raise RuntimeError("gh issue create returned no output") + # output is URL: https://github.com/owner/repo/issues/123 + return int(output.rstrip("/").split("/")[-1]) + + def create_release_tracking_issue(self, version: str, template_content: str) -> int: + """Creates a release tracking issue from a template. + + Args: + version: Release version string (e.g., "1.0.0"). + template_content: Content of the issue template markdown file. + + Returns: + The created issue number. + """ + title = f"Release {version}" + # Strip YAML frontmatter if present + issue_body = template_content + if template_content.startswith("---"): + parts = template_content.split("---", 2) + if len(parts) >= 3: + issue_body = parts[2].strip() + + return self.create_issue(title=title, body=issue_body, labels=[RELEASE_LABEL]) + + def get_issue_body(self, issue_num: int) -> str: + """Gets the body content of an issue. + + Args: + issue_num: The issue number. + + Returns: + The body string of the issue. + """ + output = self._gh_issue("view", str(issue_num), "--json=body") + if not output: + return "" + data = json.loads(output) + return data.get("body", "") + + def get_issue_title(self, issue_num: int) -> str: + """Gets the title of an issue. + + Args: + issue_num: The issue number. + + Returns: + The title string of the issue. + """ + output = self._gh_issue("view", str(issue_num), "--json=title") + if not output: + return "" + data = json.loads(output) + return data.get("title", "") + + def update_issue_body(self, issue_num: int, body: str) -> None: + """Updates the body of an issue. + + Args: + issue_num: The issue number. + body: The new body content. + """ + with tempfile.NamedTemporaryFile(mode="w", delete=False, encoding="utf-8") as f: + f.write(body) + f.flush() + temp_path = f.name + + try: + self._gh_issue( + "edit", str(issue_num), f"--body-file={temp_path}", capture_output=False + ) + finally: + if os.path.exists(temp_path): + os.remove(temp_path) + + def resolve_pr_number(self, pr_ref: str) -> int: + """Resolves a PR reference (number, #number, or GitHub URL) to a PR number. + + Args: + pr_ref: PR number string (e.g., "123", "#123") or URL. + + Returns: + The integer PR number. + + Raises: + ValueError: If the PR reference cannot be resolved or is for another repo. + """ + clean_ref = pr_ref.lstrip("#") + if clean_ref.isdigit(): + return int(clean_ref) + + if pr_ref.startswith("http"): + pattern = rf"github\.com/{re.escape(self.repo)}/pull/(\d+)(/|\?|\Z)" + match = re.search(pattern, pr_ref, re.IGNORECASE) + if match: + return int(match.group(1)) + raise ValueError( + f"URL is not for the configured repository ({self.repo}): {pr_ref}" + ) + + raise ValueError(f"Could not resolve PR reference: {pr_ref}") + + def get_pr_info(self, pr_num: int) -> PrDict: + """Gets info about a PR using gh CLI. + + Args: + pr_num: The PR number. + + Returns: + Dictionary containing PR fields (state, isDraft, mergeCommit, etc.). + """ + output = self._gh_pr("view", str(pr_num), "--json=state,isDraft,mergeCommit") + return json.loads(output) if output else {} + + def get_pr_comments(self, pr_num: int) -> list[dict]: + """Gets all comments for a PR using gh CLI. + + Args: + pr_num: The PR number. + + Returns: + List of comment objects (with body, author, etc.). + """ + output = self._gh_pr("view", str(pr_num), "--json=comments") + if not output: + return [] + data = json.loads(output) + return data.get("comments", []) + + def create_pr( + self, + title: str, + body: str, + base: str = "main", + labels: list[str] | None = None, + ) -> str: + """Creates a pull request. + + Args: + title: Title of the PR. + body: Body of the PR. + base: Base branch to merge into (default: "main"). + labels: Optional list of labels to add. + + Returns: + The URL of the created PR. + """ + cmd = [ + "create", + f"--title={title}", + f"--body={body}", + f"--base={base}", + ] + if labels: + for label in labels: + cmd.append(f"--label={label}") + output = self._gh_pr(*cmd) + return output if output else "" + + def enable_auto_merge(self, pr_num: int, method: str = "squash") -> None: + """Enables auto-merge for a PR. + + Args: + pr_num: The PR number. + method: The merge method ('squash', 'rebase', or 'merge'). + """ + cmd = ["merge", str(pr_num), "--auto"] + if method == "squash": + cmd.append("--squash") + elif method == "rebase": + cmd.append("--rebase") + elif method == "merge": + cmd.append("--merge") + self._gh_pr(*cmd, capture_output=False) + + def get_open_pr(self, branch_name: str) -> PrDict | None: + """Finds an open PR for the given branch. + + Args: + branch_name: The head branch name to search for. + + Returns: + Dictionary with 'number' and 'url' if an open PR exists, else None. + """ + cmd = [ + "list", + f"--head={branch_name}", + "--state=open", + "--json=number,url", + ] + output = self._gh_pr(*cmd) + prs = json.loads(output) if output else [] + return prs[0] if prs else None + + def post_issue_comment(self, issue_num: int, comment_body: str) -> None: + """Posts a comment on an issue or PR. + + Args: + issue_num: The issue or PR number. + comment_body: The body content of the comment. + """ + self._gh_issue( + "comment", + str(issue_num), + f"--body={comment_body}", + capture_output=False, + ) + + def add_comment_reaction(self, comment_id: int, reaction: str) -> None: + """Adds a reaction to an issue or PR comment. + + Args: + comment_id: The comment ID (note: gh api endpoint needed for comment reactions). + reaction: The reaction type (e.g., "+1", "-1", "rocket"). + """ + self._run_gh( + "api", + f"repos/{self.repo}/issues/comments/{comment_id}/reactions", + "-f", + f"content={reaction}", + capture_output=False, + ) + + def get_merge_commits_for_prs( + self, pending_items: list[BackportTask] + ) -> list[BackportTask]: + """Resolves PR references in pending backports to their merge commit SHAs. + + Updates item.status based on PR state if it cannot be resolved. + + Args: + pending_items: A list of BackportTask items to resolve. + + Returns: + The list of resolved BackportTask items. + """ + return resolve_merge_commits_for_prs(self, pending_items) + + +def resolve_merge_commits_for_prs( + gh_client: GitHub, pending_items: list[BackportTask] +) -> list[BackportTask]: + """Resolves PR references in pending backports to their merge commit SHAs. + + Updates item.status based on PR state if it cannot be resolved. + + Args: + gh_client: The GitHub client. + pending_items: A list of BackportTask items to resolve. + + Returns: + The list of resolved BackportTask items. + """ + resolved_items = [] + for item in pending_items: + pr_num = int(item.pr_ref.lstrip("#")) + print(f"Resolving PR #{pr_num} to merge commit...") + try: + pr_info = gh_client.get_pr_info(pr_num) + if not pr_info: + print(f"PR #{pr_num} not found. Gating.") + item.status = BackportTaskStatus.ERROR_NOT_FOUND + else: + state = pr_info.get("state") + is_draft = pr_info.get("isDraft", False) + if state == "OPEN" or is_draft: + print( + f"PR #{pr_num} is open or draft (state: {state}," + f" draft: {is_draft}). Ignoring." + ) + item.status = ( + BackportTaskStatus.OPEN_PR + if not is_draft + else BackportTaskStatus.DRAFT_PR + ) + elif state == "CLOSED": + print(f"PR #{pr_num} is closed but not merged. Gating.") + item.status = BackportTaskStatus.ERROR_CLOSED_PR + elif state == "MERGED": + merge_commit = pr_info.get("mergeCommit") + if merge_commit and "oid" in merge_commit: + item.commit = merge_commit["oid"] + item.status = BackportTaskStatus.RESOLVED + else: + print(f"PR #{pr_num} has no merge commit SHA. Gating.") + item.status = BackportTaskStatus.ERROR_NO_MERGE_COMMIT + else: + print(f"PR #{pr_num} has unknown state: {state}. Gating.") + item.status = BackportTaskStatus.ERROR_UNKNOWN + except Exception as e: + print(f"Error resolving PR #{pr_num}: {e}. Gating.") + item.status = BackportTaskStatus.ERROR_RESOLUTION_FAILED + resolved_items.append(item) + return resolved_items diff --git a/tools/private/release/git.py b/tools/private/release/git.py new file mode 100644 index 0000000000..3e7e3cb893 --- /dev/null +++ b/tools/private/release/git.py @@ -0,0 +1,421 @@ +"""Git helper functions for the release tool.""" + +import subprocess + +from tools.private.release.shell import run_cmd + + +class Git: + """Git helper class for the release tool. + + Operates on a specific git repository path. + """ + + def __init__(self, repo: str): + """Initializes the Git helper. + + Args: + repo: The path to the git repository. + """ + self._repo = repo + + def _run_git( + self, *args: str, check: bool = True, capture_output: bool = True + ) -> str | None: + """Runs a git command in the repository directory. + + Args: + *args: Arguments passed to the git command. + check: If True, raises CalledProcessError on failure. + capture_output: If True, captures and returns stdout. + + Returns: + The stdout of the command, stripped, or None if capture_output is + False. + """ + return run_cmd( + "git", + *args, + check=check, + capture_output=capture_output, + cwd=self._repo, + ) + + def get_tags(self) -> list[str]: + """Returns a list of all git tags in the repository. + + Returns: + A list of tag names (strings). + """ + output = self._run_git("tag") + return output.splitlines() if output else [] + + def checkout( + self, + ref: str, + create_branch: bool = False, + track_remote: str | None = None, + ) -> None: + """Checks out a git reference (tag, branch, or commit). + + Args: + ref: The git reference (tag, branch, or commit) to checkout. + create_branch: If True, creates the branch before checking it out. + track_remote: If specified, checks out the branch tracking this + remote's corresponding branch. + """ + cmd = ["checkout"] + if create_branch: + cmd.append("-b") + + should_reset_hard = False + if track_remote: + if self.branch_exists(ref): + cmd.append(ref) + should_reset_hard = True + else: + cmd.extend(["--track", f"{track_remote}/{ref}"]) + else: + cmd.append(ref) + self._run_git(*cmd, capture_output=False) + + if should_reset_hard: + self.reset_hard(reset_to=f"{track_remote}/{ref}") + + def add(self, *files: str) -> None: + """Stages files for commit. + + Args: + *files: Paths to files to stage. + """ + self._run_git("add", *files, capture_output=False) + + def add_modified_and_deleted(self) -> None: + """Stages all modified and deleted tracked files.""" + self._run_git("add", "--update", capture_output=False) + + def commit(self, message: str, amend: bool = False, no_edit: bool = False) -> None: + """Commits staged changes, optionally amending the previous commit. + + Args: + message: The commit message. + amend: If True, amends the previous commit. + no_edit: If True, uses the existing commit message without editing. + """ + cmd = ["commit"] + if amend: + cmd.append("--amend") + if no_edit: + cmd.append("--no-edit") + if message: + cmd.extend(["-m", message]) + self._run_git(*cmd, capture_output=False) + + def push( + self, + remote: str, + ref: str, + set_upstream: bool = False, + force: bool = False, + ) -> None: + """Pushes a reference to a remote repository. + + Args: + remote: The remote repository name (e.g., 'origin'). + ref: The reference to push (e.g., a branch name). + set_upstream: If True, sets the upstream tracking branch. + force: If True, force pushes the changes. + """ + cmd = ["push"] + if set_upstream: + cmd.append("--set-upstream") + if force: + cmd.append("--force") + cmd.extend([remote, ref]) + self._run_git(*cmd, capture_output=False) + + def fetch( + self, + remote: str = "origin", + refspec: str | None = None, + tags: bool = False, + force: bool = False, + ) -> None: + """Fetches updates from a remote repository. + + Args: + remote: The remote repository name. Defaults to 'origin'. + refspec: The refspec to fetch. + tags: If True, fetches all tags. + force: If True, force fetches updates. + """ + cmd = ["fetch", remote] + if refspec: + cmd.append(refspec) + if tags: + cmd.append("--tags") + if force: + cmd.append("--force") + self._run_git(*cmd, capture_output=False) + + def merge(self, commit_ref: str, ff_only: bool = True) -> None: + """Merges a commit into the current branch. + + Args: + commit_ref: The commit reference to merge. + ff_only: If True, only allows fast-forward merges. + """ + cmd = ["merge", commit_ref] + if ff_only: + cmd.append("--ff-only") + self._run_git(*cmd, capture_output=False) + + def tag(self, tag_name: str, commit_ref: str) -> None: + """Creates a local tag pointing to a specific commit. + + Args: + tag_name: The name of the tag to create. + commit_ref: The commit reference the tag should point to. + """ + self._run_git("tag", tag_name, commit_ref, capture_output=False) + + def cherry_pick(self, sha: str) -> None: + """Cherry-picks a commit. + + Args: + sha: The commit SHA to cherry-pick. + """ + self._run_git("cherry-pick", "-x", sha, capture_output=False) + + def cherry_pick_abort(self) -> None: + """Aborts an in-progress cherry-pick operation.""" + self._run_git("cherry-pick", "--abort", capture_output=False) + + def reset_hard(self, *, reset_to: str = "HEAD") -> None: + """Resets the index and working tree to a specific reference. + + Args: + reset_to: The git reference to reset to. Defaults to 'HEAD'. + """ + self._run_git("reset", "--hard", reset_to, capture_output=False) + + def status(self) -> str: + """Returns the output of git status --porcelain. + + Returns: + The porcelain status output. + """ + output = self._run_git("status", "--porcelain") + return output if output else "" + + def get_commit_sha(self, ref: str = "HEAD", short: bool = False) -> str: + """Returns the commit SHA of a given reference. + + Args: + ref: The git reference. Defaults to 'HEAD'. + short: If True, returns a short SHA. + + Returns: + The commit SHA. + """ + cmd = ["rev-parse"] + if short: + cmd.append("--short") + cmd.append(ref) + output = self._run_git(*cmd) + return output if output else "" + + def get_commit_message(self, ref: str = "HEAD") -> str: + """Returns the commit message of a given reference. + + Args: + ref: The git reference. Defaults to 'HEAD'. + + Returns: + The commit message. + """ + output = self._run_git("log", "-1", "--format=%B", ref) + return output if output else "" + + def branch_exists(self, branch_name: str) -> bool: + """Returns True if a local branch exists. + + Args: + branch_name: The name of the branch to check. + + Returns: + True if the branch exists, False otherwise. + """ + try: + self._run_git("show-ref", "--verify", f"refs/heads/{branch_name}") + return True + except subprocess.CalledProcessError: + return False + + def tag_exists(self, tag_name: str) -> bool: + """Returns True if a local tag exists. + + Args: + tag_name: The name of the tag to check. + + Returns: + True if the tag exists, False otherwise. + """ + try: + self._run_git("show-ref", "--verify", f"refs/tags/{tag_name}") + return True + except subprocess.CalledProcessError: + return False + + def sort_commits_chronologically(self, shas: list[str]) -> list[str]: + """Sorts a list of commit SHAs chronologically (oldest first). + + Args: + shas: A list of commit SHAs to sort. + + Returns: + The sorted list of commit SHAs. + """ + output = self._run_git("log", "--no-walk", "--reverse", "--format=%H", *shas) + return output.splitlines() if output else [] + + def get_current_branch(self) -> str: + """Returns the current git branch name. + + Returns: + The current branch name. + """ + output = self._run_git("rev-parse", "--abbrev-ref", "HEAD") + return output if output else "" + + def remote_branch_exists(self, remote: str, branch_name: str) -> bool: + """Returns True if a remote branch exists. + + Args: + remote: The name of the remote. + branch_name: The name of the branch. + + Returns: + True if the remote branch exists, False otherwise. + """ + try: + self._run_git( + "show-ref", + "--verify", + f"refs/remotes/{remote}/{branch_name}", + ) + return True + except subprocess.CalledProcessError: + return False + + def is_ancestor(self, ancestor: str, descendant: str) -> bool: + """Returns True if ancestor is an ancestor of descendant. + + Args: + ancestor: The commit reference that might be an ancestor. + descendant: The commit reference that might be a descendant. + + Returns: + True if ancestor is an ancestor of descendant, False otherwise. + """ + try: + self._run_git("merge-base", "--is-ancestor", ancestor, descendant) + return True + except subprocess.CalledProcessError: + return False + + def get_remote_tags(self, remote: str) -> list[str]: + """Returns a list of tags present on the specified remote repository. + + Args: + remote: The name of the git remote to query (e.g., 'origin', + 'upstream'). + + Returns: + A list of tag names (strings) found on the remote, excluding peeled + tags. + """ + output = self._run_git("ls-remote", "--tags", remote) + tags = [] + if not output: + return tags + for line in output.splitlines(): + if not line: + continue + parts = line.split() + if len(parts) < 2: + continue + ref = parts[1] + if ref.startswith("refs/tags/"): + tag = ref[len("refs/tags/") :] + # Skip peeled tags (e.g. tag^{}) to avoid + # duplicate tag names in the output. + if not tag.endswith("^{}"): + tags.append(tag) + return tags + + def get_modified_files(self, ref: str) -> list[str]: + """Returns a list of files modified in a given reference. + + Args: + ref: The git reference. + + Returns: + A list of file paths. + """ + output = self._run_git("show", "--name-only", "--format=", ref) + return [line for line in output.splitlines() if line.strip()] if output else [] + + def diff(self) -> str: + """Returns the diff of unstaged changes. + + Returns: + The diff output as a string. + """ + output = self._run_git("diff") + return output if output else "" + + def apply(self, patch_file: str) -> None: + """Applies a patch file. + + Args: + patch_file: The path to the patch file. + """ + self._run_git("apply", patch_file, capture_output=False) + + def apply_check(self, patch_file: str) -> bool: + """Verifies if a patch can be applied cleanly. + + Args: + patch_file: The path to the patch file. + + Returns: + True if the patch can be applied cleanly, False otherwise. + """ + try: + self._run_git("apply", "--check", patch_file, capture_output=False) + return True + except subprocess.CalledProcessError: + return False + + def get_remote_branches(self, remote: str = "origin") -> list[str]: + """Returns a list of remote branches. + + Args: + remote: The name of the remote. + + Returns: + A list of branch names (without the remote prefix). + """ + output = self._run_git("branch", "-r") + branches = [] + if not output: + return branches + for line in output.splitlines(): + line = line.strip() + if "->" in line: + continue + parts = line.split("/") + if len(parts) >= 2 and parts[0] == remote: + branches.append("/".join(parts[1:])) + return branches diff --git a/tools/private/release/mock_gh.py b/tools/private/release/mock_gh.py new file mode 100644 index 0000000000..e5def53799 --- /dev/null +++ b/tools/private/release/mock_gh.py @@ -0,0 +1,161 @@ +"""In-memory fake for GitHub API.""" + +import re + +from tools.private.release.gh import ( + RELEASE_LABEL, + IssueDict, + MultipleTrackingIssuesError, + NoTrackingIssueError, + PrDict, + resolve_merge_commits_for_prs, +) + + +class MockGitHub: + def __init__(self, repo: str = "bazel-contrib/rules_python"): + self.repo = repo + self.issues: dict[int, IssueDict] = {} + self.next_issue_num = 1001 + self.prs: dict[int, PrDict] = {} # num -> pr_info + self.issue_comments: dict[int, list[str]] = {} + self.reactions: dict[int, list[str]] = {} + self.pr_comments: dict[int, list[dict]] = {} + + def post_issue_comment(self, issue_num: int, comment_body: str) -> None: + self.issue_comments.setdefault(issue_num, []).append(comment_body) + + def add_comment_reaction(self, comment_id: int, reaction: str) -> None: + self.reactions.setdefault(comment_id, []).append(reaction) + + def enable_auto_merge(self, pr_num: int, method: str = "squash") -> None: + if pr_num not in self.prs: + self.create_pr(title="", body="") + self.prs[pr_num]["auto_merge"] = {"merge_method": method} + + def create_issue( + self, title: str, body: str, labels: list[str] | None = None + ) -> int: + issue_num = self.next_issue_num + self.next_issue_num += 1 + self.issues[issue_num] = { + "title": title, + "body": body, + "labels": labels or [], + "number": issue_num, + "url": f"https://github.com/{self.repo}/issues/{issue_num}", + } + return issue_num + + def create_release_tracking_issue(self, version: str, template_content: str) -> int: + # Strip YAML frontmatter if present (simplified copy from gh.py) + issue_body = template_content + if template_content.startswith("---"): + parts = template_content.split("---", 2) + if len(parts) >= 3: + issue_body = parts[2].strip() + + return self.create_issue( + title=f"Release {version}", body=issue_body, labels=[RELEASE_LABEL] + ) + + def get_issue_body(self, issue_num: int) -> str: + if issue_num not in self.issues: + raise ValueError(f"Issue #{issue_num} not found in MockGitHub") + return self.issues[issue_num]["body"] + + def get_issue_title(self, issue_num: int) -> str: + if issue_num not in self.issues: + raise ValueError(f"Issue #{issue_num} not found in MockGitHub") + return self.issues[issue_num]["title"] + + def update_issue_body(self, issue_num: int, body: str): + if issue_num not in self.issues: + raise ValueError(f"Issue #{issue_num} not found in MockGitHub") + self.issues[issue_num]["body"] = body + + def resolve_pr_number(self, pr_ref: str) -> int: + # Real algorithm copy (doesn't require RPCs) + clean_ref = pr_ref.lstrip("#") + if clean_ref.isdigit(): + return int(clean_ref) + + if pr_ref.startswith("http"): + pattern = rf"github\.com/{re.escape(self.repo)}/pull/(\d+)(/|\?|\Z)" + match = re.search(pattern, pr_ref, re.IGNORECASE) + if match: + return int(match.group(1)) + raise ValueError( + f"URL is not for the configured repository ({self.repo}): {pr_ref}" + ) + raise ValueError(f"Could not resolve PR ref: {pr_ref}") + + def get_release_tracking_issue(self, version: str) -> int: + search_title = f"Release {version}" + matching = [ + num + for num, issue in self.issues.items() + if issue["title"] == search_title + and RELEASE_LABEL in issue.get("labels", []) + ] + if not matching: + raise NoTrackingIssueError( + f"No open tracking issue found for Release {version}" + ) + if len(matching) > 1: + raise MultipleTrackingIssuesError( + f"Multiple open tracking issues found for Release {version}" + ) + return matching[0] + + def create_pr( + self, + title: str, + body: str, + base: str = "main", + labels: list[str] | None = None, + ) -> str: + pr_num = self.next_issue_num + self.next_issue_num += 1 + url = f"https://github.com/{self.repo}/pull/{pr_num}" + self.prs[pr_num] = { + "title": title, + "body": body, + "base": base, + "labels": labels or [], + "number": pr_num, + "url": url, + "state": "OPEN", + } + return url + + def get_open_pr(self, branch_name: str) -> PrDict | None: + for pr in self.prs.values(): + if pr.get("head") == branch_name and pr.get("state") == "OPEN": + return pr + return None + + def get_open_tracking_issues(self, version: str | None = None) -> list[IssueDict]: + results = [] + for issue in self.issues.values(): + if RELEASE_LABEL in issue["labels"]: + if version: + if issue["title"] == f"Release {version}": + results.append(issue) + else: + results.append(issue) + return results + + def get_pr_info(self, pr_num: int) -> PrDict: + if pr_num in self.prs: + return self.prs[pr_num] + return { + "state": "MERGED", + "mergeCommit": {"oid": f"mock_merge_sha_{pr_num}"}, + } + + def get_pr_comments(self, pr_num: int) -> list[dict]: + return self.pr_comments.get(pr_num, []) + + def get_merge_commits_for_prs(self, pending_items: list) -> list: + return resolve_merge_commits_for_prs(self, pending_items) diff --git a/tools/private/release/on_pr_merged.py b/tools/private/release/on_pr_merged.py new file mode 100644 index 0000000000..8d451580fb --- /dev/null +++ b/tools/private/release/on_pr_merged.py @@ -0,0 +1,113 @@ +"""Subcommand to handle PR merge event by processing backports.""" + +import argparse +import re + +from tools.private.release.gh import GitHub +from tools.private.release.git import Git +from tools.private.release.process_backports import ProcessBackports +from tools.private.release.release_issue import parse_backports + + +class OnPrMerged: + """Class to handle PR merge event.""" + + def __init__(self, args, git: Git, gh: GitHub): + self.args = args + self.git = git + self.gh = gh + + def run(self) -> int: + """Executes the on-pr-merged subcommand.""" + args = self.args + pr_num = args.pr + pr_ref = f"#{pr_num}" + + print(f"Verifying PR {pr_ref} has backport comment...") + try: + comments = self.gh.get_pr_comments(pr_num) + has_comment = any( + re.match( + r"^\s*/backport(\s|$)", + comment.get("body") or "", + re.IGNORECASE, + ) + for comment in comments + ) + if not has_comment: + print(f"PR {pr_ref} does not have a /backport comment. Skipping.") + return 1 + except Exception as e: + print(f"Error checking PR comments: {e}") + return 1 + + print(f"Searching for active release tracking issue containing PR {pr_ref}...") + open_issues = self.gh.get_open_tracking_issues() + if not open_issues: + print("No open release tracking issues found.") + return 1 + + found_issue = None + for issue in open_issues: + issue_num = issue["number"] + body = self.gh.get_issue_body(issue_num) + backports = parse_backports(body) + + if any(item.pr_ref == pr_ref for item in backports): + if found_issue: + print( + f"Error: PR {pr_ref} found in multiple open release" + f" tracking issues: #{found_issue} and #{issue_num}" + ) + return 1 + found_issue = issue_num + + if not found_issue: + print(f"PR {pr_ref} not found in any active release tracking issue.") + return 1 + + print(f"Found PR {pr_ref} in tracking issue #{found_issue}") + + # Now run ProcessBackports for this issue + process_args = argparse.Namespace( + issue=found_issue, + remote=args.remote, + add=None, + triggering_comment=None, + dry_run=args.dry_run, + ) + print(f"Processing backports for issue #{found_issue}...") + return ProcessBackports(process_args, self.git, self.gh).run() + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for on-pr-merged subcommand.""" + parser = subparsers.add_parser( + "on-pr-merged", + help="Handle PR merge event by processing backports.", + ) + parser.add_argument( + "pr", + type=int, + help="PR number that was merged.", + ) + parser.add_argument( + "--remote", + type=str, + required=True, + help="The git remote to push changes to (required).", + ) + parser.add_argument( + "--dry-run", + action=argparse.BooleanOptionalAction, + default=True, + help="Perform a dry run (default: True). Use --no-dry-run to actually execute.", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + git = Git(".") + gh = GitHub() + return cls(args, git, gh).run() diff --git a/tools/private/release/prepare.py b/tools/private/release/prepare.py new file mode 100644 index 0000000000..00412a8600 --- /dev/null +++ b/tools/private/release/prepare.py @@ -0,0 +1,240 @@ +"""Subcommand to prepare the release (updates changelog, placeholders).""" + +import argparse +import datetime +import pathlib + +from tools.private.release import changelog_news +from tools.private.release.gh import ( + RELEASE_PREPARED_LABEL, + GitHub, + MultipleTrackingIssuesError, + NoTrackingIssueError, +) +from tools.private.release.git import Git +from tools.private.release.release_issue import ( + parse_checklist_state, + update_task_in_body, +) +from tools.private.release.utils import ( + determine_next_version, + replace_version_next, + semver_type, +) + + +class Prepare: + """Class to prepare the release.""" + + def __init__(self, args, git: Git, gh: GitHub): + self.args = args + self.git = git + self.gh = gh + + def run(self) -> int: + """Executes the prepare subcommand.""" + args = self.args + print("Fetching upstream to verify fresh release history...") + self.git.fetch(tags=True, force=True) + + # Run pre-check: verify there are no local edits + status = self.git.status() + if status: + print( + "Error: Local edits detected. Workspace must be completely clean" + " before running release preparation." + ) + for line in status.splitlines(): + print(f" {line}") + return 1 + print("Pre-check passed: Workspace is clean.") + + version = args.version + if version is None: + version = determine_next_version() + + print(f"Running preparation pipeline for {version}...") + + # 1. Find or create tracking issue (EARLY) + # We do this before any write operations (branch creation, commit, push) + issue_num = args.issue + + if not issue_num: + try: + issue_num = self.gh.get_release_tracking_issue(version) + print(f"Tracking issue: #{issue_num}") + except MultipleTrackingIssuesError as e: + print(f"Error: {e}") + return 1 + except NoTrackingIssueError: + # Not found, we need the template + template_path = pathlib.Path( + ".github/ISSUE_TEMPLATE/release_tracking_template.md" + ) + if not template_path.exists(): + raise FileNotFoundError( + f"Template file not found at {template_path}" + ) + template_content = template_path.read_text(encoding="utf-8") + + if args.dry_run: + print( + f"[DRY RUN] No active tracking issue found for" + f" {version}. Would create a new one." + ) + print(f"[DRY RUN] Title: Release {version}\n{template_content}") + issue_num = None # Keep it None for dry-run prints later + else: + print( + f"No active tracking issue found for {version}." + " Creating a new one..." + ) + issue_num = self.gh.create_release_tracking_issue( + version, template_content + ) + print(f"Tracking issue: #{issue_num}") + else: + print(f"Tracking issue: #{issue_num}") + + branch_name = f"prepare-{version}" + + # 2. Interleaved git and write operations + + # --- Branch selection/creation --- + if self.git.branch_exists(branch_name): + if args.dry_run: + print( + f"[DRY RUN] Branch {branch_name} already exists. Would" + " checkout existing branch." + ) + else: + print(f"Branch {branch_name} already exists. Checking it out...") + self.git.checkout(branch_name) + else: + if args.dry_run: + print(f"[DRY RUN] Would create and checkout branch {branch_name}") + else: + self.git.checkout(branch_name, create_branch=True) + + # --- Update files --- + if args.dry_run: + print( + f"[DRY RUN] Would update CHANGELOG.md and version placeholders" + f" for {version}" + ) + else: + print("Updating changelog and placeholders...") + release_date = datetime.date.today().strftime("%Y-%m-%d") + changelog_news.update_changelog(version, release_date) + replace_version_next(version) + + # --- Commit and Push --- + if args.dry_run: + print(f"[DRY RUN] Would push branch {branch_name} to origin") + else: + modified_files = self.git.status() + if modified_files: + # Stage all modified and deleted tracked files + self.git.add_modified_and_deleted() + self.git.commit(f"Prepare release {version}") + else: + print("No files modified by the release tool. Nothing to commit.") + + print(f"Pushing branch {branch_name} to origin...") + # Force push to overwrite the remote branch if it already exists (e.g. from a previous run) + self.git.push("origin", branch_name, set_upstream=True, force=True) + + # --- Create PR --- + # Determine if we need to create a PR or reuse an existing one + open_pr = self.gh.get_open_pr(branch_name) + associated_pr = None + + if not open_pr and issue_num: + body = self.gh.get_issue_body(issue_num) + state = parse_checklist_state(body) + associated_pr = state["prepare_release"].pr + + if open_pr: + pr_num = open_pr["number"] + pr_url = open_pr["url"] + print(f"Open Pull Request already exists: {pr_url} (PR #{pr_num})") + elif associated_pr: + pr_num = associated_pr.lstrip("#") + pr_url = f"https://github.com/bazel-contrib/rules_python/pull/{pr_num}" + print( + f"PR #{pr_num} is already associated in tracking issue" + f" #{issue_num}. Using it." + ) + else: + if args.dry_run: + target_issue = f"#{issue_num}" if issue_num else "" + print( + f"[DRY RUN] Would create Pull Request for branch" + f" {branch_name} targeting issue {target_issue}" + ) + pr_num = "" + else: + pr_url = self.gh.create_pr( + title=f"Prepare release v{version}", + body=f"Work towards #{issue_num}", + base="main", + labels=[RELEASE_PREPARED_LABEL], + ) + pr_num = pr_url.split("/")[-1] + print(f"Created Pull Request: {pr_url} (PR #{pr_num})") + + # --- Update checklist --- + if args.dry_run: + target_issue = f"#{issue_num}" if issue_num else "" + print( + f"[DRY RUN] Would update tracking issue {target_issue} checklist" + " 'Prepare Release' task status to PENDING" + ) + else: + print( + f"Updating tracking issue #{issue_num} checklist 'Prepare" + " Release' task status to PENDING..." + ) + body = self.gh.get_issue_body(issue_num) + metadata = {"status": "pending", "pr": f"#{pr_num}"} + updated_body = update_task_in_body( + body, "Prepare Release", checked=False, metadata=metadata + ) + self.gh.update_issue_body(issue_num, updated_body) + print("Preparation pipeline completed successfully!") + + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for prepare subcommand.""" + parser = subparsers.add_parser( + "prepare", + help="Prepare the release (updates changelog, placeholders).", + ) + parser.add_argument( + "version", + nargs="?", + type=semver_type, + help="The new release version (e.g., 0.28.0). If not provided, " + "it will be determined automatically.", + ) + parser.add_argument( + "--issue", + type=int, + help="The tracking issue number (optional, triggers automated branch/PR pipeline).", + ) + parser.add_argument( + "--dry-run", + action=argparse.BooleanOptionalAction, + default=True, + help="Perform a dry run (default: True). Use --no-dry-run to actually execute.", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + git = Git(".") + gh = GitHub() + return cls(args, git, gh).run() diff --git a/tools/private/release/process_backports.py b/tools/private/release/process_backports.py new file mode 100644 index 0000000000..27eb3226b1 --- /dev/null +++ b/tools/private/release/process_backports.py @@ -0,0 +1,631 @@ +"""Subcommand to process pending backports.""" + +import argparse +import datetime +import hashlib +import os +import tempfile +from dataclasses import dataclass +from typing import Any + +from tools.private.release import changelog_news +from tools.private.release.gh import GH_REACTION_THUMBS_DOWN, GitHub +from tools.private.release.git import Git +from tools.private.release.release_issue import ( + RELEASE_TITLE_RE, + add_backports_to_body, + add_rc_task_to_body, + add_sync_changelog_task_to_body, + parse_backports, + parse_checklist_state, + update_task_in_body, +) +from tools.private.release.utils import ( + get_latest_rc_tag, + parse_pr_list, + replace_version_next, +) + + +@dataclass +class CherryPickAndUpdatePrsResult: + # List of PR references that failed to cherry-pick. + failed_prs: list[str] + # List of news files collected from the successful cherry-picks. + collected_news_files: list[str] + # List of PR numbers that were successfully cherry-picked. + successful_pr_nums: list[int] + # List of tuples mapping successful PR numbers to their version marker diffs. + collected_diffs: list[tuple[int, str]] + # The updated checklist body for the release tracking issue. + body: str + + +class ProcessBackports: + """Class to process pending backports.""" + + def __init__(self, args, git: Git, gh: GitHub): + self.args = args + self.git = git + self.gh = gh + + def _process_pr_commit_infos( + self, pr_commit_infos, body, issue, dry_run + ) -> tuple[list[str], dict[str, Any], list[str], list[str], str]: + shas = [] + sha_to_item = {} + failed_prs = [] + ignored_prs = [] + for item in pr_commit_infos: + if item.commit: + sha = item.commit + sha_to_item[sha] = item + shas.append(sha) + elif item.status in ("open-pr", "draft-pr"): + print(f"PR {item.pr_ref} is open or draft. Ignoring.") + ignored_prs.append(item.pr_ref) + else: + failed_prs.append(item.pr_ref) + status_to_set = item.status or "error-unmerged-pr" + if dry_run: + print( + f"[DRY RUN] Would update tracking issue checklist for" + f" unresolved PR {item.pr_ref} to status={status_to_set}" + ) + else: + print( + f"Updating tracking issue checklist for unresolved PR" + f" {item.pr_ref}..." + ) + try: + body = update_task_in_body( + body, + item.pr_ref, + checked=False, + metadata={"status": status_to_set}, + ) + self.gh.update_issue_body(issue, body) + except Exception as e: + print( + f"ERROR: Failed to update tracking issue for" + f" unresolved PR {item.pr_ref}: {e}" + ) + return shas, sha_to_item, failed_prs, ignored_prs, body + + def _cherry_pick_and_update_prs( + self, + sorted_shas, + sha_to_item, + body, + issue, + remote, + dry_run, + version, + branch_name, + next_rc_suffix, + ) -> CherryPickAndUpdatePrsResult: + failed_prs = [] + collected_news_files = [] + successful_pr_nums = [] + collected_diffs = [] + for sha in sorted_shas: + item = sha_to_item[sha] + print(f"Cherry-picking {item.pr_ref} / {sha}...") + try: + self.git.cherry_pick(sha) + + # Collect news files before they are deleted by update_changelog + modified_files = self.git.get_modified_files("HEAD") + for f in modified_files: + if changelog_news.is_news_file(f): + collected_news_files.append(f) + + # Replace version markers FIRST to isolate diff + print(f"Replacing version markers for PR {item.pr_ref}...") + replace_version_next(version) + + # Get diff of unstaged changes (version marker replacement) + diff_content = self.git.diff() + + # Perform news processing (merging news/ files into the changelog) + print(f"Merging news fragments into changelog for PR {item.pr_ref}...") + release_date = datetime.date.today().strftime("%Y-%m-%d") + changelog_news.update_changelog(version, release_date) + + # Stage changelog changes, news/ deletions, and version placeholder updates + self.git.add_modified_and_deleted() + + # Amend cherry-pick commit to include news merging and deletions, + # and reference the release tracking issue. + print(f"Amending cherry-pick commit for PR {item.pr_ref}...") + current_msg = self.git.get_commit_message("HEAD") + new_msg = f"{current_msg.strip()}\n\nWork towards #{issue}" + self.git.commit(new_msg, amend=True) + + try: + pr_num = self.gh.resolve_pr_number(item.pr_ref) + if diff_content: + collected_diffs.append((pr_num, diff_content)) + successful_pr_nums.append(pr_num) + except Exception as e: + print( + f"Warning: Failed to resolve PR number for {item.pr_ref}: {e}" + ) + + if not dry_run: + # Push amended commit + self.git.push(remote, branch_name) + + new_sha = self.git.get_commit_sha("HEAD", short=True) + metadata = { + "status": "done", + "rc": next_rc_suffix, + "commit": new_sha, + } + print(f"Updating tracking issue checklist for PR {item.pr_ref}...") + try: + body = update_task_in_body( + body, item.pr_ref, checked=True, metadata=metadata + ) + self.gh.update_issue_body(issue, body) + except Exception as e: + print( + f"ERROR: Failed to update tracking issue for PR" + f" {item.pr_ref}: {e}" + ) + print(f"Success: backported {item.pr_ref} / {sha} to {branch_name}") + else: + print( + f"[DRY RUN] Success: {item.pr_ref} / {sha} can be" + f" backported without error." + ) + print( + f"[DRY RUN] Would update tracking issue checklist for" + f" PR {item.pr_ref} to status=done" + ) + except Exception as e: + print(f"ERROR: Conflict or error on {sha}: {e}. Aborting.") + try: + self.git.cherry_pick_abort() + except Exception: + pass + failed_prs.append(item.pr_ref) + + if dry_run: + print( + f"[DRY RUN] Would update tracking issue checklist for" + f" failed PR {item.pr_ref} to status=error-merge-conflict" + ) + else: + print( + f"Updating tracking issue checklist for failed PR" + f" {item.pr_ref}..." + ) + try: + body = update_task_in_body( + body, + item.pr_ref, + checked=False, + metadata={"status": "error-merge-conflict"}, + ) + self.gh.update_issue_body(issue, body) + print( + f"Updated back port of {item.pr_ref} to" + f" status=error-merge-conflict (unchecked)" + ) + except Exception as e: + print( + f"ERROR: Failed to update tracking issue for" + f" failed PR {item.pr_ref}: {e}" + ) + return CherryPickAndUpdatePrsResult( + failed_prs=failed_prs, + collected_news_files=collected_news_files, + successful_pr_nums=successful_pr_nums, + collected_diffs=collected_diffs, + body=body, + ) + + def _sync_changelog_to_main( + self, + version: str, + collected_news_files: list[str], + successful_pr_nums: list[int], + collected_diffs: list[tuple[int, str]], + release_branch: str, + ) -> None: + args = self.args + sorted_prs = sorted(successful_pr_nums) + prs_str = ",".join(str(n) for n in sorted_prs) + prs_hash = hashlib.sha256(prs_str.encode()).hexdigest()[:7] + + main_branch = "main" + backport_branch = f"prepare-{version}-backports-{prs_hash}" + + print(f"Syncing changelog to {main_branch} via branch {backport_branch}...") + + self.git.fetch(args.remote, refspec=main_branch) + self.git.checkout(main_branch, track_remote=args.remote) + main_start_sha = self.git.get_commit_sha("HEAD") + + failed_version_sync_prs = [] + try: + if args.dry_run: + print( + f"[DRY RUN] Would create and checkout branch {backport_branch} from {main_branch}" + ) + else: + if self.git.branch_exists(backport_branch): + self.git.checkout(backport_branch) + self.git.reset_hard(reset_to=main_branch) + else: + self.git.checkout(backport_branch, create_branch=True) + + print( + f"Updating CHANGELOG.md and removing news files on {backport_branch}..." + ) + release_date = datetime.date.today().strftime("%Y-%m-%d") + changelog_news.update_changelog( + version, + release_date, + news_files=collected_news_files, + delete_news=True, + ) + + # Apply version marker diffs + failed_version_sync_prs = self._apply_version_marker_diffs(collected_diffs) + + if args.dry_run: + print( + f"[DRY RUN] Would commit: 'chore(release): sync changelog for v{version} backports'" + ) + print(f"[DRY RUN] Would push {backport_branch} to {args.remote}") + print( + f"[DRY RUN] Would create PR to {main_branch} with label 'type: sync-changelog'" + ) + print( + f"[DRY RUN] Would update tracking issue #{args.issue} checklist tasks 'Sync Changelog #' to PENDING" + ) + print("[DRY RUN] Diff of changes:") + print(self.git.status()) + else: + self.git.add_modified_and_deleted() + self.git.commit( + f"chore(release): sync changelog for v{version} backports" + ) + self.git.push( + args.remote, backport_branch, set_upstream=True, force=True + ) + + pr_title = f"chore(release): sync changelog for v{version} backports" + pr_body_lines = [ + "Updates CHANGELOG.md and removes news files for backports:", + ] + for pr_num in sorted_prs: + pr_body_lines.append(f"- #{pr_num}") + + if failed_version_sync_prs: + pr_body_lines.append("") + pr_body_lines.append( + "Warning: These PRs failed to update their version markers:" + ) + for pr_num in sorted(failed_version_sync_prs): + pr_body_lines.append(f"- #{pr_num}") + + pr_body_lines.append("") + pr_body_lines.append(f"Work towards #{args.issue}") + pr_body_lines.append(f"Release-Tracking-Issue: #{args.issue}") + pr_body = "\n".join(pr_body_lines) + + print(f"Creating PR to {main_branch}...") + pr_url = self.gh.create_pr( + title=pr_title, + body=pr_body, + base=main_branch, + labels=["type: sync-changelog"], + ) + print(f"Created PR: {pr_url}") + + try: + pr_num = int(pr_url.split("/")[-1]) + print(f"Enabling auto-merge for PR #{pr_num}...") + self.gh.enable_auto_merge(pr_num) + + print( + f"Updating tracking issue #{args.issue} checklist with" + " Sync Changelog tasks..." + ) + issue_body = self.gh.get_issue_body(args.issue) + for pr in successful_pr_nums: + task_name = f"Sync Changelog #{pr}" + metadata = {"status": "pending", "pr": f"#{pr_num}"} + issue_body = update_task_in_body( + issue_body, + task_name, + checked=False, + metadata=metadata, + ) + self.gh.update_issue_body(args.issue, issue_body) + except Exception as e: + print( + f"Warning: Failed to update tracking issue or enable" + f" auto-merge: {e}" + ) + finally: + if args.dry_run: + self.git.reset_hard(reset_to=main_start_sha) + self.git.checkout(release_branch) + + def _apply_version_marker_diffs( + self, + collected_diffs: list[tuple[int, str]], + ) -> list[int]: + """Applies version marker diffs on main branch and returns failed PR numbers.""" + args = self.args + failed_version_sync_prs = [] + if not collected_diffs: + return failed_version_sync_prs + + with tempfile.TemporaryDirectory() as temp_dir: + print(f"Applying {len(collected_diffs)} version marker patches...") + for pr_num, diff_content in collected_diffs: + if args.dry_run: + print( + f"[DRY RUN] Would check and apply version marker patch for PR #{pr_num}" + ) + + patch_filepath = os.path.join(temp_dir, f"{pr_num}.patch") + with open(patch_filepath, "w", encoding="utf-8") as f: + f.write(diff_content) + + if self.git.apply_check(patch_filepath): + if args.dry_run: + print( + f"[DRY RUN] Version marker patch for PR #{pr_num} applies cleanly." + ) + else: + print(f"Applying version marker patch for PR #{pr_num}...") + self.git.apply(patch_filepath) + else: + print( + f"Warning: Version marker patch for PR #{pr_num} could not be applied cleanly to main. Skipping." + ) + failed_version_sync_prs.append(pr_num) + return failed_version_sync_prs + + def run(self) -> int: + """Executes the process-backports subcommand.""" + args = self.args + exit_code = 0 + try: + exit_code = self._run_internal() + except Exception as e: + print(f"Unexpected error: {e}") + exit_code = 1 + + if exit_code != 0 and args.triggering_comment: + print(f"Reacting with thumbs-down to comment {args.triggering_comment}...") + try: + self.gh.add_comment_reaction( + args.triggering_comment, GH_REACTION_THUMBS_DOWN + ) + except Exception as e: + print(f"Failed to add reaction to comment: {e}") + + return exit_code + + def _run_internal(self) -> int: + """Internal implementation of process-backports.""" + args = self.args + body = self.gh.get_issue_body(args.issue) + + if args.add: + items_to_add = [] + for pr_ref in args.add: + try: + pr_num = self.gh.resolve_pr_number(pr_ref) + items_to_add.append({"ref": f"#{pr_num}"}) + except Exception as e: + print(f"Warning: PR ref '{pr_ref}' is invalid: {e}") + items_to_add.append( + { + "ref": pr_ref, + "metadata": {"status": "error-invalid-pr"}, + } + ) + + print(f"Adding backports {items_to_add} to tracking issue #{args.issue}...") + try: + body = add_backports_to_body(body, items_to_add) + for item in items_to_add: + if ( + "metadata" in item + and item["metadata"].get("status") == "error-invalid-pr" + ): + continue + pr_num = int(item["ref"].lstrip("#")) + body = add_sync_changelog_task_to_body(body, pr_num) + state = parse_checklist_state(body) + rc_tags = state.get("rc_tags", {}) + has_pending_rc = any( + not task.checked and task.status != "done" + for task in rc_tags.values() + ) + next_rc_num = max(rc_tags.keys()) + 1 if rc_tags else 0 + if not has_pending_rc: + print( + f"No pending RC task found. Adding 'Tag" + f" RC{next_rc_num}' to checklist..." + ) + body = add_rc_task_to_body(body, next_rc_num) + except ValueError as e: + print(f"Error: {e}") + return 1 + + if not args.dry_run: + self.gh.update_issue_body(args.issue, body) + print("Successfully updated tracking issue checklist.") + else: + print( + "[DRY RUN] Would update tracking issue checklist with new" + " backports." + ) + if not has_pending_rc: + print(f"[DRY RUN] Would add 'Tag RC{next_rc_num}' to checklist.") + + items = parse_backports(body) + + pending_items = [ + item + for item in items + if not item.checked and not item.status.startswith("error-") + ] + + if not pending_items: + print("No pending backports found.") + return 0 + + print(f"Found {len(pending_items)} pending backports to process.") + + # Determine branch name from issue title + issue_title = self.gh.get_issue_title(args.issue) + version_match = RELEASE_TITLE_RE.search(issue_title) + if not version_match: + print(f"Error: Could not parse version from issue title: {issue_title}") + return 1 + + version = version_match.group(1) + branch_version = ".".join(version.split(".")[:2]) + branch_name = f"release/{branch_version}" + + # Determine next RC tag to write to backport metadata + self.git.fetch(args.remote, tags=True, force=True) + latest_rc = get_latest_rc_tag(version, remote=args.remote) + if not latest_rc: + next_rc_suffix = "rc0" + else: + rc_num = int(latest_rc.split("-rc")[-1]) + next_rc_suffix = f"rc{rc_num + 1}" + + # Resolve PRs to merge commits using gh helper. + pr_commit_infos = self.gh.get_merge_commits_for_prs(pending_items) + + shas, sha_to_item, failed_prs, ignored_prs, body = ( + self._process_pr_commit_infos( + pr_commit_infos, body, args.issue, args.dry_run + ) + ) + + if not shas: + print("No valid merge commits to process.") + if failed_prs: + print("Failed PRs:") + for pr in failed_prs: + print(f"- {pr}") + return 1 + return 0 + + # Verify workspace is clean before proceeding + if self.git.status(): + print( + "ERROR: Git workspace is dirty. Please commit or stash changes" + " before running backports." + ) + return 1 + + # Sort chronologically using git helper + sorted_shas = self.git.sort_commits_chronologically(shas) + + self.git.fetch(args.remote) + self.git.checkout(branch_name, track_remote=args.remote) + start_sha = self.git.get_commit_sha("HEAD") + + collected_news_files = [] + successful_pr_nums = [] + collected_diffs = [] + try: + result = self._cherry_pick_and_update_prs( + sorted_shas, + sha_to_item, + body, + args.issue, + args.remote, + args.dry_run, + version, + branch_name, + next_rc_suffix, + ) + failed_prs.extend(result.failed_prs) + collected_news_files.extend(result.collected_news_files) + successful_pr_nums.extend(result.successful_pr_nums) + collected_diffs.extend(result.collected_diffs) + body = result.body + finally: + if args.dry_run: + print(f"[DRY RUN] Resetting branch {branch_name} to {start_sha}") + self.git.reset_hard(reset_to=start_sha) + + if successful_pr_nums: + self._sync_changelog_to_main( + version, + collected_news_files, + successful_pr_nums, + collected_diffs, + branch_name, + ) + + if failed_prs: + print("ERROR: One or more cherry-picks/resolutions failed:") + for pr in failed_prs: + print(f"- {pr}") + return 1 + + if args.dry_run: + print("Dry run completed successfully. No errors found.") + else: + print("All backports successfully processed!") + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for process-backports subcommand.""" + parser = subparsers.add_parser( + "process-backports", + help="Cherry-pick pending backports listed in the tracking issue.", + ) + parser.add_argument( + "--issue", + type=int, + required=True, + help="The tracking issue number (required).", + ) + parser.add_argument( + "--remote", + type=str, + required=True, + help="The git remote to push changes to (required).", + ) + parser.add_argument( + "--add", + type=parse_pr_list, + help="PR references (numbers, #numbers, or URLs, comma/space separated) to add before processing.", + ) + parser.add_argument( + "--triggering-comment", + type=int, + help="The ID of the comment that triggered this run (optional).", + ) + parser.add_argument( + "--dry-run", + action=argparse.BooleanOptionalAction, + default=True, + help="Perform a dry run (default: True). Use --no-dry-run to actually execute.", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + git = Git(".") + gh = GitHub() + return cls(args, git, gh).run() diff --git a/tools/private/release/promote.py b/tools/private/release/promote.py new file mode 100644 index 0000000000..560c531383 --- /dev/null +++ b/tools/private/release/promote.py @@ -0,0 +1,243 @@ +"""Subcommand to promote a release candidate to final release.""" + +import argparse +import os +import urllib.parse + +from tools.private.release.gh import GitHub +from tools.private.release.git import Git +from tools.private.release.release_issue import ( + RELEASE_TITLE_RE, + update_task_in_body, +) +from tools.private.release.utils import ( + REPO_URL, + determine_next_version, + get_latest_rc_tag, + semver_type, + set_github_output, +) + + +class Promote: + """Class to promote a release candidate to final release.""" + + def __init__(self, args, git: Git, gh: GitHub): + self.args = args + self.git = git + self.gh = gh + + def run(self) -> int: + """Executes the promote-rc subcommand (Phase 3).""" + args = self.args + # Fetch from remote to ensure we have the latest tags + self.git.fetch(args.remote, tags=True, force=True) + + version = args.version + if version is None: + if args.issue: + issue_title = self.gh.get_issue_title(args.issue) + version_match = RELEASE_TITLE_RE.search(issue_title) + if version_match: + version = version_match.group(1) + print( + f"Resolved version {version} from tracking issue" + f" #{args.issue} title." + ) + else: + print( + f"Error: Could not parse version from issue title:" + f" {issue_title}" + ) + return 1 + else: + version = determine_next_version() + + # Verify final tag doesn't already exist + if self.git.tag_exists(version): + print(f"Error: Final tag {version} already exists.") + return 1 + + is_first_release = version.endswith(".0") + + # Verify issue can be found + issue_num = args.issue + if not issue_num: + try: + issue_num = self.gh.get_release_tracking_issue(version) + except ValueError as e: + print(f"Error: {e}") + return 1 + except Exception as e: + print(f"Error: Unexpected error finding tracking issue: {e}") + return 1 + + if is_first_release: + latest_rc = get_latest_rc_tag(version, remote=args.remote) + if not latest_rc: + print(f"Error: No release candidate tags found matching {version}-rc*") + return 1 + commit_sha = self.git.get_commit_sha(latest_rc) + else: + latest_rc = None + + # Verify issue can be found and read it early + print(f"Verifying tracking issue #{issue_num} format...") + body = self.gh.get_issue_body(issue_num) + + # Determine release branch name + branch_version = ".".join(version.split(".")[:2]) + branch_name = f"release/{branch_version}" + remote_branch = f"{args.remote}/{branch_name}" + + print(f"Fetching remote branch {remote_branch}...") + self.git.fetch(args.remote, refspec=branch_name) + try: + branch_sha = self.git.get_commit_sha(remote_branch) + except Exception as e: + print( + f"Error: Could not get commit SHA for remote branch" + f" {remote_branch}: {e}" + ) + return 1 + + if is_first_release: + if commit_sha != branch_sha: + print( + f"Error: The latest RC tag {latest_rc} ({commit_sha[:8]}) is not at" + f" the head of release branch {remote_branch} ({branch_sha[:8]})." + ) + metadata = { + "status": "error-rc-tag-not-branch-head", + "rc": latest_rc, + "branch_commit": branch_sha[:8], + "tag_commit": commit_sha[:8], + } + try: + updated_body = update_task_in_body( + body, "Tag Final", checked=False, metadata=metadata + ) + except ValueError as e: + print(f"Error: Tracking issue #{issue_num} is malformed: {e}") + return 1 + + if not args.dry_run: + self.gh.update_issue_body(issue_num, updated_body) + print(f"Updated tracking issue #{issue_num} with error status.") + else: + print( + f"[DRY RUN] Would update tracking issue #{issue_num} with" + f" error status." + ) + return 1 + else: + # Patch release: tag branch head directly + commit_sha = branch_sha + latest_rc = None + + # Verify issue is in the right format by trying to prepare the update (for success case) + metadata = {"status": "done", "tag": version, "commit": commit_sha[:8]} + try: + updated_body = update_task_in_body( + body, "Tag Final", checked=True, metadata=metadata + ) + except ValueError as e: + print(f"Error: Tracking issue #{issue_num} is malformed: {e}") + return 1 + + # All pre-conditions met, perform modifications + promote_source = ( + latest_rc + if is_first_release + else f"head of {branch_name} ({commit_sha[:8]})" + ) + if args.dry_run: + print( + f"[DRY RUN] Pre-conditions passed successfully for promoting" + f" {promote_source} to {version}." + ) + print(f"[DRY RUN] Would tag commit {commit_sha[:8]} as {version}") + print(f"[DRY RUN] Would push tag {version} to {args.remote}") + print(f"[DRY RUN] Would update tracking issue #{issue_num} checklist") + print(f"[DRY RUN] Would post comment to tracking issue #{issue_num}") + return 0 + + print( + f"Promoting {promote_source} to final release {version} (commit" + f" {commit_sha[:8]}) using tracking issue #{issue_num}..." + ) + + # Tag the specific commit without checkout, and push to remote + self.git.tag(version, commit_sha) + self.git.push(args.remote, version) + + set_github_output("version", version) + + print(f"Updating tracking issue #{issue_num} checklist...") + self.gh.update_issue_body(issue_num, updated_body) + + print(f"Posting comment to tracking issue #{issue_num}...") + + branch_url = f"{REPO_URL}/tree/{branch_name}" + release_url = f"{REPO_URL}/releases/tag/{version}" + bcr_entry_url = f"https://registry.bazel.build/modules/rules_python/{version}" + bcr_query = ( + f'is:pr ("bazel-contrib/rules_python" in:title) ("@{version}" in:title)' + ) + bcr_search_url = f"https://github.com/bazelbuild/bazel-central-registry/pulls?q={urllib.parse.quote(bcr_query)}" + + if run_id := os.environ.get("GITHUB_RUN_ID"): + release_workflow_url = f"{REPO_URL}/actions/runs/{run_id}" + else: + release_workflow_url = f"{REPO_URL}/actions/workflows/release_promote.yaml" + + comment_body = f"""**New Release Tagged!** 🐍🌿 + +Version **{version}** has been successfully generated and tagged on branch [`{branch_name}`]({branch_url}). + +- [Github Release {version}]({release_url}) +- [BCR Entry {version}]({bcr_entry_url}) +- [BCR PRs]({bcr_search_url}) +- [Release workflow status]({release_workflow_url})""" + self.gh.post_issue_comment(issue_num, comment_body) + + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for promote-rc subcommand.""" + parser = subparsers.add_parser( + "promote", + help="Promote the latest RC to final release.", + ) + parser.add_argument( + "version", + nargs="?", + type=semver_type, + help="The final version to release (e.g., 0.38.0).", + ) + parser.add_argument( + "--issue", + type=int, + help="The tracking issue number (optional).", + ) + parser.add_argument( + "--remote", + type=str, + required=True, + help="The git remote to push the final tag to (required).", + ) + parser.add_argument( + "--dry-run", + action=argparse.BooleanOptionalAction, + default=True, + help="Perform a dry run (default: True). Use --no-dry-run to actually execute.", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + git = Git(".") + gh = GitHub() + return cls(args, git, gh).run() diff --git a/tools/private/release/release.py b/tools/private/release/release.py new file mode 100644 index 0000000000..7f1896ecf4 --- /dev/null +++ b/tools/private/release/release.py @@ -0,0 +1,78 @@ +"""A tool to perform release steps.""" + +import argparse +import os +import sys + +from tools.private.release.add_backports import AddBackports +from tools.private.release.backport_create_releases import BackportCreateReleases +from tools.private.release.backport_prepare import BackportPrepare +from tools.private.release.complete_prepare import CompletePrepare +from tools.private.release.complete_sync_changelog import CompleteSyncChangelog +from tools.private.release.create_rc import CreateRc +from tools.private.release.create_release_branch import CreateReleaseBranch +from tools.private.release.create_release_issue import CreateReleaseIssue +from tools.private.release.determine_next_version import DetermineNextVersion +from tools.private.release.on_pr_merged import OnPrMerged +from tools.private.release.prepare import Prepare +from tools.private.release.process_backports import ProcessBackports +from tools.private.release.promote import Promote + +cmds = [ + DetermineNextVersion, + CreateReleaseIssue, + Prepare, + CompletePrepare, + CompleteSyncChangelog, + CreateReleaseBranch, + AddBackports, + ProcessBackports, + OnPrMerged, + CreateRc, + Promote, + BackportPrepare, + BackportCreateReleases, +] + + +def create_parser(): + """Creates the argument parser with subcommands.""" + parser = argparse.ArgumentParser( + description="Automate release steps for rules_python." + ) + + subparsers = parser.add_subparsers( + dest="command", required=True, help="Subcommands" + ) + + for cmd in cmds: + cmd.add_parser(subparsers) + + return parser + + +def main(): + print(f"sys.argv: {sys.argv}") + if "BUILD_WORKSPACE_DIRECTORY" in os.environ: + os.chdir(os.environ["BUILD_WORKSPACE_DIRECTORY"]) + + parser = create_parser() + args = parser.parse_args() + + exit_code = 1 + try: + # args.command is the run_from_args classmethod of the selected command + exit_code = args.command(args) + except Exception as e: + sys.stdout.flush() + print(f"Fatal error: {e}", file=sys.stderr) + if hasattr(e, "__notes__"): + for note in e.__notes__: + print(note, file=sys.stderr) + sys.exit(1) + + sys.exit(exit_code if exit_code is not None else 0) + + +if __name__ == "__main__": + main() diff --git a/tools/private/release/release_issue.py b/tools/private/release/release_issue.py new file mode 100644 index 0000000000..220348bd47 --- /dev/null +++ b/tools/private/release/release_issue.py @@ -0,0 +1,388 @@ +import re + + +class BackportTask: + """Represents a backport task from the tracking issue checklist.""" + + def __init__( + self, + pr_ref: str, + checked: bool, + status: str, + rc: str | None = None, + commit: str | None = None, + metadata: dict[str, str] | None = None, + ): + """Initializes a BackportTask. + + Args: + pr_ref: The PR reference (e.g. '#123'). + checked: Whether the checklist item is checked. + status: The status of the backport (e.g. 'pending', 'done', + 'error-merge-conflict'). + rc: The release candidate version this PR was backported to. + commit: The cherry-pick commit SHA. + metadata: Raw metadata parsed from the checklist line. + """ + self.pr_ref = pr_ref + self.checked = checked + self.status = status + self.rc = rc + self.commit = commit + self.metadata = metadata or {} + + def __repr__(self): + return ( + f"BackportTask(pr_ref={self.pr_ref!r}, checked={self.checked!r}, " + f"status={self.status!r}, rc={self.rc!r}, commit={self.commit!r})" + ) + + +class ReleaseTask: + """Represents a release task from the tracking issue checklist.""" + + def __init__( + self, + name: str, + checked: bool, + status: str | None = None, + pr: str | None = None, + commit: str | None = None, + branch: str | None = None, + tag: str | None = None, + metadata: dict[str, str] | None = None, + ): + """Initializes a ReleaseTask. + + Args: + name: The name of the task (e.g. 'Prepare Release'). + checked: Whether the checklist item is checked. + status: The status of the task (e.g. 'pending', 'done'). + pr: The associated PR reference (e.g. '#123'). + commit: The associated commit SHA. + branch: The associated branch name. + tag: The associated tag name. + metadata: Raw metadata parsed from the checklist line. + """ + self.name = name + self.checked = checked + self.status = status + self.pr = pr + self.commit = commit + self.branch = branch + self.tag = tag + self.metadata = metadata or {} + + def __repr__(self): + return ( + f"ReleaseTask(name={self.name!r}, checked={self.checked!r}, " + f"status={self.status!r}, pr={self.pr!r}, commit={self.commit!r}, " + f"branch={self.branch!r}, tag={self.tag!r})" + ) + + +RELEASE_TITLE_RE = re.compile(r"Release (\d+\.\d+\.\d+)", re.IGNORECASE) + + +def parse_metadata_line(line): + """Parses a checklist line with optional | key=value metadata.""" + match = re.match(r"^\s*-\s*\[([ xX])\]\s+([^|]+)(?:\s*\|\s*(.*))?$", line) + if not match: + return None + + checked = match.group(1).lower() == "x" + name = match.group(2).strip() + metadata_str = match.group(3) + + metadata = {} + if metadata_str: + for k, v in re.findall(r"(\w+)\s*=\s*(\S+)", metadata_str): + metadata[k] = v + + return { + "checked": checked, + "name": name, + "metadata": metadata, + "original_line": line, + } + + +def format_metadata_line(checked, name, metadata): + """Formats a checklist line with space-separated key=value metadata.""" + check_str = "x" if checked else " " + if not metadata: + return f"- [{check_str}] {name}" + + metadata_pairs = [] + for k, v in metadata.items(): + if k == "commit" or k.endswith("_commit"): + # The 'commit' key (and keys ending with '_commit') is special-cased with + # a space after '=' so that GitHub autolinks the commit SHA. Autolinking + # requires certain characters to precede the value. + metadata_pairs.append(f"{k}= {v}") + else: + metadata_pairs.append(f"{k}={v}") + metadata_str = " ".join(metadata_pairs) + return f"- [{check_str}] {name} | {metadata_str}" + + +def update_task_in_body(body, task_name, checked, metadata): + """Updates a specific task's checked state and metadata in the issue body.""" + lines = body.splitlines() + updated_lines = [] + found = False + + for line in lines: + parsed = parse_metadata_line(line) + if parsed and parsed["name"].lower() == task_name.lower(): + updated_lines.append(format_metadata_line(checked, task_name, metadata)) + found = True + else: + updated_lines.append(line) + + if not found: + raise ValueError( + f"Task '{task_name}' not found in issue body. " + f"Expected format: '- [ ] {task_name}' or '- [x] {task_name}' (optionally followed by '| key=value')" + ) + + return "\n".join(updated_lines) + + +def parse_checklist_state(body): + """Parses the main checklist tasks and their metadata. + + Returns: + A dict containing ReleaseTask objects for 'prepare_release', + 'create_branch', 'tag_final', and a dict of RC tags. + """ + state = { + "prepare_release": ReleaseTask("Prepare Release", False), + "create_branch": ReleaseTask("Create Release branch", False), + "tag_final": ReleaseTask("Tag Final", False), + "rc_tags": {}, # Dynamically mapped: int -> ReleaseTask + "sync_changelogs": {}, # Dynamically mapped: int -> ReleaseTask + } + + lines = body.splitlines() + for line in lines: + parsed = parse_metadata_line(line) + if not parsed: + continue + + name = parsed["name"].strip() + meta = parsed["metadata"] + checked = parsed["checked"] + name_lower = name.lower() + + if "prepare release" in name_lower: + state["prepare_release"] = ReleaseTask( + name=name, + checked=checked, + status=meta.get("status"), + pr=meta.get("pr"), + commit=meta.get("commit"), + metadata=meta, + ) + elif "create release branch" in name_lower: + state["create_branch"] = ReleaseTask( + name=name, + checked=checked, + status=meta.get("status"), + branch=meta.get("branch"), + commit=meta.get("commit"), + metadata=meta, + ) + elif "tag final" in name_lower: + state["tag_final"] = ReleaseTask( + name=name, + checked=checked, + status=meta.get("status"), + tag=meta.get("tag"), + commit=meta.get("commit"), + metadata=meta, + ) + else: + # Match Tag RC + rc_match = re.match(r"Tag RC(\d+)", name, re.IGNORECASE) + if rc_match: + rc_num = int(rc_match.group(1)) + state["rc_tags"][rc_num] = ReleaseTask( + name=name, + checked=checked, + status=meta.get("status"), + tag=meta.get("tag"), + commit=meta.get("commit"), + metadata=meta, + ) + else: + # Match Sync Changelog # + sync_match = re.match(r"Sync Changelog #(\d+)", name, re.IGNORECASE) + if sync_match: + pr_num = int(sync_match.group(1)) + state["sync_changelogs"][pr_num] = ReleaseTask( + name=name, + checked=checked, + status=meta.get("status"), + pr=meta.get("pr"), + commit=meta.get("commit"), + metadata=meta, + ) + + return state + + +def parse_backports(body): + """Parses the ## Backports checklist section.""" + body = body.replace("\r\n", "\n") + match = re.search( + r"## Backports\n(.*?)(?=\n##|\n---|\Z)", body, re.DOTALL | re.IGNORECASE + ) + if not match: + return [] + + section_content = match.group(1) + items = [] + lines = section_content.splitlines() + + for line in lines: + parsed = parse_metadata_line(line) + if parsed: + items.append( + BackportTask( + pr_ref=parsed["name"], + checked=parsed["checked"], + status=parsed["metadata"].get("status", "pending"), + rc=parsed["metadata"].get("rc"), + commit=parsed["metadata"].get("commit"), + metadata=parsed["metadata"], + ) + ) + return items + + +def add_backports_to_body(body: str, items: list[dict]) -> str: + """Adds new backport checklist items to the ## Backports section. + + Args: + body: The issue body. + items: A list of dicts, where each dict has a 'ref' key (str) and + optional 'metadata' key (dict). + """ + body = body.replace("\r\n", "\n") + # Find the Backports section + pattern = r"(## Backports\n)(.*?)(?=\n##|\n---|\Z)" + match = re.search(pattern, body, re.DOTALL | re.IGNORECASE) + if not match: + raise ValueError("Could not find '## Backports' section in issue body.") + + section_content = match.group(2) + + # Parse existing backports to avoid duplicates + existing_items = parse_backports(body) + existing_refs = {item.pr_ref for item in existing_items} + + new_lines = [] + for item in items: + ref = item["ref"] + # Normalize numeric refs to #numeric + if not ref.startswith("#") and ref.isdigit(): + ref = f"#{ref}" + + if ref in existing_refs: + print(f"PR {ref} is already in the backports list. Skipping.") + continue + existing_refs.add(ref) + + metadata = item.get("metadata", {}) + new_lines.append( + format_metadata_line(checked=False, name=ref, metadata=metadata) + ) + + if not new_lines: + return body + + # Append new lines to the section content. + section_content_clean = section_content.rstrip("\n") + separator = "\n" if section_content_clean else "" + updated_section = section_content_clean + separator + "\n".join(new_lines) + "\n\n" + + # Replace the old section with the updated one + start, end = match.span(2) + return body[:start] + updated_section + body[end:] + + +def add_rc_task_to_body(body: str, rc_num: int) -> str: + """Adds a new 'Tag RC' task to the checklist in the issue body.""" + body = body.replace("\r\n", "\n") + lines = body.splitlines() + + # Find the index of the last "Tag RC" line + last_rc_idx = -1 + for i, line in enumerate(lines): + parsed = parse_metadata_line(line) + if parsed and re.match(r"Tag RC\d+", parsed["name"], re.IGNORECASE): + last_rc_idx = i + + if last_rc_idx == -1: + # If no RC task found (unexpected, but fallback to before "Tag Final") + for i, line in enumerate(lines): + parsed = parse_metadata_line(line) + if parsed and parsed["name"].lower() == "tag final": + last_rc_idx = i - 1 + break + + if last_rc_idx == -1: + raise ValueError("Could not find a place to insert the new RC task.") + + new_task_line = f"- [ ] Tag RC{rc_num}" + lines.insert(last_rc_idx + 1, new_task_line) + + return "\n".join(lines) + + +def add_sync_changelog_task_to_body(body: str, pr_num: int) -> str: + """Adds a new 'Sync Changelog #' task to the checklist in the issue body.""" + body = body.replace("\r\n", "\n") + + # Check if already exists + task_name = f"Sync Changelog #{pr_num}" + lines = body.splitlines() + for line in lines: + parsed = parse_metadata_line(line) + if parsed and parsed["name"].lower() == task_name.lower(): + print(f"Task '{task_name}' already exists. Skipping.") + return body + + # Find the index of the last "Sync Changelog #" line + last_sync_idx = -1 + for i, line in enumerate(lines): + parsed = parse_metadata_line(line) + if parsed and re.match(r"Sync Changelog #\d+", parsed["name"], re.IGNORECASE): + last_sync_idx = i + + if last_sync_idx == -1: + # If no Sync Changelog task found, insert before "Tag Final" + for i, line in enumerate(lines): + parsed = parse_metadata_line(line) + if parsed and parsed["name"].lower() == "tag final": + last_sync_idx = i - 1 + break + + if last_sync_idx == -1: + # If "Tag Final" not found, insert after "Create Release branch" + for i, line in enumerate(lines): + parsed = parse_metadata_line(line) + if parsed and parsed["name"].lower() == "create release branch": + last_sync_idx = i + break + + if last_sync_idx == -1: + raise ValueError( + "Could not find a place to insert the new Sync Changelog task." + ) + + new_task_line = f"- [ ] Sync Changelog #{pr_num}" + lines.insert(last_sync_idx + 1, new_task_line) + + return "\n".join(lines) diff --git a/tools/private/release/shell.py b/tools/private/release/shell.py new file mode 100644 index 0000000000..0093a28574 --- /dev/null +++ b/tools/private/release/shell.py @@ -0,0 +1,31 @@ +"""Shell utility functions for the release tool.""" + +import shlex +import subprocess + + +def run_cmd(*args, check=True, capture_output=True, cwd=None): + """Runs a command as a subprocess with separate arguments (prints command). + + If the command fails, it raises the CalledProcessError after attaching + a detailed note explaining the failure to preserve the stack trace. + """ + cmd = [str(arg) for arg in args] + cwd_suffix = f" (cwd: {cwd})" if cwd else "" + print(f"> {shlex.join(cmd)}{cwd_suffix}") + try: + result = subprocess.run( + cmd, + check=check, + stdout=subprocess.PIPE if capture_output else None, + stderr=subprocess.PIPE if capture_output else None, + universal_newlines=True, + cwd=cwd, + ) + return result.stdout.strip() if capture_output else None + except subprocess.CalledProcessError as e: + note = f"Error running command: {shlex.join(cmd)}{cwd_suffix}" + if capture_output: + note += f"\nStdout: {e.stdout}\nStderr: {e.stderr}" + e.add_note(note) + raise diff --git a/tools/private/release/utils.py b/tools/private/release/utils.py new file mode 100644 index 0000000000..ab1e0acef3 --- /dev/null +++ b/tools/private/release/utils.py @@ -0,0 +1,193 @@ +"""Utility functions for the release tool.""" + +import argparse +import fnmatch +import os +import re + +from packaging.version import parse as parse_version + +from tools.private.release.git import Git + +REPO_URL = "https://github.com/bazel-contrib/rules_python" + + +def semver_type(value): + """Argparse type validator for semantic versions.""" + if not re.match(r"^\d+\.\d+\.\d+(rc\d+)?$", value): + raise argparse.ArgumentTypeError( + f"'{value}' is not a valid semantic version (X.Y.Z or X.Y.ZrcN)" + ) + return value + + +_EXCLUDE_PATTERNS = [ + "./.agents/*", + "./.git/*", + "./.github/*", + "./.bazelci/*", + "./.bcr/*", + "./bazel-*/*", + "./CONTRIBUTING.md", + "./RELEASING.md", + "./tools/private/release/*", + "./tests/tools/private/release/*", +] + + +def _iter_version_placeholder_files(): + for root, dirs, files in os.walk(".", topdown=True): + # Filter directories + dirs[:] = [ + d + for d in dirs + if not any( + fnmatch.fnmatch(os.path.join(root, d), pattern) + for pattern in _EXCLUDE_PATTERNS + ) + ] + + for filename in files: + filepath = os.path.join(root, filename) + if any(fnmatch.fnmatch(filepath, pattern) for pattern in _EXCLUDE_PATTERNS): + continue + + yield filepath + + +def get_latest_version(): + """Gets the latest version from git tags.""" + git = Git(".") + tags = git.get_tags() + versions = [ + (tag, parse_version(tag)) + for tag in tags + if re.match(r"^\d+\.\d+\.\d+(rc\d+)?$", tag.strip()) + ] + if not versions: + raise RuntimeError("No git tags found matching X.Y.Z or X.Y.ZrcN format.") + + versions.sort(key=lambda v: v[1]) + latest_tag, latest_version = versions[-1] + + if latest_version.is_prerelease: + raise ValueError(f"The latest version is a pre-release version: {latest_tag}") + + stable_versions = [tag for tag, version in versions if not version.is_prerelease] + if not stable_versions: + raise ValueError("No stable git tags found matching X.Y.Z format.") + + return stable_versions[-1] + + +def get_latest_rc_tag(version, remote=None): + """Queries git tags and returns the highest RC tag for the version.""" + git = Git(".") + if remote: + tags = git.get_remote_tags(remote) + else: + tags = git.get_tags() + pattern = rf"^{re.escape(version)}-rc\d+$" + rc_tags = [tag.strip() for tag in tags if re.match(pattern, tag.strip())] + if not rc_tags: + return None + rc_tags.sort(key=parse_version) + return rc_tags[-1] + + +def should_increment_minor(): + """Checks if the minor version should be incremented.""" + for filepath in _iter_version_placeholder_files(): + try: + with open(filepath, "r") as f: + content = f.read() + except (IOError, UnicodeDecodeError): + continue + + if "VERSION_NEXT_FEATURE" in content: + return True + return False + + +def determine_next_version(branch_name=None): + """Determines the next version based on git tags and the current branch.""" + git = Git(".") + if branch_name is None: + branch_name = git.get_current_branch() + + if branch_name: + release_match = re.match(r"^release/(\d+)\.(\d+)$", branch_name) + if release_match: + branch_major = int(release_match.group(1)) + branch_minor = int(release_match.group(2)) + print( + f"Detected release branch: {branch_name} (targeting" + f" {branch_major}.{branch_minor}.x)" + ) + + tags = git.get_tags() + matching_patches = [] + for tag in tags: + tag = tag.strip() + m = re.match(rf"^{branch_major}\.{branch_minor}\.(\d+)$", tag) + if m: + matching_patches.append(int(m.group(1))) + + if matching_patches: + latest_patch = max(matching_patches) + next_version = f"{branch_major}.{branch_minor}.{latest_patch + 1}" + print( + f"Latest tag on this branch is" + f" {branch_major}.{branch_minor}.{latest_patch}. Next" + f" version: {next_version}" + ) + return next_version + else: + next_version = f"{branch_major}.{branch_minor}.0" + print( + f"No stable tags found for {branch_major}.{branch_minor}.x." + f" Next version: {next_version}" + ) + return next_version + + latest_version = get_latest_version() + major, minor, patch = [int(n) for n in latest_version.split(".")] + + if should_increment_minor(): + return f"{major}.{minor + 1}.0" + else: + return f"{major}.{minor}.{patch + 1}" + + +def replace_version_next(version): + """Replaces all VERSION_NEXT_* placeholders with the new version.""" + for filepath in _iter_version_placeholder_files(): + try: + with open(filepath, "r") as f: + content = f.read() + except (IOError, UnicodeDecodeError): + continue + + if "VERSION_NEXT_FEATURE" in content or "VERSION_NEXT_PATCH" in content: + new_content = content.replace("VERSION_NEXT_FEATURE", version) + new_content = new_content.replace("VERSION_NEXT_PATCH", version) + with open(filepath, "w") as f: + f.write(new_content) + + +def parse_pr_list(value: str) -> list[str]: + """Parses a comma or space separated list of PR references. + + PR references can be numbers (optionally prefixed with '#') or URLs. + """ + if not value: + return [] + # Split by space and/or comma + return [p for p in re.split(r"[\s,]+", value.strip()) if p] + + +def set_github_output(name: str, value: str) -> None: + """Sets a GitHub Actions output parameter if GITHUB_OUTPUT is set.""" + if github_output := os.environ.get("GITHUB_OUTPUT"): + with open(github_output, "a", encoding="utf-8") as f: + f.write(f"{name}={value}\n") diff --git a/tools/private/reviewbot/antigravity_review.py b/tools/private/reviewbot/antigravity_review.py new file mode 100644 index 0000000000..4ae0ef1690 --- /dev/null +++ b/tools/private/reviewbot/antigravity_review.py @@ -0,0 +1,88 @@ +# /// script +# requires-python = ">=3.11" +# dependencies = [ +# "google-antigravity", +# ] +# /// +import argparse +import asyncio +import subprocess +from pathlib import Path + +from google.antigravity import Agent, CapabilitiesConfig, LocalAgentConfig +from google.antigravity.models import GeminiAPIEndpoint, ModelTarget +from google.antigravity.types import BuiltinTools + + +def parse_args(): + parser = argparse.ArgumentParser() + parser.add_argument("--prompt", required=True, help="Path to prompt file") + parser.add_argument("--diff-file", help="Path to pre-computed diff file") + return parser.parse_args() + + +def get_pr_diff(diff_file: str | None = None) -> str: + """Fetches the git diff for the current pull request against origin/main.""" + if diff_file and Path(diff_file).exists(): + return Path(diff_file).read_text() + try: + return subprocess.check_output( + ["git", "diff", "origin/main...HEAD"], text=True, stderr=subprocess.DEVNULL + ) + except Exception: + return "No diff could be automatically extracted via git commands." + + +async def main(): + args = parse_args() + + # Read prompt file and pre-hydrate with the exact PR code diff + base_prompt = Path(args.prompt).read_text() + diff_text = get_pr_diff(args.diff_file) + prompt = ( + f"{base_prompt}\n\n## Pull Request Git Diff\n" + f"Here is the exact code diff for this pull request:\n```diff\n{diff_text}\n```" + ) + + # General coordinator instructions for the reviewer agent. + system_instructions = ( + "You are a code review assistant. Use your available skills to perform " + "reviews on pull requests." + ) + + # Create the default endpoint picking up GEMINI_API_KEY from the environment. + endpoint = GeminiAPIEndpoint() + + # Initialize the Antigravity Agent in read-only mode for security. + # Register the review-pr skill from the local reviewbot folder. + # Provide a comprehensive prioritized cascade across Gemini 3 models + # to automatically fall back if any model hits free-tier quota limits (429) + # or temporary unavailability. + config = LocalAgentConfig( + models=[ + ModelTarget(name="gemini-3.5-flash", endpoint=endpoint), + ModelTarget(name="gemini-3.1-pro-preview", endpoint=endpoint), + ModelTarget(name="gemini-3.1-flash-lite", endpoint=endpoint), + ModelTarget(name="gemini-3-pro-preview", endpoint=endpoint), + ModelTarget(name="gemini-flash-latest", endpoint=endpoint), + ModelTarget(name="gemini-pro-latest", endpoint=endpoint), + ], + system_instructions=system_instructions, + skills_paths=[str(Path(__file__).parent / "skills" / "review-pr" / "SKILL.md")], + capabilities=CapabilitiesConfig( + enabled_tools=BuiltinTools.read_only(), + ), + ) + + async with Agent(config) as agent: + response = await agent.chat(prompt) + report = await response.text() + + print("--- REVIEW REPORT GENERATED ---") + print(report) + + # TODO: Use GITHUB_TOKEN to post the report back to the PR comments. + + +if __name__ == "__main__": + asyncio.run(main()) diff --git a/tools/private/reviewbot/prompt.txt b/tools/private/reviewbot/prompt.txt new file mode 100644 index 0000000000..e406164af1 --- /dev/null +++ b/tools/private/reviewbot/prompt.txt @@ -0,0 +1,3 @@ +Use the review-pr skill to review the files modified in this pull request. +Summarize your findings and suggest specific, actionable improvements. +Group your findings into clear, descriptive nits or suggestions. diff --git a/tools/private/reviewbot/skills/review-pr/SKILL.md b/tools/private/reviewbot/skills/review-pr/SKILL.md new file mode 100644 index 0000000000..915a526686 --- /dev/null +++ b/tools/private/reviewbot/skills/review-pr/SKILL.md @@ -0,0 +1,54 @@ +--- +name: review-pr +description: Perform a read-only code review on a pull request. +--- + +# review-pr + +IMPORTANT: The exact git diff for the pull request is pre-provided right in your prompt. Analyze this provided diff directly in a single pass without calling exploratory directory listing or file reading tools unless you specifically need surrounding lines of context from a modified file. + +You are an expert Starlark, Python, and Bazel code reviewer. Analyze the changed files for +correctness, edge cases, and performance. Focus strictly on logical +correctness, concurrency safety, system architecture, performance bottlenecks, +and resource management. Do not comment on style nits or formatting issues +that an automated formatter can handle. Be constructive and concise. + +For every issue or improvement you identify, you MUST output the finding in the +GitHub Actions workflow command warning format. Specify the exact file path +and line numbers that the comment applies to. + +Format each finding exactly as a single line to stdout matching this template: +`::warning file={file_path},line={line_number},endLine={end_line},title={category}::{comment_body}` + +Where: +* `file_path` is the relative file path from the repository root. +* `line_number` is the starting line number in the file where the comment applies. +* `end_line` is the ending line number in the file where the comment applies (equal to line_number if the issue is on a single line). +* `category` is a short tag for the type of issue (e.g., "Error Handling", "Correctness", "Performance"). +* `comment_body` is your constructive and concise feedback. + +Do not write any markdown commentary outside of these GHA command formatted lines. + +Follow these checklists during your review: + +### General Quality & Architecture Checklist +* **PR Description Audit**: Verify the description contains the Why + (business/technical reason), a brief high level overview of changes, + Issue/Bug Link, and explicit Testing Evidence. +* **Separation of Concerns**: Suggest extracting large hardcoded data structures + (e.g., massive templates, complex regexes) to resource files. +* **Logic Correctness**: Verify calculations, negative values, division-by-zero, + and null safety before member access. +* **Error Handling**: Flag silent failures (e.g., empty except blocks) and + unconditional defaults that override configs. +* **Deterministic Operations**: Sort collections/keys to guarantee + reproducible/deterministic execution. + +### Skeptical Critic (Adversarial Specialist Review) +* **Dynamic Filtering**: Filter the PR diff and run only the specialist checks + that have relevant files changed (e.g. skip the C++ checks if only Python + files are modified). +* **Specialist Review Pillars**: Run parallel audits focusing on: + 1. Crash Regression: Null safety and resource lifecycle. + 2. Performance & Latency: Thread bottlenecks, locks, and network calls. + 3. Test Integrity: Coverage validity, change detectors defense. diff --git a/tools/private/sync_downloader_configs.py b/tools/private/sync_downloader_configs.py new file mode 100755 index 0000000000..7e8527b989 --- /dev/null +++ b/tools/private/sync_downloader_configs.py @@ -0,0 +1,37 @@ +#!/usr/bin/env python3 + +"""Synchronizes downloader_config.cfg across subworkspaces.""" + +import sys +from pathlib import Path + + +def main(): + repo_root = Path(__file__).resolve().parent.parent.parent + canonical = repo_root / "downloader_config.cfg" + + subworkspaces = [ + repo_root / "gazelle" / "downloader_config.cfg", + repo_root / "sphinxdocs" / "downloader_config.cfg", + ] + + with open(canonical, "r", encoding="utf-8") as f: + canonical_content = f.read() + + changed = False + for sub in subworkspaces: + if sub.exists(): + with open(sub, "r", encoding="utf-8") as f: + old_content = f.read() + if old_content != canonical_content: + with open(sub, "w", encoding="utf-8") as f: + f.write(canonical_content) + print(f"Updated {sub.relative_to(repo_root)}") + changed = True + + if changed: + sys.exit(1) + + +if __name__ == "__main__": + main() diff --git a/tools/private/update_deps/BUILD.bazel b/tools/private/update_deps/BUILD.bazel index beecf82189..8746d82c17 100644 --- a/tools/private/update_deps/BUILD.bazel +++ b/tools/private/update_deps/BUILD.bazel @@ -13,7 +13,8 @@ # limitations under the License. load("//python:py_binary.bzl", "py_binary") load("//python:py_library.bzl", "py_library") -load("//python:py_test.bzl", "py_test") +load("//tests/support:support.bzl", "SUPPORTS_BZLMOD") +load("//tests/support/pytest_test:pytest_test.bzl", "pytest_test") licenses(["notice"]) @@ -34,10 +35,10 @@ py_binary( name = "update_coverage_deps", srcs = ["update_coverage_deps.py"], data = [ - "//python/private:coverage_deps", + "//python/private:coverage_deps_filegroup", ], env = { - "UPDATE_FILE": "$(rlocationpath //python/private:coverage_deps)", + "UPDATE_FILE": "$(rlocationpath //python/private:coverage_deps_filegroup)", }, imports = ["../../.."], deps = [ @@ -65,10 +66,11 @@ py_binary( ], ) -py_test( +pytest_test( name = "update_file_test", srcs = ["update_file_test.py"], imports = ["../../.."], + target_compatible_with = SUPPORTS_BZLMOD, deps = [ ":update_file", ], diff --git a/tools/private/update_deps/update_coverage_deps.py b/tools/private/update_deps/update_coverage_deps.py index bbff67e927..8a4ccb41ba 100755 --- a/tools/private/update_deps/update_coverage_deps.py +++ b/tools/private/update_deps/update_coverage_deps.py @@ -20,11 +20,8 @@ # NOTE @aignas 2023-01-09: We should only depend on core Python 3 packages. import argparse -import difflib import json import os -import pathlib -import sys import textwrap from collections import defaultdict from dataclasses import dataclass @@ -38,14 +35,18 @@ _supported_platforms = { # Windows is unsupported right now # "win_amd64": "x86_64-pc-windows-msvc", + "manylinux1_x86_64": "x86_64-unknown-linux-gnu", "manylinux2014_x86_64": "x86_64-unknown-linux-gnu", "manylinux2014_aarch64": "aarch64-unknown-linux-gnu", "macosx_11_0_arm64": "aarch64-apple-darwin", "macosx_10_9_x86_64": "x86_64-apple-darwin", + "macosx_10_13_x86_64": "x86_64-apple-darwin", + ("t", "manylinux1_x86_64"): "x86_64-unknown-linux-gnu-freethreaded", ("t", "manylinux2014_x86_64"): "x86_64-unknown-linux-gnu-freethreaded", ("t", "manylinux2014_aarch64"): "aarch64-unknown-linux-gnu-freethreaded", ("t", "macosx_11_0_arm64"): "aarch64-apple-darwin-freethreaded", ("t", "macosx_10_9_x86_64"): "x86_64-apple-darwin-freethreaded", + ("t", "macosx_10_13_x86_64"): "x86_64-apple-darwin-freethreaded", } @@ -114,12 +115,12 @@ def _map( platform: str, **kwargs: Any, ): - if platform not in _supported_platforms: + if platform and platform not in _supported_platforms: return None return Dep( name=name, - platform=_supported_platforms[platform], + platform=_supported_platforms[platform] if platform else "", python=python_version, url=url, sha256=digests["sha256"], @@ -143,7 +144,7 @@ def _parse_args() -> argparse.Namespace: "--py", nargs="+", type=str, - default=["cp38", "cp39", "cp310", "cp311", "cp312", "cp313"], + default=["cp39", "cp310", "cp311", "cp312", "cp313", "cp314"], help="Supported python versions", ) parser.add_argument( @@ -169,6 +170,7 @@ def main(): data = json.loads(response.read().decode("utf-8")) urls = [] + default_url = None for u in data["urls"]: if u["yanked"]: continue @@ -176,10 +178,14 @@ def main(): if not u["filename"].endswith(".whl"): continue + if u["filename"].endswith("py3-none-any.whl"): + default_url = _map(name=args.name, platform="", **u) + continue + if u["python_version"] not in args.py: continue - if f'_{u["python_version"]}m_' in u["filename"]: + if f"_{u['python_version']}m_" in u["filename"]: continue platforms = _get_platforms( @@ -195,7 +201,13 @@ def main(): # Update the coverage_deps, which are used to register deps update_file( path=args.update_file, - snippet=f"_coverage_deps = {repr(Deps(urls))}\n", + snippet="\n".join( + [ + f"_default = {repr(default_url)}", + f"_coverage_deps = {repr(Deps(urls))}", + "", + ] + ), start_marker="# START: maintained by 'bazel run //tools/private/update_deps:update_coverage_deps '", end_marker="# END: maintained by 'bazel run //tools/private/update_deps:update_coverage_deps '", dry_run=args.dry_run, diff --git a/tools/private/update_deps/update_file.py b/tools/private/update_deps/update_file.py index ab3e8a817e..cbf4c32bd1 100644 --- a/tools/private/update_deps/update_file.py +++ b/tools/private/update_deps/update_file.py @@ -17,10 +17,8 @@ This is reused in other files updating coverage deps and pip deps. """ -import argparse import difflib import pathlib -import sys def _writelines(path: pathlib.Path, out: str): diff --git a/tools/private/update_deps/update_file_test.py b/tools/private/update_deps/update_file_test.py index 01c6ec74b0..b1d6545ead 100644 --- a/tools/private/update_deps/update_file_test.py +++ b/tools/private/update_deps/update_file_test.py @@ -12,14 +12,13 @@ # See the License for the specific language governing permissions and # limitations under the License. -import unittest +import pytest from tools.private.update_deps.update_file import replace_snippet, unified_diff -class TestReplaceSnippet(unittest.TestCase): - def test_replace_simple(self): - current = """\ +def test_replace_simple(): + current = """\ Before the snippet # Start marker @@ -30,15 +29,14 @@ def test_replace_simple(self): After the snippet """ - snippet = "Replaced" - got = replace_snippet( - current=current, - snippet="Replaced", - start_marker="# Start marker", - end_marker="# End marker", - ) - - want = """\ + got = replace_snippet( + current=current, + snippet="Replaced", + start_marker="# Start marker", + end_marker="# End marker", + ) + + want = """\ Before the snippet # Start marker @@ -47,10 +45,11 @@ def test_replace_simple(self): After the snippet """ - self.assertEqual(want, got) + assert got == want - def test_replace_indented(self): - current = """\ + +def test_replace_indented(): + current = """\ Before the snippet # Start marker @@ -59,14 +58,14 @@ def test_replace_indented(self): After the snippet """ - got = replace_snippet( - current=current, - snippet=" Replaced", - start_marker="# Start marker", - end_marker="# End marker", - ) - - want = """\ + got = replace_snippet( + current=current, + snippet=" Replaced", + start_marker="# Start marker", + end_marker="# End marker", + ) + + want = """\ Before the snippet # Start marker @@ -75,45 +74,42 @@ def test_replace_indented(self): After the snippet """ - self.assertEqual(want, got) - - def test_raises_if_start_is_not_found(self): - with self.assertRaises(RuntimeError) as exc: - replace_snippet( - current="foo", - snippet="", - start_marker="start", - end_marker="end", - ) - - self.assertEqual(exc.exception.args[0], "Start marker 'start' was not found") - - def test_raises_if_end_is_not_found(self): - with self.assertRaises(RuntimeError) as exc: - replace_snippet( - current="start", - snippet="", - start_marker="start", - end_marker="end", - ) - - self.assertEqual(exc.exception.args[0], "End marker 'end' was not found") - - -class TestUnifiedDiff(unittest.TestCase): - def test_diff(self): - give_a = """\ + assert got == want + + +def test_raises_if_start_is_not_found(): + with pytest.raises(RuntimeError, match="Start marker 'start' was not found"): + replace_snippet( + current="foo", + snippet="", + start_marker="start", + end_marker="end", + ) + + +def test_raises_if_end_is_not_found(): + with pytest.raises(RuntimeError, match="End marker 'end' was not found"): + replace_snippet( + current="start", + snippet="", + start_marker="start", + end_marker="end", + ) + + +def test_diff(): + give_a = """\ First line second line Third line """ - give_b = """\ + give_b = """\ First line Second line Third line """ - got = unified_diff("filename", give_a, give_b) - want = """\ + got = unified_diff("filename", give_a, give_b) + want = """\ --- a/filename +++ b/filename @@ -1,3 +1,3 @@ @@ -121,8 +117,4 @@ def test_diff(self): -second line +Second line Third line""" - self.assertEqual(want, got) - - -if __name__ == "__main__": - unittest.main() + assert got == want diff --git a/tools/private/update_deps/update_pip_deps.py b/tools/private/update_deps/update_pip_deps.py index 1034382f0d..406697bc4d 100755 --- a/tools/private/update_deps/update_pip_deps.py +++ b/tools/private/update_deps/update_pip_deps.py @@ -96,7 +96,7 @@ def _get_deps(report: dict) -> list[Dep]: url=dep["download_info"]["url"], sha256=dep["download_info"]["archive_info"]["hash"][len("sha256=") :], ) - except: + except Exception: debug_dep = textwrap.indent(json.dumps(dep, indent=4), " " * 4) print(f"Could not parse the response from 'pip':\n{debug_dep}") raise diff --git a/tools/private/zipapp/BUILD.bazel b/tools/private/zipapp/BUILD.bazel new file mode 100644 index 0000000000..45bd126c6e --- /dev/null +++ b/tools/private/zipapp/BUILD.bazel @@ -0,0 +1,51 @@ +load("//python:py_library.bzl", "py_library") +load("//python/private:py_interpreter_program.bzl", "py_interpreter_program") # buildifier: disable=bzl-visibility +load("//python/private:visibility.bzl", "NOT_ACTUALLY_PUBLIC") # buildifier: disable=bzl-visibility + +package( + default_visibility = ["//:__subpackages__"], +) + +py_interpreter_program( + name = "zipper", + main = "zipper.py", + # Not actually public. Only public so rules_python-generated toolchains + # are able to reference it. + visibility = NOT_ACTUALLY_PUBLIC, +) + +py_library( + name = "zipper_lib", + srcs = ["zipper.py"], +) + +py_interpreter_program( + name = "exe_zip_maker", + main = "exe_zip_maker.py", + # Not actually public. Only public so rules_python-generated toolchains + # are able to reference it. + visibility = NOT_ACTUALLY_PUBLIC, +) + +py_library( + name = "exe_zip_maker_lib", + srcs = ["exe_zip_maker.py"], +) + +py_interpreter_program( + name = "zip_main_maker", + main = "zip_main_maker.py", + # Not actually public. Only public so rules_python-generated toolchains + # are able to reference it. + visibility = NOT_ACTUALLY_PUBLIC, +) + +py_library( + name = "zip_main_maker_lib", + srcs = ["zip_main_maker.py"], +) + +filegroup( + name = "distribution", + srcs = glob(["**"]), +) diff --git a/tools/private/zipapp/exe_zip_maker.py b/tools/private/zipapp/exe_zip_maker.py new file mode 100644 index 0000000000..29391c86da --- /dev/null +++ b/tools/private/zipapp/exe_zip_maker.py @@ -0,0 +1,40 @@ +import hashlib +import os +import shutil +import stat +import sys + +BLOCK_SIZE = 256 * 1024 + + +def create_exe_zip(preamble_path, zip_path, output_path): + sha256_hash = hashlib.sha256() + with open(zip_path, "rb", buffering=BLOCK_SIZE) as f: + for byte_block in iter(lambda: f.read(BLOCK_SIZE), b""): + sha256_hash.update(byte_block) + zip_hash = sha256_hash.hexdigest() + + with open(preamble_path, "rb") as f: + preamble_content = f.read() + + preamble_content = preamble_content.replace(b"%ZIP_HASH%", zip_hash.encode("utf-8")) + + with open(output_path, "wb") as out_f: + out_f.write(preamble_content) + with open(zip_path, "rb") as zip_f: + shutil.copyfileobj(zip_f, out_f, length=BLOCK_SIZE) + + st = os.stat(output_path) + os.chmod(output_path, st.st_mode | stat.S_IEXEC) + + +def main(): + if len(sys.argv) != 4: + print(f"Usage: {sys.argv[0]} ", file=sys.stderr) + sys.exit(1) + + create_exe_zip(sys.argv[1], sys.argv[2], sys.argv[3]) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/private/zipapp/zip_main_maker.py b/tools/private/zipapp/zip_main_maker.py new file mode 100644 index 0000000000..ae112ffc66 --- /dev/null +++ b/tools/private/zipapp/zip_main_maker.py @@ -0,0 +1,94 @@ +"""Creates the __main__.py for a zipapp by populating a template. + +This program also calculates a hash of the application files to include in +the template, which allows making the extraction directory unique to the +content of the zipapp. +""" + +import argparse +import hashlib +import os + +BLOCK_SIZE = 256 * 1024 + + +def create_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser(fromfile_prefix_chars="@") + parser.add_argument("--template", required=True) + parser.add_argument("--output", required=True) + parser.add_argument("--substitution", action="append", default=[]) + parser.add_argument( + "--hash_files_manifest", + required=True, + help="A file containing lines in rf-XXX formats (rf-empty, rf-file, rf-symlink, etc.)", + ) + return parser + + +def compute_inputs_hash(manifest_path: str) -> str: + h = hashlib.sha256() + with open(manifest_path, "r", encoding="utf-8") as f: + manifest_lines = f.read().splitlines() + + # Sort lines for determinism. Hash the paths (to capture structure) and the + # content. + for line in sorted(manifest_lines): + type_, _, rest = line.partition("|") + h.update(rest.encode("utf-8")) + parts = rest.split("|") + + if type_ == "rf-empty": + continue + if type_ == "symlink": + # The symlink path and the target it points to + # are captured by hashing the entire line above. + continue + + is_symlink_str = parts[0] + path = parts[-1] + + if is_symlink_str == "-1": + is_symlink = not os.path.exists(path) + else: + is_symlink = is_symlink_str == "1" + + if is_symlink: + h.update(os.readlink(path).encode("utf-8")) + else: + with open(path, "rb") as f: + while True: + chunk = f.read(BLOCK_SIZE) + if not chunk: + break + h.update(chunk) + + return h.hexdigest() + + +def expand_template(template_path: str, output_path: str, substitutions: dict) -> None: + with open(template_path, "r", encoding="utf-8") as f: + content = f.read() + + for key, val in substitutions.items(): + content = content.replace(key, val) + + with open(output_path, "w", encoding="utf-8") as f: + f.write(content) + + +def main(): + parser = create_parser() + args = parser.parse_args() + + app_hash = compute_inputs_hash(args.hash_files_manifest) + + substitutions = {"%APP_HASH%": app_hash} + for s in args.substitution: + key, val = s.split("=", 1) + substitutions[key] = val + + expand_template(args.template, args.output, substitutions) + + +if __name__ == "__main__": + main() diff --git a/tools/private/zipapp/zipper.py b/tools/private/zipapp/zipper.py new file mode 100644 index 0000000000..5a8eb8c6fd --- /dev/null +++ b/tools/private/zipapp/zipper.py @@ -0,0 +1,286 @@ +import argparse +import os +import shutil +import sys +import zipfile +from os.path import dirname + +# Unix permission bit for symlink (S_IFLNK) +# S_IFLNK is usually 0o120000 +S_IFLNK = 0o120000 + + +def unix_join(*parts): + return "/".join(parts) + + +def _get_zip_runfiles_path( + path, workspace_name, legacy_external_runfiles, runfiles_dir +): + if legacy_external_runfiles and path.startswith("external/"): + path = path[len("external/") :] + elif path.startswith("../"): + path = path[3:] + else: + path = unix_join(workspace_name, path) + return unix_join(runfiles_dir, path) + + +def _parse_entry( + line, + line_idx, + workspace_name, + legacy_external_runfiles, + runfiles_dir, +): + line = line.strip() + if not line: + return None + + parts = line.split("|") + type_ = parts[0] + + if type_ == "regular": + _, is_symlink_str, zip_path, content_path = parts + elif type_ == "rf-empty": + _, runfile_path = parts + zip_path = _get_zip_runfiles_path( + runfile_path, workspace_name, legacy_external_runfiles, runfiles_dir + ) + content_path = None # Empty file + is_symlink_str = "0" + elif type_ == "rf-file": + _, is_symlink_str, runfile_path, content_path = parts + zip_path = _get_zip_runfiles_path( + runfile_path, workspace_name, legacy_external_runfiles, runfiles_dir + ) + elif type_ == "rf-symlink": + _, is_symlink_str, runfile_path, content_path = parts + zip_path = unix_join(runfiles_dir, workspace_name, runfile_path) + elif type_ == "rf-root-symlink": + _, is_symlink_str, runfile_path, content_path = parts + zip_path = unix_join(runfiles_dir, runfile_path) + elif type_ == "symlink": + _, runfile_path, link_to_rf_path = parts + zip_path = unix_join(runfiles_dir, runfile_path) + link_to_rf_path = unix_join(runfiles_dir, link_to_rf_path) + content_path = os.path.relpath(link_to_rf_path, start=dirname(zip_path)) + is_symlink_str = "2" + else: + raise ValueError( + f"Error: Unknown entry type or invalid format at line {line_idx + 1}: {line}" + ) + + return type_, is_symlink_str, zip_path, content_path + + +def read_manifest( + manifest_path, workspace_name, legacy_external_runfiles, runfiles_dir +): + with open(manifest_path, "r") as f: + entries = [] + for line_idx, line in enumerate(f): + try: + entry = _parse_entry( + line, + line_idx, + workspace_name, + legacy_external_runfiles, + runfiles_dir, + ) + if entry: + entries.append(entry) + except ValueError as e: + e.add_note(f"Error processing line {line_idx + 1}: {line.strip()}") + raise + + # Sort symlink entries first so they have precedence. + # Then sort by zip path + entries.sort(key=lambda x: (x[2], 0 if x[0] == "symlink" else 1)) + return entries + + +def convert_symlink_target(path, platform_pathsep): + """Converts the path a symlink points to the target-platform format. + + On Windows, relative symlinks must use backslashes. + """ + if platform_pathsep == "/": + # Convert Windows to Unix + return path.replace("\\", platform_pathsep) + else: + # Convert Unix to Windows + return path.replace("/", "\\") + + +# Zip files use forward slash for the entries, even on Windows +def normalize_zip_path(path): + return path.replace("\\", "/") + + +def _write_entry(zf, entry, compress_type, seen, platform_pathsep): + type_, is_symlink_str, zip_path, content_path = entry + # Normalize slashes, otherwise the `seen` logic doesn't + # work correctly. + zip_path = normalize_zip_path(zip_path) + if zip_path in seen: + # This can occur because symlink entries have precedence + # over non-symlink entries. + return + seen.add(zip_path) + + if type_ == "rf-empty": + zi = zipfile.ZipInfo(zip_path) + zi.date_time = (1980, 1, 1, 0, 0, 0) + zi.create_system = 3 # Unix + zi.compress_type = compress_type + # Create empty file + zi.external_attr = (0o644 & 0xFFFF) << 16 + zf.writestr(zi, "") + return + if type_ == "symlink": + zi = zipfile.ZipInfo(zip_path) + zi.date_time = (1980, 1, 1, 0, 0, 0) + zi.create_system = 3 # Unix + zi.compress_type = compress_type + target = convert_symlink_target(content_path, platform_pathsep) + # Set permissions to 777 for symlink (standard) + zi.external_attr = (S_IFLNK | 0o777) << 16 + zf.writestr(zi, target) + return + + if is_symlink_str == "-1": + if not os.path.exists(content_path): + is_symlink_str = "1" + else: + is_symlink_str = "0" + + is_symlink = is_symlink_str == "1" + + if is_symlink: + zi = zipfile.ZipInfo(zip_path) + zi.date_time = (1980, 1, 1, 0, 0, 0) + zi.create_system = 3 # Unix + zi.compress_type = compress_type + target = convert_symlink_target(os.readlink(content_path), platform_pathsep) + # Set permissions to 777 for symlink (standard) + zi.external_attr = (S_IFLNK | 0o777) << 16 + zf.writestr(zi, target) + else: + st = os.stat(content_path) + zi = zipfile.ZipInfo(zip_path) + zi.date_time = (1980, 1, 1, 0, 0, 0) + zi.create_system = 3 # Unix + zi.compress_type = compress_type + # Preserve permissions, otherwise execute is dropped. + zi.external_attr = (st.st_mode & 0xFFFF) << 16 + with open(content_path, "rb") as src, zf.open(zi, "w") as dst: + shutil.copyfileobj(src, dst) + + +def create_zip( + *, + manifest_path, + output_zip, + compress_level, + workspace_name, + legacy_external_runfiles, + runfiles_dir, + platform_pathsep, +): + compress_type = zipfile.ZIP_STORED if compress_level == 0 else zipfile.ZIP_DEFLATED + zf_level = compress_level if compress_level != 0 else None + + entries = read_manifest( + manifest_path, workspace_name, legacy_external_runfiles, runfiles_dir + ) + + seen = set() + with zipfile.ZipFile( + output_zip, "w", compress_type, allowZip64=True, compresslevel=zf_level + ) as zf: + for entry in entries: + _write_entry(zf, entry, compress_type, seen, platform_pathsep) + + +def main(): + parser = argparse.ArgumentParser(description="Create a zip file from a manifest.") + parser.add_argument( + "manifest", + help=""" +Path to the manifest file. Lines have one of the following formats: + +1. `regular|is_symlink|zip_path|content_path`: This form stores the `zip_path` + in the zip, whose content is taken from `content_path` + +2. `rf-empty|runfile_path`: A `runfiles.empty_filenames` value. The stored + zip path is computed from `runfile_path` + +3. `rf-file|is_symlink|runfile_path|content_path`: Store a file in + the zip. The zip path is computed from `runfile_path`. + +4. `rf-symlink|is_symlink|runfile_symlink_path|content_path`: Store a + main-repo-relative path in the zip. + +5. `rf-root-symlink|is_symlink|runfile_root_path|content_path`: Store a + runfiles-root-relative path in the zip. + +6. `symlink|runfile_root_path|link_to_path_rf_path`: Store a symlink that + stores a relative path from `runfile_root_path` to `link_to_rf_path` + +In all cases, `is_symlink` has the following values: +* `1` means it should be stored as a symlink whose value is read + (using `readlink()`) from `content_path`. +* `0` means to store it as a regular file, read from `content_path` +* `-1` occurs with Bazel 7 (because it lacks `File.is_symlink`), which means + to infer whether it's a symlink (files to be stored as symlinks can be + determined by looking for symlinks that point to non-existent files). + +For runfiles entries, they have `--runfiles-dir` prepended to their computed +zip path. + +Compute `zip_path` from `runfile_path`: Computing the final zip path for +runfiles entries is a bit complicated, but boils down to computing what the +runfiles-root-relative path would be, with `--legacy-external-runfiles` taken +into account. +""", + ) + parser.add_argument("output", help="Path to the output zip file.") + parser.add_argument( + "--compression", + type=int, + default=0, + help="Compression level (0 for stored, others for deflated)", + ) + parser.add_argument("--workspace-name", default="", help="Name of the workspace") + parser.add_argument( + "--legacy-external-runfiles", + default="0", + choices=["0", "1"], + help="Whether to use legacy external runfiles behavior", + ) + parser.add_argument( + "--runfiles-dir", default="runfiles", help="Name of the runfiles directory" + ) + parser.add_argument( + "--target-platform-pathsep", help="The path separator for the target platform" + ) + args = parser.parse_args() + + try: + create_zip( + manifest_path=args.manifest, + output_zip=args.output, + compress_level=args.compression, + workspace_name=args.workspace_name, + legacy_external_runfiles=args.legacy_external_runfiles == "1", + runfiles_dir=args.runfiles_dir, + platform_pathsep=args.target_platform_pathsep, + ) + except Exception as e: + e.add_note(f"Error creating zip {args.output}") + raise + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/publish/BUILD.bazel b/tools/publish/BUILD.bazel index 1648ac85df..2f02809ccd 100644 --- a/tools/publish/BUILD.bazel +++ b/tools/publish/BUILD.bazel @@ -1,14 +1,10 @@ -load("//python/config_settings:transition.bzl", "py_binary") load("//python/entry_points:py_console_script_binary.bzl", "py_console_script_binary") load("//tools/private:publish_deps.bzl", "publish_deps") py_console_script_binary( name = "twine", - # We use a py_binary rule with version transitions to ensure that we do not - # rely on the default version of the registered python toolchain. What is more - # we are using this instead of `@python_versions//3.11:defs.bzl` because loading - # that file relies on bzlmod being enabled. - binary_rule = py_binary, + # We transition to a specific python version in order to ensure that we + # don't rely on the default version configured by the root module. pkg = "@rules_python_publish_deps//twine", python_version = "3.11", script = "twine", @@ -37,6 +33,9 @@ publish_deps( "requirements_universal.txt": "", # universal "requirements_windows.txt": "windows", }, - upgrade = True, + args = [ + "--emit-index-url", + "--upgrade", # always upgrade + ], visibility = ["//private:__pkg__"], ) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index 2f8088e358..5022924d80 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -6,196 +6,197 @@ backports-tarfile==1.2.0 \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2024.8.30 \ - --hash=sha256:922820b53db7a7257ffbda3f597266d435245903d80737e34f8a45ff3e3230d8 \ - --hash=sha256:bec941d2aa8195e248a60b31ff9f0558284cf01a52591ceda73ea9afffd69fd9 +certifi==2026.7.22 \ + --hash=sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775 \ + --hash=sha256:741e2c3b351ddf169a738da9f2c048608ff7f2c5cc02f1ebc6b118bb090d5d55 # via requests -charset-normalizer==3.4.0 \ - --hash=sha256:0099d79bdfcf5c1f0c2c72f91516702ebf8b0b8ddd8905f97a8aecf49712c621 \ - --hash=sha256:0713f3adb9d03d49d365b70b84775d0a0d18e4ab08d12bc46baa6132ba78aaf6 \ - --hash=sha256:07afec21bbbbf8a5cc3651aa96b980afe2526e7f048fdfb7f1014d84acc8b6d8 \ - --hash=sha256:0b309d1747110feb25d7ed6b01afdec269c647d382c857ef4663bbe6ad95a912 \ - --hash=sha256:0d99dd8ff461990f12d6e42c7347fd9ab2532fb70e9621ba520f9e8637161d7c \ - --hash=sha256:0de7b687289d3c1b3e8660d0741874abe7888100efe14bd0f9fd7141bcbda92b \ - --hash=sha256:1110e22af8ca26b90bd6364fe4c763329b0ebf1ee213ba32b68c73de5752323d \ - --hash=sha256:130272c698667a982a5d0e626851ceff662565379baf0ff2cc58067b81d4f11d \ - --hash=sha256:136815f06a3ae311fae551c3df1f998a1ebd01ddd424aa5603a4336997629e95 \ - --hash=sha256:14215b71a762336254351b00ec720a8e85cada43b987da5a042e4ce3e82bd68e \ - --hash=sha256:1db4e7fefefd0f548d73e2e2e041f9df5c59e178b4c72fbac4cc6f535cfb1565 \ - --hash=sha256:1ffd9493de4c922f2a38c2bf62b831dcec90ac673ed1ca182fe11b4d8e9f2a64 \ - --hash=sha256:2006769bd1640bdf4d5641c69a3d63b71b81445473cac5ded39740a226fa88ab \ - --hash=sha256:20587d20f557fe189b7947d8e7ec5afa110ccf72a3128d61a2a387c3313f46be \ - --hash=sha256:223217c3d4f82c3ac5e29032b3f1c2eb0fb591b72161f86d93f5719079dae93e \ - --hash=sha256:27623ba66c183eca01bf9ff833875b459cad267aeeb044477fedac35e19ba907 \ - --hash=sha256:285e96d9d53422efc0d7a17c60e59f37fbf3dfa942073f666db4ac71e8d726d0 \ - --hash=sha256:2de62e8801ddfff069cd5c504ce3bc9672b23266597d4e4f50eda28846c322f2 \ - --hash=sha256:2f6c34da58ea9c1a9515621f4d9ac379871a8f21168ba1b5e09d74250de5ad62 \ - --hash=sha256:309a7de0a0ff3040acaebb35ec45d18db4b28232f21998851cfa709eeff49d62 \ - --hash=sha256:35c404d74c2926d0287fbd63ed5d27eb911eb9e4a3bb2c6d294f3cfd4a9e0c23 \ - --hash=sha256:3710a9751938947e6327ea9f3ea6332a09bf0ba0c09cae9cb1f250bd1f1549bc \ - --hash=sha256:3d59d125ffbd6d552765510e3f31ed75ebac2c7470c7274195b9161a32350284 \ - --hash=sha256:40d3ff7fc90b98c637bda91c89d51264a3dcf210cade3a2c6f838c7268d7a4ca \ - --hash=sha256:425c5f215d0eecee9a56cdb703203dda90423247421bf0d67125add85d0c4455 \ - --hash=sha256:43193c5cda5d612f247172016c4bb71251c784d7a4d9314677186a838ad34858 \ - --hash=sha256:44aeb140295a2f0659e113b31cfe92c9061622cadbc9e2a2f7b8ef6b1e29ef4b \ - --hash=sha256:47334db71978b23ebcf3c0f9f5ee98b8d65992b65c9c4f2d34c2eaf5bcaf0594 \ - --hash=sha256:4796efc4faf6b53a18e3d46343535caed491776a22af773f366534056c4e1fbc \ - --hash=sha256:4a51b48f42d9358460b78725283f04bddaf44a9358197b889657deba38f329db \ - --hash=sha256:4b67fdab07fdd3c10bb21edab3cbfe8cf5696f453afce75d815d9d7223fbe88b \ - --hash=sha256:4ec9dd88a5b71abfc74e9df5ebe7921c35cbb3b641181a531ca65cdb5e8e4dea \ - --hash=sha256:4f9fc98dad6c2eaa32fc3af1417d95b5e3d08aff968df0cd320066def971f9a6 \ - --hash=sha256:54b6a92d009cbe2fb11054ba694bc9e284dad30a26757b1e372a1fdddaf21920 \ - --hash=sha256:55f56e2ebd4e3bc50442fbc0888c9d8c94e4e06a933804e2af3e89e2f9c1c749 \ - --hash=sha256:5726cf76c982532c1863fb64d8c6dd0e4c90b6ece9feb06c9f202417a31f7dd7 \ - --hash=sha256:5d447056e2ca60382d460a604b6302d8db69476fd2015c81e7c35417cfabe4cd \ - --hash=sha256:5ed2e36c3e9b4f21dd9422f6893dec0abf2cca553af509b10cd630f878d3eb99 \ - --hash=sha256:5ff2ed8194587faf56555927b3aa10e6fb69d931e33953943bc4f837dfee2242 \ - --hash=sha256:62f60aebecfc7f4b82e3f639a7d1433a20ec32824db2199a11ad4f5e146ef5ee \ - --hash=sha256:63bc5c4ae26e4bc6be6469943b8253c0fd4e4186c43ad46e713ea61a0ba49129 \ - --hash=sha256:6b40e8d38afe634559e398cc32b1472f376a4099c75fe6299ae607e404c033b2 \ - --hash=sha256:6b493a043635eb376e50eedf7818f2f322eabbaa974e948bd8bdd29eb7ef2a51 \ - --hash=sha256:6dba5d19c4dfab08e58d5b36304b3f92f3bd5d42c1a3fa37b5ba5cdf6dfcbcee \ - --hash=sha256:6fd30dc99682dc2c603c2b315bded2799019cea829f8bf57dc6b61efde6611c8 \ - --hash=sha256:707b82d19e65c9bd28b81dde95249b07bf9f5b90ebe1ef17d9b57473f8a64b7b \ - --hash=sha256:7706f5850360ac01d80c89bcef1640683cc12ed87f42579dab6c5d3ed6888613 \ - --hash=sha256:7782afc9b6b42200f7362858f9e73b1f8316afb276d316336c0ec3bd73312742 \ - --hash=sha256:79983512b108e4a164b9c8d34de3992f76d48cadc9554c9e60b43f308988aabe \ - --hash=sha256:7f683ddc7eedd742e2889d2bfb96d69573fde1d92fcb811979cdb7165bb9c7d3 \ - --hash=sha256:82357d85de703176b5587dbe6ade8ff67f9f69a41c0733cf2425378b49954de5 \ - --hash=sha256:84450ba661fb96e9fd67629b93d2941c871ca86fc38d835d19d4225ff946a631 \ - --hash=sha256:86f4e8cca779080f66ff4f191a685ced73d2f72d50216f7112185dc02b90b9b7 \ - --hash=sha256:8cda06946eac330cbe6598f77bb54e690b4ca93f593dee1568ad22b04f347c15 \ - --hash=sha256:8ce7fd6767a1cc5a92a639b391891bf1c268b03ec7e021c7d6d902285259685c \ - --hash=sha256:8ff4e7cdfdb1ab5698e675ca622e72d58a6fa2a8aa58195de0c0061288e6e3ea \ - --hash=sha256:9289fd5dddcf57bab41d044f1756550f9e7cf0c8e373b8cdf0ce8773dc4bd417 \ - --hash=sha256:92a7e36b000bf022ef3dbb9c46bfe2d52c047d5e3f3343f43204263c5addc250 \ - --hash=sha256:92db3c28b5b2a273346bebb24857fda45601aef6ae1c011c0a997106581e8a88 \ - --hash=sha256:95c3c157765b031331dd4db3c775e58deaee050a3042fcad72cbc4189d7c8dca \ - --hash=sha256:980b4f289d1d90ca5efcf07958d3eb38ed9c0b7676bf2831a54d4f66f9c27dfa \ - --hash=sha256:9ae4ef0b3f6b41bad6366fb0ea4fc1d7ed051528e113a60fa2a65a9abb5b1d99 \ - --hash=sha256:9c98230f5042f4945f957d006edccc2af1e03ed5e37ce7c373f00a5a4daa6149 \ - --hash=sha256:9fa2566ca27d67c86569e8c85297aaf413ffab85a8960500f12ea34ff98e4c41 \ - --hash=sha256:a14969b8691f7998e74663b77b4c36c0337cb1df552da83d5c9004a93afdb574 \ - --hash=sha256:a8aacce6e2e1edcb6ac625fb0f8c3a9570ccc7bfba1f63419b3769ccf6a00ed0 \ - --hash=sha256:a8e538f46104c815be19c975572d74afb53f29650ea2025bbfaef359d2de2f7f \ - --hash=sha256:aa41e526a5d4a9dfcfbab0716c7e8a1b215abd3f3df5a45cf18a12721d31cb5d \ - --hash=sha256:aa693779a8b50cd97570e5a0f343538a8dbd3e496fa5dcb87e29406ad0299654 \ - --hash=sha256:ab22fbd9765e6954bc0bcff24c25ff71dcbfdb185fcdaca49e81bac68fe724d3 \ - --hash=sha256:ab2e5bef076f5a235c3774b4f4028a680432cded7cad37bba0fd90d64b187d19 \ - --hash=sha256:ab973df98fc99ab39080bfb0eb3a925181454d7c3ac8a1e695fddfae696d9e90 \ - --hash=sha256:af73657b7a68211996527dbfeffbb0864e043d270580c5aef06dc4b659a4b578 \ - --hash=sha256:b197e7094f232959f8f20541ead1d9862ac5ebea1d58e9849c1bf979255dfac9 \ - --hash=sha256:b295729485b06c1a0683af02a9e42d2caa9db04a373dc38a6a58cdd1e8abddf1 \ - --hash=sha256:b8831399554b92b72af5932cdbbd4ddc55c55f631bb13ff8fe4e6536a06c5c51 \ - --hash=sha256:b8dcd239c743aa2f9c22ce674a145e0a25cb1566c495928440a181ca1ccf6719 \ - --hash=sha256:bcb4f8ea87d03bc51ad04add8ceaf9b0f085ac045ab4d74e73bbc2dc033f0236 \ - --hash=sha256:bd7af3717683bea4c87acd8c0d3d5b44d56120b26fd3f8a692bdd2d5260c620a \ - --hash=sha256:bf4475b82be41b07cc5e5ff94810e6a01f276e37c2d55571e3fe175e467a1a1c \ - --hash=sha256:c3e446d253bd88f6377260d07c895816ebf33ffffd56c1c792b13bff9c3e1ade \ - --hash=sha256:c57516e58fd17d03ebe67e181a4e4e2ccab1168f8c2976c6a334d4f819fe5944 \ - --hash=sha256:c94057af19bc953643a33581844649a7fdab902624d2eb739738a30e2b3e60fc \ - --hash=sha256:cab5d0b79d987c67f3b9e9c53f54a61360422a5a0bc075f43cab5621d530c3b6 \ - --hash=sha256:ce031db0408e487fd2775d745ce30a7cd2923667cf3b69d48d219f1d8f5ddeb6 \ - --hash=sha256:cee4373f4d3ad28f1ab6290684d8e2ebdb9e7a1b74fdc39e4c211995f77bec27 \ - --hash=sha256:d5b054862739d276e09928de37c79ddeec42a6e1bfc55863be96a36ba22926f6 \ - --hash=sha256:dbe03226baf438ac4fda9e2d0715022fd579cb641c4cf639fa40d53b2fe6f3e2 \ - --hash=sha256:dc15e99b2d8a656f8e666854404f1ba54765871104e50c8e9813af8a7db07f12 \ - --hash=sha256:dcaf7c1524c0542ee2fc82cc8ec337f7a9f7edee2532421ab200d2b920fc97cf \ - --hash=sha256:dd4eda173a9fcccb5f2e2bd2a9f423d180194b1bf17cf59e3269899235b2a114 \ - --hash=sha256:dd9a8bd8900e65504a305bf8ae6fa9fbc66de94178c420791d0293702fce2df7 \ - --hash=sha256:de7376c29d95d6719048c194a9cf1a1b0393fbe8488a22008610b0361d834ecf \ - --hash=sha256:e7fdd52961feb4c96507aa649550ec2a0d527c086d284749b2f582f2d40a2e0d \ - --hash=sha256:e91f541a85298cf35433bf66f3fab2a4a2cff05c127eeca4af174f6d497f0d4b \ - --hash=sha256:e9e3c4c9e1ed40ea53acf11e2a386383c3304212c965773704e4603d589343ed \ - --hash=sha256:ee803480535c44e7f5ad00788526da7d85525cfefaf8acf8ab9a310000be4b03 \ - --hash=sha256:f09cb5a7bbe1ecae6e87901a2eb23e0256bb524a79ccc53eb0b7629fbe7677c4 \ - --hash=sha256:f19c1585933c82098c2a520f8ec1227f20e339e33aca8fa6f956f6691b784e67 \ - --hash=sha256:f1a2f519ae173b5b6a2c9d5fa3116ce16e48b3462c8b96dfdded11055e3d6365 \ - --hash=sha256:f28f891ccd15c514a0981f3b9db9aa23d62fe1a99997512b0491d2ed323d229a \ - --hash=sha256:f3e73a4255342d4eb26ef6df01e3962e73aa29baa3124a8e824c5d3364a65748 \ - --hash=sha256:f606a1881d2663630ea5b8ce2efe2111740df4b687bd78b34a8131baa007f79b \ - --hash=sha256:fe9f97feb71aa9896b81973a7bbada8c49501dc73e58a10fcef6663af95e5079 \ - --hash=sha256:ffc519621dce0c767e96b9c53f09c5d215578e10b02c285809f76509a3931482 +charset-normalizer==3.4.9 \ + --hash=sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380 \ + --hash=sha256:03d07803992c6c7bbc976327f34b18b6160327fc81cb82c9d504720ac0be3b62 \ + --hash=sha256:04ce310cb89c15df659582aee80a0603788732a5e017d5bd5c81158106ce249c \ + --hash=sha256:0d861473f743244d349b50f850d10eb87aeb22bbdcc8e64f79273c94af5a8226 \ + --hash=sha256:0e94703ec9684807f20cfb5eed95c70f67f2a8f21ad620146d7b5a13677b93e5 \ + --hash=sha256:0fa1aec2d32bcc03c8fa0f6f1712caad1adc38509f31142112e5c9daf5b9c833 \ + --hash=sha256:16b65ea0f2465b6fb52aa22de5eca612aa964ddfec00a912e26f4656cbef890b \ + --hash=sha256:16d10d789dd9bcca1173c95af82c58433122564b7bc39385124be735a35cbe99 \ + --hash=sha256:19ac87f93086ce37b86e098888555c4b4bc48102279bae3350098c0ed664b501 \ + --hash=sha256:1d22856ffbe153a602df38e4a5464f0b748a54002e0d69ac6d2ad0a197cc99ec \ + --hash=sha256:21e764fd1e70b6a3e205a0e46f3051701f98a8cb3fad66eeb80e48bb502f8698 \ + --hash=sha256:231ddcbb35e2ff8973e1365db41fe0572662893b99a05deb183b68ad4c0c8bd4 \ + --hash=sha256:253a4a220747e8b5faf57ec320c4f5efb0cef05f647420bf267143ec15dba10a \ + --hash=sha256:280081916dc341820640489a66e4696049401ef1cf6dd672f672e70ad915aca3 \ + --hash=sha256:2a441ea71902098ffe78c5abe6c494f44160b4af614ed16c3d9a3b1d17fd8ee2 \ + --hash=sha256:304b13570067b2547562e308af560b3963857b1fa90bd6afd978130130fe2d6a \ + --hash=sha256:32286a2c8d167e897177b673176c1e3e00d4057caf5d2b64eef9a3666b03018e \ + --hash=sha256:33bdcc2a32c0a0e861f60841a512c8acc658c87c2ac59d89e3a46dacf7d866e4 \ + --hash=sha256:375b83ed0aecfce76c16d198fbc21f3b11b337d68662bea0a995046682a11419 \ + --hash=sha256:3c09a49d6cde137258beb3d551994a2927fd35ad5cf96aed573f61bbd67c5f84 \ + --hash=sha256:3d92613ec25e43b05f042302531ec0f00b8445190e43325880cbd6ab7c2581da \ + --hash=sha256:40a126142a56b2dfc0aacbad1de8310cbf60da7656db0e6b16eebd48e3e93519 \ + --hash=sha256:416c229f77e5ea25b3dfd4b582f8d73d7e43c22320302b9ab128a2d3a0b38efe \ + --hash=sha256:432786d3561e69aeeae6c7e8648964ce0ad05736120135601f87ac26b9c83381 \ + --hash=sha256:43b9e366a31fdd1c87d0eb08f579b4a82b723ea54338f040d6b4e518a026ea29 \ + --hash=sha256:440eede837960000d74978f0eba527be106b5b9aee0daf779d395276ed0b0614 \ + --hash=sha256:45b0cc4e3556cd875e09102988d1ab8356c998b596c9fced84547c8138b487a0 \ + --hash=sha256:476743fe6dfe14a2da12e3ac79125dc84a3b2cf8094369a47a1529b0cd8549fe \ + --hash=sha256:4773092f8019072343a7447203308b176e10199920eb02d6195e81bbb3274c29 \ + --hash=sha256:4b3dac63058cc36820b0dd072f89898604e2d39686fe05321729d00d8ac185a0 \ + --hash=sha256:4d1c96a7a18b9690a4d46df09e3e3382406ae3213727cd1019ebade1c4a81917 \ + --hash=sha256:51307f5c71007673a2bf8232ad973483d281e74cb99c8c5a990af1eefa6277d9 \ + --hash=sha256:51447e9aa2684679af07ca5021c3db526e0284347ebf4ffcec1154c3350cfe32 \ + --hash=sha256:58150c9f9b9a552505912d182ccdf26f6396fb6094816ceebcbb20eecabaed94 \ + --hash=sha256:5b10cd92fc5c498b35a8635df6d5a100207f88b63a4dc1de7ef9a548e1e2cd63 \ + --hash=sha256:5e226f6218febc71f6c1fc2fafb91c226f75bdc1d8fb12d66823716e891608fd \ + --hash=sha256:609b3ba8fcc0fb5ab7af00719d0fb6ad0cb518e48e7712d12fd68f1327951198 \ + --hash=sha256:60f44ade2cf573dad7a277e6f8ca9a51a21dda572b13bd7d8539bb3cd5dbedde \ + --hash=sha256:611057cc5d5c0afc743ba8be6bd828c17e0aaa8643f9d0a9b9bb7dea80eb8012 \ + --hash=sha256:6366a16e1a25018694d6a5d784d09b046edc9eac40ea2b54065c3052672516a1 \ + --hash=sha256:65a7ff3f705e57d392f7261b6d0550fe137c3019477431f1c355e0db0a7d3e15 \ + --hash=sha256:673611bbd43f0810bec0b0f028ddeaaa501190339cac411f347ac76917c3ae7b \ + --hash=sha256:67830fc78e67501f47bb950471b2dcb9b35b140084429318e862895a8e89c993 \ + --hash=sha256:68ce9f4d6b26d5ccbf7fd4459bf75f74a0a146677ebba80597df60cbdb20e6f4 \ + --hash=sha256:68e5f26a1ad57ded6d1cfb85331d1c1a195314756471d97758c48498bb4dcdf5 \ + --hash=sha256:69b157c5d3292bcd443faca052f3096f637f1e074b98212a933c074ae23dc3b8 \ + --hash=sha256:75286256590a6320cf106a0d28970d3560aad9ee09aa7b34fb40524792436d35 \ + --hash=sha256:78841cccf1af7b40f6f716338d50c0902dbe88d9f800b3c973b7a9a0a693a642 \ + --hash=sha256:78fa18e436a1a0e58dbd7e02fc4473f3f32cceb12df9dfca542d075961c307d2 \ + --hash=sha256:79580094b00d1789d1f93ea55bc43cb2f611910c72235b7657f3482ddcc1b22d \ + --hash=sha256:7b86a2b16095d250c6f58b3d9b2eee6f4147754344f3dab0922f7c9bf7d226c9 \ + --hash=sha256:83aed2c10721ddd90f68140685391b50811a880af20654c59af6b6c66c40513c \ + --hash=sha256:84fd18bcc17526fc2b3c1af7d2b9217d32c9c04448c16ec693b9b4f1985c3d33 \ + --hash=sha256:871ff67ea1aad4dfd91736464934d56b32dac49f9fbe16cddba36198a7b3a0db \ + --hash=sha256:898f0e9068ca27d37f8e83a5b962821df851532e6c4a7d615c1c033f9da6eedf \ + --hash=sha256:8a79d9f4d8001473a30c163556b3c3bfebec837495a412dde78b51672f6134f9 \ + --hash=sha256:8c041122946b7ba21bb32c45b1aa57b1be35527690aeb3c5c234521085632eee \ + --hash=sha256:90c44bc373b7687f6948b693cceaea1348ae0975d7474746559494468e3c1d84 \ + --hash=sha256:9104ed0bd76a429d46f9ec0dbc9b08ad1d2dcdf2b00a5a0daa1c145329b35b44 \ + --hash=sha256:920079c3f7456fa213e0829ed2073aaa727fd39d889ead5b4f35d0de5460d04f \ + --hash=sha256:93d59d504b230e83c7a843251681959a0b6a9cd76f6e146ce1b8a80eb8739af9 \ + --hash=sha256:9b2aff1c7b3884512b9512c3eaadd9bab39fb45042ffaaa1dd08ff2b9f8109d9 \ + --hash=sha256:9b8e0f3107e2200b76f6054de99016eac3ee6762713587b36baaa7e4bd2ae177 \ + --hash=sha256:9bb41182d93ea91f60b4bc8fbf4c820c69ef8a12ab2d917f3f1834f1acad07e8 \ + --hash=sha256:9cdef90ae47919cae358d8ab15797a800ed41da7aba5d72419fb510729e2ed4b \ + --hash=sha256:a1786910334ed46ab1dd73222f2cd1e05c2c3bb39f6dddb4f8b36fc382058a39 \ + --hash=sha256:a4cfde78a9f2880208d16a93b795726a3017d5977e08d1e162a7a31322479c41 \ + --hash=sha256:a4fbdde9dd4a9ce5fd52c2b3a347bb50cc89483ef783f1cb00d408c13f7a96c0 \ + --hash=sha256:aa99adc8f081b475a12843953db36831eaf83ec33eb46a90629ca6a5de45a616 \ + --hash=sha256:ac351b3b8014eead140e77e9717e2992c6bbe30b63bc3422422eb84865412e3d \ + --hash=sha256:ad41ba96094304aa090f5a30cb6e4fb3b3f1c264c523394b4c39bbacc4dc92ba \ + --hash=sha256:b5314963fce9b0b12743891de876e724997864ee22aa496f903f426c7e2fa5b2 \ + --hash=sha256:bcf74c1df76758a395bf0af608c04c82257523f55c9868b334f06270d0f2112b \ + --hash=sha256:bd47ba7fc3ca94896759ea0109775132d3e7ab921fbf54038e1bab2e46c313c9 \ + --hash=sha256:c0323c9daef75ef2e5083624b4585018a0c9d5e3b40f607eed81a311270b934b \ + --hash=sha256:c1225416b463483160e4af85d5fc3a9690ccb53fd4b1865a6437825f5ede3209 \ + --hash=sha256:c1c948747b03be832dceed96ca815cef7360de9aa19d37c730f8e3f6101aca48 \ + --hash=sha256:c25fe15c70c59eb7c5ce8c06a1f3fa1da0ecc5ea1e7a5922c40fd2fa9b0d5046 \ + --hash=sha256:cc1b0fff8ead343dae06305f954eb8468ba0ec1a97881f42489d198e4ce3c632 \ + --hash=sha256:cd6280cf040f233bd7d3407b743b4b4c74f70e8e1c4199cb112a62c941c0772a \ + --hash=sha256:cd6c3d4b783c556fa00bf540854e42f135e2f256abd29669fcd0da0f2dec79c2 \ + --hash=sha256:d4d6fcde76f94f5cb9e43e9e9a61f16dacefd228cbbf6f1a09bd9b219a92f1a1 \ + --hash=sha256:ddf4af30b417d9fe16481e9b81c27ab2a7cde1ff7ba3e85653b02db7d145dc7b \ + --hash=sha256:df115d4d83168fdf2cae48ef1ff6d1cb4c466364e30861b37121de0f3bf1b990 \ + --hash=sha256:df7276909358e5635ae203673ab7e509ddd224225a8d6b0790bf13eb2bde1cc5 \ + --hash=sha256:e4fd89cc178bced6ad29cb3e6dd4aa63fa5017c3524dbd0b25998fb64a87cc8b \ + --hash=sha256:e9701d0049d92c16703a42771b98d560b95248949f23f8cf7b4eddd201814fb9 \ + --hash=sha256:ee2f2a527e3c1a6e6411eb4209642e138b544a2d72fe5d0d76daf77b24063534 \ + --hash=sha256:f7fb7d750cfa0a070d2c24e831fd3481019a60dd317ea2b39acbcebc08b6ed81 \ + --hash=sha256:f840ed6d8ecba8255df8c42b87fadeda98ddfc6eeec05e2dc66e26d46dd6f58a \ + --hash=sha256:f86c6358749bd4fda175388691e3ba8c46e24c5347d0afd20f9b7edfc9faf07d \ + --hash=sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf \ + --hash=sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115 # via requests -docutils==0.21.2 \ - --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ - --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2 +docutils==0.23 \ + --hash=sha256:25d013af9bf23bc1c7b2b093dff4208166c53a94786c9e447808335ef1185fea \ + --hash=sha256:746f5060322511280a1e50eb76846ed6bf2342984b2ac04dc42caa1a8d78799e # via readme-renderer -idna==3.10 \ - --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ - --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3 +id==1.6.1 \ + --hash=sha256:d0732d624fb46fd4e7bc4e5152f00214450953b9e772c182c1c22964def1a069 \ + --hash=sha256:f5ec41ed2629a508f5d0988eda142e190c9c6da971100612c4de9ad9f9b237ca + # via twine +idna==3.18 \ + --hash=sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2 \ + --hash=sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848 # via requests -importlib-metadata==8.5.0 \ - --hash=sha256:45e54197d28b7a7f1559e60b95e7c567032b602131fbd588f1497f47880aa68b \ - --hash=sha256:71522656f0abace1d072b9e5481a48f07c138e00f079c38c8f883823f9c26bd7 - # via - # keyring - # twine +importlib-metadata==9.0.0 \ + --hash=sha256:2d21d1cc5a017bd0559e36150c21c830ab1dc304dedd1b7ea85d20f45ef3edd7 \ + --hash=sha256:a4f57ab599e6a2e3016d7595cfd72eb4661a5106e787a95bcc90c7105b831efc + # via keyring jaraco-classes==3.4.0 \ --hash=sha256:47a024b51d0239c0dd8c8540c6c7f484be3b8fcf0b2d85c13825780d3b3f3acd \ --hash=sha256:f662826b6bed8cace05e7ff873ce0f9283b5c924470fe664fff1c2f00f581790 # via keyring -jaraco-context==6.0.1 \ - --hash=sha256:9bae4ea555cf0b14938dc0aee7c9f32ed303aa20a3b73e7dc80111628792d1b3 \ - --hash=sha256:f797fc481b490edb305122c9181830a3a5b76d84ef6d1aef2fb9b47ab956f9e4 +jaraco-context==6.1.2 \ + --hash=sha256:bf8150b79a2d5d91ae48629d8b427a8f7ba0e1097dd6202a9059f29a36379535 \ + --hash=sha256:f1a6c9d391e661cc5b8d39861ff077a7dc24dc23833ccee564b234b81c82dfe3 # via keyring -jaraco-functools==4.1.0 \ - --hash=sha256:70f7e0e2ae076498e212562325e805204fc092d7b4c17e0e86c959e249701a9d \ - --hash=sha256:ad159f13428bc4acbf5541ad6dec511f91573b90fba04df61dafa2a1231cf649 +jaraco-functools==4.6.0 \ + --hash=sha256:880c577ec9720b3a052d5bc611fb9f2269b3d87902ef42440df443b88e443280 \ + --hash=sha256:99e3dc0060c5cbe8fcd1cdb36258e2a65ca40f1566b2033b12abb1bb44dd3c30 # via keyring -keyring==25.4.1 \ - --hash=sha256:5426f817cf7f6f007ba5ec722b1bcad95a75b27d780343772ad76b17cb47b0bf \ - --hash=sha256:b07ebc55f3e8ed86ac81dd31ef14e81ace9dd9c3d4b5d77a6e9a2016d0d71a1b +keyring==25.7.0 \ + --hash=sha256:be4a0b195f149690c166e850609a477c532ddbfbaed96a404d4e43f8d5e2689f \ + --hash=sha256:fe01bd85eb3f8fb3dd0405defdeac9a5b4f6f0439edbb3149577f244a2e8245b # via twine -markdown-it-py==3.0.0 \ - --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ - --hash=sha256:e3f60a94fa066dc52ec76661e37c851cb232d92f9886b15cb560aaada2df8feb +markdown-it-py==4.2.0 \ + --hash=sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49 \ + --hash=sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a # via rich mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba # via markdown-it-py -more-itertools==10.5.0 \ - --hash=sha256:037b0d3203ce90cca8ab1defbbdac29d5f993fc20131f3664dc8d6acfa872aef \ - --hash=sha256:5482bfef7849c25dc3c6dd53a6173ae4795da2a41a80faea6700d9f5846c5da6 +more-itertools==11.1.0 \ + --hash=sha256:48e8f4d9e7e5878571ecf6f2b4e57634f93cd474cc8cfbd2376f2d11b396e30d \ + --hash=sha256:4b65538ae22f6fed0ce4874efd317463a7489796a0939fa66824dd542125a192 # via # jaraco-classes # jaraco-functools -nh3==0.2.18 \ - --hash=sha256:0411beb0589eacb6734f28d5497ca2ed379eafab8ad8c84b31bb5c34072b7164 \ - --hash=sha256:14c5a72e9fe82aea5fe3072116ad4661af5cf8e8ff8fc5ad3450f123e4925e86 \ - --hash=sha256:19aaba96e0f795bd0a6c56291495ff59364f4300d4a39b29a0abc9cb3774a84b \ - --hash=sha256:34c03fa78e328c691f982b7c03d4423bdfd7da69cd707fe572f544cf74ac23ad \ - --hash=sha256:36c95d4b70530b320b365659bb5034341316e6a9b30f0b25fa9c9eff4c27a204 \ - --hash=sha256:3a157ab149e591bb638a55c8c6bcb8cdb559c8b12c13a8affaba6cedfe51713a \ - --hash=sha256:42c64511469005058cd17cc1537578eac40ae9f7200bedcfd1fc1a05f4f8c200 \ - --hash=sha256:5f36b271dae35c465ef5e9090e1fdaba4a60a56f0bb0ba03e0932a66f28b9189 \ - --hash=sha256:6955369e4d9f48f41e3f238a9e60f9410645db7e07435e62c6a9ea6135a4907f \ - --hash=sha256:7b7c2a3c9eb1a827d42539aa64091640bd275b81e097cd1d8d82ef91ffa2e811 \ - --hash=sha256:8ce0f819d2f1933953fca255db2471ad58184a60508f03e6285e5114b6254844 \ - --hash=sha256:94a166927e53972a9698af9542ace4e38b9de50c34352b962f4d9a7d4c927af4 \ - --hash=sha256:a7f1b5b2c15866f2db413a3649a8fe4fd7b428ae58be2c0f6bca5eefd53ca2be \ - --hash=sha256:c8b3a1cebcba9b3669ed1a84cc65bf005728d2f0bc1ed2a6594a992e817f3a50 \ - --hash=sha256:de3ceed6e661954871d6cd78b410213bdcb136f79aafe22aa7182e028b8c7307 \ - --hash=sha256:f0eca9ca8628dbb4e916ae2491d72957fdd35f7a5d326b7032a345f111ac07fe +nh3==0.3.6 \ + --hash=sha256:082675ff87b9385ec430ffe6d5847ba7456cc39b73720cd4add472f9f4cffd56 \ + --hash=sha256:2411e8c3cee81a1ddd62c2a5d50585c28aa5566d373ad1db92536b95ddb24ef2 \ + --hash=sha256:25c733bee928530556b1db0ea46c52cf5aa686146e38e60a6fc7cb801ef91cec \ + --hash=sha256:2f90d9a0cfdbee218994fdaaeeb5a0fde62d08f35e4eef0378ec1e2200172fd0 \ + --hash=sha256:34d2b0d934156b87ee114f599a3ba9b8b9e17b5d79652ba3a13fa50903de965e \ + --hash=sha256:36d06341bd501240d320f5942481ed5e6846136b666e1ba4faf802b78ebc875f \ + --hash=sha256:43bc1ed3fa0716295fabee29ba42b2667e4a51d140b0a68e092170a765474fa6 \ + --hash=sha256:44673b27010051ab5a5e438a86ec31bbda61d4a77d7e900af6b7be3037c1abae \ + --hash=sha256:455469a29951edc92bc48b47ac2281c3f2609e6c4f6a047056449f8c2c23facf \ + --hash=sha256:4713502748f564fee0633b37b3403783ce0a3af3a3d148ad91025a5bdadb7bc6 \ + --hash=sha256:5276ef17bdba9ad8040575c74072008b13aae429436e9d0429e718bb5f90f4da \ + --hash=sha256:597a8e843bea00b2eb5520658dc24a9bb032e7fc9e7c2c0c4cd29420220c9796 \ + --hash=sha256:69bbb92865a693d909db3a700d3c01537533844d0948c1e9323561ce06ecda41 \ + --hash=sha256:69f365963f63a1e9bff53bdbb3c542c7c2efed3e163c9d5d83a772a2ac468c21 \ + --hash=sha256:82ca5bf427ad1b216b65ede1a2e2d87dc49bec417ceba0f297213107d3cd9d78 \ + --hash=sha256:889932a97fb4abb6f95fef1914c0d269ebfb60011e67121c1163059b9449dbb4 \ + --hash=sha256:905f877dc66dd7aea4a76e54bcb26acb5ff8216f720c0017ccf63e0e6035698e \ + --hash=sha256:a43ebd7543555c3ac1bc353023d0794e75cb76f6f18f19c32e95441496c0cc25 \ + --hash=sha256:d14bf7982e7a77c0c775634c29c07ce08b38a046df73e1c1f139b3e82f18a38e \ + --hash=sha256:e196fa70c2ff2eb4de7d3df3108f8f358c1d69dff20d45b11f20a5aa227ffb6d \ + --hash=sha256:e1b160831c9cdb06a6c79c2f9cdb11386602938f9af260d1c457a85add4f6f69 \ + --hash=sha256:e6b7beece07525dc6e6b0fc2f104442de2ba328360ad00e50cbe2e1fd620447d \ + --hash=sha256:edb2b4a1a27523e6cc7c417f8d21ce3d005243548b93e56b762b66b0c7f589f9 \ + --hash=sha256:f2f14b7ae1fca99c4a66c981aac3974e7fbc1ca30a12673d223ae1df76680917 \ + --hash=sha256:f338ac7d594c067679f1e99b4f5ec3906842979560f9d8f15d6bdfa39a353b10 \ + --hash=sha256:f3736c9dd3d1856f80cd031715b84ca75cda2bbb1ac802c3da26bfce590838d7 \ + --hash=sha256:f5ed5fe84aee7f39db95c214a7421bf0499fbf500fec6d86a4e29bfc37971438 # via readme-renderer -pkginfo==1.10.0 \ - --hash=sha256:5df73835398d10db79f8eecd5cd86b1f6d29317589ea70796994d49399af6297 \ - --hash=sha256:889a6da2ed7ffc58ab5b900d888ddce90bce912f2d2de1dc1c26f4cb9fe65097 +packaging==26.2 \ + --hash=sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e \ + --hash=sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661 # via twine -pygments==2.18.0 \ - --hash=sha256:786ff802f32e91311bff3889f6e9a86e81505fe99f2735bb6d60ae0c5004f199 \ - --hash=sha256:b8e6aca0523f3ab76fee51799c488e38782ac06eafcf95e7ba832985c8e7b13a +pygments==2.20.0 \ + --hash=sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f \ + --hash=sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176 # via # readme-renderer # rich -readme-renderer==44.0 \ - --hash=sha256:2fbca89b81a08526aadf1357a8c2ae889ec05fb03f5da67f9769c9a592166151 \ - --hash=sha256:8712034eabbfa6805cacf1402b4eeb2a73028f72d1166d6f5cb7f9c047c5d1e1 +readme-renderer==45.0 \ + --hash=sha256:030a8fac74904f8fba11ad1bb6964e3f76e896dc7e5e71f16af190c9056696d1 \ + --hash=sha256:3385ed220117104a2bceb4a9dac8c5fdf6d1f96890d7ea2a9c7174fd5c84091f # via twine -requests==2.32.3 \ - --hash=sha256:55365417734eb18255590a9ff9eb97e9e1da868d4ccd6402399eaf68af20a760 \ - --hash=sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6 +requests==2.34.2 \ + --hash=sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0 \ + --hash=sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed # via # requests-toolbelt # twine @@ -207,21 +208,22 @@ rfc3986==2.0.0 \ --hash=sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd \ --hash=sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c # via twine -rich==13.9.3 \ - --hash=sha256:9836f5096eb2172c9e77df411c1b009bace4193d6a481d534fea75ebba758283 \ - --hash=sha256:bc1e01b899537598cf02579d2b9f4a415104d3fc439313a7a2c165d76557a08e +rich==15.0.0 \ + --hash=sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb \ + --hash=sha256:edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36 # via twine -twine==5.1.1 \ - --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ - --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db +twine==7.0.0 \ + --hash=sha256:85cdb29c518efef867360ae4acd4b0dfd61c8654a22fca08e6f8539f05022177 \ + --hash=sha256:b854164df26db268af05f49aa5c0344b10e27a494343ff05b1e0bad3b135f5a7 # via -r tools/publish/requirements.in -urllib3==2.2.3 \ - --hash=sha256:ca899ca043dcb1bafa3e262d73aa25c465bfb49e0bd9dd5d59f1d0acba2f8fac \ - --hash=sha256:e7d814a81dad81e6caf2ec9fdedb284ecc9c73076b62654547cc64ccdcae26e9 +urllib3==2.7.0 \ + --hash=sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c \ + --hash=sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897 # via + # id # requests # twine -zipp==3.20.2 \ - --hash=sha256:a817ac80d6cf4b23bf7f2828b7cabf326f15a001bea8b1f9b49631780ba28350 \ - --hash=sha256:bc9eb26f4506fda01b81bcde0ca78103b6e62f991b381fec825435c836edbc29 +zipp==4.1.0 \ + --hash=sha256:25ad4e16390cd314347dd8f1de67a2ac538ae658ed4ab9db16029c07c188e97f \ + --hash=sha256:4cb57381f544315db7688e976e922a2b18cdb513d21cc194eb42232ba2a3e602 # via importlib-metadata diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 785af7f9af..b786f5d029 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -6,304 +6,357 @@ backports-tarfile==1.2.0 \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2024.8.30 \ - --hash=sha256:922820b53db7a7257ffbda3f597266d435245903d80737e34f8a45ff3e3230d8 \ - --hash=sha256:bec941d2aa8195e248a60b31ff9f0558284cf01a52591ceda73ea9afffd69fd9 +certifi==2026.7.22 \ + --hash=sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775 \ + --hash=sha256:741e2c3b351ddf169a738da9f2c048608ff7f2c5cc02f1ebc6b118bb090d5d55 # via requests -cffi==1.17.1 \ - --hash=sha256:045d61c734659cc045141be4bae381a41d89b741f795af1dd018bfb532fd0df8 \ - --hash=sha256:0984a4925a435b1da406122d4d7968dd861c1385afe3b45ba82b750f229811e2 \ - --hash=sha256:0e2b1fac190ae3ebfe37b979cc1ce69c81f4e4fe5746bb401dca63a9062cdaf1 \ - --hash=sha256:0f048dcf80db46f0098ccac01132761580d28e28bc0f78ae0d58048063317e15 \ - --hash=sha256:1257bdabf294dceb59f5e70c64a3e2f462c30c7ad68092d01bbbfb1c16b1ba36 \ - --hash=sha256:1c39c6016c32bc48dd54561950ebd6836e1670f2ae46128f67cf49e789c52824 \ - --hash=sha256:1d599671f396c4723d016dbddb72fe8e0397082b0a77a4fab8028923bec050e8 \ - --hash=sha256:28b16024becceed8c6dfbc75629e27788d8a3f9030691a1dbf9821a128b22c36 \ - --hash=sha256:2bb1a08b8008b281856e5971307cc386a8e9c5b625ac297e853d36da6efe9c17 \ - --hash=sha256:30c5e0cb5ae493c04c8b42916e52ca38079f1b235c2f8ae5f4527b963c401caf \ - --hash=sha256:31000ec67d4221a71bd3f67df918b1f88f676f1c3b535a7eb473255fdc0b83fc \ - --hash=sha256:386c8bf53c502fff58903061338ce4f4950cbdcb23e2902d86c0f722b786bbe3 \ - --hash=sha256:3edc8d958eb099c634dace3c7e16560ae474aa3803a5df240542b305d14e14ed \ - --hash=sha256:45398b671ac6d70e67da8e4224a065cec6a93541bb7aebe1b198a61b58c7b702 \ - --hash=sha256:46bf43160c1a35f7ec506d254e5c890f3c03648a4dbac12d624e4490a7046cd1 \ - --hash=sha256:4ceb10419a9adf4460ea14cfd6bc43d08701f0835e979bf821052f1805850fe8 \ - --hash=sha256:51392eae71afec0d0c8fb1a53b204dbb3bcabcb3c9b807eedf3e1e6ccf2de903 \ - --hash=sha256:5da5719280082ac6bd9aa7becb3938dc9f9cbd57fac7d2871717b1feb0902ab6 \ - --hash=sha256:610faea79c43e44c71e1ec53a554553fa22321b65fae24889706c0a84d4ad86d \ - --hash=sha256:636062ea65bd0195bc012fea9321aca499c0504409f413dc88af450b57ffd03b \ - --hash=sha256:6883e737d7d9e4899a8a695e00ec36bd4e5e4f18fabe0aca0efe0a4b44cdb13e \ - --hash=sha256:6b8b4a92e1c65048ff98cfe1f735ef8f1ceb72e3d5f0c25fdb12087a23da22be \ - --hash=sha256:6f17be4345073b0a7b8ea599688f692ac3ef23ce28e5df79c04de519dbc4912c \ - --hash=sha256:706510fe141c86a69c8ddc029c7910003a17353970cff3b904ff0686a5927683 \ - --hash=sha256:72e72408cad3d5419375fc87d289076ee319835bdfa2caad331e377589aebba9 \ - --hash=sha256:733e99bc2df47476e3848417c5a4540522f234dfd4ef3ab7fafdf555b082ec0c \ - --hash=sha256:7596d6620d3fa590f677e9ee430df2958d2d6d6de2feeae5b20e82c00b76fbf8 \ - --hash=sha256:78122be759c3f8a014ce010908ae03364d00a1f81ab5c7f4a7a5120607ea56e1 \ - --hash=sha256:805b4371bf7197c329fcb3ead37e710d1bca9da5d583f5073b799d5c5bd1eee4 \ - --hash=sha256:85a950a4ac9c359340d5963966e3e0a94a676bd6245a4b55bc43949eee26a655 \ - --hash=sha256:8f2cdc858323644ab277e9bb925ad72ae0e67f69e804f4898c070998d50b1a67 \ - --hash=sha256:9755e4345d1ec879e3849e62222a18c7174d65a6a92d5b346b1863912168b595 \ - --hash=sha256:98e3969bcff97cae1b2def8ba499ea3d6f31ddfdb7635374834cf89a1a08ecf0 \ - --hash=sha256:a08d7e755f8ed21095a310a693525137cfe756ce62d066e53f502a83dc550f65 \ - --hash=sha256:a1ed2dd2972641495a3ec98445e09766f077aee98a1c896dcb4ad0d303628e41 \ - --hash=sha256:a24ed04c8ffd54b0729c07cee15a81d964e6fee0e3d4d342a27b020d22959dc6 \ - --hash=sha256:a45e3c6913c5b87b3ff120dcdc03f6131fa0065027d0ed7ee6190736a74cd401 \ - --hash=sha256:a9b15d491f3ad5d692e11f6b71f7857e7835eb677955c00cc0aefcd0669adaf6 \ - --hash=sha256:ad9413ccdeda48c5afdae7e4fa2192157e991ff761e7ab8fdd8926f40b160cc3 \ - --hash=sha256:b2ab587605f4ba0bf81dc0cb08a41bd1c0a5906bd59243d56bad7668a6fc6c16 \ - --hash=sha256:b62ce867176a75d03a665bad002af8e6d54644fad99a3c70905c543130e39d93 \ - --hash=sha256:c03e868a0b3bc35839ba98e74211ed2b05d2119be4e8a0f224fba9384f1fe02e \ - --hash=sha256:c59d6e989d07460165cc5ad3c61f9fd8f1b4796eacbd81cee78957842b834af4 \ - --hash=sha256:c7eac2ef9b63c79431bc4b25f1cd649d7f061a28808cbc6c47b534bd789ef964 \ - --hash=sha256:c9c3d058ebabb74db66e431095118094d06abf53284d9c81f27300d0e0d8bc7c \ - --hash=sha256:ca74b8dbe6e8e8263c0ffd60277de77dcee6c837a3d0881d8c1ead7268c9e576 \ - --hash=sha256:caaf0640ef5f5517f49bc275eca1406b0ffa6aa184892812030f04c2abf589a0 \ - --hash=sha256:cdf5ce3acdfd1661132f2a9c19cac174758dc2352bfe37d98aa7512c6b7178b3 \ - --hash=sha256:d016c76bdd850f3c626af19b0542c9677ba156e4ee4fccfdd7848803533ef662 \ - --hash=sha256:d01b12eeeb4427d3110de311e1774046ad344f5b1a7403101878976ecd7a10f3 \ - --hash=sha256:d63afe322132c194cf832bfec0dc69a99fb9bb6bbd550f161a49e9e855cc78ff \ - --hash=sha256:da95af8214998d77a98cc14e3a3bd00aa191526343078b530ceb0bd710fb48a5 \ - --hash=sha256:dd398dbc6773384a17fe0d3e7eeb8d1a21c2200473ee6806bb5e6a8e62bb73dd \ - --hash=sha256:de2ea4b5833625383e464549fec1bc395c1bdeeb5f25c4a3a82b5a8c756ec22f \ - --hash=sha256:de55b766c7aa2e2a3092c51e0483d700341182f08e67c63630d5b6f200bb28e5 \ - --hash=sha256:df8b1c11f177bc2313ec4b2d46baec87a5f3e71fc8b45dab2ee7cae86d9aba14 \ - --hash=sha256:e03eab0a8677fa80d646b5ddece1cbeaf556c313dcfac435ba11f107ba117b5d \ - --hash=sha256:e221cf152cff04059d011ee126477f0d9588303eb57e88923578ace7baad17f9 \ - --hash=sha256:e31ae45bc2e29f6b2abd0de1cc3b9d5205aa847cafaecb8af1476a609a2f6eb7 \ - --hash=sha256:edae79245293e15384b51f88b00613ba9f7198016a5948b5dddf4917d4d26382 \ - --hash=sha256:f1e22e8c4419538cb197e4dd60acc919d7696e5ef98ee4da4e01d3f8cfa4cc5a \ - --hash=sha256:f3a2b4222ce6b60e2e8b337bb9596923045681d71e5a082783484d845390938e \ - --hash=sha256:f6a16c31041f09ead72d69f583767292f750d24913dadacf5756b966aacb3f1a \ - --hash=sha256:f75c7ab1f9e4aca5414ed4d8e5c0e303a34f4421f8a0d47a4d019ceff0ab6af4 \ - --hash=sha256:f79fc4fc25f1c8698ff97788206bb3c2598949bfe0fef03d299eb1b5356ada99 \ - --hash=sha256:f7f5baafcc48261359e14bcd6d9bff6d4b28d9103847c9e136694cb0501aef87 \ - --hash=sha256:fc48c783f9c87e60831201f2cce7f3b2e4846bf4d8728eabe54d60700b318a0b +cffi==2.1.0 \ + --hash=sha256:02cb7ff33ded4f1532476731f89ede53e2e488a8e6205515a82144246ffa7dcc \ + --hash=sha256:03e9810d18c646077e501f661b682fbf5dee4676048527ca3cffe66faa9960dd \ + --hash=sha256:0520e1f4c35f44e209cbbb421b67eec42e6a157f59444dfb6058874ff3610e5d \ + --hash=sha256:0582a58f3051372229ca8e7f5f589f9e5632678208d8636fea3676711fdf7fe5 \ + --hash=sha256:0611e7ebf90573a535ebdc33ae9da222d037853983e13359f580fab781ca017f \ + --hash=sha256:0a42c688d19fca6e095a53c6a6e2295a5b050a8b289f109adab02a9e61a25de6 \ + --hash=sha256:0a96b74cda968eebbad56d973efe5098974f0a9fb323865bf99ea1fd24e3e64c \ + --hash=sha256:10537b1df4967ca26d21e5072d7d54188354483b91dc75058968d3f0cf13fbda \ + --hash=sha256:11b3fb55f4f8ad92274ed26705f65d8f91457de71f5380061eb6d125a768fecd \ + --hash=sha256:15faec4adfff450819f3aee0e2e02c812de6edb88203aa58807955db2003472a \ + --hash=sha256:164bff1657b2a74f0b6d54e11c9b375bc97b931f2ca9c43fcf875838da1570dd \ + --hash=sha256:1854b724d00f6654c742097d5387569021be12d3a0f770eae1df8f8acfcc6acd \ + --hash=sha256:19c54ac121cad98450b4896fa9a43ee0180d57bc4bc911a33db6cab1efab6cd3 \ + --hash=sha256:1b96bfe2c4bd825681b7d311ad6d9b7280a091f43e8f63da5729638083cd3bfb \ + --hash=sha256:1e9f50d192a3e525b15a75ab5114e442d83d657b7ec29182a991bc9a88fd3a66 \ + --hash=sha256:1ff3456eab0d889592d1936d6125bbfbc7ae4d3354a700f8bd80450a66445d4d \ + --hash=sha256:2282cd5e38aa8accd03e99d1256af8411c84cdbee6a89d841b563fdbd1f3e50f \ + --hash=sha256:276f20fffd7b396e12516ba8edf9509210ac248cbbc5acbc39cd512f9f59ebe6 \ + --hash=sha256:2b71d409cccee78310ab5dec549aed052aaea483346e282c7b02362596e01bb0 \ + --hash=sha256:2e9dabb9abcb7ad15938c7196ad5c1718a4e6d33cc79b4c0209bdb64c4a54a5c \ + --hash=sha256:30b65779d598c370374fefabf138d456fd6f3216bfa7bedfab1ba82025b0cd93 \ + --hash=sha256:33eb1ad83ebe8f313e0df035c406227d55a79456704a863fad9842136af5ad7d \ + --hash=sha256:35aaea0c7ee0e58a5cd8c2fd1a48fdf7ece0d2699b7ecdda08194e9ce5dd9b3d \ + --hash=sha256:3681e031db29958a7502f5c0c9d6bbc4c36cb20f7b104086fa642d1799631ff8 \ + --hash=sha256:379de10ce1ba048b1448599d1b37b24caee16309d1ac98d3982fc997f768700b \ + --hash=sha256:37f525a7e7e50c017fdebe58b787be310ad59357ae43a053943a6e1a6c526001 \ + --hash=sha256:3b926723c13eba9f81d2ef3820d63aeceec3b2d4639906047bf675cb8a7a500d \ + --hash=sha256:3d7f118b5adbfdfead90c25822690b02bc8074fba949bb7858bec4ebd55adb43 \ + --hash=sha256:46b1c8db8f6122420f32d02fffb924c2fe9bc772d228c7c711748fff56aabb2b \ + --hash=sha256:47ff3a8bfd8cb9da1af7524b965127095055654c177fcfc7578debcb015eecd0 \ + --hash=sha256:4d433a51f1870e43a13b6732f92aaf540ff77c2015097c78556f75a2d6c030e0 \ + --hash=sha256:4f26194e3d95e06501b942642855aed4f953d55e95d7d01b7c4483db3ecff458 \ + --hash=sha256:510aeeeac94811b138077451da1fb18b308a5feab47dd2b603af55804155e1c8 \ + --hash=sha256:5972433ad71a9e46516584ef60a0fda12d9dc459938d1539c3ddecf9bdc1368d \ + --hash=sha256:5ecbd0499275d57506d397eebe1981cee87b47fcd9ef5c22cab7ed7644a39a94 \ + --hash=sha256:6274dcb2d15cef48daa73ed1be5a40d501d74dccd0cd6db364776d12cb6ba022 \ + --hash=sha256:63960549e4f8dc41e31accb97b975abaecfc44c03e396c093a6436763c2ea7db \ + --hash=sha256:64c753a0f87a256020004f37a1c8c02c480e725f910f0b2a0f3f07debd1b2479 \ + --hash=sha256:6af371f3767faeffc6ac1ef57cdfd25844403e9d3f476c5537caee499de96376 \ + --hash=sha256:6ca4919c6e4f89aa99c42510b42cf54596892c00b3f9077f6bdd1505e24b9c8d \ + --hash=sha256:6d194185eabd279f1c05ebe3504265ddfc5ad2b58d0714f7db9f01da592e9eb6 \ + --hash=sha256:702c436735fbe99d59ada02a1f65cfc0d31c0ee8b7290912f8fbc5cd1e4b16c3 \ + --hash=sha256:716ff8ec22f20b4d988b12884086bcef0fc99737043e503f7a3935a6be99b1ea \ + --hash=sha256:762f99479dcb369f60ab9017ad4ab97a36a1dd7c1ee5a3b15db0f4b8659120cd \ + --hash=sha256:7762faa47e8ff7eb80bd261d9a7d8eea2d8baa69de5e95b70c1f338bbe712f02 \ + --hash=sha256:78474632761faa0fb96f30b1c928c84ebcf68713cbb80d15bab09dfe61640fde \ + --hash=sha256:799416bae98336e400981ff6e532d67d5c709cfb30afb79865a1315f94b0e224 \ + --hash=sha256:7d034dcffa09e9a46c93fa3a3be402096cb5354ac6e41ab8e5cc9cd8b642ad76 \ + --hash=sha256:7d28dff1db6764108bc30788d85d61c876beff416d9a49cb9dd7c5a9f34f5804 \ + --hash=sha256:7d3538f9c0e50670f4deb93dbb696576e60590369cae2faf7de681e597a8a1f1 \ + --hash=sha256:7d5980a3433d4b71a5e120f9dd551403d7824e31e2e67124fe2769c404c06913 \ + --hash=sha256:7ea6b3e2c4250ff1de21c630fe72d0f63eb95c2c32ffbf64a358cf4a8836d714 \ + --hash=sha256:86cf8755a791f72c85dc287128cc62d4f24d392e3f1e15837245623f4a33cccc \ + --hash=sha256:88023dfe18799507b73f1dbb0d14326a17465de1bc9c9c7655c22845e9ddc3a2 \ + --hash=sha256:89095c1968b4ba8285840e131bf2891b09ae137fe2146905acae0354fbce1b5e \ + --hash=sha256:8d35c139744adb3e727cd51b1a18324bbe44b8bd41bf8322bca4d41289f48eda \ + --hash=sha256:8e74a6135550c4748af665b1b1118b6aab33b1fc6a16f9aff630af107c3b4512 \ + --hash=sha256:8f9ec95b8a043d3dfbc74d9abc6f7baf524dd27a8dc160b0a32ff9cdab650c28 \ + --hash=sha256:90bec57cf82089383bd06a605b3eb8daebf7e5a668520beaf6e327a83a947699 \ + --hash=sha256:95f2954c2c9473d892eca6e0409f3568b37ab62a8eedb122461f73cc273476e3 \ + --hash=sha256:961be50688f7fba2fa65f63712d3b9b341a22311f5253460ce933f52f0de1c8c \ + --hash=sha256:98fff996e983a36d3aa2eca83af40c5821202e7e6f32d13ae94e3d2286f10cfe \ + --hash=sha256:9b8f0f26ca4e7513c534d351eca551947d053fac438f2a04ac96d882909b0d3a \ + --hash=sha256:9d72af0cf10a76a600a9690078fe31c63b9588c8e86bf9fd353f713c84b5db0f \ + --hash=sha256:9d8272c0e483b024e1b9ad029821470ed8ec65631dbd90217469da0e7cd89f1c \ + --hash=sha256:a016194dbe13d14ee9556e734b772d8d67b947092b268d757fd4290e3ba2dfc2 \ + --hash=sha256:a5781494d4d400a3f47f8f1da94b324f6e6b440a53387774002890a2a2f4b50f \ + --hash=sha256:a95b05f9baf29b91171b3a8bd2020b028835243e7b0ff6bb23e2a3c228518b1b \ + --hash=sha256:aa7a1b53a2a4452ada2d1b5dade9960b2522f1e61293a811a077439e39029565 \ + --hash=sha256:ac0f1a2d0cfa7eea3f2aaf006ab6e70e8feeb16b75d65b7e5939982ca2f11056 \ + --hash=sha256:af5e2915d41fe6c961694d7bfdc8562942638200f3ce2765dfb8b745cf997629 \ + --hash=sha256:b6422532152adf4e59b110cb2808cee7a033800952f5c036b4af047ee43199e7 \ + --hash=sha256:b65f590ef2a44640f9a05dbb548a429b4ade77913ce683ac8b1480777658a6c0 \ + --hash=sha256:ba00f661f8ba35d075c937174e27c2c421cec3942fd2e0ea3e66996757c0fdd9 \ + --hash=sha256:bccbbb5ee76a61f9d99b5bf3846a51d7fca4b6a732fe46f89295610edaf41853 \ + --hash=sha256:bf01d8c84cbea96b944c73b22182e6c7c432b3475632b8111dbfdc95ddad6e13 \ + --hash=sha256:bf5c6cf48238b0eb4c086978c492ad1cbc22373fc5b2d7353b3a598ce6db887a \ + --hash=sha256:c16914df9fb7f500e440e6875fa23ff5e0b31db01fa9c06af98d59a91f0dc2e4 \ + --hash=sha256:c351efb95e832a853a29361675f33a7ce53de1a109cd73fd47af0712213aa4ce \ + --hash=sha256:c4165821e131d6d4ca444347c2b694e2311bcfa3fe5a861cc72968f28867beac \ + --hash=sha256:c5f5df567f6eb216de69be06ce55c8b714090fae02b18a3b40da8163b8c5fa9c \ + --hash=sha256:c941bb58d5a6e1c3892d86e42927ed6c180302f07e6d395d08c416e594b98b46 \ + --hash=sha256:c97f080ea627e2863524c5af3836e2270b5f5dfff1f104392b959f8df0c5d384 \ + --hash=sha256:cb96698e3c7413d906ce83f8ffd245ec1bd94707541f299d0ce4d6b0193e982b \ + --hash=sha256:cbb7640ce37159548d2147b5b8c241f962143d4c71231431820783f4dc78f210 \ + --hash=sha256:cdf2448aab5f661c9315308ec8b93f4e8a1a67a3c733f8631067a2b67d5913dc \ + --hash=sha256:d2117334c3af3bdcb9a88522b844a2bdb5efdc4f71c6c822df55486ae1c3347a \ + --hash=sha256:d53d10f7da99ae46f7373b9150393e9c5eab9b224909982b43832668de4779f5 \ + --hash=sha256:d9fafc5aa2e2a39aaf7f8cc0c1f044a9b07fca12e558dca53a3cc5c654ad67a7 \ + --hash=sha256:db3eb7d46527159a878ec3460e9d40615bc25ba337d477db681aea6e4f05c5d2 \ + --hash=sha256:dbf7c7a88e2bac086f06d14577332760bdeecc42bdec8ac4077f6260557d9326 \ + --hash=sha256:df2b82571a1b30f58a87bf4e5a9e78d2b1eff6c6ce8fd3aa3757221f93f0863f \ + --hash=sha256:df92f2aba50eb4d96718b68ef76f2e57a57b54f2fa62333496d16c6d585a85ca \ + --hash=sha256:eb4e8997a49aa2c08a3e43c9045d224448b8941d88e7ac163c7d383e560cbf98 \ + --hash=sha256:efc1cdd798b1aaf39b4610bba7aad28c9bea9b910f25c784ccf9ec1fa719d1f9 \ + --hash=sha256:f146d154428a2523f9cc7936c02353c2459b8f6cf07d3cd1ee1c0a611109c5d5 \ + --hash=sha256:f5bce581e6b8c235e566a14768a943b172ada3ed73537bb0c0be1edee312d4e7 \ + --hash=sha256:f9912624a0c0b834b7520d7769b3644453aabc0a7e1c839da7359f050750e9bc \ + --hash=sha256:fb62edb5bb52cca65fab91a63afa7561607120d26090a7e8fda6fb9f064726da \ + --hash=sha256:ff067a8d8d880e7809e4ac88eb009bb848870115317b306666502ccad30b147f # via cryptography -charset-normalizer==3.4.0 \ - --hash=sha256:0099d79bdfcf5c1f0c2c72f91516702ebf8b0b8ddd8905f97a8aecf49712c621 \ - --hash=sha256:0713f3adb9d03d49d365b70b84775d0a0d18e4ab08d12bc46baa6132ba78aaf6 \ - --hash=sha256:07afec21bbbbf8a5cc3651aa96b980afe2526e7f048fdfb7f1014d84acc8b6d8 \ - --hash=sha256:0b309d1747110feb25d7ed6b01afdec269c647d382c857ef4663bbe6ad95a912 \ - --hash=sha256:0d99dd8ff461990f12d6e42c7347fd9ab2532fb70e9621ba520f9e8637161d7c \ - --hash=sha256:0de7b687289d3c1b3e8660d0741874abe7888100efe14bd0f9fd7141bcbda92b \ - --hash=sha256:1110e22af8ca26b90bd6364fe4c763329b0ebf1ee213ba32b68c73de5752323d \ - --hash=sha256:130272c698667a982a5d0e626851ceff662565379baf0ff2cc58067b81d4f11d \ - --hash=sha256:136815f06a3ae311fae551c3df1f998a1ebd01ddd424aa5603a4336997629e95 \ - --hash=sha256:14215b71a762336254351b00ec720a8e85cada43b987da5a042e4ce3e82bd68e \ - --hash=sha256:1db4e7fefefd0f548d73e2e2e041f9df5c59e178b4c72fbac4cc6f535cfb1565 \ - --hash=sha256:1ffd9493de4c922f2a38c2bf62b831dcec90ac673ed1ca182fe11b4d8e9f2a64 \ - --hash=sha256:2006769bd1640bdf4d5641c69a3d63b71b81445473cac5ded39740a226fa88ab \ - --hash=sha256:20587d20f557fe189b7947d8e7ec5afa110ccf72a3128d61a2a387c3313f46be \ - --hash=sha256:223217c3d4f82c3ac5e29032b3f1c2eb0fb591b72161f86d93f5719079dae93e \ - --hash=sha256:27623ba66c183eca01bf9ff833875b459cad267aeeb044477fedac35e19ba907 \ - --hash=sha256:285e96d9d53422efc0d7a17c60e59f37fbf3dfa942073f666db4ac71e8d726d0 \ - --hash=sha256:2de62e8801ddfff069cd5c504ce3bc9672b23266597d4e4f50eda28846c322f2 \ - --hash=sha256:2f6c34da58ea9c1a9515621f4d9ac379871a8f21168ba1b5e09d74250de5ad62 \ - --hash=sha256:309a7de0a0ff3040acaebb35ec45d18db4b28232f21998851cfa709eeff49d62 \ - --hash=sha256:35c404d74c2926d0287fbd63ed5d27eb911eb9e4a3bb2c6d294f3cfd4a9e0c23 \ - --hash=sha256:3710a9751938947e6327ea9f3ea6332a09bf0ba0c09cae9cb1f250bd1f1549bc \ - --hash=sha256:3d59d125ffbd6d552765510e3f31ed75ebac2c7470c7274195b9161a32350284 \ - --hash=sha256:40d3ff7fc90b98c637bda91c89d51264a3dcf210cade3a2c6f838c7268d7a4ca \ - --hash=sha256:425c5f215d0eecee9a56cdb703203dda90423247421bf0d67125add85d0c4455 \ - --hash=sha256:43193c5cda5d612f247172016c4bb71251c784d7a4d9314677186a838ad34858 \ - --hash=sha256:44aeb140295a2f0659e113b31cfe92c9061622cadbc9e2a2f7b8ef6b1e29ef4b \ - --hash=sha256:47334db71978b23ebcf3c0f9f5ee98b8d65992b65c9c4f2d34c2eaf5bcaf0594 \ - --hash=sha256:4796efc4faf6b53a18e3d46343535caed491776a22af773f366534056c4e1fbc \ - --hash=sha256:4a51b48f42d9358460b78725283f04bddaf44a9358197b889657deba38f329db \ - --hash=sha256:4b67fdab07fdd3c10bb21edab3cbfe8cf5696f453afce75d815d9d7223fbe88b \ - --hash=sha256:4ec9dd88a5b71abfc74e9df5ebe7921c35cbb3b641181a531ca65cdb5e8e4dea \ - --hash=sha256:4f9fc98dad6c2eaa32fc3af1417d95b5e3d08aff968df0cd320066def971f9a6 \ - --hash=sha256:54b6a92d009cbe2fb11054ba694bc9e284dad30a26757b1e372a1fdddaf21920 \ - --hash=sha256:55f56e2ebd4e3bc50442fbc0888c9d8c94e4e06a933804e2af3e89e2f9c1c749 \ - --hash=sha256:5726cf76c982532c1863fb64d8c6dd0e4c90b6ece9feb06c9f202417a31f7dd7 \ - --hash=sha256:5d447056e2ca60382d460a604b6302d8db69476fd2015c81e7c35417cfabe4cd \ - --hash=sha256:5ed2e36c3e9b4f21dd9422f6893dec0abf2cca553af509b10cd630f878d3eb99 \ - --hash=sha256:5ff2ed8194587faf56555927b3aa10e6fb69d931e33953943bc4f837dfee2242 \ - --hash=sha256:62f60aebecfc7f4b82e3f639a7d1433a20ec32824db2199a11ad4f5e146ef5ee \ - --hash=sha256:63bc5c4ae26e4bc6be6469943b8253c0fd4e4186c43ad46e713ea61a0ba49129 \ - --hash=sha256:6b40e8d38afe634559e398cc32b1472f376a4099c75fe6299ae607e404c033b2 \ - --hash=sha256:6b493a043635eb376e50eedf7818f2f322eabbaa974e948bd8bdd29eb7ef2a51 \ - --hash=sha256:6dba5d19c4dfab08e58d5b36304b3f92f3bd5d42c1a3fa37b5ba5cdf6dfcbcee \ - --hash=sha256:6fd30dc99682dc2c603c2b315bded2799019cea829f8bf57dc6b61efde6611c8 \ - --hash=sha256:707b82d19e65c9bd28b81dde95249b07bf9f5b90ebe1ef17d9b57473f8a64b7b \ - --hash=sha256:7706f5850360ac01d80c89bcef1640683cc12ed87f42579dab6c5d3ed6888613 \ - --hash=sha256:7782afc9b6b42200f7362858f9e73b1f8316afb276d316336c0ec3bd73312742 \ - --hash=sha256:79983512b108e4a164b9c8d34de3992f76d48cadc9554c9e60b43f308988aabe \ - --hash=sha256:7f683ddc7eedd742e2889d2bfb96d69573fde1d92fcb811979cdb7165bb9c7d3 \ - --hash=sha256:82357d85de703176b5587dbe6ade8ff67f9f69a41c0733cf2425378b49954de5 \ - --hash=sha256:84450ba661fb96e9fd67629b93d2941c871ca86fc38d835d19d4225ff946a631 \ - --hash=sha256:86f4e8cca779080f66ff4f191a685ced73d2f72d50216f7112185dc02b90b9b7 \ - --hash=sha256:8cda06946eac330cbe6598f77bb54e690b4ca93f593dee1568ad22b04f347c15 \ - --hash=sha256:8ce7fd6767a1cc5a92a639b391891bf1c268b03ec7e021c7d6d902285259685c \ - --hash=sha256:8ff4e7cdfdb1ab5698e675ca622e72d58a6fa2a8aa58195de0c0061288e6e3ea \ - --hash=sha256:9289fd5dddcf57bab41d044f1756550f9e7cf0c8e373b8cdf0ce8773dc4bd417 \ - --hash=sha256:92a7e36b000bf022ef3dbb9c46bfe2d52c047d5e3f3343f43204263c5addc250 \ - --hash=sha256:92db3c28b5b2a273346bebb24857fda45601aef6ae1c011c0a997106581e8a88 \ - --hash=sha256:95c3c157765b031331dd4db3c775e58deaee050a3042fcad72cbc4189d7c8dca \ - --hash=sha256:980b4f289d1d90ca5efcf07958d3eb38ed9c0b7676bf2831a54d4f66f9c27dfa \ - --hash=sha256:9ae4ef0b3f6b41bad6366fb0ea4fc1d7ed051528e113a60fa2a65a9abb5b1d99 \ - --hash=sha256:9c98230f5042f4945f957d006edccc2af1e03ed5e37ce7c373f00a5a4daa6149 \ - --hash=sha256:9fa2566ca27d67c86569e8c85297aaf413ffab85a8960500f12ea34ff98e4c41 \ - --hash=sha256:a14969b8691f7998e74663b77b4c36c0337cb1df552da83d5c9004a93afdb574 \ - --hash=sha256:a8aacce6e2e1edcb6ac625fb0f8c3a9570ccc7bfba1f63419b3769ccf6a00ed0 \ - --hash=sha256:a8e538f46104c815be19c975572d74afb53f29650ea2025bbfaef359d2de2f7f \ - --hash=sha256:aa41e526a5d4a9dfcfbab0716c7e8a1b215abd3f3df5a45cf18a12721d31cb5d \ - --hash=sha256:aa693779a8b50cd97570e5a0f343538a8dbd3e496fa5dcb87e29406ad0299654 \ - --hash=sha256:ab22fbd9765e6954bc0bcff24c25ff71dcbfdb185fcdaca49e81bac68fe724d3 \ - --hash=sha256:ab2e5bef076f5a235c3774b4f4028a680432cded7cad37bba0fd90d64b187d19 \ - --hash=sha256:ab973df98fc99ab39080bfb0eb3a925181454d7c3ac8a1e695fddfae696d9e90 \ - --hash=sha256:af73657b7a68211996527dbfeffbb0864e043d270580c5aef06dc4b659a4b578 \ - --hash=sha256:b197e7094f232959f8f20541ead1d9862ac5ebea1d58e9849c1bf979255dfac9 \ - --hash=sha256:b295729485b06c1a0683af02a9e42d2caa9db04a373dc38a6a58cdd1e8abddf1 \ - --hash=sha256:b8831399554b92b72af5932cdbbd4ddc55c55f631bb13ff8fe4e6536a06c5c51 \ - --hash=sha256:b8dcd239c743aa2f9c22ce674a145e0a25cb1566c495928440a181ca1ccf6719 \ - --hash=sha256:bcb4f8ea87d03bc51ad04add8ceaf9b0f085ac045ab4d74e73bbc2dc033f0236 \ - --hash=sha256:bd7af3717683bea4c87acd8c0d3d5b44d56120b26fd3f8a692bdd2d5260c620a \ - --hash=sha256:bf4475b82be41b07cc5e5ff94810e6a01f276e37c2d55571e3fe175e467a1a1c \ - --hash=sha256:c3e446d253bd88f6377260d07c895816ebf33ffffd56c1c792b13bff9c3e1ade \ - --hash=sha256:c57516e58fd17d03ebe67e181a4e4e2ccab1168f8c2976c6a334d4f819fe5944 \ - --hash=sha256:c94057af19bc953643a33581844649a7fdab902624d2eb739738a30e2b3e60fc \ - --hash=sha256:cab5d0b79d987c67f3b9e9c53f54a61360422a5a0bc075f43cab5621d530c3b6 \ - --hash=sha256:ce031db0408e487fd2775d745ce30a7cd2923667cf3b69d48d219f1d8f5ddeb6 \ - --hash=sha256:cee4373f4d3ad28f1ab6290684d8e2ebdb9e7a1b74fdc39e4c211995f77bec27 \ - --hash=sha256:d5b054862739d276e09928de37c79ddeec42a6e1bfc55863be96a36ba22926f6 \ - --hash=sha256:dbe03226baf438ac4fda9e2d0715022fd579cb641c4cf639fa40d53b2fe6f3e2 \ - --hash=sha256:dc15e99b2d8a656f8e666854404f1ba54765871104e50c8e9813af8a7db07f12 \ - --hash=sha256:dcaf7c1524c0542ee2fc82cc8ec337f7a9f7edee2532421ab200d2b920fc97cf \ - --hash=sha256:dd4eda173a9fcccb5f2e2bd2a9f423d180194b1bf17cf59e3269899235b2a114 \ - --hash=sha256:dd9a8bd8900e65504a305bf8ae6fa9fbc66de94178c420791d0293702fce2df7 \ - --hash=sha256:de7376c29d95d6719048c194a9cf1a1b0393fbe8488a22008610b0361d834ecf \ - --hash=sha256:e7fdd52961feb4c96507aa649550ec2a0d527c086d284749b2f582f2d40a2e0d \ - --hash=sha256:e91f541a85298cf35433bf66f3fab2a4a2cff05c127eeca4af174f6d497f0d4b \ - --hash=sha256:e9e3c4c9e1ed40ea53acf11e2a386383c3304212c965773704e4603d589343ed \ - --hash=sha256:ee803480535c44e7f5ad00788526da7d85525cfefaf8acf8ab9a310000be4b03 \ - --hash=sha256:f09cb5a7bbe1ecae6e87901a2eb23e0256bb524a79ccc53eb0b7629fbe7677c4 \ - --hash=sha256:f19c1585933c82098c2a520f8ec1227f20e339e33aca8fa6f956f6691b784e67 \ - --hash=sha256:f1a2f519ae173b5b6a2c9d5fa3116ce16e48b3462c8b96dfdded11055e3d6365 \ - --hash=sha256:f28f891ccd15c514a0981f3b9db9aa23d62fe1a99997512b0491d2ed323d229a \ - --hash=sha256:f3e73a4255342d4eb26ef6df01e3962e73aa29baa3124a8e824c5d3364a65748 \ - --hash=sha256:f606a1881d2663630ea5b8ce2efe2111740df4b687bd78b34a8131baa007f79b \ - --hash=sha256:fe9f97feb71aa9896b81973a7bbada8c49501dc73e58a10fcef6663af95e5079 \ - --hash=sha256:ffc519621dce0c767e96b9c53f09c5d215578e10b02c285809f76509a3931482 +charset-normalizer==3.4.9 \ + --hash=sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380 \ + --hash=sha256:03d07803992c6c7bbc976327f34b18b6160327fc81cb82c9d504720ac0be3b62 \ + --hash=sha256:04ce310cb89c15df659582aee80a0603788732a5e017d5bd5c81158106ce249c \ + --hash=sha256:0d861473f743244d349b50f850d10eb87aeb22bbdcc8e64f79273c94af5a8226 \ + --hash=sha256:0e94703ec9684807f20cfb5eed95c70f67f2a8f21ad620146d7b5a13677b93e5 \ + --hash=sha256:0fa1aec2d32bcc03c8fa0f6f1712caad1adc38509f31142112e5c9daf5b9c833 \ + --hash=sha256:16b65ea0f2465b6fb52aa22de5eca612aa964ddfec00a912e26f4656cbef890b \ + --hash=sha256:16d10d789dd9bcca1173c95af82c58433122564b7bc39385124be735a35cbe99 \ + --hash=sha256:19ac87f93086ce37b86e098888555c4b4bc48102279bae3350098c0ed664b501 \ + --hash=sha256:1d22856ffbe153a602df38e4a5464f0b748a54002e0d69ac6d2ad0a197cc99ec \ + --hash=sha256:21e764fd1e70b6a3e205a0e46f3051701f98a8cb3fad66eeb80e48bb502f8698 \ + --hash=sha256:231ddcbb35e2ff8973e1365db41fe0572662893b99a05deb183b68ad4c0c8bd4 \ + --hash=sha256:253a4a220747e8b5faf57ec320c4f5efb0cef05f647420bf267143ec15dba10a \ + --hash=sha256:280081916dc341820640489a66e4696049401ef1cf6dd672f672e70ad915aca3 \ + --hash=sha256:2a441ea71902098ffe78c5abe6c494f44160b4af614ed16c3d9a3b1d17fd8ee2 \ + --hash=sha256:304b13570067b2547562e308af560b3963857b1fa90bd6afd978130130fe2d6a \ + --hash=sha256:32286a2c8d167e897177b673176c1e3e00d4057caf5d2b64eef9a3666b03018e \ + --hash=sha256:33bdcc2a32c0a0e861f60841a512c8acc658c87c2ac59d89e3a46dacf7d866e4 \ + --hash=sha256:375b83ed0aecfce76c16d198fbc21f3b11b337d68662bea0a995046682a11419 \ + --hash=sha256:3c09a49d6cde137258beb3d551994a2927fd35ad5cf96aed573f61bbd67c5f84 \ + --hash=sha256:3d92613ec25e43b05f042302531ec0f00b8445190e43325880cbd6ab7c2581da \ + --hash=sha256:40a126142a56b2dfc0aacbad1de8310cbf60da7656db0e6b16eebd48e3e93519 \ + --hash=sha256:416c229f77e5ea25b3dfd4b582f8d73d7e43c22320302b9ab128a2d3a0b38efe \ + --hash=sha256:432786d3561e69aeeae6c7e8648964ce0ad05736120135601f87ac26b9c83381 \ + --hash=sha256:43b9e366a31fdd1c87d0eb08f579b4a82b723ea54338f040d6b4e518a026ea29 \ + --hash=sha256:440eede837960000d74978f0eba527be106b5b9aee0daf779d395276ed0b0614 \ + --hash=sha256:45b0cc4e3556cd875e09102988d1ab8356c998b596c9fced84547c8138b487a0 \ + --hash=sha256:476743fe6dfe14a2da12e3ac79125dc84a3b2cf8094369a47a1529b0cd8549fe \ + --hash=sha256:4773092f8019072343a7447203308b176e10199920eb02d6195e81bbb3274c29 \ + --hash=sha256:4b3dac63058cc36820b0dd072f89898604e2d39686fe05321729d00d8ac185a0 \ + --hash=sha256:4d1c96a7a18b9690a4d46df09e3e3382406ae3213727cd1019ebade1c4a81917 \ + --hash=sha256:51307f5c71007673a2bf8232ad973483d281e74cb99c8c5a990af1eefa6277d9 \ + --hash=sha256:51447e9aa2684679af07ca5021c3db526e0284347ebf4ffcec1154c3350cfe32 \ + --hash=sha256:58150c9f9b9a552505912d182ccdf26f6396fb6094816ceebcbb20eecabaed94 \ + --hash=sha256:5b10cd92fc5c498b35a8635df6d5a100207f88b63a4dc1de7ef9a548e1e2cd63 \ + --hash=sha256:5e226f6218febc71f6c1fc2fafb91c226f75bdc1d8fb12d66823716e891608fd \ + --hash=sha256:609b3ba8fcc0fb5ab7af00719d0fb6ad0cb518e48e7712d12fd68f1327951198 \ + --hash=sha256:60f44ade2cf573dad7a277e6f8ca9a51a21dda572b13bd7d8539bb3cd5dbedde \ + --hash=sha256:611057cc5d5c0afc743ba8be6bd828c17e0aaa8643f9d0a9b9bb7dea80eb8012 \ + --hash=sha256:6366a16e1a25018694d6a5d784d09b046edc9eac40ea2b54065c3052672516a1 \ + --hash=sha256:65a7ff3f705e57d392f7261b6d0550fe137c3019477431f1c355e0db0a7d3e15 \ + --hash=sha256:673611bbd43f0810bec0b0f028ddeaaa501190339cac411f347ac76917c3ae7b \ + --hash=sha256:67830fc78e67501f47bb950471b2dcb9b35b140084429318e862895a8e89c993 \ + --hash=sha256:68ce9f4d6b26d5ccbf7fd4459bf75f74a0a146677ebba80597df60cbdb20e6f4 \ + --hash=sha256:68e5f26a1ad57ded6d1cfb85331d1c1a195314756471d97758c48498bb4dcdf5 \ + --hash=sha256:69b157c5d3292bcd443faca052f3096f637f1e074b98212a933c074ae23dc3b8 \ + --hash=sha256:75286256590a6320cf106a0d28970d3560aad9ee09aa7b34fb40524792436d35 \ + --hash=sha256:78841cccf1af7b40f6f716338d50c0902dbe88d9f800b3c973b7a9a0a693a642 \ + --hash=sha256:78fa18e436a1a0e58dbd7e02fc4473f3f32cceb12df9dfca542d075961c307d2 \ + --hash=sha256:79580094b00d1789d1f93ea55bc43cb2f611910c72235b7657f3482ddcc1b22d \ + --hash=sha256:7b86a2b16095d250c6f58b3d9b2eee6f4147754344f3dab0922f7c9bf7d226c9 \ + --hash=sha256:83aed2c10721ddd90f68140685391b50811a880af20654c59af6b6c66c40513c \ + --hash=sha256:84fd18bcc17526fc2b3c1af7d2b9217d32c9c04448c16ec693b9b4f1985c3d33 \ + --hash=sha256:871ff67ea1aad4dfd91736464934d56b32dac49f9fbe16cddba36198a7b3a0db \ + --hash=sha256:898f0e9068ca27d37f8e83a5b962821df851532e6c4a7d615c1c033f9da6eedf \ + --hash=sha256:8a79d9f4d8001473a30c163556b3c3bfebec837495a412dde78b51672f6134f9 \ + --hash=sha256:8c041122946b7ba21bb32c45b1aa57b1be35527690aeb3c5c234521085632eee \ + --hash=sha256:90c44bc373b7687f6948b693cceaea1348ae0975d7474746559494468e3c1d84 \ + --hash=sha256:9104ed0bd76a429d46f9ec0dbc9b08ad1d2dcdf2b00a5a0daa1c145329b35b44 \ + --hash=sha256:920079c3f7456fa213e0829ed2073aaa727fd39d889ead5b4f35d0de5460d04f \ + --hash=sha256:93d59d504b230e83c7a843251681959a0b6a9cd76f6e146ce1b8a80eb8739af9 \ + --hash=sha256:9b2aff1c7b3884512b9512c3eaadd9bab39fb45042ffaaa1dd08ff2b9f8109d9 \ + --hash=sha256:9b8e0f3107e2200b76f6054de99016eac3ee6762713587b36baaa7e4bd2ae177 \ + --hash=sha256:9bb41182d93ea91f60b4bc8fbf4c820c69ef8a12ab2d917f3f1834f1acad07e8 \ + --hash=sha256:9cdef90ae47919cae358d8ab15797a800ed41da7aba5d72419fb510729e2ed4b \ + --hash=sha256:a1786910334ed46ab1dd73222f2cd1e05c2c3bb39f6dddb4f8b36fc382058a39 \ + --hash=sha256:a4cfde78a9f2880208d16a93b795726a3017d5977e08d1e162a7a31322479c41 \ + --hash=sha256:a4fbdde9dd4a9ce5fd52c2b3a347bb50cc89483ef783f1cb00d408c13f7a96c0 \ + --hash=sha256:aa99adc8f081b475a12843953db36831eaf83ec33eb46a90629ca6a5de45a616 \ + --hash=sha256:ac351b3b8014eead140e77e9717e2992c6bbe30b63bc3422422eb84865412e3d \ + --hash=sha256:ad41ba96094304aa090f5a30cb6e4fb3b3f1c264c523394b4c39bbacc4dc92ba \ + --hash=sha256:b5314963fce9b0b12743891de876e724997864ee22aa496f903f426c7e2fa5b2 \ + --hash=sha256:bcf74c1df76758a395bf0af608c04c82257523f55c9868b334f06270d0f2112b \ + --hash=sha256:bd47ba7fc3ca94896759ea0109775132d3e7ab921fbf54038e1bab2e46c313c9 \ + --hash=sha256:c0323c9daef75ef2e5083624b4585018a0c9d5e3b40f607eed81a311270b934b \ + --hash=sha256:c1225416b463483160e4af85d5fc3a9690ccb53fd4b1865a6437825f5ede3209 \ + --hash=sha256:c1c948747b03be832dceed96ca815cef7360de9aa19d37c730f8e3f6101aca48 \ + --hash=sha256:c25fe15c70c59eb7c5ce8c06a1f3fa1da0ecc5ea1e7a5922c40fd2fa9b0d5046 \ + --hash=sha256:cc1b0fff8ead343dae06305f954eb8468ba0ec1a97881f42489d198e4ce3c632 \ + --hash=sha256:cd6280cf040f233bd7d3407b743b4b4c74f70e8e1c4199cb112a62c941c0772a \ + --hash=sha256:cd6c3d4b783c556fa00bf540854e42f135e2f256abd29669fcd0da0f2dec79c2 \ + --hash=sha256:d4d6fcde76f94f5cb9e43e9e9a61f16dacefd228cbbf6f1a09bd9b219a92f1a1 \ + --hash=sha256:ddf4af30b417d9fe16481e9b81c27ab2a7cde1ff7ba3e85653b02db7d145dc7b \ + --hash=sha256:df115d4d83168fdf2cae48ef1ff6d1cb4c466364e30861b37121de0f3bf1b990 \ + --hash=sha256:df7276909358e5635ae203673ab7e509ddd224225a8d6b0790bf13eb2bde1cc5 \ + --hash=sha256:e4fd89cc178bced6ad29cb3e6dd4aa63fa5017c3524dbd0b25998fb64a87cc8b \ + --hash=sha256:e9701d0049d92c16703a42771b98d560b95248949f23f8cf7b4eddd201814fb9 \ + --hash=sha256:ee2f2a527e3c1a6e6411eb4209642e138b544a2d72fe5d0d76daf77b24063534 \ + --hash=sha256:f7fb7d750cfa0a070d2c24e831fd3481019a60dd317ea2b39acbcebc08b6ed81 \ + --hash=sha256:f840ed6d8ecba8255df8c42b87fadeda98ddfc6eeec05e2dc66e26d46dd6f58a \ + --hash=sha256:f86c6358749bd4fda175388691e3ba8c46e24c5347d0afd20f9b7edfc9faf07d \ + --hash=sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf \ + --hash=sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115 # via requests -cryptography==43.0.3 \ - --hash=sha256:0c580952eef9bf68c4747774cde7ec1d85a6e61de97281f2dba83c7d2c806362 \ - --hash=sha256:0f996e7268af62598f2fc1204afa98a3b5712313a55c4c9d434aef49cadc91d4 \ - --hash=sha256:1ec0bcf7e17c0c5669d881b1cd38c4972fade441b27bda1051665faaa89bdcaa \ - --hash=sha256:281c945d0e28c92ca5e5930664c1cefd85efe80e5c0d2bc58dd63383fda29f83 \ - --hash=sha256:2ce6fae5bdad59577b44e4dfed356944fbf1d925269114c28be377692643b4ff \ - --hash=sha256:315b9001266a492a6ff443b61238f956b214dbec9910a081ba5b6646a055a805 \ - --hash=sha256:443c4a81bb10daed9a8f334365fe52542771f25aedaf889fd323a853ce7377d6 \ - --hash=sha256:4a02ded6cd4f0a5562a8887df8b3bd14e822a90f97ac5e544c162899bc467664 \ - --hash=sha256:53a583b6637ab4c4e3591a15bc9db855b8d9dee9a669b550f311480acab6eb08 \ - --hash=sha256:63efa177ff54aec6e1c0aefaa1a241232dcd37413835a9b674b6e3f0ae2bfd3e \ - --hash=sha256:74f57f24754fe349223792466a709f8e0c093205ff0dca557af51072ff47ab18 \ - --hash=sha256:7e1ce50266f4f70bf41a2c6dc4358afadae90e2a1e5342d3c08883df1675374f \ - --hash=sha256:81ef806b1fef6b06dcebad789f988d3b37ccaee225695cf3e07648eee0fc6b73 \ - --hash=sha256:846da004a5804145a5f441b8530b4bf35afbf7da70f82409f151695b127213d5 \ - --hash=sha256:8ac43ae87929a5982f5948ceda07001ee5e83227fd69cf55b109144938d96984 \ - --hash=sha256:9762ea51a8fc2a88b70cf2995e5675b38d93bf36bd67d91721c309df184f49bd \ - --hash=sha256:a2a431ee15799d6db9fe80c82b055bae5a752bef645bba795e8e52687c69efe3 \ - --hash=sha256:bf7a1932ac4176486eab36a19ed4c0492da5d97123f1406cf15e41b05e787d2e \ - --hash=sha256:c2e6fc39c4ab499049df3bdf567f768a723a5e8464816e8f009f121a5a9f4405 \ - --hash=sha256:cbeb489927bd7af4aa98d4b261af9a5bc025bd87f0e3547e11584be9e9427be2 \ - --hash=sha256:d03b5621a135bffecad2c73e9f4deb1a0f977b9a8ffe6f8e002bf6c9d07b918c \ - --hash=sha256:d56e96520b1020449bbace2b78b603442e7e378a9b3bd68de65c782db1507995 \ - --hash=sha256:df6b6c6d742395dd77a23ea3728ab62f98379eff8fb61be2744d4679ab678f73 \ - --hash=sha256:e1be4655c7ef6e1bbe6b5d0403526601323420bcf414598955968c9ef3eb7d16 \ - --hash=sha256:f18c716be16bc1fea8e95def49edf46b82fccaa88587a45f8dc0ff6ab5d8e0a7 \ - --hash=sha256:f46304d6f0c6ab8e52770addfa2fc41e6629495548862279641972b6215451cd \ - --hash=sha256:f7b178f11ed3664fd0e995a47ed2b5ff0a12d893e41dd0494f406d1cf555cab7 +cryptography==50.0.0 \ + --hash=sha256:031e2d5dd4bb9caa3ca9c82e5a197fd8ae680232cee62603d1a813f3f07e3d03 \ + --hash=sha256:06a32a980526a6ab9a4b9bf8f7385800791e2bb960903cb6b530e4817509a3b7 \ + --hash=sha256:07479a1cb08219ab719147e742e76090c9c773321959bb94946fffdd397a6437 \ + --hash=sha256:07949c449a1abcf60d1ee6e88956d89404c7df3c8258f46589e912988e551987 \ + --hash=sha256:105110f43a471dbd0060b9c9516cb8a6a79233631a04cc2ba16f28323ac6e025 \ + --hash=sha256:11b74db56cdbe3cdee6e3f6982ecb70334fa10dce99ed58bf7894aaaa3b2a037 \ + --hash=sha256:12b9c6996425c76ea6c457ace4f3073e715b8c545add07cd1a8f3a4f90691269 \ + --hash=sha256:1489e263a8048bb8b6a8bac662eb2d402ea5d2b7b4699b72f385f1e2772db105 \ + --hash=sha256:19736989797678c6af1e55cd49055cdbcb55d8f6b5583ac5335f933aba9101dc \ + --hash=sha256:1b4a266766514614f8aa60416e71f2fc6e575d36e7bdc90f644fadb2f4b75b95 \ + --hash=sha256:2a8183b489dc1f7f80f135780fadc1108f14b31b8a40411c7a5b17425f65f28b \ + --hash=sha256:37fdb0d0111f1e2ff07139dfb79f1b49531f8e213c46f1163dd7642979b58c47 \ + --hash=sha256:3f5735ffe4996d28b809371756219f5354864902a3b9e7c0b9ee87041209fc9c \ + --hash=sha256:49e7d93abdbd2990caced757e5fade25302f719c3c8fb6e6fff2dde98999fc41 \ + --hash=sha256:5e34edd123674534acd70147f0ca331eaa2c74e6325fb2028c886aa26ba0b68c \ + --hash=sha256:62598a8a57f815db4c6259a4e97d857dab56697e7de8e8ab02352ab74da1995d \ + --hash=sha256:65c2c3add92b45fd0709db8594536aea39c2a67af0e27ffcf049c498501140b7 \ + --hash=sha256:6ba6a53445bd3cfa809ef3ef5f1589aa6ba08784a1d962bf47d0940e871dab1c \ + --hash=sha256:6e7d61120573a7f2cd94cc095f9e81f6967c61ccdf194285aa143ecec8e0b708 \ + --hash=sha256:7cec5b856506da6defb290f30c9ee687d5f5e8cb0bd3f6459dde43b0b4fa40ef \ + --hash=sha256:80b63928fa35083b33966ce1efb70e5b9607181e49dcd1c22c8c005e319f667f \ + --hash=sha256:82148ec5bddac30b51a5b3c1945075f896fa022cb93f8e4a01e9f6ee95292c5f \ + --hash=sha256:828743d939e9629bc267b8e2d08d8bb67cd4319c771a33d4b18b22dd8fb7440a \ + --hash=sha256:8d89f3976b10b4ce31118de72329025f70d2c6ead14a8217c5514dd2c6d5a78f \ + --hash=sha256:8eb5e1172eb569ea8a872796576e6a67c276351728b6455d5beb01242b027c6a \ + --hash=sha256:900131fafd8aead39ac7dd3a7e833be754c17a95cfd91221636949fe4eb0aa8a \ + --hash=sha256:910d11e1a385c654bf738bf3e6b8e6ed5de0f5610fcae2be9e5b398d8081d20e \ + --hash=sha256:910e1d2668e7de9648f2bcee30e180db2a6b15c30f887d7c4c93ddf96e3992e3 \ + --hash=sha256:9aa87839c383bdbab6ef865787a1fb877af8dd03464c4400322726feaaadfc6d \ + --hash=sha256:a1b30560f2acc95aa8b2e06e716a13dbfc97314747b80d9707e307f77b40d6b3 \ + --hash=sha256:a91296cb61e8df6f86d0c19cc4068228da256bf59bf86049fbd821084565327f \ + --hash=sha256:b42a28c1844fd9de8f3f7d540e36b66f3a9c83fceac7170ebc7a6a19edd9dcae \ + --hash=sha256:bd1c592e4d5974f0d08d4888e432157adba757c66da0246918e43677fafa2d30 \ + --hash=sha256:c87f62a3d3b9888ed0fdde100ec06aa61ca9cd44bad9057d1dff9a516b5f5bb9 \ + --hash=sha256:c99c003e088647b8a5b7c145d6f78c335f6348332b62e142d411c4b63d1460b9 \ + --hash=sha256:ccdc4a71a4dabae05de219404f9f4abc38e3b58422177ff93d0da05967dafa07 \ + --hash=sha256:d24fead1d4d076e1bfb006dcec392074a3cd8d7b4fc8a595aa64073b2b7a96ba \ + --hash=sha256:d58c3db7cd6eed54e6c06744db55456b65ebd7492ddeae9c1e93cfca7aa857d3 \ + --hash=sha256:d764dcf130c428ef66786f866dd750f53182bc608813489915e9fc106bb0c82f \ + --hash=sha256:df2a58a472f332225671c35b0a830208b86d004f82baa8530fa3782c85646533 \ + --hash=sha256:e722f16708d854fe924790e051061f6704a472c3bac347b6fd88033ea8dd0dc5 \ + --hash=sha256:ecfed7367f965a0328cfbdd70da860f15441f002f613185668c6e6ebf5a0ac11 \ + --hash=sha256:eeac2acb5a20ed25e0ad6d1df9891a520b78b404266b6d11778f25d5d691a6c9 \ + --hash=sha256:f59e38625469987d7ef6d495323c55e7db6c212eaf6112267e0d3b565a2e9c9f \ + --hash=sha256:f89831ef99dd7dd169ab06d63a831adb9e20a87aac6d380266bbda5823349169 \ + --hash=sha256:fd9192b7b70c573d7f214eb1ae35e00d359f6f5e4b27c7e21e30de1fc6204645 # via secretstorage -docutils==0.21.2 \ - --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ - --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2 +docutils==0.23 \ + --hash=sha256:25d013af9bf23bc1c7b2b093dff4208166c53a94786c9e447808335ef1185fea \ + --hash=sha256:746f5060322511280a1e50eb76846ed6bf2342984b2ac04dc42caa1a8d78799e # via readme-renderer -idna==3.10 \ - --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ - --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3 +id==1.6.1 \ + --hash=sha256:d0732d624fb46fd4e7bc4e5152f00214450953b9e772c182c1c22964def1a069 \ + --hash=sha256:f5ec41ed2629a508f5d0988eda142e190c9c6da971100612c4de9ad9f9b237ca + # via twine +idna==3.18 \ + --hash=sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2 \ + --hash=sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848 # via requests -importlib-metadata==8.5.0 \ - --hash=sha256:45e54197d28b7a7f1559e60b95e7c567032b602131fbd588f1497f47880aa68b \ - --hash=sha256:71522656f0abace1d072b9e5481a48f07c138e00f079c38c8f883823f9c26bd7 - # via - # keyring - # twine +importlib-metadata==9.0.0 \ + --hash=sha256:2d21d1cc5a017bd0559e36150c21c830ab1dc304dedd1b7ea85d20f45ef3edd7 \ + --hash=sha256:a4f57ab599e6a2e3016d7595cfd72eb4661a5106e787a95bcc90c7105b831efc + # via keyring jaraco-classes==3.4.0 \ --hash=sha256:47a024b51d0239c0dd8c8540c6c7f484be3b8fcf0b2d85c13825780d3b3f3acd \ --hash=sha256:f662826b6bed8cace05e7ff873ce0f9283b5c924470fe664fff1c2f00f581790 # via keyring -jaraco-context==6.0.1 \ - --hash=sha256:9bae4ea555cf0b14938dc0aee7c9f32ed303aa20a3b73e7dc80111628792d1b3 \ - --hash=sha256:f797fc481b490edb305122c9181830a3a5b76d84ef6d1aef2fb9b47ab956f9e4 +jaraco-context==6.1.2 \ + --hash=sha256:bf8150b79a2d5d91ae48629d8b427a8f7ba0e1097dd6202a9059f29a36379535 \ + --hash=sha256:f1a6c9d391e661cc5b8d39861ff077a7dc24dc23833ccee564b234b81c82dfe3 # via keyring -jaraco-functools==4.1.0 \ - --hash=sha256:70f7e0e2ae076498e212562325e805204fc092d7b4c17e0e86c959e249701a9d \ - --hash=sha256:ad159f13428bc4acbf5541ad6dec511f91573b90fba04df61dafa2a1231cf649 +jaraco-functools==4.6.0 \ + --hash=sha256:880c577ec9720b3a052d5bc611fb9f2269b3d87902ef42440df443b88e443280 \ + --hash=sha256:99e3dc0060c5cbe8fcd1cdb36258e2a65ca40f1566b2033b12abb1bb44dd3c30 # via keyring -jeepney==0.8.0 \ - --hash=sha256:5efe48d255973902f6badc3ce55e2aa6c5c3b3bc642059ef3a91247bcfcc5806 \ - --hash=sha256:c0a454ad016ca575060802ee4d590dd912e35c122fa04e70306de3d076cce755 +jeepney==0.9.0 \ + --hash=sha256:97e5714520c16fc0a45695e5365a2e11b81ea79bba796e26f9f1d178cb182683 \ + --hash=sha256:cf0e9e845622b81e4a28df94c40345400256ec608d0e55bb8a3feaa9163f5732 # via # keyring # secretstorage -keyring==25.4.1 \ - --hash=sha256:5426f817cf7f6f007ba5ec722b1bcad95a75b27d780343772ad76b17cb47b0bf \ - --hash=sha256:b07ebc55f3e8ed86ac81dd31ef14e81ace9dd9c3d4b5d77a6e9a2016d0d71a1b +keyring==25.7.0 \ + --hash=sha256:be4a0b195f149690c166e850609a477c532ddbfbaed96a404d4e43f8d5e2689f \ + --hash=sha256:fe01bd85eb3f8fb3dd0405defdeac9a5b4f6f0439edbb3149577f244a2e8245b # via twine -markdown-it-py==3.0.0 \ - --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ - --hash=sha256:e3f60a94fa066dc52ec76661e37c851cb232d92f9886b15cb560aaada2df8feb +markdown-it-py==4.2.0 \ + --hash=sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49 \ + --hash=sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a # via rich mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba # via markdown-it-py -more-itertools==10.5.0 \ - --hash=sha256:037b0d3203ce90cca8ab1defbbdac29d5f993fc20131f3664dc8d6acfa872aef \ - --hash=sha256:5482bfef7849c25dc3c6dd53a6173ae4795da2a41a80faea6700d9f5846c5da6 +more-itertools==11.1.0 \ + --hash=sha256:48e8f4d9e7e5878571ecf6f2b4e57634f93cd474cc8cfbd2376f2d11b396e30d \ + --hash=sha256:4b65538ae22f6fed0ce4874efd317463a7489796a0939fa66824dd542125a192 # via # jaraco-classes # jaraco-functools -nh3==0.2.18 \ - --hash=sha256:0411beb0589eacb6734f28d5497ca2ed379eafab8ad8c84b31bb5c34072b7164 \ - --hash=sha256:14c5a72e9fe82aea5fe3072116ad4661af5cf8e8ff8fc5ad3450f123e4925e86 \ - --hash=sha256:19aaba96e0f795bd0a6c56291495ff59364f4300d4a39b29a0abc9cb3774a84b \ - --hash=sha256:34c03fa78e328c691f982b7c03d4423bdfd7da69cd707fe572f544cf74ac23ad \ - --hash=sha256:36c95d4b70530b320b365659bb5034341316e6a9b30f0b25fa9c9eff4c27a204 \ - --hash=sha256:3a157ab149e591bb638a55c8c6bcb8cdb559c8b12c13a8affaba6cedfe51713a \ - --hash=sha256:42c64511469005058cd17cc1537578eac40ae9f7200bedcfd1fc1a05f4f8c200 \ - --hash=sha256:5f36b271dae35c465ef5e9090e1fdaba4a60a56f0bb0ba03e0932a66f28b9189 \ - --hash=sha256:6955369e4d9f48f41e3f238a9e60f9410645db7e07435e62c6a9ea6135a4907f \ - --hash=sha256:7b7c2a3c9eb1a827d42539aa64091640bd275b81e097cd1d8d82ef91ffa2e811 \ - --hash=sha256:8ce0f819d2f1933953fca255db2471ad58184a60508f03e6285e5114b6254844 \ - --hash=sha256:94a166927e53972a9698af9542ace4e38b9de50c34352b962f4d9a7d4c927af4 \ - --hash=sha256:a7f1b5b2c15866f2db413a3649a8fe4fd7b428ae58be2c0f6bca5eefd53ca2be \ - --hash=sha256:c8b3a1cebcba9b3669ed1a84cc65bf005728d2f0bc1ed2a6594a992e817f3a50 \ - --hash=sha256:de3ceed6e661954871d6cd78b410213bdcb136f79aafe22aa7182e028b8c7307 \ - --hash=sha256:f0eca9ca8628dbb4e916ae2491d72957fdd35f7a5d326b7032a345f111ac07fe +nh3==0.3.6 \ + --hash=sha256:082675ff87b9385ec430ffe6d5847ba7456cc39b73720cd4add472f9f4cffd56 \ + --hash=sha256:2411e8c3cee81a1ddd62c2a5d50585c28aa5566d373ad1db92536b95ddb24ef2 \ + --hash=sha256:25c733bee928530556b1db0ea46c52cf5aa686146e38e60a6fc7cb801ef91cec \ + --hash=sha256:2f90d9a0cfdbee218994fdaaeeb5a0fde62d08f35e4eef0378ec1e2200172fd0 \ + --hash=sha256:34d2b0d934156b87ee114f599a3ba9b8b9e17b5d79652ba3a13fa50903de965e \ + --hash=sha256:36d06341bd501240d320f5942481ed5e6846136b666e1ba4faf802b78ebc875f \ + --hash=sha256:43bc1ed3fa0716295fabee29ba42b2667e4a51d140b0a68e092170a765474fa6 \ + --hash=sha256:44673b27010051ab5a5e438a86ec31bbda61d4a77d7e900af6b7be3037c1abae \ + --hash=sha256:455469a29951edc92bc48b47ac2281c3f2609e6c4f6a047056449f8c2c23facf \ + --hash=sha256:4713502748f564fee0633b37b3403783ce0a3af3a3d148ad91025a5bdadb7bc6 \ + --hash=sha256:5276ef17bdba9ad8040575c74072008b13aae429436e9d0429e718bb5f90f4da \ + --hash=sha256:597a8e843bea00b2eb5520658dc24a9bb032e7fc9e7c2c0c4cd29420220c9796 \ + --hash=sha256:69bbb92865a693d909db3a700d3c01537533844d0948c1e9323561ce06ecda41 \ + --hash=sha256:69f365963f63a1e9bff53bdbb3c542c7c2efed3e163c9d5d83a772a2ac468c21 \ + --hash=sha256:82ca5bf427ad1b216b65ede1a2e2d87dc49bec417ceba0f297213107d3cd9d78 \ + --hash=sha256:889932a97fb4abb6f95fef1914c0d269ebfb60011e67121c1163059b9449dbb4 \ + --hash=sha256:905f877dc66dd7aea4a76e54bcb26acb5ff8216f720c0017ccf63e0e6035698e \ + --hash=sha256:a43ebd7543555c3ac1bc353023d0794e75cb76f6f18f19c32e95441496c0cc25 \ + --hash=sha256:d14bf7982e7a77c0c775634c29c07ce08b38a046df73e1c1f139b3e82f18a38e \ + --hash=sha256:e196fa70c2ff2eb4de7d3df3108f8f358c1d69dff20d45b11f20a5aa227ffb6d \ + --hash=sha256:e1b160831c9cdb06a6c79c2f9cdb11386602938f9af260d1c457a85add4f6f69 \ + --hash=sha256:e6b7beece07525dc6e6b0fc2f104442de2ba328360ad00e50cbe2e1fd620447d \ + --hash=sha256:edb2b4a1a27523e6cc7c417f8d21ce3d005243548b93e56b762b66b0c7f589f9 \ + --hash=sha256:f2f14b7ae1fca99c4a66c981aac3974e7fbc1ca30a12673d223ae1df76680917 \ + --hash=sha256:f338ac7d594c067679f1e99b4f5ec3906842979560f9d8f15d6bdfa39a353b10 \ + --hash=sha256:f3736c9dd3d1856f80cd031715b84ca75cda2bbb1ac802c3da26bfce590838d7 \ + --hash=sha256:f5ed5fe84aee7f39db95c214a7421bf0499fbf500fec6d86a4e29bfc37971438 # via readme-renderer -pkginfo==1.10.0 \ - --hash=sha256:5df73835398d10db79f8eecd5cd86b1f6d29317589ea70796994d49399af6297 \ - --hash=sha256:889a6da2ed7ffc58ab5b900d888ddce90bce912f2d2de1dc1c26f4cb9fe65097 +packaging==26.2 \ + --hash=sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e \ + --hash=sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661 # via twine -pycparser==2.22 \ - --hash=sha256:491c8be9c040f5390f5bf44a5b07752bd07f56edf992381b05c701439eec10f6 \ - --hash=sha256:c3702b6d3dd8c7abc1afa565d7e63d53a1d0bd86cdc24edd75470f4de499cfcc +pycparser==3.0 \ + --hash=sha256:600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29 \ + --hash=sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992 # via cffi -pygments==2.18.0 \ - --hash=sha256:786ff802f32e91311bff3889f6e9a86e81505fe99f2735bb6d60ae0c5004f199 \ - --hash=sha256:b8e6aca0523f3ab76fee51799c488e38782ac06eafcf95e7ba832985c8e7b13a +pygments==2.20.0 \ + --hash=sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f \ + --hash=sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176 # via # readme-renderer # rich -readme-renderer==44.0 \ - --hash=sha256:2fbca89b81a08526aadf1357a8c2ae889ec05fb03f5da67f9769c9a592166151 \ - --hash=sha256:8712034eabbfa6805cacf1402b4eeb2a73028f72d1166d6f5cb7f9c047c5d1e1 +readme-renderer==45.0 \ + --hash=sha256:030a8fac74904f8fba11ad1bb6964e3f76e896dc7e5e71f16af190c9056696d1 \ + --hash=sha256:3385ed220117104a2bceb4a9dac8c5fdf6d1f96890d7ea2a9c7174fd5c84091f # via twine -requests==2.32.3 \ - --hash=sha256:55365417734eb18255590a9ff9eb97e9e1da868d4ccd6402399eaf68af20a760 \ - --hash=sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6 +requests==2.34.2 \ + --hash=sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0 \ + --hash=sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed # via # requests-toolbelt # twine @@ -315,25 +368,26 @@ rfc3986==2.0.0 \ --hash=sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd \ --hash=sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c # via twine -rich==13.9.3 \ - --hash=sha256:9836f5096eb2172c9e77df411c1b009bace4193d6a481d534fea75ebba758283 \ - --hash=sha256:bc1e01b899537598cf02579d2b9f4a415104d3fc439313a7a2c165d76557a08e +rich==15.0.0 \ + --hash=sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb \ + --hash=sha256:edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36 # via twine -secretstorage==3.3.3 \ - --hash=sha256:2403533ef369eca6d2ba81718576c5e0f564d5cca1b58f73a8b23e7d4eeebd77 \ - --hash=sha256:f356e6628222568e3af06f2eba8df495efa13b3b63081dafd4f7d9a7b7bc9f99 +secretstorage==3.5.0 \ + --hash=sha256:0ce65888c0725fcb2c5bc0fdb8e5438eece02c523557ea40ce0703c266248137 \ + --hash=sha256:f04b8e4689cbce351744d5537bf6b1329c6fc68f91fa666f60a380edddcd11be # via keyring -twine==5.1.1 \ - --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ - --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db +twine==7.0.0 \ + --hash=sha256:85cdb29c518efef867360ae4acd4b0dfd61c8654a22fca08e6f8539f05022177 \ + --hash=sha256:b854164df26db268af05f49aa5c0344b10e27a494343ff05b1e0bad3b135f5a7 # via -r tools/publish/requirements.in -urllib3==2.2.3 \ - --hash=sha256:ca899ca043dcb1bafa3e262d73aa25c465bfb49e0bd9dd5d59f1d0acba2f8fac \ - --hash=sha256:e7d814a81dad81e6caf2ec9fdedb284ecc9c73076b62654547cc64ccdcae26e9 +urllib3==2.7.0 \ + --hash=sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c \ + --hash=sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897 # via + # id # requests # twine -zipp==3.20.2 \ - --hash=sha256:a817ac80d6cf4b23bf7f2828b7cabf326f15a001bea8b1f9b49631780ba28350 \ - --hash=sha256:bc9eb26f4506fda01b81bcde0ca78103b6e62f991b381fec825435c836edbc29 +zipp==4.1.0 \ + --hash=sha256:25ad4e16390cd314347dd8f1de67a2ac538ae658ed4ab9db16029c07c188e97f \ + --hash=sha256:4cb57381f544315db7688e976e922a2b18cdb513d21cc194eb42232ba2a3e602 # via importlib-metadata diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 06f93286f5..99d317f1a7 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -2,312 +2,365 @@ # bazel run //tools/publish:requirements_universal.update --index-url https://pypi.org/simple -backports-tarfile==1.2.0 ; python_full_version < '3.12' \ +backports-tarfile==1.2.0 ; python_full_version < '3.12' and platform_machine != 'ppc64le' and platform_machine != 's390x' \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2024.8.30 \ - --hash=sha256:922820b53db7a7257ffbda3f597266d435245903d80737e34f8a45ff3e3230d8 \ - --hash=sha256:bec941d2aa8195e248a60b31ff9f0558284cf01a52591ceda73ea9afffd69fd9 +certifi==2026.7.22 \ + --hash=sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775 \ + --hash=sha256:741e2c3b351ddf169a738da9f2c048608ff7f2c5cc02f1ebc6b118bb090d5d55 # via requests -cffi==1.17.1 ; platform_python_implementation != 'PyPy' and sys_platform == 'linux' \ - --hash=sha256:045d61c734659cc045141be4bae381a41d89b741f795af1dd018bfb532fd0df8 \ - --hash=sha256:0984a4925a435b1da406122d4d7968dd861c1385afe3b45ba82b750f229811e2 \ - --hash=sha256:0e2b1fac190ae3ebfe37b979cc1ce69c81f4e4fe5746bb401dca63a9062cdaf1 \ - --hash=sha256:0f048dcf80db46f0098ccac01132761580d28e28bc0f78ae0d58048063317e15 \ - --hash=sha256:1257bdabf294dceb59f5e70c64a3e2f462c30c7ad68092d01bbbfb1c16b1ba36 \ - --hash=sha256:1c39c6016c32bc48dd54561950ebd6836e1670f2ae46128f67cf49e789c52824 \ - --hash=sha256:1d599671f396c4723d016dbddb72fe8e0397082b0a77a4fab8028923bec050e8 \ - --hash=sha256:28b16024becceed8c6dfbc75629e27788d8a3f9030691a1dbf9821a128b22c36 \ - --hash=sha256:2bb1a08b8008b281856e5971307cc386a8e9c5b625ac297e853d36da6efe9c17 \ - --hash=sha256:30c5e0cb5ae493c04c8b42916e52ca38079f1b235c2f8ae5f4527b963c401caf \ - --hash=sha256:31000ec67d4221a71bd3f67df918b1f88f676f1c3b535a7eb473255fdc0b83fc \ - --hash=sha256:386c8bf53c502fff58903061338ce4f4950cbdcb23e2902d86c0f722b786bbe3 \ - --hash=sha256:3edc8d958eb099c634dace3c7e16560ae474aa3803a5df240542b305d14e14ed \ - --hash=sha256:45398b671ac6d70e67da8e4224a065cec6a93541bb7aebe1b198a61b58c7b702 \ - --hash=sha256:46bf43160c1a35f7ec506d254e5c890f3c03648a4dbac12d624e4490a7046cd1 \ - --hash=sha256:4ceb10419a9adf4460ea14cfd6bc43d08701f0835e979bf821052f1805850fe8 \ - --hash=sha256:51392eae71afec0d0c8fb1a53b204dbb3bcabcb3c9b807eedf3e1e6ccf2de903 \ - --hash=sha256:5da5719280082ac6bd9aa7becb3938dc9f9cbd57fac7d2871717b1feb0902ab6 \ - --hash=sha256:610faea79c43e44c71e1ec53a554553fa22321b65fae24889706c0a84d4ad86d \ - --hash=sha256:636062ea65bd0195bc012fea9321aca499c0504409f413dc88af450b57ffd03b \ - --hash=sha256:6883e737d7d9e4899a8a695e00ec36bd4e5e4f18fabe0aca0efe0a4b44cdb13e \ - --hash=sha256:6b8b4a92e1c65048ff98cfe1f735ef8f1ceb72e3d5f0c25fdb12087a23da22be \ - --hash=sha256:6f17be4345073b0a7b8ea599688f692ac3ef23ce28e5df79c04de519dbc4912c \ - --hash=sha256:706510fe141c86a69c8ddc029c7910003a17353970cff3b904ff0686a5927683 \ - --hash=sha256:72e72408cad3d5419375fc87d289076ee319835bdfa2caad331e377589aebba9 \ - --hash=sha256:733e99bc2df47476e3848417c5a4540522f234dfd4ef3ab7fafdf555b082ec0c \ - --hash=sha256:7596d6620d3fa590f677e9ee430df2958d2d6d6de2feeae5b20e82c00b76fbf8 \ - --hash=sha256:78122be759c3f8a014ce010908ae03364d00a1f81ab5c7f4a7a5120607ea56e1 \ - --hash=sha256:805b4371bf7197c329fcb3ead37e710d1bca9da5d583f5073b799d5c5bd1eee4 \ - --hash=sha256:85a950a4ac9c359340d5963966e3e0a94a676bd6245a4b55bc43949eee26a655 \ - --hash=sha256:8f2cdc858323644ab277e9bb925ad72ae0e67f69e804f4898c070998d50b1a67 \ - --hash=sha256:9755e4345d1ec879e3849e62222a18c7174d65a6a92d5b346b1863912168b595 \ - --hash=sha256:98e3969bcff97cae1b2def8ba499ea3d6f31ddfdb7635374834cf89a1a08ecf0 \ - --hash=sha256:a08d7e755f8ed21095a310a693525137cfe756ce62d066e53f502a83dc550f65 \ - --hash=sha256:a1ed2dd2972641495a3ec98445e09766f077aee98a1c896dcb4ad0d303628e41 \ - --hash=sha256:a24ed04c8ffd54b0729c07cee15a81d964e6fee0e3d4d342a27b020d22959dc6 \ - --hash=sha256:a45e3c6913c5b87b3ff120dcdc03f6131fa0065027d0ed7ee6190736a74cd401 \ - --hash=sha256:a9b15d491f3ad5d692e11f6b71f7857e7835eb677955c00cc0aefcd0669adaf6 \ - --hash=sha256:ad9413ccdeda48c5afdae7e4fa2192157e991ff761e7ab8fdd8926f40b160cc3 \ - --hash=sha256:b2ab587605f4ba0bf81dc0cb08a41bd1c0a5906bd59243d56bad7668a6fc6c16 \ - --hash=sha256:b62ce867176a75d03a665bad002af8e6d54644fad99a3c70905c543130e39d93 \ - --hash=sha256:c03e868a0b3bc35839ba98e74211ed2b05d2119be4e8a0f224fba9384f1fe02e \ - --hash=sha256:c59d6e989d07460165cc5ad3c61f9fd8f1b4796eacbd81cee78957842b834af4 \ - --hash=sha256:c7eac2ef9b63c79431bc4b25f1cd649d7f061a28808cbc6c47b534bd789ef964 \ - --hash=sha256:c9c3d058ebabb74db66e431095118094d06abf53284d9c81f27300d0e0d8bc7c \ - --hash=sha256:ca74b8dbe6e8e8263c0ffd60277de77dcee6c837a3d0881d8c1ead7268c9e576 \ - --hash=sha256:caaf0640ef5f5517f49bc275eca1406b0ffa6aa184892812030f04c2abf589a0 \ - --hash=sha256:cdf5ce3acdfd1661132f2a9c19cac174758dc2352bfe37d98aa7512c6b7178b3 \ - --hash=sha256:d016c76bdd850f3c626af19b0542c9677ba156e4ee4fccfdd7848803533ef662 \ - --hash=sha256:d01b12eeeb4427d3110de311e1774046ad344f5b1a7403101878976ecd7a10f3 \ - --hash=sha256:d63afe322132c194cf832bfec0dc69a99fb9bb6bbd550f161a49e9e855cc78ff \ - --hash=sha256:da95af8214998d77a98cc14e3a3bd00aa191526343078b530ceb0bd710fb48a5 \ - --hash=sha256:dd398dbc6773384a17fe0d3e7eeb8d1a21c2200473ee6806bb5e6a8e62bb73dd \ - --hash=sha256:de2ea4b5833625383e464549fec1bc395c1bdeeb5f25c4a3a82b5a8c756ec22f \ - --hash=sha256:de55b766c7aa2e2a3092c51e0483d700341182f08e67c63630d5b6f200bb28e5 \ - --hash=sha256:df8b1c11f177bc2313ec4b2d46baec87a5f3e71fc8b45dab2ee7cae86d9aba14 \ - --hash=sha256:e03eab0a8677fa80d646b5ddece1cbeaf556c313dcfac435ba11f107ba117b5d \ - --hash=sha256:e221cf152cff04059d011ee126477f0d9588303eb57e88923578ace7baad17f9 \ - --hash=sha256:e31ae45bc2e29f6b2abd0de1cc3b9d5205aa847cafaecb8af1476a609a2f6eb7 \ - --hash=sha256:edae79245293e15384b51f88b00613ba9f7198016a5948b5dddf4917d4d26382 \ - --hash=sha256:f1e22e8c4419538cb197e4dd60acc919d7696e5ef98ee4da4e01d3f8cfa4cc5a \ - --hash=sha256:f3a2b4222ce6b60e2e8b337bb9596923045681d71e5a082783484d845390938e \ - --hash=sha256:f6a16c31041f09ead72d69f583767292f750d24913dadacf5756b966aacb3f1a \ - --hash=sha256:f75c7ab1f9e4aca5414ed4d8e5c0e303a34f4421f8a0d47a4d019ceff0ab6af4 \ - --hash=sha256:f79fc4fc25f1c8698ff97788206bb3c2598949bfe0fef03d299eb1b5356ada99 \ - --hash=sha256:f7f5baafcc48261359e14bcd6d9bff6d4b28d9103847c9e136694cb0501aef87 \ - --hash=sha256:fc48c783f9c87e60831201f2cce7f3b2e4846bf4d8728eabe54d60700b318a0b +cffi==2.1.0 ; platform_machine != 'ppc64le' and platform_machine != 's390x' and platform_python_implementation != 'PyPy' and sys_platform == 'linux' \ + --hash=sha256:02cb7ff33ded4f1532476731f89ede53e2e488a8e6205515a82144246ffa7dcc \ + --hash=sha256:03e9810d18c646077e501f661b682fbf5dee4676048527ca3cffe66faa9960dd \ + --hash=sha256:0520e1f4c35f44e209cbbb421b67eec42e6a157f59444dfb6058874ff3610e5d \ + --hash=sha256:0582a58f3051372229ca8e7f5f589f9e5632678208d8636fea3676711fdf7fe5 \ + --hash=sha256:0611e7ebf90573a535ebdc33ae9da222d037853983e13359f580fab781ca017f \ + --hash=sha256:0a42c688d19fca6e095a53c6a6e2295a5b050a8b289f109adab02a9e61a25de6 \ + --hash=sha256:0a96b74cda968eebbad56d973efe5098974f0a9fb323865bf99ea1fd24e3e64c \ + --hash=sha256:10537b1df4967ca26d21e5072d7d54188354483b91dc75058968d3f0cf13fbda \ + --hash=sha256:11b3fb55f4f8ad92274ed26705f65d8f91457de71f5380061eb6d125a768fecd \ + --hash=sha256:15faec4adfff450819f3aee0e2e02c812de6edb88203aa58807955db2003472a \ + --hash=sha256:164bff1657b2a74f0b6d54e11c9b375bc97b931f2ca9c43fcf875838da1570dd \ + --hash=sha256:1854b724d00f6654c742097d5387569021be12d3a0f770eae1df8f8acfcc6acd \ + --hash=sha256:19c54ac121cad98450b4896fa9a43ee0180d57bc4bc911a33db6cab1efab6cd3 \ + --hash=sha256:1b96bfe2c4bd825681b7d311ad6d9b7280a091f43e8f63da5729638083cd3bfb \ + --hash=sha256:1e9f50d192a3e525b15a75ab5114e442d83d657b7ec29182a991bc9a88fd3a66 \ + --hash=sha256:1ff3456eab0d889592d1936d6125bbfbc7ae4d3354a700f8bd80450a66445d4d \ + --hash=sha256:2282cd5e38aa8accd03e99d1256af8411c84cdbee6a89d841b563fdbd1f3e50f \ + --hash=sha256:276f20fffd7b396e12516ba8edf9509210ac248cbbc5acbc39cd512f9f59ebe6 \ + --hash=sha256:2b71d409cccee78310ab5dec549aed052aaea483346e282c7b02362596e01bb0 \ + --hash=sha256:2e9dabb9abcb7ad15938c7196ad5c1718a4e6d33cc79b4c0209bdb64c4a54a5c \ + --hash=sha256:30b65779d598c370374fefabf138d456fd6f3216bfa7bedfab1ba82025b0cd93 \ + --hash=sha256:33eb1ad83ebe8f313e0df035c406227d55a79456704a863fad9842136af5ad7d \ + --hash=sha256:35aaea0c7ee0e58a5cd8c2fd1a48fdf7ece0d2699b7ecdda08194e9ce5dd9b3d \ + --hash=sha256:3681e031db29958a7502f5c0c9d6bbc4c36cb20f7b104086fa642d1799631ff8 \ + --hash=sha256:379de10ce1ba048b1448599d1b37b24caee16309d1ac98d3982fc997f768700b \ + --hash=sha256:37f525a7e7e50c017fdebe58b787be310ad59357ae43a053943a6e1a6c526001 \ + --hash=sha256:3b926723c13eba9f81d2ef3820d63aeceec3b2d4639906047bf675cb8a7a500d \ + --hash=sha256:3d7f118b5adbfdfead90c25822690b02bc8074fba949bb7858bec4ebd55adb43 \ + --hash=sha256:46b1c8db8f6122420f32d02fffb924c2fe9bc772d228c7c711748fff56aabb2b \ + --hash=sha256:47ff3a8bfd8cb9da1af7524b965127095055654c177fcfc7578debcb015eecd0 \ + --hash=sha256:4d433a51f1870e43a13b6732f92aaf540ff77c2015097c78556f75a2d6c030e0 \ + --hash=sha256:4f26194e3d95e06501b942642855aed4f953d55e95d7d01b7c4483db3ecff458 \ + --hash=sha256:510aeeeac94811b138077451da1fb18b308a5feab47dd2b603af55804155e1c8 \ + --hash=sha256:5972433ad71a9e46516584ef60a0fda12d9dc459938d1539c3ddecf9bdc1368d \ + --hash=sha256:5ecbd0499275d57506d397eebe1981cee87b47fcd9ef5c22cab7ed7644a39a94 \ + --hash=sha256:6274dcb2d15cef48daa73ed1be5a40d501d74dccd0cd6db364776d12cb6ba022 \ + --hash=sha256:63960549e4f8dc41e31accb97b975abaecfc44c03e396c093a6436763c2ea7db \ + --hash=sha256:64c753a0f87a256020004f37a1c8c02c480e725f910f0b2a0f3f07debd1b2479 \ + --hash=sha256:6af371f3767faeffc6ac1ef57cdfd25844403e9d3f476c5537caee499de96376 \ + --hash=sha256:6ca4919c6e4f89aa99c42510b42cf54596892c00b3f9077f6bdd1505e24b9c8d \ + --hash=sha256:6d194185eabd279f1c05ebe3504265ddfc5ad2b58d0714f7db9f01da592e9eb6 \ + --hash=sha256:702c436735fbe99d59ada02a1f65cfc0d31c0ee8b7290912f8fbc5cd1e4b16c3 \ + --hash=sha256:716ff8ec22f20b4d988b12884086bcef0fc99737043e503f7a3935a6be99b1ea \ + --hash=sha256:762f99479dcb369f60ab9017ad4ab97a36a1dd7c1ee5a3b15db0f4b8659120cd \ + --hash=sha256:7762faa47e8ff7eb80bd261d9a7d8eea2d8baa69de5e95b70c1f338bbe712f02 \ + --hash=sha256:78474632761faa0fb96f30b1c928c84ebcf68713cbb80d15bab09dfe61640fde \ + --hash=sha256:799416bae98336e400981ff6e532d67d5c709cfb30afb79865a1315f94b0e224 \ + --hash=sha256:7d034dcffa09e9a46c93fa3a3be402096cb5354ac6e41ab8e5cc9cd8b642ad76 \ + --hash=sha256:7d28dff1db6764108bc30788d85d61c876beff416d9a49cb9dd7c5a9f34f5804 \ + --hash=sha256:7d3538f9c0e50670f4deb93dbb696576e60590369cae2faf7de681e597a8a1f1 \ + --hash=sha256:7d5980a3433d4b71a5e120f9dd551403d7824e31e2e67124fe2769c404c06913 \ + --hash=sha256:7ea6b3e2c4250ff1de21c630fe72d0f63eb95c2c32ffbf64a358cf4a8836d714 \ + --hash=sha256:86cf8755a791f72c85dc287128cc62d4f24d392e3f1e15837245623f4a33cccc \ + --hash=sha256:88023dfe18799507b73f1dbb0d14326a17465de1bc9c9c7655c22845e9ddc3a2 \ + --hash=sha256:89095c1968b4ba8285840e131bf2891b09ae137fe2146905acae0354fbce1b5e \ + --hash=sha256:8d35c139744adb3e727cd51b1a18324bbe44b8bd41bf8322bca4d41289f48eda \ + --hash=sha256:8e74a6135550c4748af665b1b1118b6aab33b1fc6a16f9aff630af107c3b4512 \ + --hash=sha256:8f9ec95b8a043d3dfbc74d9abc6f7baf524dd27a8dc160b0a32ff9cdab650c28 \ + --hash=sha256:90bec57cf82089383bd06a605b3eb8daebf7e5a668520beaf6e327a83a947699 \ + --hash=sha256:95f2954c2c9473d892eca6e0409f3568b37ab62a8eedb122461f73cc273476e3 \ + --hash=sha256:961be50688f7fba2fa65f63712d3b9b341a22311f5253460ce933f52f0de1c8c \ + --hash=sha256:98fff996e983a36d3aa2eca83af40c5821202e7e6f32d13ae94e3d2286f10cfe \ + --hash=sha256:9b8f0f26ca4e7513c534d351eca551947d053fac438f2a04ac96d882909b0d3a \ + --hash=sha256:9d72af0cf10a76a600a9690078fe31c63b9588c8e86bf9fd353f713c84b5db0f \ + --hash=sha256:9d8272c0e483b024e1b9ad029821470ed8ec65631dbd90217469da0e7cd89f1c \ + --hash=sha256:a016194dbe13d14ee9556e734b772d8d67b947092b268d757fd4290e3ba2dfc2 \ + --hash=sha256:a5781494d4d400a3f47f8f1da94b324f6e6b440a53387774002890a2a2f4b50f \ + --hash=sha256:a95b05f9baf29b91171b3a8bd2020b028835243e7b0ff6bb23e2a3c228518b1b \ + --hash=sha256:aa7a1b53a2a4452ada2d1b5dade9960b2522f1e61293a811a077439e39029565 \ + --hash=sha256:ac0f1a2d0cfa7eea3f2aaf006ab6e70e8feeb16b75d65b7e5939982ca2f11056 \ + --hash=sha256:af5e2915d41fe6c961694d7bfdc8562942638200f3ce2765dfb8b745cf997629 \ + --hash=sha256:b6422532152adf4e59b110cb2808cee7a033800952f5c036b4af047ee43199e7 \ + --hash=sha256:b65f590ef2a44640f9a05dbb548a429b4ade77913ce683ac8b1480777658a6c0 \ + --hash=sha256:ba00f661f8ba35d075c937174e27c2c421cec3942fd2e0ea3e66996757c0fdd9 \ + --hash=sha256:bccbbb5ee76a61f9d99b5bf3846a51d7fca4b6a732fe46f89295610edaf41853 \ + --hash=sha256:bf01d8c84cbea96b944c73b22182e6c7c432b3475632b8111dbfdc95ddad6e13 \ + --hash=sha256:bf5c6cf48238b0eb4c086978c492ad1cbc22373fc5b2d7353b3a598ce6db887a \ + --hash=sha256:c16914df9fb7f500e440e6875fa23ff5e0b31db01fa9c06af98d59a91f0dc2e4 \ + --hash=sha256:c351efb95e832a853a29361675f33a7ce53de1a109cd73fd47af0712213aa4ce \ + --hash=sha256:c4165821e131d6d4ca444347c2b694e2311bcfa3fe5a861cc72968f28867beac \ + --hash=sha256:c5f5df567f6eb216de69be06ce55c8b714090fae02b18a3b40da8163b8c5fa9c \ + --hash=sha256:c941bb58d5a6e1c3892d86e42927ed6c180302f07e6d395d08c416e594b98b46 \ + --hash=sha256:c97f080ea627e2863524c5af3836e2270b5f5dfff1f104392b959f8df0c5d384 \ + --hash=sha256:cb96698e3c7413d906ce83f8ffd245ec1bd94707541f299d0ce4d6b0193e982b \ + --hash=sha256:cbb7640ce37159548d2147b5b8c241f962143d4c71231431820783f4dc78f210 \ + --hash=sha256:cdf2448aab5f661c9315308ec8b93f4e8a1a67a3c733f8631067a2b67d5913dc \ + --hash=sha256:d2117334c3af3bdcb9a88522b844a2bdb5efdc4f71c6c822df55486ae1c3347a \ + --hash=sha256:d53d10f7da99ae46f7373b9150393e9c5eab9b224909982b43832668de4779f5 \ + --hash=sha256:d9fafc5aa2e2a39aaf7f8cc0c1f044a9b07fca12e558dca53a3cc5c654ad67a7 \ + --hash=sha256:db3eb7d46527159a878ec3460e9d40615bc25ba337d477db681aea6e4f05c5d2 \ + --hash=sha256:dbf7c7a88e2bac086f06d14577332760bdeecc42bdec8ac4077f6260557d9326 \ + --hash=sha256:df2b82571a1b30f58a87bf4e5a9e78d2b1eff6c6ce8fd3aa3757221f93f0863f \ + --hash=sha256:df92f2aba50eb4d96718b68ef76f2e57a57b54f2fa62333496d16c6d585a85ca \ + --hash=sha256:eb4e8997a49aa2c08a3e43c9045d224448b8941d88e7ac163c7d383e560cbf98 \ + --hash=sha256:efc1cdd798b1aaf39b4610bba7aad28c9bea9b910f25c784ccf9ec1fa719d1f9 \ + --hash=sha256:f146d154428a2523f9cc7936c02353c2459b8f6cf07d3cd1ee1c0a611109c5d5 \ + --hash=sha256:f5bce581e6b8c235e566a14768a943b172ada3ed73537bb0c0be1edee312d4e7 \ + --hash=sha256:f9912624a0c0b834b7520d7769b3644453aabc0a7e1c839da7359f050750e9bc \ + --hash=sha256:fb62edb5bb52cca65fab91a63afa7561607120d26090a7e8fda6fb9f064726da \ + --hash=sha256:ff067a8d8d880e7809e4ac88eb009bb848870115317b306666502ccad30b147f # via cryptography -charset-normalizer==3.4.0 \ - --hash=sha256:0099d79bdfcf5c1f0c2c72f91516702ebf8b0b8ddd8905f97a8aecf49712c621 \ - --hash=sha256:0713f3adb9d03d49d365b70b84775d0a0d18e4ab08d12bc46baa6132ba78aaf6 \ - --hash=sha256:07afec21bbbbf8a5cc3651aa96b980afe2526e7f048fdfb7f1014d84acc8b6d8 \ - --hash=sha256:0b309d1747110feb25d7ed6b01afdec269c647d382c857ef4663bbe6ad95a912 \ - --hash=sha256:0d99dd8ff461990f12d6e42c7347fd9ab2532fb70e9621ba520f9e8637161d7c \ - --hash=sha256:0de7b687289d3c1b3e8660d0741874abe7888100efe14bd0f9fd7141bcbda92b \ - --hash=sha256:1110e22af8ca26b90bd6364fe4c763329b0ebf1ee213ba32b68c73de5752323d \ - --hash=sha256:130272c698667a982a5d0e626851ceff662565379baf0ff2cc58067b81d4f11d \ - --hash=sha256:136815f06a3ae311fae551c3df1f998a1ebd01ddd424aa5603a4336997629e95 \ - --hash=sha256:14215b71a762336254351b00ec720a8e85cada43b987da5a042e4ce3e82bd68e \ - --hash=sha256:1db4e7fefefd0f548d73e2e2e041f9df5c59e178b4c72fbac4cc6f535cfb1565 \ - --hash=sha256:1ffd9493de4c922f2a38c2bf62b831dcec90ac673ed1ca182fe11b4d8e9f2a64 \ - --hash=sha256:2006769bd1640bdf4d5641c69a3d63b71b81445473cac5ded39740a226fa88ab \ - --hash=sha256:20587d20f557fe189b7947d8e7ec5afa110ccf72a3128d61a2a387c3313f46be \ - --hash=sha256:223217c3d4f82c3ac5e29032b3f1c2eb0fb591b72161f86d93f5719079dae93e \ - --hash=sha256:27623ba66c183eca01bf9ff833875b459cad267aeeb044477fedac35e19ba907 \ - --hash=sha256:285e96d9d53422efc0d7a17c60e59f37fbf3dfa942073f666db4ac71e8d726d0 \ - --hash=sha256:2de62e8801ddfff069cd5c504ce3bc9672b23266597d4e4f50eda28846c322f2 \ - --hash=sha256:2f6c34da58ea9c1a9515621f4d9ac379871a8f21168ba1b5e09d74250de5ad62 \ - --hash=sha256:309a7de0a0ff3040acaebb35ec45d18db4b28232f21998851cfa709eeff49d62 \ - --hash=sha256:35c404d74c2926d0287fbd63ed5d27eb911eb9e4a3bb2c6d294f3cfd4a9e0c23 \ - --hash=sha256:3710a9751938947e6327ea9f3ea6332a09bf0ba0c09cae9cb1f250bd1f1549bc \ - --hash=sha256:3d59d125ffbd6d552765510e3f31ed75ebac2c7470c7274195b9161a32350284 \ - --hash=sha256:40d3ff7fc90b98c637bda91c89d51264a3dcf210cade3a2c6f838c7268d7a4ca \ - --hash=sha256:425c5f215d0eecee9a56cdb703203dda90423247421bf0d67125add85d0c4455 \ - --hash=sha256:43193c5cda5d612f247172016c4bb71251c784d7a4d9314677186a838ad34858 \ - --hash=sha256:44aeb140295a2f0659e113b31cfe92c9061622cadbc9e2a2f7b8ef6b1e29ef4b \ - --hash=sha256:47334db71978b23ebcf3c0f9f5ee98b8d65992b65c9c4f2d34c2eaf5bcaf0594 \ - --hash=sha256:4796efc4faf6b53a18e3d46343535caed491776a22af773f366534056c4e1fbc \ - --hash=sha256:4a51b48f42d9358460b78725283f04bddaf44a9358197b889657deba38f329db \ - --hash=sha256:4b67fdab07fdd3c10bb21edab3cbfe8cf5696f453afce75d815d9d7223fbe88b \ - --hash=sha256:4ec9dd88a5b71abfc74e9df5ebe7921c35cbb3b641181a531ca65cdb5e8e4dea \ - --hash=sha256:4f9fc98dad6c2eaa32fc3af1417d95b5e3d08aff968df0cd320066def971f9a6 \ - --hash=sha256:54b6a92d009cbe2fb11054ba694bc9e284dad30a26757b1e372a1fdddaf21920 \ - --hash=sha256:55f56e2ebd4e3bc50442fbc0888c9d8c94e4e06a933804e2af3e89e2f9c1c749 \ - --hash=sha256:5726cf76c982532c1863fb64d8c6dd0e4c90b6ece9feb06c9f202417a31f7dd7 \ - --hash=sha256:5d447056e2ca60382d460a604b6302d8db69476fd2015c81e7c35417cfabe4cd \ - --hash=sha256:5ed2e36c3e9b4f21dd9422f6893dec0abf2cca553af509b10cd630f878d3eb99 \ - --hash=sha256:5ff2ed8194587faf56555927b3aa10e6fb69d931e33953943bc4f837dfee2242 \ - --hash=sha256:62f60aebecfc7f4b82e3f639a7d1433a20ec32824db2199a11ad4f5e146ef5ee \ - --hash=sha256:63bc5c4ae26e4bc6be6469943b8253c0fd4e4186c43ad46e713ea61a0ba49129 \ - --hash=sha256:6b40e8d38afe634559e398cc32b1472f376a4099c75fe6299ae607e404c033b2 \ - --hash=sha256:6b493a043635eb376e50eedf7818f2f322eabbaa974e948bd8bdd29eb7ef2a51 \ - --hash=sha256:6dba5d19c4dfab08e58d5b36304b3f92f3bd5d42c1a3fa37b5ba5cdf6dfcbcee \ - --hash=sha256:6fd30dc99682dc2c603c2b315bded2799019cea829f8bf57dc6b61efde6611c8 \ - --hash=sha256:707b82d19e65c9bd28b81dde95249b07bf9f5b90ebe1ef17d9b57473f8a64b7b \ - --hash=sha256:7706f5850360ac01d80c89bcef1640683cc12ed87f42579dab6c5d3ed6888613 \ - --hash=sha256:7782afc9b6b42200f7362858f9e73b1f8316afb276d316336c0ec3bd73312742 \ - --hash=sha256:79983512b108e4a164b9c8d34de3992f76d48cadc9554c9e60b43f308988aabe \ - --hash=sha256:7f683ddc7eedd742e2889d2bfb96d69573fde1d92fcb811979cdb7165bb9c7d3 \ - --hash=sha256:82357d85de703176b5587dbe6ade8ff67f9f69a41c0733cf2425378b49954de5 \ - --hash=sha256:84450ba661fb96e9fd67629b93d2941c871ca86fc38d835d19d4225ff946a631 \ - --hash=sha256:86f4e8cca779080f66ff4f191a685ced73d2f72d50216f7112185dc02b90b9b7 \ - --hash=sha256:8cda06946eac330cbe6598f77bb54e690b4ca93f593dee1568ad22b04f347c15 \ - --hash=sha256:8ce7fd6767a1cc5a92a639b391891bf1c268b03ec7e021c7d6d902285259685c \ - --hash=sha256:8ff4e7cdfdb1ab5698e675ca622e72d58a6fa2a8aa58195de0c0061288e6e3ea \ - --hash=sha256:9289fd5dddcf57bab41d044f1756550f9e7cf0c8e373b8cdf0ce8773dc4bd417 \ - --hash=sha256:92a7e36b000bf022ef3dbb9c46bfe2d52c047d5e3f3343f43204263c5addc250 \ - --hash=sha256:92db3c28b5b2a273346bebb24857fda45601aef6ae1c011c0a997106581e8a88 \ - --hash=sha256:95c3c157765b031331dd4db3c775e58deaee050a3042fcad72cbc4189d7c8dca \ - --hash=sha256:980b4f289d1d90ca5efcf07958d3eb38ed9c0b7676bf2831a54d4f66f9c27dfa \ - --hash=sha256:9ae4ef0b3f6b41bad6366fb0ea4fc1d7ed051528e113a60fa2a65a9abb5b1d99 \ - --hash=sha256:9c98230f5042f4945f957d006edccc2af1e03ed5e37ce7c373f00a5a4daa6149 \ - --hash=sha256:9fa2566ca27d67c86569e8c85297aaf413ffab85a8960500f12ea34ff98e4c41 \ - --hash=sha256:a14969b8691f7998e74663b77b4c36c0337cb1df552da83d5c9004a93afdb574 \ - --hash=sha256:a8aacce6e2e1edcb6ac625fb0f8c3a9570ccc7bfba1f63419b3769ccf6a00ed0 \ - --hash=sha256:a8e538f46104c815be19c975572d74afb53f29650ea2025bbfaef359d2de2f7f \ - --hash=sha256:aa41e526a5d4a9dfcfbab0716c7e8a1b215abd3f3df5a45cf18a12721d31cb5d \ - --hash=sha256:aa693779a8b50cd97570e5a0f343538a8dbd3e496fa5dcb87e29406ad0299654 \ - --hash=sha256:ab22fbd9765e6954bc0bcff24c25ff71dcbfdb185fcdaca49e81bac68fe724d3 \ - --hash=sha256:ab2e5bef076f5a235c3774b4f4028a680432cded7cad37bba0fd90d64b187d19 \ - --hash=sha256:ab973df98fc99ab39080bfb0eb3a925181454d7c3ac8a1e695fddfae696d9e90 \ - --hash=sha256:af73657b7a68211996527dbfeffbb0864e043d270580c5aef06dc4b659a4b578 \ - --hash=sha256:b197e7094f232959f8f20541ead1d9862ac5ebea1d58e9849c1bf979255dfac9 \ - --hash=sha256:b295729485b06c1a0683af02a9e42d2caa9db04a373dc38a6a58cdd1e8abddf1 \ - --hash=sha256:b8831399554b92b72af5932cdbbd4ddc55c55f631bb13ff8fe4e6536a06c5c51 \ - --hash=sha256:b8dcd239c743aa2f9c22ce674a145e0a25cb1566c495928440a181ca1ccf6719 \ - --hash=sha256:bcb4f8ea87d03bc51ad04add8ceaf9b0f085ac045ab4d74e73bbc2dc033f0236 \ - --hash=sha256:bd7af3717683bea4c87acd8c0d3d5b44d56120b26fd3f8a692bdd2d5260c620a \ - --hash=sha256:bf4475b82be41b07cc5e5ff94810e6a01f276e37c2d55571e3fe175e467a1a1c \ - --hash=sha256:c3e446d253bd88f6377260d07c895816ebf33ffffd56c1c792b13bff9c3e1ade \ - --hash=sha256:c57516e58fd17d03ebe67e181a4e4e2ccab1168f8c2976c6a334d4f819fe5944 \ - --hash=sha256:c94057af19bc953643a33581844649a7fdab902624d2eb739738a30e2b3e60fc \ - --hash=sha256:cab5d0b79d987c67f3b9e9c53f54a61360422a5a0bc075f43cab5621d530c3b6 \ - --hash=sha256:ce031db0408e487fd2775d745ce30a7cd2923667cf3b69d48d219f1d8f5ddeb6 \ - --hash=sha256:cee4373f4d3ad28f1ab6290684d8e2ebdb9e7a1b74fdc39e4c211995f77bec27 \ - --hash=sha256:d5b054862739d276e09928de37c79ddeec42a6e1bfc55863be96a36ba22926f6 \ - --hash=sha256:dbe03226baf438ac4fda9e2d0715022fd579cb641c4cf639fa40d53b2fe6f3e2 \ - --hash=sha256:dc15e99b2d8a656f8e666854404f1ba54765871104e50c8e9813af8a7db07f12 \ - --hash=sha256:dcaf7c1524c0542ee2fc82cc8ec337f7a9f7edee2532421ab200d2b920fc97cf \ - --hash=sha256:dd4eda173a9fcccb5f2e2bd2a9f423d180194b1bf17cf59e3269899235b2a114 \ - --hash=sha256:dd9a8bd8900e65504a305bf8ae6fa9fbc66de94178c420791d0293702fce2df7 \ - --hash=sha256:de7376c29d95d6719048c194a9cf1a1b0393fbe8488a22008610b0361d834ecf \ - --hash=sha256:e7fdd52961feb4c96507aa649550ec2a0d527c086d284749b2f582f2d40a2e0d \ - --hash=sha256:e91f541a85298cf35433bf66f3fab2a4a2cff05c127eeca4af174f6d497f0d4b \ - --hash=sha256:e9e3c4c9e1ed40ea53acf11e2a386383c3304212c965773704e4603d589343ed \ - --hash=sha256:ee803480535c44e7f5ad00788526da7d85525cfefaf8acf8ab9a310000be4b03 \ - --hash=sha256:f09cb5a7bbe1ecae6e87901a2eb23e0256bb524a79ccc53eb0b7629fbe7677c4 \ - --hash=sha256:f19c1585933c82098c2a520f8ec1227f20e339e33aca8fa6f956f6691b784e67 \ - --hash=sha256:f1a2f519ae173b5b6a2c9d5fa3116ce16e48b3462c8b96dfdded11055e3d6365 \ - --hash=sha256:f28f891ccd15c514a0981f3b9db9aa23d62fe1a99997512b0491d2ed323d229a \ - --hash=sha256:f3e73a4255342d4eb26ef6df01e3962e73aa29baa3124a8e824c5d3364a65748 \ - --hash=sha256:f606a1881d2663630ea5b8ce2efe2111740df4b687bd78b34a8131baa007f79b \ - --hash=sha256:fe9f97feb71aa9896b81973a7bbada8c49501dc73e58a10fcef6663af95e5079 \ - --hash=sha256:ffc519621dce0c767e96b9c53f09c5d215578e10b02c285809f76509a3931482 +charset-normalizer==3.4.9 \ + --hash=sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380 \ + --hash=sha256:03d07803992c6c7bbc976327f34b18b6160327fc81cb82c9d504720ac0be3b62 \ + --hash=sha256:04ce310cb89c15df659582aee80a0603788732a5e017d5bd5c81158106ce249c \ + --hash=sha256:0d861473f743244d349b50f850d10eb87aeb22bbdcc8e64f79273c94af5a8226 \ + --hash=sha256:0e94703ec9684807f20cfb5eed95c70f67f2a8f21ad620146d7b5a13677b93e5 \ + --hash=sha256:0fa1aec2d32bcc03c8fa0f6f1712caad1adc38509f31142112e5c9daf5b9c833 \ + --hash=sha256:16b65ea0f2465b6fb52aa22de5eca612aa964ddfec00a912e26f4656cbef890b \ + --hash=sha256:16d10d789dd9bcca1173c95af82c58433122564b7bc39385124be735a35cbe99 \ + --hash=sha256:19ac87f93086ce37b86e098888555c4b4bc48102279bae3350098c0ed664b501 \ + --hash=sha256:1d22856ffbe153a602df38e4a5464f0b748a54002e0d69ac6d2ad0a197cc99ec \ + --hash=sha256:21e764fd1e70b6a3e205a0e46f3051701f98a8cb3fad66eeb80e48bb502f8698 \ + --hash=sha256:231ddcbb35e2ff8973e1365db41fe0572662893b99a05deb183b68ad4c0c8bd4 \ + --hash=sha256:253a4a220747e8b5faf57ec320c4f5efb0cef05f647420bf267143ec15dba10a \ + --hash=sha256:280081916dc341820640489a66e4696049401ef1cf6dd672f672e70ad915aca3 \ + --hash=sha256:2a441ea71902098ffe78c5abe6c494f44160b4af614ed16c3d9a3b1d17fd8ee2 \ + --hash=sha256:304b13570067b2547562e308af560b3963857b1fa90bd6afd978130130fe2d6a \ + --hash=sha256:32286a2c8d167e897177b673176c1e3e00d4057caf5d2b64eef9a3666b03018e \ + --hash=sha256:33bdcc2a32c0a0e861f60841a512c8acc658c87c2ac59d89e3a46dacf7d866e4 \ + --hash=sha256:375b83ed0aecfce76c16d198fbc21f3b11b337d68662bea0a995046682a11419 \ + --hash=sha256:3c09a49d6cde137258beb3d551994a2927fd35ad5cf96aed573f61bbd67c5f84 \ + --hash=sha256:3d92613ec25e43b05f042302531ec0f00b8445190e43325880cbd6ab7c2581da \ + --hash=sha256:40a126142a56b2dfc0aacbad1de8310cbf60da7656db0e6b16eebd48e3e93519 \ + --hash=sha256:416c229f77e5ea25b3dfd4b582f8d73d7e43c22320302b9ab128a2d3a0b38efe \ + --hash=sha256:432786d3561e69aeeae6c7e8648964ce0ad05736120135601f87ac26b9c83381 \ + --hash=sha256:43b9e366a31fdd1c87d0eb08f579b4a82b723ea54338f040d6b4e518a026ea29 \ + --hash=sha256:440eede837960000d74978f0eba527be106b5b9aee0daf779d395276ed0b0614 \ + --hash=sha256:45b0cc4e3556cd875e09102988d1ab8356c998b596c9fced84547c8138b487a0 \ + --hash=sha256:476743fe6dfe14a2da12e3ac79125dc84a3b2cf8094369a47a1529b0cd8549fe \ + --hash=sha256:4773092f8019072343a7447203308b176e10199920eb02d6195e81bbb3274c29 \ + --hash=sha256:4b3dac63058cc36820b0dd072f89898604e2d39686fe05321729d00d8ac185a0 \ + --hash=sha256:4d1c96a7a18b9690a4d46df09e3e3382406ae3213727cd1019ebade1c4a81917 \ + --hash=sha256:51307f5c71007673a2bf8232ad973483d281e74cb99c8c5a990af1eefa6277d9 \ + --hash=sha256:51447e9aa2684679af07ca5021c3db526e0284347ebf4ffcec1154c3350cfe32 \ + --hash=sha256:58150c9f9b9a552505912d182ccdf26f6396fb6094816ceebcbb20eecabaed94 \ + --hash=sha256:5b10cd92fc5c498b35a8635df6d5a100207f88b63a4dc1de7ef9a548e1e2cd63 \ + --hash=sha256:5e226f6218febc71f6c1fc2fafb91c226f75bdc1d8fb12d66823716e891608fd \ + --hash=sha256:609b3ba8fcc0fb5ab7af00719d0fb6ad0cb518e48e7712d12fd68f1327951198 \ + --hash=sha256:60f44ade2cf573dad7a277e6f8ca9a51a21dda572b13bd7d8539bb3cd5dbedde \ + --hash=sha256:611057cc5d5c0afc743ba8be6bd828c17e0aaa8643f9d0a9b9bb7dea80eb8012 \ + --hash=sha256:6366a16e1a25018694d6a5d784d09b046edc9eac40ea2b54065c3052672516a1 \ + --hash=sha256:65a7ff3f705e57d392f7261b6d0550fe137c3019477431f1c355e0db0a7d3e15 \ + --hash=sha256:673611bbd43f0810bec0b0f028ddeaaa501190339cac411f347ac76917c3ae7b \ + --hash=sha256:67830fc78e67501f47bb950471b2dcb9b35b140084429318e862895a8e89c993 \ + --hash=sha256:68ce9f4d6b26d5ccbf7fd4459bf75f74a0a146677ebba80597df60cbdb20e6f4 \ + --hash=sha256:68e5f26a1ad57ded6d1cfb85331d1c1a195314756471d97758c48498bb4dcdf5 \ + --hash=sha256:69b157c5d3292bcd443faca052f3096f637f1e074b98212a933c074ae23dc3b8 \ + --hash=sha256:75286256590a6320cf106a0d28970d3560aad9ee09aa7b34fb40524792436d35 \ + --hash=sha256:78841cccf1af7b40f6f716338d50c0902dbe88d9f800b3c973b7a9a0a693a642 \ + --hash=sha256:78fa18e436a1a0e58dbd7e02fc4473f3f32cceb12df9dfca542d075961c307d2 \ + --hash=sha256:79580094b00d1789d1f93ea55bc43cb2f611910c72235b7657f3482ddcc1b22d \ + --hash=sha256:7b86a2b16095d250c6f58b3d9b2eee6f4147754344f3dab0922f7c9bf7d226c9 \ + --hash=sha256:83aed2c10721ddd90f68140685391b50811a880af20654c59af6b6c66c40513c \ + --hash=sha256:84fd18bcc17526fc2b3c1af7d2b9217d32c9c04448c16ec693b9b4f1985c3d33 \ + --hash=sha256:871ff67ea1aad4dfd91736464934d56b32dac49f9fbe16cddba36198a7b3a0db \ + --hash=sha256:898f0e9068ca27d37f8e83a5b962821df851532e6c4a7d615c1c033f9da6eedf \ + --hash=sha256:8a79d9f4d8001473a30c163556b3c3bfebec837495a412dde78b51672f6134f9 \ + --hash=sha256:8c041122946b7ba21bb32c45b1aa57b1be35527690aeb3c5c234521085632eee \ + --hash=sha256:90c44bc373b7687f6948b693cceaea1348ae0975d7474746559494468e3c1d84 \ + --hash=sha256:9104ed0bd76a429d46f9ec0dbc9b08ad1d2dcdf2b00a5a0daa1c145329b35b44 \ + --hash=sha256:920079c3f7456fa213e0829ed2073aaa727fd39d889ead5b4f35d0de5460d04f \ + --hash=sha256:93d59d504b230e83c7a843251681959a0b6a9cd76f6e146ce1b8a80eb8739af9 \ + --hash=sha256:9b2aff1c7b3884512b9512c3eaadd9bab39fb45042ffaaa1dd08ff2b9f8109d9 \ + --hash=sha256:9b8e0f3107e2200b76f6054de99016eac3ee6762713587b36baaa7e4bd2ae177 \ + --hash=sha256:9bb41182d93ea91f60b4bc8fbf4c820c69ef8a12ab2d917f3f1834f1acad07e8 \ + --hash=sha256:9cdef90ae47919cae358d8ab15797a800ed41da7aba5d72419fb510729e2ed4b \ + --hash=sha256:a1786910334ed46ab1dd73222f2cd1e05c2c3bb39f6dddb4f8b36fc382058a39 \ + --hash=sha256:a4cfde78a9f2880208d16a93b795726a3017d5977e08d1e162a7a31322479c41 \ + --hash=sha256:a4fbdde9dd4a9ce5fd52c2b3a347bb50cc89483ef783f1cb00d408c13f7a96c0 \ + --hash=sha256:aa99adc8f081b475a12843953db36831eaf83ec33eb46a90629ca6a5de45a616 \ + --hash=sha256:ac351b3b8014eead140e77e9717e2992c6bbe30b63bc3422422eb84865412e3d \ + --hash=sha256:ad41ba96094304aa090f5a30cb6e4fb3b3f1c264c523394b4c39bbacc4dc92ba \ + --hash=sha256:b5314963fce9b0b12743891de876e724997864ee22aa496f903f426c7e2fa5b2 \ + --hash=sha256:bcf74c1df76758a395bf0af608c04c82257523f55c9868b334f06270d0f2112b \ + --hash=sha256:bd47ba7fc3ca94896759ea0109775132d3e7ab921fbf54038e1bab2e46c313c9 \ + --hash=sha256:c0323c9daef75ef2e5083624b4585018a0c9d5e3b40f607eed81a311270b934b \ + --hash=sha256:c1225416b463483160e4af85d5fc3a9690ccb53fd4b1865a6437825f5ede3209 \ + --hash=sha256:c1c948747b03be832dceed96ca815cef7360de9aa19d37c730f8e3f6101aca48 \ + --hash=sha256:c25fe15c70c59eb7c5ce8c06a1f3fa1da0ecc5ea1e7a5922c40fd2fa9b0d5046 \ + --hash=sha256:cc1b0fff8ead343dae06305f954eb8468ba0ec1a97881f42489d198e4ce3c632 \ + --hash=sha256:cd6280cf040f233bd7d3407b743b4b4c74f70e8e1c4199cb112a62c941c0772a \ + --hash=sha256:cd6c3d4b783c556fa00bf540854e42f135e2f256abd29669fcd0da0f2dec79c2 \ + --hash=sha256:d4d6fcde76f94f5cb9e43e9e9a61f16dacefd228cbbf6f1a09bd9b219a92f1a1 \ + --hash=sha256:ddf4af30b417d9fe16481e9b81c27ab2a7cde1ff7ba3e85653b02db7d145dc7b \ + --hash=sha256:df115d4d83168fdf2cae48ef1ff6d1cb4c466364e30861b37121de0f3bf1b990 \ + --hash=sha256:df7276909358e5635ae203673ab7e509ddd224225a8d6b0790bf13eb2bde1cc5 \ + --hash=sha256:e4fd89cc178bced6ad29cb3e6dd4aa63fa5017c3524dbd0b25998fb64a87cc8b \ + --hash=sha256:e9701d0049d92c16703a42771b98d560b95248949f23f8cf7b4eddd201814fb9 \ + --hash=sha256:ee2f2a527e3c1a6e6411eb4209642e138b544a2d72fe5d0d76daf77b24063534 \ + --hash=sha256:f7fb7d750cfa0a070d2c24e831fd3481019a60dd317ea2b39acbcebc08b6ed81 \ + --hash=sha256:f840ed6d8ecba8255df8c42b87fadeda98ddfc6eeec05e2dc66e26d46dd6f58a \ + --hash=sha256:f86c6358749bd4fda175388691e3ba8c46e24c5347d0afd20f9b7edfc9faf07d \ + --hash=sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf \ + --hash=sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115 # via requests -cryptography==43.0.3 ; sys_platform == 'linux' \ - --hash=sha256:0c580952eef9bf68c4747774cde7ec1d85a6e61de97281f2dba83c7d2c806362 \ - --hash=sha256:0f996e7268af62598f2fc1204afa98a3b5712313a55c4c9d434aef49cadc91d4 \ - --hash=sha256:1ec0bcf7e17c0c5669d881b1cd38c4972fade441b27bda1051665faaa89bdcaa \ - --hash=sha256:281c945d0e28c92ca5e5930664c1cefd85efe80e5c0d2bc58dd63383fda29f83 \ - --hash=sha256:2ce6fae5bdad59577b44e4dfed356944fbf1d925269114c28be377692643b4ff \ - --hash=sha256:315b9001266a492a6ff443b61238f956b214dbec9910a081ba5b6646a055a805 \ - --hash=sha256:443c4a81bb10daed9a8f334365fe52542771f25aedaf889fd323a853ce7377d6 \ - --hash=sha256:4a02ded6cd4f0a5562a8887df8b3bd14e822a90f97ac5e544c162899bc467664 \ - --hash=sha256:53a583b6637ab4c4e3591a15bc9db855b8d9dee9a669b550f311480acab6eb08 \ - --hash=sha256:63efa177ff54aec6e1c0aefaa1a241232dcd37413835a9b674b6e3f0ae2bfd3e \ - --hash=sha256:74f57f24754fe349223792466a709f8e0c093205ff0dca557af51072ff47ab18 \ - --hash=sha256:7e1ce50266f4f70bf41a2c6dc4358afadae90e2a1e5342d3c08883df1675374f \ - --hash=sha256:81ef806b1fef6b06dcebad789f988d3b37ccaee225695cf3e07648eee0fc6b73 \ - --hash=sha256:846da004a5804145a5f441b8530b4bf35afbf7da70f82409f151695b127213d5 \ - --hash=sha256:8ac43ae87929a5982f5948ceda07001ee5e83227fd69cf55b109144938d96984 \ - --hash=sha256:9762ea51a8fc2a88b70cf2995e5675b38d93bf36bd67d91721c309df184f49bd \ - --hash=sha256:a2a431ee15799d6db9fe80c82b055bae5a752bef645bba795e8e52687c69efe3 \ - --hash=sha256:bf7a1932ac4176486eab36a19ed4c0492da5d97123f1406cf15e41b05e787d2e \ - --hash=sha256:c2e6fc39c4ab499049df3bdf567f768a723a5e8464816e8f009f121a5a9f4405 \ - --hash=sha256:cbeb489927bd7af4aa98d4b261af9a5bc025bd87f0e3547e11584be9e9427be2 \ - --hash=sha256:d03b5621a135bffecad2c73e9f4deb1a0f977b9a8ffe6f8e002bf6c9d07b918c \ - --hash=sha256:d56e96520b1020449bbace2b78b603442e7e378a9b3bd68de65c782db1507995 \ - --hash=sha256:df6b6c6d742395dd77a23ea3728ab62f98379eff8fb61be2744d4679ab678f73 \ - --hash=sha256:e1be4655c7ef6e1bbe6b5d0403526601323420bcf414598955968c9ef3eb7d16 \ - --hash=sha256:f18c716be16bc1fea8e95def49edf46b82fccaa88587a45f8dc0ff6ab5d8e0a7 \ - --hash=sha256:f46304d6f0c6ab8e52770addfa2fc41e6629495548862279641972b6215451cd \ - --hash=sha256:f7b178f11ed3664fd0e995a47ed2b5ff0a12d893e41dd0494f406d1cf555cab7 +cryptography==50.0.0 ; platform_machine != 'ppc64le' and platform_machine != 's390x' and sys_platform == 'linux' \ + --hash=sha256:031e2d5dd4bb9caa3ca9c82e5a197fd8ae680232cee62603d1a813f3f07e3d03 \ + --hash=sha256:06a32a980526a6ab9a4b9bf8f7385800791e2bb960903cb6b530e4817509a3b7 \ + --hash=sha256:07479a1cb08219ab719147e742e76090c9c773321959bb94946fffdd397a6437 \ + --hash=sha256:07949c449a1abcf60d1ee6e88956d89404c7df3c8258f46589e912988e551987 \ + --hash=sha256:105110f43a471dbd0060b9c9516cb8a6a79233631a04cc2ba16f28323ac6e025 \ + --hash=sha256:11b74db56cdbe3cdee6e3f6982ecb70334fa10dce99ed58bf7894aaaa3b2a037 \ + --hash=sha256:12b9c6996425c76ea6c457ace4f3073e715b8c545add07cd1a8f3a4f90691269 \ + --hash=sha256:1489e263a8048bb8b6a8bac662eb2d402ea5d2b7b4699b72f385f1e2772db105 \ + --hash=sha256:19736989797678c6af1e55cd49055cdbcb55d8f6b5583ac5335f933aba9101dc \ + --hash=sha256:1b4a266766514614f8aa60416e71f2fc6e575d36e7bdc90f644fadb2f4b75b95 \ + --hash=sha256:2a8183b489dc1f7f80f135780fadc1108f14b31b8a40411c7a5b17425f65f28b \ + --hash=sha256:37fdb0d0111f1e2ff07139dfb79f1b49531f8e213c46f1163dd7642979b58c47 \ + --hash=sha256:3f5735ffe4996d28b809371756219f5354864902a3b9e7c0b9ee87041209fc9c \ + --hash=sha256:49e7d93abdbd2990caced757e5fade25302f719c3c8fb6e6fff2dde98999fc41 \ + --hash=sha256:5e34edd123674534acd70147f0ca331eaa2c74e6325fb2028c886aa26ba0b68c \ + --hash=sha256:62598a8a57f815db4c6259a4e97d857dab56697e7de8e8ab02352ab74da1995d \ + --hash=sha256:65c2c3add92b45fd0709db8594536aea39c2a67af0e27ffcf049c498501140b7 \ + --hash=sha256:6ba6a53445bd3cfa809ef3ef5f1589aa6ba08784a1d962bf47d0940e871dab1c \ + --hash=sha256:6e7d61120573a7f2cd94cc095f9e81f6967c61ccdf194285aa143ecec8e0b708 \ + --hash=sha256:7cec5b856506da6defb290f30c9ee687d5f5e8cb0bd3f6459dde43b0b4fa40ef \ + --hash=sha256:80b63928fa35083b33966ce1efb70e5b9607181e49dcd1c22c8c005e319f667f \ + --hash=sha256:82148ec5bddac30b51a5b3c1945075f896fa022cb93f8e4a01e9f6ee95292c5f \ + --hash=sha256:828743d939e9629bc267b8e2d08d8bb67cd4319c771a33d4b18b22dd8fb7440a \ + --hash=sha256:8d89f3976b10b4ce31118de72329025f70d2c6ead14a8217c5514dd2c6d5a78f \ + --hash=sha256:8eb5e1172eb569ea8a872796576e6a67c276351728b6455d5beb01242b027c6a \ + --hash=sha256:900131fafd8aead39ac7dd3a7e833be754c17a95cfd91221636949fe4eb0aa8a \ + --hash=sha256:910d11e1a385c654bf738bf3e6b8e6ed5de0f5610fcae2be9e5b398d8081d20e \ + --hash=sha256:910e1d2668e7de9648f2bcee30e180db2a6b15c30f887d7c4c93ddf96e3992e3 \ + --hash=sha256:9aa87839c383bdbab6ef865787a1fb877af8dd03464c4400322726feaaadfc6d \ + --hash=sha256:a1b30560f2acc95aa8b2e06e716a13dbfc97314747b80d9707e307f77b40d6b3 \ + --hash=sha256:a91296cb61e8df6f86d0c19cc4068228da256bf59bf86049fbd821084565327f \ + --hash=sha256:b42a28c1844fd9de8f3f7d540e36b66f3a9c83fceac7170ebc7a6a19edd9dcae \ + --hash=sha256:bd1c592e4d5974f0d08d4888e432157adba757c66da0246918e43677fafa2d30 \ + --hash=sha256:c87f62a3d3b9888ed0fdde100ec06aa61ca9cd44bad9057d1dff9a516b5f5bb9 \ + --hash=sha256:c99c003e088647b8a5b7c145d6f78c335f6348332b62e142d411c4b63d1460b9 \ + --hash=sha256:ccdc4a71a4dabae05de219404f9f4abc38e3b58422177ff93d0da05967dafa07 \ + --hash=sha256:d24fead1d4d076e1bfb006dcec392074a3cd8d7b4fc8a595aa64073b2b7a96ba \ + --hash=sha256:d58c3db7cd6eed54e6c06744db55456b65ebd7492ddeae9c1e93cfca7aa857d3 \ + --hash=sha256:d764dcf130c428ef66786f866dd750f53182bc608813489915e9fc106bb0c82f \ + --hash=sha256:df2a58a472f332225671c35b0a830208b86d004f82baa8530fa3782c85646533 \ + --hash=sha256:e722f16708d854fe924790e051061f6704a472c3bac347b6fd88033ea8dd0dc5 \ + --hash=sha256:ecfed7367f965a0328cfbdd70da860f15441f002f613185668c6e6ebf5a0ac11 \ + --hash=sha256:eeac2acb5a20ed25e0ad6d1df9891a520b78b404266b6d11778f25d5d691a6c9 \ + --hash=sha256:f59e38625469987d7ef6d495323c55e7db6c212eaf6112267e0d3b565a2e9c9f \ + --hash=sha256:f89831ef99dd7dd169ab06d63a831adb9e20a87aac6d380266bbda5823349169 \ + --hash=sha256:fd9192b7b70c573d7f214eb1ae35e00d359f6f5e4b27c7e21e30de1fc6204645 # via secretstorage -docutils==0.21.2 \ - --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ - --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2 +docutils==0.23 \ + --hash=sha256:25d013af9bf23bc1c7b2b093dff4208166c53a94786c9e447808335ef1185fea \ + --hash=sha256:746f5060322511280a1e50eb76846ed6bf2342984b2ac04dc42caa1a8d78799e # via readme-renderer -idna==3.10 \ - --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ - --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3 +id==1.6.1 \ + --hash=sha256:d0732d624fb46fd4e7bc4e5152f00214450953b9e772c182c1c22964def1a069 \ + --hash=sha256:f5ec41ed2629a508f5d0988eda142e190c9c6da971100612c4de9ad9f9b237ca + # via twine +idna==3.18 \ + --hash=sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2 \ + --hash=sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848 # via requests -importlib-metadata==8.5.0 \ - --hash=sha256:45e54197d28b7a7f1559e60b95e7c567032b602131fbd588f1497f47880aa68b \ - --hash=sha256:71522656f0abace1d072b9e5481a48f07c138e00f079c38c8f883823f9c26bd7 - # via - # keyring - # twine -jaraco-classes==3.4.0 \ +importlib-metadata==9.0.0 ; python_full_version < '3.12' and platform_machine != 'ppc64le' and platform_machine != 's390x' \ + --hash=sha256:2d21d1cc5a017bd0559e36150c21c830ab1dc304dedd1b7ea85d20f45ef3edd7 \ + --hash=sha256:a4f57ab599e6a2e3016d7595cfd72eb4661a5106e787a95bcc90c7105b831efc + # via keyring +jaraco-classes==3.4.0 ; platform_machine != 'ppc64le' and platform_machine != 's390x' \ --hash=sha256:47a024b51d0239c0dd8c8540c6c7f484be3b8fcf0b2d85c13825780d3b3f3acd \ --hash=sha256:f662826b6bed8cace05e7ff873ce0f9283b5c924470fe664fff1c2f00f581790 # via keyring -jaraco-context==6.0.1 \ - --hash=sha256:9bae4ea555cf0b14938dc0aee7c9f32ed303aa20a3b73e7dc80111628792d1b3 \ - --hash=sha256:f797fc481b490edb305122c9181830a3a5b76d84ef6d1aef2fb9b47ab956f9e4 +jaraco-context==6.1.2 ; platform_machine != 'ppc64le' and platform_machine != 's390x' \ + --hash=sha256:bf8150b79a2d5d91ae48629d8b427a8f7ba0e1097dd6202a9059f29a36379535 \ + --hash=sha256:f1a6c9d391e661cc5b8d39861ff077a7dc24dc23833ccee564b234b81c82dfe3 # via keyring -jaraco-functools==4.1.0 \ - --hash=sha256:70f7e0e2ae076498e212562325e805204fc092d7b4c17e0e86c959e249701a9d \ - --hash=sha256:ad159f13428bc4acbf5541ad6dec511f91573b90fba04df61dafa2a1231cf649 +jaraco-functools==4.6.0 ; platform_machine != 'ppc64le' and platform_machine != 's390x' \ + --hash=sha256:880c577ec9720b3a052d5bc611fb9f2269b3d87902ef42440df443b88e443280 \ + --hash=sha256:99e3dc0060c5cbe8fcd1cdb36258e2a65ca40f1566b2033b12abb1bb44dd3c30 # via keyring -jeepney==0.8.0 ; sys_platform == 'linux' \ - --hash=sha256:5efe48d255973902f6badc3ce55e2aa6c5c3b3bc642059ef3a91247bcfcc5806 \ - --hash=sha256:c0a454ad016ca575060802ee4d590dd912e35c122fa04e70306de3d076cce755 +jeepney==0.9.0 ; platform_machine != 'ppc64le' and platform_machine != 's390x' and sys_platform == 'linux' \ + --hash=sha256:97e5714520c16fc0a45695e5365a2e11b81ea79bba796e26f9f1d178cb182683 \ + --hash=sha256:cf0e9e845622b81e4a28df94c40345400256ec608d0e55bb8a3feaa9163f5732 # via # keyring # secretstorage -keyring==25.4.1 \ - --hash=sha256:5426f817cf7f6f007ba5ec722b1bcad95a75b27d780343772ad76b17cb47b0bf \ - --hash=sha256:b07ebc55f3e8ed86ac81dd31ef14e81ace9dd9c3d4b5d77a6e9a2016d0d71a1b +keyring==25.7.0 ; platform_machine != 'ppc64le' and platform_machine != 's390x' \ + --hash=sha256:be4a0b195f149690c166e850609a477c532ddbfbaed96a404d4e43f8d5e2689f \ + --hash=sha256:fe01bd85eb3f8fb3dd0405defdeac9a5b4f6f0439edbb3149577f244a2e8245b # via twine -markdown-it-py==3.0.0 \ - --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ - --hash=sha256:e3f60a94fa066dc52ec76661e37c851cb232d92f9886b15cb560aaada2df8feb +markdown-it-py==4.2.0 \ + --hash=sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49 \ + --hash=sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a # via rich mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba # via markdown-it-py -more-itertools==10.5.0 \ - --hash=sha256:037b0d3203ce90cca8ab1defbbdac29d5f993fc20131f3664dc8d6acfa872aef \ - --hash=sha256:5482bfef7849c25dc3c6dd53a6173ae4795da2a41a80faea6700d9f5846c5da6 +more-itertools==11.1.0 ; platform_machine != 'ppc64le' and platform_machine != 's390x' \ + --hash=sha256:48e8f4d9e7e5878571ecf6f2b4e57634f93cd474cc8cfbd2376f2d11b396e30d \ + --hash=sha256:4b65538ae22f6fed0ce4874efd317463a7489796a0939fa66824dd542125a192 # via # jaraco-classes # jaraco-functools -nh3==0.2.18 \ - --hash=sha256:0411beb0589eacb6734f28d5497ca2ed379eafab8ad8c84b31bb5c34072b7164 \ - --hash=sha256:14c5a72e9fe82aea5fe3072116ad4661af5cf8e8ff8fc5ad3450f123e4925e86 \ - --hash=sha256:19aaba96e0f795bd0a6c56291495ff59364f4300d4a39b29a0abc9cb3774a84b \ - --hash=sha256:34c03fa78e328c691f982b7c03d4423bdfd7da69cd707fe572f544cf74ac23ad \ - --hash=sha256:36c95d4b70530b320b365659bb5034341316e6a9b30f0b25fa9c9eff4c27a204 \ - --hash=sha256:3a157ab149e591bb638a55c8c6bcb8cdb559c8b12c13a8affaba6cedfe51713a \ - --hash=sha256:42c64511469005058cd17cc1537578eac40ae9f7200bedcfd1fc1a05f4f8c200 \ - --hash=sha256:5f36b271dae35c465ef5e9090e1fdaba4a60a56f0bb0ba03e0932a66f28b9189 \ - --hash=sha256:6955369e4d9f48f41e3f238a9e60f9410645db7e07435e62c6a9ea6135a4907f \ - --hash=sha256:7b7c2a3c9eb1a827d42539aa64091640bd275b81e097cd1d8d82ef91ffa2e811 \ - --hash=sha256:8ce0f819d2f1933953fca255db2471ad58184a60508f03e6285e5114b6254844 \ - --hash=sha256:94a166927e53972a9698af9542ace4e38b9de50c34352b962f4d9a7d4c927af4 \ - --hash=sha256:a7f1b5b2c15866f2db413a3649a8fe4fd7b428ae58be2c0f6bca5eefd53ca2be \ - --hash=sha256:c8b3a1cebcba9b3669ed1a84cc65bf005728d2f0bc1ed2a6594a992e817f3a50 \ - --hash=sha256:de3ceed6e661954871d6cd78b410213bdcb136f79aafe22aa7182e028b8c7307 \ - --hash=sha256:f0eca9ca8628dbb4e916ae2491d72957fdd35f7a5d326b7032a345f111ac07fe +nh3==0.3.6 \ + --hash=sha256:082675ff87b9385ec430ffe6d5847ba7456cc39b73720cd4add472f9f4cffd56 \ + --hash=sha256:2411e8c3cee81a1ddd62c2a5d50585c28aa5566d373ad1db92536b95ddb24ef2 \ + --hash=sha256:25c733bee928530556b1db0ea46c52cf5aa686146e38e60a6fc7cb801ef91cec \ + --hash=sha256:2f90d9a0cfdbee218994fdaaeeb5a0fde62d08f35e4eef0378ec1e2200172fd0 \ + --hash=sha256:34d2b0d934156b87ee114f599a3ba9b8b9e17b5d79652ba3a13fa50903de965e \ + --hash=sha256:36d06341bd501240d320f5942481ed5e6846136b666e1ba4faf802b78ebc875f \ + --hash=sha256:43bc1ed3fa0716295fabee29ba42b2667e4a51d140b0a68e092170a765474fa6 \ + --hash=sha256:44673b27010051ab5a5e438a86ec31bbda61d4a77d7e900af6b7be3037c1abae \ + --hash=sha256:455469a29951edc92bc48b47ac2281c3f2609e6c4f6a047056449f8c2c23facf \ + --hash=sha256:4713502748f564fee0633b37b3403783ce0a3af3a3d148ad91025a5bdadb7bc6 \ + --hash=sha256:5276ef17bdba9ad8040575c74072008b13aae429436e9d0429e718bb5f90f4da \ + --hash=sha256:597a8e843bea00b2eb5520658dc24a9bb032e7fc9e7c2c0c4cd29420220c9796 \ + --hash=sha256:69bbb92865a693d909db3a700d3c01537533844d0948c1e9323561ce06ecda41 \ + --hash=sha256:69f365963f63a1e9bff53bdbb3c542c7c2efed3e163c9d5d83a772a2ac468c21 \ + --hash=sha256:82ca5bf427ad1b216b65ede1a2e2d87dc49bec417ceba0f297213107d3cd9d78 \ + --hash=sha256:889932a97fb4abb6f95fef1914c0d269ebfb60011e67121c1163059b9449dbb4 \ + --hash=sha256:905f877dc66dd7aea4a76e54bcb26acb5ff8216f720c0017ccf63e0e6035698e \ + --hash=sha256:a43ebd7543555c3ac1bc353023d0794e75cb76f6f18f19c32e95441496c0cc25 \ + --hash=sha256:d14bf7982e7a77c0c775634c29c07ce08b38a046df73e1c1f139b3e82f18a38e \ + --hash=sha256:e196fa70c2ff2eb4de7d3df3108f8f358c1d69dff20d45b11f20a5aa227ffb6d \ + --hash=sha256:e1b160831c9cdb06a6c79c2f9cdb11386602938f9af260d1c457a85add4f6f69 \ + --hash=sha256:e6b7beece07525dc6e6b0fc2f104442de2ba328360ad00e50cbe2e1fd620447d \ + --hash=sha256:edb2b4a1a27523e6cc7c417f8d21ce3d005243548b93e56b762b66b0c7f589f9 \ + --hash=sha256:f2f14b7ae1fca99c4a66c981aac3974e7fbc1ca30a12673d223ae1df76680917 \ + --hash=sha256:f338ac7d594c067679f1e99b4f5ec3906842979560f9d8f15d6bdfa39a353b10 \ + --hash=sha256:f3736c9dd3d1856f80cd031715b84ca75cda2bbb1ac802c3da26bfce590838d7 \ + --hash=sha256:f5ed5fe84aee7f39db95c214a7421bf0499fbf500fec6d86a4e29bfc37971438 # via readme-renderer -pkginfo==1.10.0 \ - --hash=sha256:5df73835398d10db79f8eecd5cd86b1f6d29317589ea70796994d49399af6297 \ - --hash=sha256:889a6da2ed7ffc58ab5b900d888ddce90bce912f2d2de1dc1c26f4cb9fe65097 +packaging==26.2 \ + --hash=sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e \ + --hash=sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661 # via twine -pycparser==2.22 ; platform_python_implementation != 'PyPy' and sys_platform == 'linux' \ - --hash=sha256:491c8be9c040f5390f5bf44a5b07752bd07f56edf992381b05c701439eec10f6 \ - --hash=sha256:c3702b6d3dd8c7abc1afa565d7e63d53a1d0bd86cdc24edd75470f4de499cfcc +pycparser==3.0 ; implementation_name != 'PyPy' and platform_machine != 'ppc64le' and platform_machine != 's390x' and platform_python_implementation != 'PyPy' and sys_platform == 'linux' \ + --hash=sha256:600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29 \ + --hash=sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992 # via cffi -pygments==2.18.0 \ - --hash=sha256:786ff802f32e91311bff3889f6e9a86e81505fe99f2735bb6d60ae0c5004f199 \ - --hash=sha256:b8e6aca0523f3ab76fee51799c488e38782ac06eafcf95e7ba832985c8e7b13a +pygments==2.20.0 \ + --hash=sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f \ + --hash=sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176 # via # readme-renderer # rich -pywin32-ctypes==0.2.3 ; sys_platform == 'win32' \ +pywin32-ctypes==0.2.3 ; platform_machine != 'ppc64le' and platform_machine != 's390x' and sys_platform == 'win32' \ --hash=sha256:8a1513379d709975552d202d942d9837758905c8d01eb82b8bcc30918929e7b8 \ --hash=sha256:d162dc04946d704503b2edc4d55f3dba5c1d539ead017afa00142c38b9885755 # via keyring -readme-renderer==44.0 \ - --hash=sha256:2fbca89b81a08526aadf1357a8c2ae889ec05fb03f5da67f9769c9a592166151 \ - --hash=sha256:8712034eabbfa6805cacf1402b4eeb2a73028f72d1166d6f5cb7f9c047c5d1e1 +readme-renderer==45.0 \ + --hash=sha256:030a8fac74904f8fba11ad1bb6964e3f76e896dc7e5e71f16af190c9056696d1 \ + --hash=sha256:3385ed220117104a2bceb4a9dac8c5fdf6d1f96890d7ea2a9c7174fd5c84091f # via twine -requests==2.32.3 \ - --hash=sha256:55365417734eb18255590a9ff9eb97e9e1da868d4ccd6402399eaf68af20a760 \ - --hash=sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6 +requests==2.34.2 \ + --hash=sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0 \ + --hash=sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed # via # requests-toolbelt # twine @@ -319,25 +372,26 @@ rfc3986==2.0.0 \ --hash=sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd \ --hash=sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c # via twine -rich==13.9.3 \ - --hash=sha256:9836f5096eb2172c9e77df411c1b009bace4193d6a481d534fea75ebba758283 \ - --hash=sha256:bc1e01b899537598cf02579d2b9f4a415104d3fc439313a7a2c165d76557a08e +rich==15.0.0 \ + --hash=sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb \ + --hash=sha256:edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36 # via twine -secretstorage==3.3.3 ; sys_platform == 'linux' \ - --hash=sha256:2403533ef369eca6d2ba81718576c5e0f564d5cca1b58f73a8b23e7d4eeebd77 \ - --hash=sha256:f356e6628222568e3af06f2eba8df495efa13b3b63081dafd4f7d9a7b7bc9f99 +secretstorage==3.5.0 ; platform_machine != 'ppc64le' and platform_machine != 's390x' and sys_platform == 'linux' \ + --hash=sha256:0ce65888c0725fcb2c5bc0fdb8e5438eece02c523557ea40ce0703c266248137 \ + --hash=sha256:f04b8e4689cbce351744d5537bf6b1329c6fc68f91fa666f60a380edddcd11be # via keyring -twine==5.1.1 \ - --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ - --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db +twine==7.0.0 \ + --hash=sha256:85cdb29c518efef867360ae4acd4b0dfd61c8654a22fca08e6f8539f05022177 \ + --hash=sha256:b854164df26db268af05f49aa5c0344b10e27a494343ff05b1e0bad3b135f5a7 # via -r tools/publish/requirements.in -urllib3==2.2.3 \ - --hash=sha256:ca899ca043dcb1bafa3e262d73aa25c465bfb49e0bd9dd5d59f1d0acba2f8fac \ - --hash=sha256:e7d814a81dad81e6caf2ec9fdedb284ecc9c73076b62654547cc64ccdcae26e9 +urllib3==2.7.0 \ + --hash=sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c \ + --hash=sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897 # via + # id # requests # twine -zipp==3.20.2 \ - --hash=sha256:a817ac80d6cf4b23bf7f2828b7cabf326f15a001bea8b1f9b49631780ba28350 \ - --hash=sha256:bc9eb26f4506fda01b81bcde0ca78103b6e62f991b381fec825435c836edbc29 +zipp==4.1.0 ; python_full_version < '3.12' and platform_machine != 'ppc64le' and platform_machine != 's390x' \ + --hash=sha256:25ad4e16390cd314347dd8f1de67a2ac538ae658ed4ab9db16029c07c188e97f \ + --hash=sha256:4cb57381f544315db7688e976e922a2b18cdb513d21cc194eb42232ba2a3e602 # via importlib-metadata diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 23d298643f..b235d95a52 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -6,186 +6,187 @@ backports-tarfile==1.2.0 \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2024.8.30 \ - --hash=sha256:922820b53db7a7257ffbda3f597266d435245903d80737e34f8a45ff3e3230d8 \ - --hash=sha256:bec941d2aa8195e248a60b31ff9f0558284cf01a52591ceda73ea9afffd69fd9 +certifi==2026.7.22 \ + --hash=sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775 \ + --hash=sha256:741e2c3b351ddf169a738da9f2c048608ff7f2c5cc02f1ebc6b118bb090d5d55 # via requests -charset-normalizer==3.4.0 \ - --hash=sha256:0099d79bdfcf5c1f0c2c72f91516702ebf8b0b8ddd8905f97a8aecf49712c621 \ - --hash=sha256:0713f3adb9d03d49d365b70b84775d0a0d18e4ab08d12bc46baa6132ba78aaf6 \ - --hash=sha256:07afec21bbbbf8a5cc3651aa96b980afe2526e7f048fdfb7f1014d84acc8b6d8 \ - --hash=sha256:0b309d1747110feb25d7ed6b01afdec269c647d382c857ef4663bbe6ad95a912 \ - --hash=sha256:0d99dd8ff461990f12d6e42c7347fd9ab2532fb70e9621ba520f9e8637161d7c \ - --hash=sha256:0de7b687289d3c1b3e8660d0741874abe7888100efe14bd0f9fd7141bcbda92b \ - --hash=sha256:1110e22af8ca26b90bd6364fe4c763329b0ebf1ee213ba32b68c73de5752323d \ - --hash=sha256:130272c698667a982a5d0e626851ceff662565379baf0ff2cc58067b81d4f11d \ - --hash=sha256:136815f06a3ae311fae551c3df1f998a1ebd01ddd424aa5603a4336997629e95 \ - --hash=sha256:14215b71a762336254351b00ec720a8e85cada43b987da5a042e4ce3e82bd68e \ - --hash=sha256:1db4e7fefefd0f548d73e2e2e041f9df5c59e178b4c72fbac4cc6f535cfb1565 \ - --hash=sha256:1ffd9493de4c922f2a38c2bf62b831dcec90ac673ed1ca182fe11b4d8e9f2a64 \ - --hash=sha256:2006769bd1640bdf4d5641c69a3d63b71b81445473cac5ded39740a226fa88ab \ - --hash=sha256:20587d20f557fe189b7947d8e7ec5afa110ccf72a3128d61a2a387c3313f46be \ - --hash=sha256:223217c3d4f82c3ac5e29032b3f1c2eb0fb591b72161f86d93f5719079dae93e \ - --hash=sha256:27623ba66c183eca01bf9ff833875b459cad267aeeb044477fedac35e19ba907 \ - --hash=sha256:285e96d9d53422efc0d7a17c60e59f37fbf3dfa942073f666db4ac71e8d726d0 \ - --hash=sha256:2de62e8801ddfff069cd5c504ce3bc9672b23266597d4e4f50eda28846c322f2 \ - --hash=sha256:2f6c34da58ea9c1a9515621f4d9ac379871a8f21168ba1b5e09d74250de5ad62 \ - --hash=sha256:309a7de0a0ff3040acaebb35ec45d18db4b28232f21998851cfa709eeff49d62 \ - --hash=sha256:35c404d74c2926d0287fbd63ed5d27eb911eb9e4a3bb2c6d294f3cfd4a9e0c23 \ - --hash=sha256:3710a9751938947e6327ea9f3ea6332a09bf0ba0c09cae9cb1f250bd1f1549bc \ - --hash=sha256:3d59d125ffbd6d552765510e3f31ed75ebac2c7470c7274195b9161a32350284 \ - --hash=sha256:40d3ff7fc90b98c637bda91c89d51264a3dcf210cade3a2c6f838c7268d7a4ca \ - --hash=sha256:425c5f215d0eecee9a56cdb703203dda90423247421bf0d67125add85d0c4455 \ - --hash=sha256:43193c5cda5d612f247172016c4bb71251c784d7a4d9314677186a838ad34858 \ - --hash=sha256:44aeb140295a2f0659e113b31cfe92c9061622cadbc9e2a2f7b8ef6b1e29ef4b \ - --hash=sha256:47334db71978b23ebcf3c0f9f5ee98b8d65992b65c9c4f2d34c2eaf5bcaf0594 \ - --hash=sha256:4796efc4faf6b53a18e3d46343535caed491776a22af773f366534056c4e1fbc \ - --hash=sha256:4a51b48f42d9358460b78725283f04bddaf44a9358197b889657deba38f329db \ - --hash=sha256:4b67fdab07fdd3c10bb21edab3cbfe8cf5696f453afce75d815d9d7223fbe88b \ - --hash=sha256:4ec9dd88a5b71abfc74e9df5ebe7921c35cbb3b641181a531ca65cdb5e8e4dea \ - --hash=sha256:4f9fc98dad6c2eaa32fc3af1417d95b5e3d08aff968df0cd320066def971f9a6 \ - --hash=sha256:54b6a92d009cbe2fb11054ba694bc9e284dad30a26757b1e372a1fdddaf21920 \ - --hash=sha256:55f56e2ebd4e3bc50442fbc0888c9d8c94e4e06a933804e2af3e89e2f9c1c749 \ - --hash=sha256:5726cf76c982532c1863fb64d8c6dd0e4c90b6ece9feb06c9f202417a31f7dd7 \ - --hash=sha256:5d447056e2ca60382d460a604b6302d8db69476fd2015c81e7c35417cfabe4cd \ - --hash=sha256:5ed2e36c3e9b4f21dd9422f6893dec0abf2cca553af509b10cd630f878d3eb99 \ - --hash=sha256:5ff2ed8194587faf56555927b3aa10e6fb69d931e33953943bc4f837dfee2242 \ - --hash=sha256:62f60aebecfc7f4b82e3f639a7d1433a20ec32824db2199a11ad4f5e146ef5ee \ - --hash=sha256:63bc5c4ae26e4bc6be6469943b8253c0fd4e4186c43ad46e713ea61a0ba49129 \ - --hash=sha256:6b40e8d38afe634559e398cc32b1472f376a4099c75fe6299ae607e404c033b2 \ - --hash=sha256:6b493a043635eb376e50eedf7818f2f322eabbaa974e948bd8bdd29eb7ef2a51 \ - --hash=sha256:6dba5d19c4dfab08e58d5b36304b3f92f3bd5d42c1a3fa37b5ba5cdf6dfcbcee \ - --hash=sha256:6fd30dc99682dc2c603c2b315bded2799019cea829f8bf57dc6b61efde6611c8 \ - --hash=sha256:707b82d19e65c9bd28b81dde95249b07bf9f5b90ebe1ef17d9b57473f8a64b7b \ - --hash=sha256:7706f5850360ac01d80c89bcef1640683cc12ed87f42579dab6c5d3ed6888613 \ - --hash=sha256:7782afc9b6b42200f7362858f9e73b1f8316afb276d316336c0ec3bd73312742 \ - --hash=sha256:79983512b108e4a164b9c8d34de3992f76d48cadc9554c9e60b43f308988aabe \ - --hash=sha256:7f683ddc7eedd742e2889d2bfb96d69573fde1d92fcb811979cdb7165bb9c7d3 \ - --hash=sha256:82357d85de703176b5587dbe6ade8ff67f9f69a41c0733cf2425378b49954de5 \ - --hash=sha256:84450ba661fb96e9fd67629b93d2941c871ca86fc38d835d19d4225ff946a631 \ - --hash=sha256:86f4e8cca779080f66ff4f191a685ced73d2f72d50216f7112185dc02b90b9b7 \ - --hash=sha256:8cda06946eac330cbe6598f77bb54e690b4ca93f593dee1568ad22b04f347c15 \ - --hash=sha256:8ce7fd6767a1cc5a92a639b391891bf1c268b03ec7e021c7d6d902285259685c \ - --hash=sha256:8ff4e7cdfdb1ab5698e675ca622e72d58a6fa2a8aa58195de0c0061288e6e3ea \ - --hash=sha256:9289fd5dddcf57bab41d044f1756550f9e7cf0c8e373b8cdf0ce8773dc4bd417 \ - --hash=sha256:92a7e36b000bf022ef3dbb9c46bfe2d52c047d5e3f3343f43204263c5addc250 \ - --hash=sha256:92db3c28b5b2a273346bebb24857fda45601aef6ae1c011c0a997106581e8a88 \ - --hash=sha256:95c3c157765b031331dd4db3c775e58deaee050a3042fcad72cbc4189d7c8dca \ - --hash=sha256:980b4f289d1d90ca5efcf07958d3eb38ed9c0b7676bf2831a54d4f66f9c27dfa \ - --hash=sha256:9ae4ef0b3f6b41bad6366fb0ea4fc1d7ed051528e113a60fa2a65a9abb5b1d99 \ - --hash=sha256:9c98230f5042f4945f957d006edccc2af1e03ed5e37ce7c373f00a5a4daa6149 \ - --hash=sha256:9fa2566ca27d67c86569e8c85297aaf413ffab85a8960500f12ea34ff98e4c41 \ - --hash=sha256:a14969b8691f7998e74663b77b4c36c0337cb1df552da83d5c9004a93afdb574 \ - --hash=sha256:a8aacce6e2e1edcb6ac625fb0f8c3a9570ccc7bfba1f63419b3769ccf6a00ed0 \ - --hash=sha256:a8e538f46104c815be19c975572d74afb53f29650ea2025bbfaef359d2de2f7f \ - --hash=sha256:aa41e526a5d4a9dfcfbab0716c7e8a1b215abd3f3df5a45cf18a12721d31cb5d \ - --hash=sha256:aa693779a8b50cd97570e5a0f343538a8dbd3e496fa5dcb87e29406ad0299654 \ - --hash=sha256:ab22fbd9765e6954bc0bcff24c25ff71dcbfdb185fcdaca49e81bac68fe724d3 \ - --hash=sha256:ab2e5bef076f5a235c3774b4f4028a680432cded7cad37bba0fd90d64b187d19 \ - --hash=sha256:ab973df98fc99ab39080bfb0eb3a925181454d7c3ac8a1e695fddfae696d9e90 \ - --hash=sha256:af73657b7a68211996527dbfeffbb0864e043d270580c5aef06dc4b659a4b578 \ - --hash=sha256:b197e7094f232959f8f20541ead1d9862ac5ebea1d58e9849c1bf979255dfac9 \ - --hash=sha256:b295729485b06c1a0683af02a9e42d2caa9db04a373dc38a6a58cdd1e8abddf1 \ - --hash=sha256:b8831399554b92b72af5932cdbbd4ddc55c55f631bb13ff8fe4e6536a06c5c51 \ - --hash=sha256:b8dcd239c743aa2f9c22ce674a145e0a25cb1566c495928440a181ca1ccf6719 \ - --hash=sha256:bcb4f8ea87d03bc51ad04add8ceaf9b0f085ac045ab4d74e73bbc2dc033f0236 \ - --hash=sha256:bd7af3717683bea4c87acd8c0d3d5b44d56120b26fd3f8a692bdd2d5260c620a \ - --hash=sha256:bf4475b82be41b07cc5e5ff94810e6a01f276e37c2d55571e3fe175e467a1a1c \ - --hash=sha256:c3e446d253bd88f6377260d07c895816ebf33ffffd56c1c792b13bff9c3e1ade \ - --hash=sha256:c57516e58fd17d03ebe67e181a4e4e2ccab1168f8c2976c6a334d4f819fe5944 \ - --hash=sha256:c94057af19bc953643a33581844649a7fdab902624d2eb739738a30e2b3e60fc \ - --hash=sha256:cab5d0b79d987c67f3b9e9c53f54a61360422a5a0bc075f43cab5621d530c3b6 \ - --hash=sha256:ce031db0408e487fd2775d745ce30a7cd2923667cf3b69d48d219f1d8f5ddeb6 \ - --hash=sha256:cee4373f4d3ad28f1ab6290684d8e2ebdb9e7a1b74fdc39e4c211995f77bec27 \ - --hash=sha256:d5b054862739d276e09928de37c79ddeec42a6e1bfc55863be96a36ba22926f6 \ - --hash=sha256:dbe03226baf438ac4fda9e2d0715022fd579cb641c4cf639fa40d53b2fe6f3e2 \ - --hash=sha256:dc15e99b2d8a656f8e666854404f1ba54765871104e50c8e9813af8a7db07f12 \ - --hash=sha256:dcaf7c1524c0542ee2fc82cc8ec337f7a9f7edee2532421ab200d2b920fc97cf \ - --hash=sha256:dd4eda173a9fcccb5f2e2bd2a9f423d180194b1bf17cf59e3269899235b2a114 \ - --hash=sha256:dd9a8bd8900e65504a305bf8ae6fa9fbc66de94178c420791d0293702fce2df7 \ - --hash=sha256:de7376c29d95d6719048c194a9cf1a1b0393fbe8488a22008610b0361d834ecf \ - --hash=sha256:e7fdd52961feb4c96507aa649550ec2a0d527c086d284749b2f582f2d40a2e0d \ - --hash=sha256:e91f541a85298cf35433bf66f3fab2a4a2cff05c127eeca4af174f6d497f0d4b \ - --hash=sha256:e9e3c4c9e1ed40ea53acf11e2a386383c3304212c965773704e4603d589343ed \ - --hash=sha256:ee803480535c44e7f5ad00788526da7d85525cfefaf8acf8ab9a310000be4b03 \ - --hash=sha256:f09cb5a7bbe1ecae6e87901a2eb23e0256bb524a79ccc53eb0b7629fbe7677c4 \ - --hash=sha256:f19c1585933c82098c2a520f8ec1227f20e339e33aca8fa6f956f6691b784e67 \ - --hash=sha256:f1a2f519ae173b5b6a2c9d5fa3116ce16e48b3462c8b96dfdded11055e3d6365 \ - --hash=sha256:f28f891ccd15c514a0981f3b9db9aa23d62fe1a99997512b0491d2ed323d229a \ - --hash=sha256:f3e73a4255342d4eb26ef6df01e3962e73aa29baa3124a8e824c5d3364a65748 \ - --hash=sha256:f606a1881d2663630ea5b8ce2efe2111740df4b687bd78b34a8131baa007f79b \ - --hash=sha256:fe9f97feb71aa9896b81973a7bbada8c49501dc73e58a10fcef6663af95e5079 \ - --hash=sha256:ffc519621dce0c767e96b9c53f09c5d215578e10b02c285809f76509a3931482 +charset-normalizer==3.4.9 \ + --hash=sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380 \ + --hash=sha256:03d07803992c6c7bbc976327f34b18b6160327fc81cb82c9d504720ac0be3b62 \ + --hash=sha256:04ce310cb89c15df659582aee80a0603788732a5e017d5bd5c81158106ce249c \ + --hash=sha256:0d861473f743244d349b50f850d10eb87aeb22bbdcc8e64f79273c94af5a8226 \ + --hash=sha256:0e94703ec9684807f20cfb5eed95c70f67f2a8f21ad620146d7b5a13677b93e5 \ + --hash=sha256:0fa1aec2d32bcc03c8fa0f6f1712caad1adc38509f31142112e5c9daf5b9c833 \ + --hash=sha256:16b65ea0f2465b6fb52aa22de5eca612aa964ddfec00a912e26f4656cbef890b \ + --hash=sha256:16d10d789dd9bcca1173c95af82c58433122564b7bc39385124be735a35cbe99 \ + --hash=sha256:19ac87f93086ce37b86e098888555c4b4bc48102279bae3350098c0ed664b501 \ + --hash=sha256:1d22856ffbe153a602df38e4a5464f0b748a54002e0d69ac6d2ad0a197cc99ec \ + --hash=sha256:21e764fd1e70b6a3e205a0e46f3051701f98a8cb3fad66eeb80e48bb502f8698 \ + --hash=sha256:231ddcbb35e2ff8973e1365db41fe0572662893b99a05deb183b68ad4c0c8bd4 \ + --hash=sha256:253a4a220747e8b5faf57ec320c4f5efb0cef05f647420bf267143ec15dba10a \ + --hash=sha256:280081916dc341820640489a66e4696049401ef1cf6dd672f672e70ad915aca3 \ + --hash=sha256:2a441ea71902098ffe78c5abe6c494f44160b4af614ed16c3d9a3b1d17fd8ee2 \ + --hash=sha256:304b13570067b2547562e308af560b3963857b1fa90bd6afd978130130fe2d6a \ + --hash=sha256:32286a2c8d167e897177b673176c1e3e00d4057caf5d2b64eef9a3666b03018e \ + --hash=sha256:33bdcc2a32c0a0e861f60841a512c8acc658c87c2ac59d89e3a46dacf7d866e4 \ + --hash=sha256:375b83ed0aecfce76c16d198fbc21f3b11b337d68662bea0a995046682a11419 \ + --hash=sha256:3c09a49d6cde137258beb3d551994a2927fd35ad5cf96aed573f61bbd67c5f84 \ + --hash=sha256:3d92613ec25e43b05f042302531ec0f00b8445190e43325880cbd6ab7c2581da \ + --hash=sha256:40a126142a56b2dfc0aacbad1de8310cbf60da7656db0e6b16eebd48e3e93519 \ + --hash=sha256:416c229f77e5ea25b3dfd4b582f8d73d7e43c22320302b9ab128a2d3a0b38efe \ + --hash=sha256:432786d3561e69aeeae6c7e8648964ce0ad05736120135601f87ac26b9c83381 \ + --hash=sha256:43b9e366a31fdd1c87d0eb08f579b4a82b723ea54338f040d6b4e518a026ea29 \ + --hash=sha256:440eede837960000d74978f0eba527be106b5b9aee0daf779d395276ed0b0614 \ + --hash=sha256:45b0cc4e3556cd875e09102988d1ab8356c998b596c9fced84547c8138b487a0 \ + --hash=sha256:476743fe6dfe14a2da12e3ac79125dc84a3b2cf8094369a47a1529b0cd8549fe \ + --hash=sha256:4773092f8019072343a7447203308b176e10199920eb02d6195e81bbb3274c29 \ + --hash=sha256:4b3dac63058cc36820b0dd072f89898604e2d39686fe05321729d00d8ac185a0 \ + --hash=sha256:4d1c96a7a18b9690a4d46df09e3e3382406ae3213727cd1019ebade1c4a81917 \ + --hash=sha256:51307f5c71007673a2bf8232ad973483d281e74cb99c8c5a990af1eefa6277d9 \ + --hash=sha256:51447e9aa2684679af07ca5021c3db526e0284347ebf4ffcec1154c3350cfe32 \ + --hash=sha256:58150c9f9b9a552505912d182ccdf26f6396fb6094816ceebcbb20eecabaed94 \ + --hash=sha256:5b10cd92fc5c498b35a8635df6d5a100207f88b63a4dc1de7ef9a548e1e2cd63 \ + --hash=sha256:5e226f6218febc71f6c1fc2fafb91c226f75bdc1d8fb12d66823716e891608fd \ + --hash=sha256:609b3ba8fcc0fb5ab7af00719d0fb6ad0cb518e48e7712d12fd68f1327951198 \ + --hash=sha256:60f44ade2cf573dad7a277e6f8ca9a51a21dda572b13bd7d8539bb3cd5dbedde \ + --hash=sha256:611057cc5d5c0afc743ba8be6bd828c17e0aaa8643f9d0a9b9bb7dea80eb8012 \ + --hash=sha256:6366a16e1a25018694d6a5d784d09b046edc9eac40ea2b54065c3052672516a1 \ + --hash=sha256:65a7ff3f705e57d392f7261b6d0550fe137c3019477431f1c355e0db0a7d3e15 \ + --hash=sha256:673611bbd43f0810bec0b0f028ddeaaa501190339cac411f347ac76917c3ae7b \ + --hash=sha256:67830fc78e67501f47bb950471b2dcb9b35b140084429318e862895a8e89c993 \ + --hash=sha256:68ce9f4d6b26d5ccbf7fd4459bf75f74a0a146677ebba80597df60cbdb20e6f4 \ + --hash=sha256:68e5f26a1ad57ded6d1cfb85331d1c1a195314756471d97758c48498bb4dcdf5 \ + --hash=sha256:69b157c5d3292bcd443faca052f3096f637f1e074b98212a933c074ae23dc3b8 \ + --hash=sha256:75286256590a6320cf106a0d28970d3560aad9ee09aa7b34fb40524792436d35 \ + --hash=sha256:78841cccf1af7b40f6f716338d50c0902dbe88d9f800b3c973b7a9a0a693a642 \ + --hash=sha256:78fa18e436a1a0e58dbd7e02fc4473f3f32cceb12df9dfca542d075961c307d2 \ + --hash=sha256:79580094b00d1789d1f93ea55bc43cb2f611910c72235b7657f3482ddcc1b22d \ + --hash=sha256:7b86a2b16095d250c6f58b3d9b2eee6f4147754344f3dab0922f7c9bf7d226c9 \ + --hash=sha256:83aed2c10721ddd90f68140685391b50811a880af20654c59af6b6c66c40513c \ + --hash=sha256:84fd18bcc17526fc2b3c1af7d2b9217d32c9c04448c16ec693b9b4f1985c3d33 \ + --hash=sha256:871ff67ea1aad4dfd91736464934d56b32dac49f9fbe16cddba36198a7b3a0db \ + --hash=sha256:898f0e9068ca27d37f8e83a5b962821df851532e6c4a7d615c1c033f9da6eedf \ + --hash=sha256:8a79d9f4d8001473a30c163556b3c3bfebec837495a412dde78b51672f6134f9 \ + --hash=sha256:8c041122946b7ba21bb32c45b1aa57b1be35527690aeb3c5c234521085632eee \ + --hash=sha256:90c44bc373b7687f6948b693cceaea1348ae0975d7474746559494468e3c1d84 \ + --hash=sha256:9104ed0bd76a429d46f9ec0dbc9b08ad1d2dcdf2b00a5a0daa1c145329b35b44 \ + --hash=sha256:920079c3f7456fa213e0829ed2073aaa727fd39d889ead5b4f35d0de5460d04f \ + --hash=sha256:93d59d504b230e83c7a843251681959a0b6a9cd76f6e146ce1b8a80eb8739af9 \ + --hash=sha256:9b2aff1c7b3884512b9512c3eaadd9bab39fb45042ffaaa1dd08ff2b9f8109d9 \ + --hash=sha256:9b8e0f3107e2200b76f6054de99016eac3ee6762713587b36baaa7e4bd2ae177 \ + --hash=sha256:9bb41182d93ea91f60b4bc8fbf4c820c69ef8a12ab2d917f3f1834f1acad07e8 \ + --hash=sha256:9cdef90ae47919cae358d8ab15797a800ed41da7aba5d72419fb510729e2ed4b \ + --hash=sha256:a1786910334ed46ab1dd73222f2cd1e05c2c3bb39f6dddb4f8b36fc382058a39 \ + --hash=sha256:a4cfde78a9f2880208d16a93b795726a3017d5977e08d1e162a7a31322479c41 \ + --hash=sha256:a4fbdde9dd4a9ce5fd52c2b3a347bb50cc89483ef783f1cb00d408c13f7a96c0 \ + --hash=sha256:aa99adc8f081b475a12843953db36831eaf83ec33eb46a90629ca6a5de45a616 \ + --hash=sha256:ac351b3b8014eead140e77e9717e2992c6bbe30b63bc3422422eb84865412e3d \ + --hash=sha256:ad41ba96094304aa090f5a30cb6e4fb3b3f1c264c523394b4c39bbacc4dc92ba \ + --hash=sha256:b5314963fce9b0b12743891de876e724997864ee22aa496f903f426c7e2fa5b2 \ + --hash=sha256:bcf74c1df76758a395bf0af608c04c82257523f55c9868b334f06270d0f2112b \ + --hash=sha256:bd47ba7fc3ca94896759ea0109775132d3e7ab921fbf54038e1bab2e46c313c9 \ + --hash=sha256:c0323c9daef75ef2e5083624b4585018a0c9d5e3b40f607eed81a311270b934b \ + --hash=sha256:c1225416b463483160e4af85d5fc3a9690ccb53fd4b1865a6437825f5ede3209 \ + --hash=sha256:c1c948747b03be832dceed96ca815cef7360de9aa19d37c730f8e3f6101aca48 \ + --hash=sha256:c25fe15c70c59eb7c5ce8c06a1f3fa1da0ecc5ea1e7a5922c40fd2fa9b0d5046 \ + --hash=sha256:cc1b0fff8ead343dae06305f954eb8468ba0ec1a97881f42489d198e4ce3c632 \ + --hash=sha256:cd6280cf040f233bd7d3407b743b4b4c74f70e8e1c4199cb112a62c941c0772a \ + --hash=sha256:cd6c3d4b783c556fa00bf540854e42f135e2f256abd29669fcd0da0f2dec79c2 \ + --hash=sha256:d4d6fcde76f94f5cb9e43e9e9a61f16dacefd228cbbf6f1a09bd9b219a92f1a1 \ + --hash=sha256:ddf4af30b417d9fe16481e9b81c27ab2a7cde1ff7ba3e85653b02db7d145dc7b \ + --hash=sha256:df115d4d83168fdf2cae48ef1ff6d1cb4c466364e30861b37121de0f3bf1b990 \ + --hash=sha256:df7276909358e5635ae203673ab7e509ddd224225a8d6b0790bf13eb2bde1cc5 \ + --hash=sha256:e4fd89cc178bced6ad29cb3e6dd4aa63fa5017c3524dbd0b25998fb64a87cc8b \ + --hash=sha256:e9701d0049d92c16703a42771b98d560b95248949f23f8cf7b4eddd201814fb9 \ + --hash=sha256:ee2f2a527e3c1a6e6411eb4209642e138b544a2d72fe5d0d76daf77b24063534 \ + --hash=sha256:f7fb7d750cfa0a070d2c24e831fd3481019a60dd317ea2b39acbcebc08b6ed81 \ + --hash=sha256:f840ed6d8ecba8255df8c42b87fadeda98ddfc6eeec05e2dc66e26d46dd6f58a \ + --hash=sha256:f86c6358749bd4fda175388691e3ba8c46e24c5347d0afd20f9b7edfc9faf07d \ + --hash=sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf \ + --hash=sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115 # via requests -docutils==0.21.2 \ - --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ - --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2 +docutils==0.23 \ + --hash=sha256:25d013af9bf23bc1c7b2b093dff4208166c53a94786c9e447808335ef1185fea \ + --hash=sha256:746f5060322511280a1e50eb76846ed6bf2342984b2ac04dc42caa1a8d78799e # via readme-renderer -idna==3.10 \ - --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ - --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3 +id==1.6.1 \ + --hash=sha256:d0732d624fb46fd4e7bc4e5152f00214450953b9e772c182c1c22964def1a069 \ + --hash=sha256:f5ec41ed2629a508f5d0988eda142e190c9c6da971100612c4de9ad9f9b237ca + # via twine +idna==3.18 \ + --hash=sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2 \ + --hash=sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848 # via requests -importlib-metadata==8.5.0 \ - --hash=sha256:45e54197d28b7a7f1559e60b95e7c567032b602131fbd588f1497f47880aa68b \ - --hash=sha256:71522656f0abace1d072b9e5481a48f07c138e00f079c38c8f883823f9c26bd7 - # via - # keyring - # twine +importlib-metadata==9.0.0 \ + --hash=sha256:2d21d1cc5a017bd0559e36150c21c830ab1dc304dedd1b7ea85d20f45ef3edd7 \ + --hash=sha256:a4f57ab599e6a2e3016d7595cfd72eb4661a5106e787a95bcc90c7105b831efc + # via keyring jaraco-classes==3.4.0 \ --hash=sha256:47a024b51d0239c0dd8c8540c6c7f484be3b8fcf0b2d85c13825780d3b3f3acd \ --hash=sha256:f662826b6bed8cace05e7ff873ce0f9283b5c924470fe664fff1c2f00f581790 # via keyring -jaraco-context==6.0.1 \ - --hash=sha256:9bae4ea555cf0b14938dc0aee7c9f32ed303aa20a3b73e7dc80111628792d1b3 \ - --hash=sha256:f797fc481b490edb305122c9181830a3a5b76d84ef6d1aef2fb9b47ab956f9e4 +jaraco-context==6.1.2 \ + --hash=sha256:bf8150b79a2d5d91ae48629d8b427a8f7ba0e1097dd6202a9059f29a36379535 \ + --hash=sha256:f1a6c9d391e661cc5b8d39861ff077a7dc24dc23833ccee564b234b81c82dfe3 # via keyring -jaraco-functools==4.1.0 \ - --hash=sha256:70f7e0e2ae076498e212562325e805204fc092d7b4c17e0e86c959e249701a9d \ - --hash=sha256:ad159f13428bc4acbf5541ad6dec511f91573b90fba04df61dafa2a1231cf649 +jaraco-functools==4.6.0 \ + --hash=sha256:880c577ec9720b3a052d5bc611fb9f2269b3d87902ef42440df443b88e443280 \ + --hash=sha256:99e3dc0060c5cbe8fcd1cdb36258e2a65ca40f1566b2033b12abb1bb44dd3c30 # via keyring -keyring==25.4.1 \ - --hash=sha256:5426f817cf7f6f007ba5ec722b1bcad95a75b27d780343772ad76b17cb47b0bf \ - --hash=sha256:b07ebc55f3e8ed86ac81dd31ef14e81ace9dd9c3d4b5d77a6e9a2016d0d71a1b +keyring==25.7.0 \ + --hash=sha256:be4a0b195f149690c166e850609a477c532ddbfbaed96a404d4e43f8d5e2689f \ + --hash=sha256:fe01bd85eb3f8fb3dd0405defdeac9a5b4f6f0439edbb3149577f244a2e8245b # via twine -markdown-it-py==3.0.0 \ - --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ - --hash=sha256:e3f60a94fa066dc52ec76661e37c851cb232d92f9886b15cb560aaada2df8feb +markdown-it-py==4.2.0 \ + --hash=sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49 \ + --hash=sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a # via rich mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba # via markdown-it-py -more-itertools==10.5.0 \ - --hash=sha256:037b0d3203ce90cca8ab1defbbdac29d5f993fc20131f3664dc8d6acfa872aef \ - --hash=sha256:5482bfef7849c25dc3c6dd53a6173ae4795da2a41a80faea6700d9f5846c5da6 +more-itertools==11.1.0 \ + --hash=sha256:48e8f4d9e7e5878571ecf6f2b4e57634f93cd474cc8cfbd2376f2d11b396e30d \ + --hash=sha256:4b65538ae22f6fed0ce4874efd317463a7489796a0939fa66824dd542125a192 # via # jaraco-classes # jaraco-functools -nh3==0.2.18 \ - --hash=sha256:0411beb0589eacb6734f28d5497ca2ed379eafab8ad8c84b31bb5c34072b7164 \ - --hash=sha256:14c5a72e9fe82aea5fe3072116ad4661af5cf8e8ff8fc5ad3450f123e4925e86 \ - --hash=sha256:19aaba96e0f795bd0a6c56291495ff59364f4300d4a39b29a0abc9cb3774a84b \ - --hash=sha256:34c03fa78e328c691f982b7c03d4423bdfd7da69cd707fe572f544cf74ac23ad \ - --hash=sha256:36c95d4b70530b320b365659bb5034341316e6a9b30f0b25fa9c9eff4c27a204 \ - --hash=sha256:3a157ab149e591bb638a55c8c6bcb8cdb559c8b12c13a8affaba6cedfe51713a \ - --hash=sha256:42c64511469005058cd17cc1537578eac40ae9f7200bedcfd1fc1a05f4f8c200 \ - --hash=sha256:5f36b271dae35c465ef5e9090e1fdaba4a60a56f0bb0ba03e0932a66f28b9189 \ - --hash=sha256:6955369e4d9f48f41e3f238a9e60f9410645db7e07435e62c6a9ea6135a4907f \ - --hash=sha256:7b7c2a3c9eb1a827d42539aa64091640bd275b81e097cd1d8d82ef91ffa2e811 \ - --hash=sha256:8ce0f819d2f1933953fca255db2471ad58184a60508f03e6285e5114b6254844 \ - --hash=sha256:94a166927e53972a9698af9542ace4e38b9de50c34352b962f4d9a7d4c927af4 \ - --hash=sha256:a7f1b5b2c15866f2db413a3649a8fe4fd7b428ae58be2c0f6bca5eefd53ca2be \ - --hash=sha256:c8b3a1cebcba9b3669ed1a84cc65bf005728d2f0bc1ed2a6594a992e817f3a50 \ - --hash=sha256:de3ceed6e661954871d6cd78b410213bdcb136f79aafe22aa7182e028b8c7307 \ - --hash=sha256:f0eca9ca8628dbb4e916ae2491d72957fdd35f7a5d326b7032a345f111ac07fe +nh3==0.3.6 \ + --hash=sha256:082675ff87b9385ec430ffe6d5847ba7456cc39b73720cd4add472f9f4cffd56 \ + --hash=sha256:2411e8c3cee81a1ddd62c2a5d50585c28aa5566d373ad1db92536b95ddb24ef2 \ + --hash=sha256:25c733bee928530556b1db0ea46c52cf5aa686146e38e60a6fc7cb801ef91cec \ + --hash=sha256:2f90d9a0cfdbee218994fdaaeeb5a0fde62d08f35e4eef0378ec1e2200172fd0 \ + --hash=sha256:34d2b0d934156b87ee114f599a3ba9b8b9e17b5d79652ba3a13fa50903de965e \ + --hash=sha256:36d06341bd501240d320f5942481ed5e6846136b666e1ba4faf802b78ebc875f \ + --hash=sha256:43bc1ed3fa0716295fabee29ba42b2667e4a51d140b0a68e092170a765474fa6 \ + --hash=sha256:44673b27010051ab5a5e438a86ec31bbda61d4a77d7e900af6b7be3037c1abae \ + --hash=sha256:455469a29951edc92bc48b47ac2281c3f2609e6c4f6a047056449f8c2c23facf \ + --hash=sha256:4713502748f564fee0633b37b3403783ce0a3af3a3d148ad91025a5bdadb7bc6 \ + --hash=sha256:5276ef17bdba9ad8040575c74072008b13aae429436e9d0429e718bb5f90f4da \ + --hash=sha256:597a8e843bea00b2eb5520658dc24a9bb032e7fc9e7c2c0c4cd29420220c9796 \ + --hash=sha256:69bbb92865a693d909db3a700d3c01537533844d0948c1e9323561ce06ecda41 \ + --hash=sha256:69f365963f63a1e9bff53bdbb3c542c7c2efed3e163c9d5d83a772a2ac468c21 \ + --hash=sha256:82ca5bf427ad1b216b65ede1a2e2d87dc49bec417ceba0f297213107d3cd9d78 \ + --hash=sha256:889932a97fb4abb6f95fef1914c0d269ebfb60011e67121c1163059b9449dbb4 \ + --hash=sha256:905f877dc66dd7aea4a76e54bcb26acb5ff8216f720c0017ccf63e0e6035698e \ + --hash=sha256:a43ebd7543555c3ac1bc353023d0794e75cb76f6f18f19c32e95441496c0cc25 \ + --hash=sha256:d14bf7982e7a77c0c775634c29c07ce08b38a046df73e1c1f139b3e82f18a38e \ + --hash=sha256:e196fa70c2ff2eb4de7d3df3108f8f358c1d69dff20d45b11f20a5aa227ffb6d \ + --hash=sha256:e1b160831c9cdb06a6c79c2f9cdb11386602938f9af260d1c457a85add4f6f69 \ + --hash=sha256:e6b7beece07525dc6e6b0fc2f104442de2ba328360ad00e50cbe2e1fd620447d \ + --hash=sha256:edb2b4a1a27523e6cc7c417f8d21ce3d005243548b93e56b762b66b0c7f589f9 \ + --hash=sha256:f2f14b7ae1fca99c4a66c981aac3974e7fbc1ca30a12673d223ae1df76680917 \ + --hash=sha256:f338ac7d594c067679f1e99b4f5ec3906842979560f9d8f15d6bdfa39a353b10 \ + --hash=sha256:f3736c9dd3d1856f80cd031715b84ca75cda2bbb1ac802c3da26bfce590838d7 \ + --hash=sha256:f5ed5fe84aee7f39db95c214a7421bf0499fbf500fec6d86a4e29bfc37971438 # via readme-renderer -pkginfo==1.10.0 \ - --hash=sha256:5df73835398d10db79f8eecd5cd86b1f6d29317589ea70796994d49399af6297 \ - --hash=sha256:889a6da2ed7ffc58ab5b900d888ddce90bce912f2d2de1dc1c26f4cb9fe65097 +packaging==26.2 \ + --hash=sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e \ + --hash=sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661 # via twine -pygments==2.18.0 \ - --hash=sha256:786ff802f32e91311bff3889f6e9a86e81505fe99f2735bb6d60ae0c5004f199 \ - --hash=sha256:b8e6aca0523f3ab76fee51799c488e38782ac06eafcf95e7ba832985c8e7b13a +pygments==2.20.0 \ + --hash=sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f \ + --hash=sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176 # via # readme-renderer # rich @@ -193,13 +194,13 @@ pywin32-ctypes==0.2.3 \ --hash=sha256:8a1513379d709975552d202d942d9837758905c8d01eb82b8bcc30918929e7b8 \ --hash=sha256:d162dc04946d704503b2edc4d55f3dba5c1d539ead017afa00142c38b9885755 # via keyring -readme-renderer==44.0 \ - --hash=sha256:2fbca89b81a08526aadf1357a8c2ae889ec05fb03f5da67f9769c9a592166151 \ - --hash=sha256:8712034eabbfa6805cacf1402b4eeb2a73028f72d1166d6f5cb7f9c047c5d1e1 +readme-renderer==45.0 \ + --hash=sha256:030a8fac74904f8fba11ad1bb6964e3f76e896dc7e5e71f16af190c9056696d1 \ + --hash=sha256:3385ed220117104a2bceb4a9dac8c5fdf6d1f96890d7ea2a9c7174fd5c84091f # via twine -requests==2.32.3 \ - --hash=sha256:55365417734eb18255590a9ff9eb97e9e1da868d4ccd6402399eaf68af20a760 \ - --hash=sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6 +requests==2.34.2 \ + --hash=sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0 \ + --hash=sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed # via # requests-toolbelt # twine @@ -211,21 +212,22 @@ rfc3986==2.0.0 \ --hash=sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd \ --hash=sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c # via twine -rich==13.9.3 \ - --hash=sha256:9836f5096eb2172c9e77df411c1b009bace4193d6a481d534fea75ebba758283 \ - --hash=sha256:bc1e01b899537598cf02579d2b9f4a415104d3fc439313a7a2c165d76557a08e +rich==15.0.0 \ + --hash=sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb \ + --hash=sha256:edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36 # via twine -twine==5.1.1 \ - --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ - --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db +twine==7.0.0 \ + --hash=sha256:85cdb29c518efef867360ae4acd4b0dfd61c8654a22fca08e6f8539f05022177 \ + --hash=sha256:b854164df26db268af05f49aa5c0344b10e27a494343ff05b1e0bad3b135f5a7 # via -r tools/publish/requirements.in -urllib3==2.2.3 \ - --hash=sha256:ca899ca043dcb1bafa3e262d73aa25c465bfb49e0bd9dd5d59f1d0acba2f8fac \ - --hash=sha256:e7d814a81dad81e6caf2ec9fdedb284ecc9c73076b62654547cc64ccdcae26e9 +urllib3==2.7.0 \ + --hash=sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c \ + --hash=sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897 # via + # id # requests # twine -zipp==3.20.2 \ - --hash=sha256:a817ac80d6cf4b23bf7f2828b7cabf326f15a001bea8b1f9b49631780ba28350 \ - --hash=sha256:bc9eb26f4506fda01b81bcde0ca78103b6e62f991b381fec825435c836edbc29 +zipp==4.1.0 \ + --hash=sha256:25ad4e16390cd314347dd8f1de67a2ac538ae658ed4ab9db16029c07c188e97f \ + --hash=sha256:4cb57381f544315db7688e976e922a2b18cdb513d21cc194eb42232ba2a3e602 # via importlib-metadata diff --git a/tools/update_deleted_packages.sh b/tools/update_deleted_packages.sh index 17e33d182a..83bed91d16 100755 --- a/tools/update_deleted_packages.sh +++ b/tools/update_deleted_packages.sh @@ -23,17 +23,19 @@ # 2. For each of the directories, get all directories that contains a BUILD.bazel file. # 3. Sort and remove duplicates. -set -euxo pipefail +set -euo pipefail DIR="$(dirname $0)/.." cd $DIR # The sed -i.bak pattern is compatible between macos and linux -sed -i.bak "/^[^#].*--deleted_packages/s#=.*#=$(\ - find examples/*/* tests/*/* \( -name WORKSPACE -or -name MODULE.bazel \) | +{ + echo "# Generated via './tools/update_deleted_packages.sh'" + find examples tests gazelle \( -name WORKSPACE -or -name MODULE.bazel \) | xargs -n 1 dirname | - xargs -n 1 -I{} find {} \( -name BUILD -or -name BUILD.bazel \) | + xargs -I{} find {} \( -name BUILD -or -name BUILD.bazel \) | xargs -n 1 dirname | + grep -v "gazelle/docs" | sort -u | - paste -sd, -\ -)#" $DIR/.bazelrc && rm .bazelrc.bak + sed 's/^/common --deleted_packages=/g' +} | tee "$DIR"/.bazelrc.deleted_packages diff --git a/tools/wheelmaker.py b/tools/wheelmaker.py index 23b18eca5f..70e375b4ee 100644 --- a/tools/wheelmaker.py +++ b/tools/wheelmaker.py @@ -98,6 +98,34 @@ def normalize_pep440(version): return str(packaging.version.Version(f"0+{sanitized}")) +def arcname_from( + name: str, + distribution_prefix: str, + strip_path_prefixes: Sequence[str] = (), # noqa: F821 + add_path_prefix: str = "", +) -> str: + """Return the within-archive name for a given file path name. + + Prefixes to strip are checked in order and only the first match will be used. + + Args: + name: The file path eg 'mylib/a/b/c/file.py' + distribution_prefix: The + strip_path_prefixes: Remove these prefixes from names. + add_path_prefix: Add prefix after stripping the path from names. + """ + # Always use unix path separators. + normalized_arcname = name.replace(os.path.sep, "/") + # Don't manipulate names filenames in the .distinfo or .data directories. + if distribution_prefix and normalized_arcname.startswith(distribution_prefix): + return normalized_arcname + for prefix in strip_path_prefixes: + if normalized_arcname.startswith(prefix): + return add_path_prefix + normalized_arcname[len(prefix) :] + + return add_path_prefix + normalized_arcname + + class _WhlFile(zipfile.ZipFile): def __init__( self, @@ -106,14 +134,20 @@ def __init__( mode, distribution_prefix: str, strip_path_prefixes=None, + add_path_prefix=None, compression=zipfile.ZIP_DEFLATED, + quote_all_filenames: bool = False, **kwargs, ): self._distribution_prefix = distribution_prefix self._strip_path_prefixes = strip_path_prefixes or [] - # Entries for the RECORD file as (filename, hash, size) tuples. - self._record = [] + self._add_path_prefix = add_path_prefix or "" + # Entries for the RECORD file as (filename, digest, size) tuples. + self._record: list[tuple[str, str, str]] = [] + # Whether to quote filenames in the RECORD file (for compatibility with + # some wheels like torch that have quoted filenames in their RECORD). + self.quote_all_filenames = quote_all_filenames super().__init__(filename, mode=mode, compression=compression, **kwargs) @@ -126,20 +160,8 @@ def data_path(self, basename): def add_file(self, package_filename, real_filename): """Add given file to the distribution.""" - def arcname_from(name): - # Always use unix path separators. - normalized_arcname = name.replace(os.path.sep, "/") - # Don't manipulate names filenames in the .distinfo or .data directories. - if normalized_arcname.startswith(self._distribution_prefix): - return normalized_arcname - for prefix in self._strip_path_prefixes: - if normalized_arcname.startswith(prefix): - return normalized_arcname[len(prefix) :] - - return normalized_arcname - if os.path.isdir(real_filename): - directory_contents = os.listdir(real_filename) + directory_contents = sorted(os.listdir(real_filename)) for file_ in directory_contents: self.add_file( "{}/{}".format(package_filename, file_), @@ -147,14 +169,19 @@ def arcname_from(name): ) return - arcname = arcname_from(package_filename) + arcname = arcname_from( + package_filename, + distribution_prefix=self._distribution_prefix, + strip_path_prefixes=self._strip_path_prefixes, + add_path_prefix=self._add_path_prefix, + ) zinfo = self._zipinfo(arcname) # Write file to the zip archive while computing the hash and length hash = hashlib.sha256() size = 0 with open(real_filename, "rb") as fsrc: - with self.open(zinfo, "w") as fdst: + with self.open(zinfo, "w", force_zip64=True) as fdst: while True: block = fsrc.read(2**20) if not block: @@ -175,16 +202,15 @@ def add_string(self, filename, contents): hash.update(contents) self._add_to_record(filename, self._serialize_digest(hash), len(contents)) - def _serialize_digest(self, hash): + def _serialize_digest(self, hash) -> str: # https://www.python.org/dev/peps/pep-0376/#record # "base64.urlsafe_b64encode(digest) with trailing = removed" digest = base64.urlsafe_b64encode(hash.digest()) digest = b"sha256=" + digest.rstrip(b"=") - return digest + return digest.decode("utf-8", "surrogateescape") - def _add_to_record(self, filename, hash, size): - size = str(size).encode("ascii") - self._record.append((filename, hash, size)) + def _add_to_record(self, filename: str, hash: str, size: int) -> None: + self._record.append((filename, hash, str(size))) def _zipinfo(self, filename): """Construct deterministic ZipInfo entry for a file named filename""" @@ -206,27 +232,27 @@ def _zipinfo(self, filename): zinfo.compress_type = self.compression return zinfo - def add_recordfile(self): + def _quote_filename(self, filename: str) -> str: + """Return a possibly quoted filename for RECORD file.""" + filename = filename.lstrip("/") + # Some RECORDs like torch have *all* filenames quoted and we must minimize diff. + # Otherwise, we quote only when necessary (e.g. for filenames with commas). + quoting = csv.QUOTE_ALL if self.quote_all_filenames else csv.QUOTE_MINIMAL + with io.StringIO() as buf: + csv.writer(buf, quoting=quoting).writerow([filename]) + return buf.getvalue().strip() + + def add_recordfile(self) -> str: """Write RECORD file to the distribution.""" record_path = self.distinfo_path("RECORD") - entries = self._record + [(record_path, b"", b"")] - with io.StringIO() as contents_io: - writer = csv.writer(contents_io, lineterminator="\n") - for filename, digest, size in entries: - if isinstance(filename, str): - filename = filename.lstrip("/") - writer.writerow( - ( - c - if isinstance(c, str) - else c.decode("utf-8", "surrogateescape") - for c in (filename, digest, size) - ) - ) - - contents = contents_io.getvalue() - self.add_string(record_path, contents) - return contents.encode("utf-8", "surrogateescape") + entries = self._record + [(record_path, "", "")] + entries = [ + (self._quote_filename(fname), digest, size) + for fname, digest, size in entries + ] + contents = "\n".join(",".join(entry) for entry in entries) + "\n" + self.add_string(record_path, contents) + return contents class WheelMaker(object): @@ -241,6 +267,7 @@ def __init__( compress, outfile=None, strip_path_prefixes=None, + add_path_prefix=None, ): self._name = name self._version = normalize_pep440(version) @@ -250,6 +277,7 @@ def __init__( self._platform = platform self._outfile = outfile self._strip_path_prefixes = strip_path_prefixes + self._add_path_prefix = add_path_prefix self._compress = compress self._wheelname_fragment_distribution_name = escape_filename_distribution_name( self._name @@ -267,7 +295,10 @@ def __enter__(self): mode="w", distribution_prefix=self._distribution_prefix, strip_path_prefixes=self._strip_path_prefixes, - compression=zipfile.ZIP_DEFLATED if self._compress else zipfile.ZIP_STORED, + add_path_prefix=self._add_path_prefix, + compression=( + zipfile.ZIP_DEFLATED if self._compress else zipfile.ZIP_STORED + ), ) return self @@ -310,9 +341,7 @@ def add_wheelfile(self): Wheel-Version: 1.0 Generator: bazel-wheelmaker 1.0 Root-Is-Purelib: {} -""".format( - "true" if self._platform == "any" else "false" - ) +""".format("true" if self._platform == "any" else "false") for tag in self.disttags(): wheel_contents += "Tag: %s\n" % tag self._whlfile.add_string(self.distinfo_path("WHEEL"), wheel_contents) @@ -345,6 +374,35 @@ def get_files_to_package(input_files): return files +def get_new_requirement_line(reqs_text: str, extra: str) -> str: + """Formats a requirement text into a Requires-Dist metadata line.""" + # This is not imported at the top of the file due to the reliance + # on this file in the `whl_library` repository rule which does not + # provide `packaging` but does import symbols defined here. + from packaging.requirements import Requirement + + req = Requirement(reqs_text.strip()) + req_extra_deps = f"[{','.join(req.extras)}]" if req.extras else "" + + # Handle URL requirements (PEP 508) + if req.url: + req_spec = f" @ {req.url}" + else: + req_spec = str(req.specifier) + + base = f"Requires-Dist: {req.name}{req_extra_deps}{req_spec}" + + if req.marker: + if extra: + return f"{base}; ({req.marker}) and {extra}" + else: + return f"{base}; {req.marker}" + elif extra: + return f"{base}; {extra}" + else: + return base + + def resolve_argument_stamp( argument: str, volatile_status_stamp: Path, stable_status_stamp: Path ) -> str: @@ -410,7 +468,7 @@ def parse_args() -> argparse.Namespace: output_group.add_argument( "--name_file", type=Path, - help="A file where the canonical name of the " "wheel will be written", + help="A file where the canonical name of the wheel will be written", ) output_group.add_argument( @@ -421,6 +479,13 @@ def parse_args() -> argparse.Namespace: help="Path prefix to be stripped from input package files' path. " "Can be supplied multiple times. Evaluated in order.", ) + output_group.add_argument( + "--path_prefix", + type=str, + default="", + help="Path prefix to be prepended to input package files' path. " + "It is prepended after stripping any specified path prefixes first.", + ) wheel_group = parser.add_argument_group("Wheel metadata") wheel_group.add_argument( @@ -433,7 +498,8 @@ def parse_args() -> argparse.Namespace: "--description_file", help="Path to the file with package description" ) wheel_group.add_argument( - "--description_content_type", help="Content type of the package description" + "--description_content_type", + help="Content type of the package description", ) wheel_group.add_argument( "--entry_points_file", @@ -482,7 +548,7 @@ def parse_args() -> argparse.Namespace: return parser.parse_args(sys.argv[1:]) -def _parse_file_pairs(content: List[str]) -> List[List[str]]: +def _parse_file_pairs(content: List[str]) -> List[List[str]]: # noqa: F821 """ Parse ; delimited lists of files into a 2D list. """ @@ -535,6 +601,7 @@ def main() -> None: platform=arguments.platform, outfile=arguments.out, strip_path_prefixes=strip_prefixes, + add_path_prefix=arguments.path_prefix, compress=not arguments.no_compress, ) as maker: for package_filename, real_filename in all_files: @@ -550,24 +617,9 @@ def main() -> None: metadata = arguments.metadata_file.read_text(encoding="utf-8") - # This is not imported at the top of the file due to the reliance - # on this file in the `whl_library` repository rule which does not - # provide `packaging` but does import symbols defined here. - from packaging.requirements import Requirement - # Search for any `Requires-Dist` entries that refer to other files and # expand them. - def get_new_requirement_line(reqs_text, extra): - req = Requirement(reqs_text.strip()) - if req.marker: - if extra: - return f"Requires-Dist: {req.name}{req.specifier}; ({req.marker}) and {extra}" - else: - return f"Requires-Dist: {req.name}{req.specifier}; {req.marker}" - else: - return f"Requires-Dist: {req.name}{req.specifier}; {extra}".strip(" ;") - for meta_line in metadata.splitlines(): if not meta_line.startswith("Requires-Dist: "): continue @@ -599,7 +651,14 @@ def get_new_requirement_line(reqs_text, extra): reqs.append(get_new_requirement_line(reqs_text, extra)) - metadata = metadata.replace(meta_line, "\n".join(reqs)) + if reqs: + metadata = metadata.replace(meta_line, "\n".join(reqs)) + # File is empty + # So replace the meta_line entirely, including removing newline chars + else: + metadata = re.sub( + re.escape(meta_line) + r"(?:\r?\n)?", "", metadata, count=1 + ) maker.add_metadata( metadata=metadata, @@ -609,7 +668,8 @@ def get_new_requirement_line(reqs_text, extra): if arguments.entry_points_file: maker.add_file( - maker.distinfo_path("entry_points.txt"), arguments.entry_points_file + maker.distinfo_path("entry_points.txt"), + arguments.entry_points_file, ) # Sort the files for reproducible order in the archive. diff --git a/version.bzl b/version.bzl index 61fb81efd4..4d85b5c420 100644 --- a/version.bzl +++ b/version.bzl @@ -17,11 +17,11 @@ # against. # This version should be updated together with the version of Bazel # in .bazelversion. -BAZEL_VERSION = "7.x" +BAZEL_VERSION = "8.x" # NOTE: Keep in sync with .bazelci/presubmit.yml # This is the minimum supported bazel version, that we have some tests for. -MINIMUM_BAZEL_VERSION = "6.4.0" +MINIMUM_BAZEL_VERSION = "7.4.1" # Versions of Bazel which users should be able to use. # Ensures we don't break backwards-compatibility, diff --git a/workspace_bazel9.bzl b/workspace_bazel9.bzl new file mode 100644 index 0000000000..dad59378a9 --- /dev/null +++ b/workspace_bazel9.bzl @@ -0,0 +1,10 @@ +"""Workaround for Bazel 9 duplicate name issue in Gazelle.""" + +def bazel_9_workaround(name = "bazel_9_workaround"): + # Necessary so that Bazel 9 recognizes this as rules_python and doesn't try + # to load the version Bazel itself uses by default. + # We hide this from Gazelle's WORKSPACE parser by putting it in a macro. + native.local_repository( + name = "rules_python", + path = ".", + )