Improve performance in _manylinux.py by cthoyt · Pull Request #869 · pypa/packaging

cthoyt · 2025-01-31T11:40:03Z

This PR makes a few performance improvements:

Reduces the doubled call to _is_compatible(arch, glibc_version)
Moves the assignment of the tag variable directly into the yield, reducing memory allocation in case this is never used when _is_compatible(arch, glibc_version) is false
Uses an assignment expression for dictionary lookup

uranusjr

Nice

cthoyt · 2025-05-19T07:04:12Z

@pradyunsg thanks for the review, do you know the best way to advocate to get this PR merged that is respectful of the time/notification inbox of the maintainers?

cthoyt · 2025-05-19T07:07:29Z

-                if glibc_version in _LEGACY_MANYLINUX_MAP:
-                    legacy_tag = _LEGACY_MANYLINUX_MAP[glibc_version]
-                    if _is_compatible(arch, glibc_version):
+                    yield "manylinux_{}_{}_{}".format(*glibc_version, arch)


this replaces the combination of tag = "manylinux_{}_{}".format(*glibc_version) and yield f"{tag}_{arch}" since these two string format operations are done in succession (without, imo, the intermediate name adding any useful context nor being reused anywhere)

cthoyt · 2025-05-19T07:08:01Z

-                # Handle the legacy manylinux1, manylinux2010, manylinux2014 tags.
-                if glibc_version in _LEGACY_MANYLINUX_MAP:
-                    legacy_tag = _LEGACY_MANYLINUX_MAP[glibc_version]
-                    if _is_compatible(arch, glibc_version):


the doubled call to _is_compatible is combine

henryiii · 2025-05-22T19:59:52Z

Thanks!

## Summary This PR makes a few performance improvements: 1. Reduces the need to unpack and repack a `_GLibCVersion` tuple 2. Reduces the doubled call to `_is_compatible(arch, glibc_version)` 3. Moves the assignment of the `tag` variable directly into the yield, reducing memory allocation in case this is never used when `_is_compatible(arch, glibc_version)` is false 4. Combines the check of the `glibc_version` being in `_LEGACY_MANYLINUX_MAP` and the assignment to the variable together. I'm not sure if this is actually better, but using the assignment expression reduces this from 4 lines to 2 ## Test Plan I upstreamed these changes in pypa/packaging#869. Otherwise, I'm pretty confident this is a minor change that works the same

This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [packaging](https://github.com/pypa/packaging) | packages | major | `==25.0` → `==26.0` | --- ### Release Notes <details> <summary>pypa/packaging (packaging)</summary> ### [`v26.0`](https://github.com/pypa/packaging/releases/tag/26.0) [Compare Source](pypa/packaging@25.0...26.0) Read about the performance improvements here: <https://iscinumpy.dev/post/packaging-faster>. #### What's Changed Features: - PEP 751: support pylock by [@sbidoul](https://github.com/sbidoul) in [#900](pypa/packaging#900) - PEP 794: import name metadata by [@brettcannon](https://github.com/brettcannon) in [#948](pypa/packaging#948) - Support writing metadata by [@henryiii](https://github.com/henryiii) in [#846](pypa/packaging#846) - Support `__replace__` for `Version` by [@henryiii](https://github.com/henryiii) in [#1003](pypa/packaging#1003) - Support positional pattern matching for `Version` and `Specifier` by [@henryiii](https://github.com/henryiii) in [#1004](pypa/packaging#1004) Behavior adaptations: - PEP 440 handling of prereleases for `Specifier.contains`, `SpecifierSet.contains`, and `SpecifierSet.filter` by [@notatallshaw](https://github.com/notatallshaw) in [#897](pypa/packaging#897) - Handle PEP 440 edge case in `SpecifierSet.filter` by [@notatallshaw](https://github.com/notatallshaw) in [#942](pypa/packaging#942) - Adjust arbitrary equality intersection preservation in `SpecifierSet` by [@notatallshaw](https://github.com/notatallshaw) in [#951](pypa/packaging#951) - Return `False` instead of raising for `.contains` with invalid version by [@Liam-DeVoe](https://github.com/Liam-DeVoe) in [#932](pypa/packaging#932) - Support arbitrary equality on arbitrary strings for `Specifier` and `SpecifierSet`'s `filter` and `contains` method. by [@notatallshaw](https://github.com/notatallshaw) in [#954](pypa/packaging#954) - Only try to parse as `Version` on certain marker keys, return `False` on unequal ordered comparsions by [@JP-Ellis](https://github.com/JP-Ellis) in [#939](pypa/packaging#939) Fixes: - Update `_hash` when unpickling `Tag()` by [@dholth](https://github.com/dholth) in [#860](pypa/packaging#860) - Correct comment and simplify implicit prerelease handling in `Specifier.prereleases` by [@notatallshaw](https://github.com/notatallshaw) in [#896](pypa/packaging#896) - Use explicit `_GLibCVersion` `NamedTuple` in `_manylinux` by [@cthoyt](https://github.com/cthoyt) in [#868](pypa/packaging#868) - Detect invalid license expressions containing `()` by [@bwoodsend](https://github.com/bwoodsend) in [#879](pypa/packaging#879) - Correct regex for metadata `'name'` format by [@di](https://github.com/di) in [#925](pypa/packaging#925) - Improve the message around expecting a semicolon by [@pradyunsg](https://github.com/pradyunsg) in [#833](pypa/packaging#833) - Support nested parens in license expressions by [@Liam-DeVoe](https://github.com/Liam-DeVoe) in [#931](pypa/packaging#931) - Add space before at symbol in `Requirements` string by [@henryiii](https://github.com/henryiii) in [#953](pypa/packaging#953) - A root logger use found by ruff LOG, use `packaging` logger instead by [@henryiii](https://github.com/henryiii) in [#965](pypa/packaging#965) - Better support for subclassing `Marker` and `Requirement` by [@henryiii](https://github.com/henryiii) in [#1022](pypa/packaging#1022) - Normalize all extras, not just if it comes first by [@henryiii](https://github.com/henryiii) in [#1024](pypa/packaging#1024) - Don't produce a broken repr if `Marker` fails to construct by [@henryiii](https://github.com/henryiii) in [#1033](pypa/packaging#1033) Performance: - Avoid recompiling regexes in the tokenizer for a 3x speedup by [@hauntsaninja](https://github.com/hauntsaninja) in [#1019](pypa/packaging#1019) - Improve performance in `_manylinux.py` by [@cthoyt](https://github.com/cthoyt) in [#869](pypa/packaging#869) - Minor cleanups to `Version` by [@bearomorphism](https://github.com/bearomorphism) in [#913](pypa/packaging#913) - Skip redundant creation of `Version`s in specifier comparison by [@notatallshaw](https://github.com/notatallshaw) in [#986](pypa/packaging#986) - Cache `Specifier`'s Version by [@notatallshaw](https://github.com/notatallshaw) in [#985](pypa/packaging#985) - Make `Version` a little faster by [@henryiii](https://github.com/henryiii) in [#987](pypa/packaging#987) - Minor `Version` regex cleanup by [@henryiii](https://github.com/henryiii) in [#990](pypa/packaging#990) - Faster regex on Python 3.11.5+ by [@henryiii](https://github.com/henryiii) in [#988](pypa/packaging#988) and [#1055](pypa/packaging#1055) - Lazily calculate `_key` in `Version` by [@notatallshaw](https://github.com/notatallshaw) in [#989](pypa/packaging#989) and regression for `packaging_legacy` fixed by [@henryiii](https://github.com/henryiii) in [#1048](pypa/packaging#1048) - Faster `canonicalize_version` by [@henryiii](https://github.com/henryiii) in [#993](pypa/packaging#993) - Use `fullmatch` in a couple more places by [@henryiii](https://github.com/henryiii) in [#992](pypa/packaging#992) - Use `fullmatch` for markers too by [@henryiii](https://github.com/henryiii) in [#1029](pypa/packaging#1029) - Use `map` instead of generator by [@henryiii](https://github.com/henryiii) in [#996](pypa/packaging#996) - Deprecate `._version` (`_Version`, a `NamedTuple`) by [@henryiii](https://github.com/henryiii) in [#995](pypa/packaging#995) and [#1062](pypa/packaging#1062) - Avoid duplicate `Version` creation in `canonicalize_version` by [@henryiii](https://github.com/henryiii) in [#994](pypa/packaging#994) - Add `__slots__` to `Version` by [@henryiii](https://github.com/henryiii) in [#1001](pypa/packaging#1001) - Add `__slots__` to `Specifier`s by [@henryiii](https://github.com/henryiii) in [#1002](pypa/packaging#1002) - Add `__slots__` to `Node`s by [@henryiii](https://github.com/henryiii) in [#1032](pypa/packaging#1032) - Use `version.__replace__` in specifier comparison by [@notatallshaw](https://github.com/notatallshaw) in [#999](pypa/packaging#999) - Use `_get_spec_version` in more places in `Specifier` by [@notatallshaw](https://github.com/notatallshaw) in [#1005](pypa/packaging#1005) - Pull `set` construction out of function by [@henryiii](https://github.com/henryiii) in [#1012](pypa/packaging#1012) - Letter normalization dict for prereleases and the like by [@henryiii](https://github.com/henryiii) in [#1014](pypa/packaging#1014) - Avoid normalizing extras again when comparing by [@henryiii](https://github.com/henryiii) in [#1028](pypa/packaging#1028) - Speed up `Version.__str__` by about 10% by [@henryiii](https://github.com/henryiii) in [#997](pypa/packaging#997) - Increase the performance of `canonicalize_name` by avoiding a regex by [@henryiii](https://github.com/henryiii) in [#1030](pypa/packaging#1030), [#1047](pypa/packaging#1047), and [#1064](pypa/packaging#1064) - Faster zero stripping by [@henryiii](https://github.com/henryiii) in [#1058](pypa/packaging#1058) Type annotations: - Fix a type annotation by [@brettcannon](https://github.com/brettcannon) in [#907](pypa/packaging#907) - Fix tags return type in `parse_wheel_filename` docs by [@ncoghlan](https://github.com/ncoghlan) in [#973](pypa/packaging#973) - Add type hint for `_version` in `.version.Version` by [@brettcannon](https://github.com/brettcannon) in [#927](pypa/packaging#927) - Changed static type annotations in prereleases setter method in `specifier.py` by [@subhajitsaha01](https://github.com/subhajitsaha01) in [#930](pypa/packaging#930) - Statically type the tests by [@henryiii](https://github.com/henryiii) in [#982](pypa/packaging#982) Internal: - Test and declare support Python 3.14 by [@henryiii](https://github.com/henryiii) in [#901](pypa/packaging#901) - Modernize and speed up tests on Python 3.14 by [@henryiii](https://github.com/henryiii) in [#903](pypa/packaging#903) - Change our license metadata to use an SPDX license expression by [@cdce8p](https://github.com/cdce8p) in [#881](pypa/packaging#881) - No need for `license-files` by [@DimitriPapadopoulos](https://github.com/DimitriPapadopoulos) in [#924](pypa/packaging#924) - Update mypy by [@hauntsaninja](https://github.com/hauntsaninja) in [#891](pypa/packaging#891) - Some config updates by [@henryiii](https://github.com/henryiii) in [#902](pypa/packaging#902) - Add spell check and rst check by [@henryiii](https://github.com/henryiii) in [#904](pypa/packaging#904) - Clean up ruff ignores by [@henryiii](https://github.com/henryiii) in [#905](pypa/packaging#905) - Update example for env marker `python_version` by [@trim21](https://github.com/trim21) in [#908](pypa/packaging#908) - Move codespell configuration into pyproject.toml by [@yarikoptic](https://github.com/yarikoptic) in [#910](pypa/packaging#910) - Check warning a little more precisely by [@henryiii](https://github.com/henryiii) in [#837](pypa/packaging#837) - Speed up mypy a little by [@henryiii](https://github.com/henryiii) in [#836](pypa/packaging#836) - Apply ruff/flake8-pyi rules (PYI) by [@DimitriPapadopoulos](https://github.com/DimitriPapadopoulos) in [#835](pypa/packaging#835) - Better local runs for codespell by [@henryiii](https://github.com/henryiii) in [#911](pypa/packaging#911) - Remove outdated/confusing Gist link by [@stefan6419846](https://github.com/stefan6419846) in [#921](pypa/packaging#921) - Fix docs and docs ci after [#897](pypa/packaging#897) landed by [@notatallshaw](https://github.com/notatallshaw) in [#926](pypa/packaging#926) - Run twine-check on push in CI by [@EpicWink](https://github.com/EpicWink) in [#922](pypa/packaging#922) - `ruff` was renamed `ruff-check` in pre-commit by [@henryiii](https://github.com/henryiii) in [#933](pypa/packaging#933) - Fix incorrectly implicitly concatenated string in specifiers test by [@notatallshaw](https://github.com/notatallshaw) in [#946](pypa/packaging#946) - Simplify conditional by [@ofek](https://github.com/ofek) in [#949](pypa/packaging#949) - Modernize nox, use dependency-groups for tests by [@henryiii](https://github.com/henryiii) in [#952](pypa/packaging#952) - Add more checks that don't affect anything by [@henryiii](https://github.com/henryiii) in [#957](pypa/packaging#957) - Enable Ruff ISC rule by [@henryiii](https://github.com/henryiii) in [#959](pypa/packaging#959) - Ruff code FLY by [@henryiii](https://github.com/henryiii) in [#963](pypa/packaging#963) - pytest `log_level` is better than `log_cli_level` by [@henryiii](https://github.com/henryiii) in [#956](pypa/packaging#956) - Ruff code TRY by [@henryiii](https://github.com/henryiii) in [#961](pypa/packaging#961) - Add the ruff PL checks by [@henryiii](https://github.com/henryiii) in [#964](pypa/packaging#964) - Enable Ruff ARG rules by [@henryiii](https://github.com/henryiii) in [#958](pypa/packaging#958) - Ruff PT code (pytest) by [@henryiii](https://github.com/henryiii) in [#960](pypa/packaging#960) - Add ruff DTZ by [@henryiii](https://github.com/henryiii) in [#968](pypa/packaging#968) - Add ruff BLE by [@henryiii](https://github.com/henryiii) in [#967](pypa/packaging#967) - Add the ruff SIM checks by [@henryiii](https://github.com/henryiii) in [#966](pypa/packaging#966) - Adding ruff PERF by [@henryiii](https://github.com/henryiii) in [#969](pypa/packaging#969) - Move some config into coverage config by [@henryiii](https://github.com/henryiii) in [#971](pypa/packaging#971) - Check ruff C4 by [@henryiii](https://github.com/henryiii) in [#962](pypa/packaging#962) - Adding ruff T20 by [@henryiii](https://github.com/henryiii) in [#972](pypa/packaging#972) - Add a tests pass job by [@henryiii](https://github.com/henryiii) in [#977](pypa/packaging#977) - Add ruff TC by [@henryiii](https://github.com/henryiii) in [#980](pypa/packaging#980) - Adding part of ruff RET by [@henryiii](https://github.com/henryiii) in [#979](pypa/packaging#979) - Reorder mypy check by [@henryiii](https://github.com/henryiii) in [#983](pypa/packaging#983) - Enable ruff ALL by [@henryiii](https://github.com/henryiii) in [#984](pypa/packaging#984) - Link back to repo/source in furo by [@henryiii](https://github.com/henryiii) in [#991](pypa/packaging#991) - Add case insensitivity tests for arbitrary equality by [@notatallshaw](https://github.com/notatallshaw) in [#975](pypa/packaging#975) - Synchronize documentation and code for markers by [@zahlman](https://github.com/zahlman) in [#1008](pypa/packaging#1008) - Use `partition` in `_parse_project_urls` by [@henryiii](https://github.com/henryiii) in [#1013](pypa/packaging#1013) - auto-skip the dependabot PRs in the release changelog generation by [@henryiii](https://github.com/henryiii) in [#1016](pypa/packaging#1016) - Update unreleased section in changelog by [@henryiii](https://github.com/henryiii) in [#1017](pypa/packaging#1017) - Fix PR role to match extlinks by [@hugovk](https://github.com/hugovk) in [#1020](pypa/packaging#1020) - Mention new parts in README by [@henryiii](https://github.com/henryiii) in [#1023](pypa/packaging#1023) - Replace a couple of asserts with else by [@henryiii](https://github.com/henryiii) in [#1027](pypa/packaging#1027) - Simplify and/or check a little more by [@henryiii](https://github.com/henryiii) in [#1031](pypa/packaging#1031) - Use slim runner for all check by [@henryiii](https://github.com/henryiii) in [#1021](pypa/packaging#1021) - Use typos instead of codespell by [@henryiii](https://github.com/henryiii) in [#1015](pypa/packaging#1015) - Update changelog with recent additions by [@henryiii](https://github.com/henryiii) in [#1034](pypa/packaging#1034) - Publish to PyPI via GitHub CI by [@EpicWink](https://github.com/EpicWink) in [#893](pypa/packaging#893) - Use prek for faster pre-commit lint step by [@henryiii](https://github.com/henryiii) in [#1037](pypa/packaging#1037) - Add help text to noxfile by [@henryiii](https://github.com/henryiii) in [#1038](pypa/packaging#1038) - Update licenses to 3.27 by [@henryiii](https://github.com/henryiii) in [#1036](pypa/packaging#1036) - Use relative import in `packaging.licenses` by [@notatallshaw](https://github.com/notatallshaw) in [#1039](pypa/packaging#1039) - Add zizmor and tighten up CI by [@henryiii](https://github.com/henryiii) in [#1035](pypa/packaging#1035) - Fix release script by [@henryiii](https://github.com/henryiii) in [#1040](pypa/packaging#1040) - Fix using a dev version (again) by [@henryiii](https://github.com/henryiii) in [#1041](pypa/packaging#1041) - Fix type hint of function used with `contextlib.contextmanager` by [@SpecLad](https://github.com/SpecLad) in [#1046](pypa/packaging#1046) - Always run tests by [@henryiii](https://github.com/henryiii) in [#1044](pypa/packaging#1044) - Fix a changelog number by [@henryiii](https://github.com/henryiii) in [#1042](pypa/packaging#1042) - Fix the publish job by [@henryiii](https://github.com/henryiii) in [#1043](pypa/packaging#1043) - Get the correct tag on publish by [@henryiii](https://github.com/henryiii) in [#1045](pypa/packaging#1045) - Test on first public release of CPython 3.11 and newer by [@henryiii](https://github.com/henryiii) in [#1056](pypa/packaging#1056) - Fix publication job (again) by [@henryiii](https://github.com/henryiii) in [#1051](pypa/packaging#1051) - Use `actionlint` to check CI workflows by [@miketheman](https://github.com/miketheman) in [#1052](pypa/packaging#1052) - Fix formatting of distribution types in metadata.rst by [@brettcannon](https://github.com/brettcannon) in [#1053](pypa/packaging#1053) #### New Contributors - [@cdce8p](https://github.com/cdce8p) made their first contribution in [#881](pypa/packaging#881) - [@dholth](https://github.com/dholth) made their first contribution in [#860](pypa/packaging#860) - [@trim21](https://github.com/trim21) made their first contribution in [#908](pypa/packaging#908) - [@yarikoptic](https://github.com/yarikoptic) made their first contribution in [#910](pypa/packaging#910) - [@cthoyt](https://github.com/cthoyt) made their first contribution in [#868](pypa/packaging#868) - [@bwoodsend](https://github.com/bwoodsend) made their first contribution in [#879](pypa/packaging#879) - [@stefan6419846](https://github.com/stefan6419846) made their first contribution in [#921](pypa/packaging#921) - [@bearomorphism](https://github.com/bearomorphism) made their first contribution in [#913](pypa/packaging#913) - [@EpicWink](https://github.com/EpicWink) made their first contribution in [#922](pypa/packaging#922) - [@Liam-DeVoe](https://github.com/Liam-DeVoe) made their first contribution in [#932](pypa/packaging#932) - [@subhajitsaha01](https://github.com/subhajitsaha01) made their first contribution in [#930](pypa/packaging#930) - [@ncoghlan](https://github.com/ncoghlan) made their first contribution in [#973](pypa/packaging#973) - [@zahlman](https://github.com/zahlman) made their first contribution in [#1008](pypa/packaging#1008) - [@JP-Ellis](https://github.com/JP-Ellis) made their first contribution in [#939](pypa/packaging#939) #### Since last RC Fixes: - Restore `._version` as a compat shim by [@henryiii](https://github.com/henryiii) in [#1062](pypa/packaging#1062) Performance: - Dual replace by [@henryiii](https://github.com/henryiii) in [#1064](pypa/packaging#1064) Documentaiton: - Prepare for 26.0 final by [@henryiii](https://github.com/henryiii) in [#1063](pypa/packaging#1063) **Full Changelog**: <pypa/packaging@26.0rc3...26.0> </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://kubara.git.onstackit.cloud/STACKIT/kubara/pulls/280

cthoyt mentioned this pull request Jan 31, 2025

Improve performance of uv-python crate's manylinux submodule astral-sh/uv#11131

Merged

uranusjr approved these changes May 8, 2025

View reviewed changes

pradyunsg approved these changes May 10, 2025

View reviewed changes

pradyunsg force-pushed the patch-2 branch from bbfb63c to 5a05a81 Compare May 10, 2025 18:00

cthoyt force-pushed the patch-2 branch from 5a05a81 to afbbacf Compare May 10, 2025 22:38

cthoyt force-pushed the patch-2 branch from afbbacf to 8c6560c Compare May 19, 2025 07:04

cthoyt commented May 19, 2025

View reviewed changes

cthoyt added 2 commits May 22, 2025 15:56

Improve performance in _manylinux.py

8495c94

Use assignment expression to reduce a line

4c2ade7

henryiii force-pushed the patch-2 branch from 8c6560c to 4c2ade7 Compare May 22, 2025 19:56

henryiii approved these changes May 22, 2025

View reviewed changes

henryiii merged commit f89652b into pypa:main May 22, 2025
38 checks passed

cthoyt deleted the patch-2 branch May 22, 2025 22:03

github-actions Bot mentioned this pull request Feb 12, 2026

Bump packaging from 25.0 to 26.0 senzing-factory/dockerhub-util#176

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve performance in _manylinux.py#869

Improve performance in _manylinux.py#869
henryiii merged 2 commits intopypa:mainfrom
cthoyt:patch-2

cthoyt commented Jan 31, 2025 •

edited

Loading

Uh oh!

uranusjr left a comment

Uh oh!

cthoyt commented May 19, 2025

Uh oh!

cthoyt May 19, 2025

Uh oh!

cthoyt May 19, 2025

Uh oh!

Uh oh!

henryiii commented May 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

cthoyt commented Jan 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

uranusjr left a comment

Choose a reason for hiding this comment

Uh oh!

cthoyt commented May 19, 2025

Uh oh!

cthoyt May 19, 2025

Choose a reason for hiding this comment

Uh oh!

cthoyt May 19, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

henryiii commented May 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

cthoyt commented Jan 31, 2025 •

edited

Loading