Skip to content

Refactor and separate concerns of external python package handling code - #953

Merged
groodt merged 4 commits into
mainfrom
groodt-refactor-tools
Jan 10, 2023
Merged

Refactor and separate concerns of external python package handling code#953
groodt merged 4 commits into
mainfrom
groodt-refactor-tools

Conversation

@groodt

@groodt groodt commented Dec 29, 2022

Copy link
Copy Markdown
Contributor

Refactors the code that deals with external python packages retrieved from repositories (i.e. PyPI, artifactory etc) into 3 main concerns:

  1. wheel_installer: The intention is for this to be only Wheel unpacking, which would be a hermetic operation and could run as a build action. However, at the moment, there is also a pip interaction that happens here to build wheels. This can be refactored out into a wheel_builder in a future PR
  2. lock_file_generator: This generates a requirements.bzl lock file for the external python packages from a fully resolved requirements.txt file (optionally generated from dependency_resolver. See 3)
  3. dependency_resolver: Fully resolves a requirements.in file of direct dependencies into a requirements.txt file.

I think the refactor makes the code easier to work with and understand. This should make further improvements easier over time.

@groodt
groodt marked this pull request as ready for review December 29, 2022 10:16
@groodt
groodt requested a review from f0rmiga December 29, 2022 10:17

@rickeylev rickeylev left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just nits, really; otherwise LGTM

@@ -0,0 +1,51 @@
load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be "@rules_python//" or just "//"?

srcs = [
"namespace_pkgs_test.py",
],
tags = ["unit"],

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this tags=unit used by something elsewhere (didn't see it)? I'd think that size=small was sufficient info to filter (small usually means unit test)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's probably unused. I saw a few test_tag_filters to exclude integration test tags. I'll try remove it and see.

@groodt
groodt merged commit 70cce26 into main Jan 10, 2023
@alexeagle
alexeagle deleted the groodt-refactor-tools branch May 23, 2023 17:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants