fix(templates): resolve core dependencies locally and batch pip installs#17032
fix(templates): resolve core dependencies locally and batch pip installs#17032ohmayr wants to merge 5 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request updates the noxfile.py template and its corresponding golden files to install core dependencies from local paths instead of remote git repositories. Feedback indicates that a comment needs updating to reflect the new variable name core_packages. Additionally, the logic for locating the packages directory is currently too simplistic and will fail for nested golden files; a more robust traversal method was suggested to ensure the correct path is resolved.
| "google-auth @ git+https://github.com/googleapis/google-cloud-python#egg=google-auth&subdirectory=packages/google-auth", | ||
| "grpc-google-iam-v1 @ git+https://github.com/googleapis/google-cloud-python#egg=grpc-google-iam-v1&subdirectory=packages/grpc-google-iam-v1", | ||
| "proto-plus @ git+https://github.com/googleapis/google-cloud-python#egg=proto-plus&subdirectory=packages/proto-plus", | ||
| core_packages = [ |
There was a problem hiding this comment.
The variable has been renamed to core_packages, but the comment on lines 596-597 still refers to the old name core_dependencies_from_source. Please update the comment to maintain consistency.
| for dep in core_dependencies_from_source: | ||
| session.install(dep, "--no-deps", "--ignore-installed") | ||
| print(f"Installed {dep}") | ||
| packages_dir = CURRENT_DIRECTORY.parent |
There was a problem hiding this comment.
Using CURRENT_DIRECTORY.parent assumes the package is always a direct child of the packages/ directory. This logic fails for generated golden files used in integration tests (e.g., packages/gapic-generator/tests/integration/goldens/asset/noxfile.py), as they are nested deeper. This will cause the core_deps_from_source session to fail when run on goldens. Consider a more robust way to locate the packages directory.
packages_dir = CURRENT_DIRECTORY.parent
while packages_dir.name != "packages" and packages_dir.parent != packages_dir:
packages_dir = packages_dir.parent
fix(templates): resolve core dependencies locally and batch pip installs
Previously, the
core_deps_from_sourceandprerelease_depsnox sessions installed core sibling packages sequentially. In a monorepo environment, this caused two major issues:pip installcommands triggered strict dependency resolution repeatedly, leading to severe CI timeouts.This updates the noxfile template to:
packages/directory to guarantee the presubmit tests the active PR's code.pip installcommands using--no-depsand--ignore-installedto bypass the resolver overhead and eliminate the sequential network bottleneck.prerelease_depsinto local vs. PyPI deployments using a regex parser, ensuring safe handling of complex version bounds (e.g.,grpcio>=1.75.1) without hardcoding multiple lists.