chore(ci): use get_package_shards.py in import-profiler for dynamic sharding - #18079
Open
hebaalazzeh wants to merge 6 commits into
Open
chore(ci): use get_package_shards.py in import-profiler for dynamic sharding#18079hebaalazzeh wants to merge 6 commits into
get_package_shards.py in import-profiler for dynamic sharding#18079hebaalazzeh wants to merge 6 commits into
Conversation
Fix module resolution for packages like google-cloud-testutils whose top-level Python package (test_utils) was being filtered out by ignored_prefixes in find_module_from_package.
Contributor
There was a problem hiding this comment.
Code Review
This pull request updates the import profiler script to add the absolute path of the package directory to sys.path and bypasses the ignore filter for the google-cloud-testutils package. It also adds a corresponding unit test. The review feedback correctly points out that the new unit test mocks filesystem functions globally instead of locally, violating the repository style guide (Rule 19), and recommends explicitly asserting the expected call sequence of mocked helpers (Rule 20).
get_package_shards.py in import-profiler for dynamic sharding
hebaalazzeh
marked this pull request as ready for review
August 11, 2026 21:46
get_package_shards.py in import-profiler for dynamic shardingget_package_shards.py in import-profiler for dynamic sharding
hebaalazzeh
enabled auto-merge (squash)
August 11, 2026 22:15
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Updates the
import-profilerworkflow to useci/get_package_shards.pyfor dynamic sharding matrix generation, bringing it in line with the sharding pattern used inlint.ymlandunittest.yml.Changes
initializeJob: Added an initialization job runningci/get_package_shards.pywithMAX_SHARDS: 8to dynamically calculate shard distribution based on modified packages.[0..7]) with dynamic shard matrix evaluation. If few or no packages changed, only the necessary number of shards are spawned instead of always spinning up 8 runners.PACKAGE_LISTdirectly toci/run_conditional_tests.sh.all-import-profilesto depend on[initialize, import-profile]and verify both job outcomes.