fix(profiler): prevent filtering target package name in module discovery - #18072
fix(profiler): prevent filtering target package name in module discovery#18072hebaalazzeh wants to merge 3 commits into
Conversation
c608b53 to
7af75c7
Compare
There was a problem hiding this comment.
Code Review
This pull request introduces PEP 0810 explicit lazy imports (lazy_modules) across several packages in google-cloud-core and updates the import profiler script to better handle namespace packages. The review feedback suggests placing the lazy_modules definitions before import statements in accordance with PEP 8, which would also eliminate the need for .flake8 E402 bypasses and typing imports. Additionally, the feedback points out a logic issue in the import profiler's package filtering that causes false positives for packages like google-cloud-build, recommending a more specific exception for google-cloud-testutils.
I am having trouble creating individual review comments. Click here to see my feedback.
scripts/import_profiler/profiler.py (413-424)
The current logic pkg_clean.endswith(top.replace("_", "")) introduces false positives for other packages whose names end with an ignored prefix.
For example:
- For
google-cloud-build,pkg_cleanis"googlecloudbuild". If a standard"build"directory exists,topis"build". Since"googlecloudbuild"ends with"build", the"build"directory will NOT be ignored, which is incorrect as it is just a build artifact directory. - For
google-cloud-notebooks, the"notebooks"directory (which contains Jupyter notebooks and is inignored_prefixes) will NOT be ignored because"googlecloudnotebooks"ends with"notebooks".
Since google-cloud-testutils is the only package in the repository whose top-level Python package is actually in ignored_prefixes (test_utils), we should make this exception specific to google-cloud-testutils and test_utils. This avoids false positives for all other packages.
ignored_prefixes = ("tests", "samples", "examples", "benchmark", "benchmarks", "third_party", "testing", "test_utils", "docs", "build", "dist", "bin", "ci", "scripts", "cloudbuild", "notebooks", "assets", "scratch", "specs")
ignored_starts = ("test_", "sample_", "bench_", "example_", "doc_", "notebook_")
filtered = []
for p in pkgs:
top = p.split(".")[0]
is_ignored_top = top in ignored_prefixes or top.startswith(ignored_starts)
if is_ignored_top and pkg == "google-cloud-testutils" and top == "test_utils":
is_ignored_top = False
if is_ignored_top or p in ("google", "google.cloud"):
continue
filtered.append(p)
7af75c7 to
4e3ba12
Compare
24d9a30 to
f24e53d
Compare
Fix module resolution for packages like
google-cloud-testutilswhose top-level Python package (test_utils) was being filtered out byignored_prefixesin find_module_from_package.