fix: Exclude external directory when generating python report (bazel-contrib#2136)

ewianda · aignas · web-flow · commit dac8a5fb7864 · 2024-08-22T11:57:38.000Z
This PR will reduce the time it take to run `bazel coverage` *Before this fix,* ``` bazel coverage --cache_test_results=no //... Elapsed time: 7.054s, Critical Path: 6.87s ``` [lcov --list bazel-out/_coverage/_coverage_report.dat](https://github.com/user-attachments/files/16681828/lcov.log) *After* ``` bazel coverage --cache_test_results=no //... Elapsed time: 2.474s, Critical Path: 1.18s $ lcov --list bazel-out/_coverage/_coverage_report.dat Reading tracefile bazel-out/_coverage/_coverage_report.dat |Lines |Functions |Branches Filename |Rate Num|Rate Num|Rate Num ================================================================================ [/home/ewianda/.cache/bazel/_bazel_ewianda/da4b4cc49e0e621570c9e24d6f1eab95/execroot/_main/bazel-out/k8-fastbuild/bin/benchsci/ml/nlp/] test_tokenizer_stage2_bootstrap.py | 6.0% 250| - 0| - 0 [benchsci/] devtools/python/pytest_helper.py |90.5% 21| - 0| - 0 ml/nlp/test_tokenizer.py | 100% 13| - 0| - 0 ml/nlp/tokenizer.py |61.8% 76| - 0| - 0 ================================================================================ Total:|26.1% 360| - 0| - 0 ``` Related to bazel-contrib#1434 --------- Co-authored-by: aignas <240938+aignas@users.noreply.github.com>
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -29,6 +29,8 @@ A brief description of the categories of changes:
 
 ### Fixed
 * (gazelle): Fix incorrect use of `t.Fatal`/`t.Fatalf` in tests.
+* (toolchain) Omit third-party python packages from coverage reports from
+  stage2 bootstrap template.
 
 ### Added
 * Nothing yet
diff --git a/python/private/stage2_bootstrap_template.py b/python/private/stage2_bootstrap_template.py
@@ -364,6 +364,14 @@ def _maybe_collect_coverage(enable):
                 # Pipes can't be read back later, which can cause coverage to
                 # throw an error when trying to get its source code.
                 "/dev/fd/*",
+                # The mechanism for finding third-party packages in coverage-py
+                # only works for installed packages, not for runfiles. e.g:
+                #'$HOME/.local/lib/python3.10/site-packages',
+                # '/usr/lib/python',
+                # '/usr/lib/python3.10/site-packages',
+                # '/usr/local/lib/python3.10/dist-packages'
+                # see https://github.com/nedbat/coveragepy/blob/bfb0c708fdd8182b2a9f0fc403596693ef65e475/coverage/inorout.py#L153-L164
+                "*/external/*",
             ],
         )
         cov.start()
