Skip to content

fix: Resolve kserve with pip --dry-run instead of installing it - #6733

Open
larrysingleton007 wants to merge 1 commit into
feast-dev:masterfrom
larrysingleton007:fix/kserve-test-isolation
Open

fix: Resolve kserve with pip --dry-run instead of installing it#6733
larrysingleton007 wants to merge 1 commit into
feast-dev:masterfrom
larrysingleton007:fix/kserve-test-isolation

Conversation

@larrysingleton007

Copy link
Copy Markdown
Contributor

What this PR does / why we need it

test_install_kserve_with_feast installs kserve into the interpreter running the unit suite, with no isolation and no cleanup, so it mutates the environment every other test is using.

Feast pins psutil==5.9.0 and kserve 0.15.2 requires psutil<6.0.0,>=5.9.6, so pip cannot leave the installed version in place — it uninstalls psutil, then installs 5.9.8:

Attempting uninstall: psutil
  Found existing installation: psutil 5.9.0
  Uninstalling psutil-5.9.0:
    Successfully uninstalled psutil-5.9.0
Installing collected packages: … psutil …

make test-python-unit runs pytest -n 8, so eight workers share one environment. Any test importing psutil during that window fails, including every test that shells out to the CLI, since feast/metrics.py imports it at module scope. That is how an unrelated docs PR picked up a red unit-test-python (3.12, ubuntu-latest) on test_3rd_party_providers with ModuleNotFoundError: No module named 'psutil'. Because it depends on worker timing, a re-run usually passes and it reads as flake.

kserve also pulls protobuf down to 4.25.x, which leaves the environment broken for the next run: the installed grpcio-health-checking ships protobuf 6.x generated code importing google.protobuf.runtime_version. The first run passes, because collection imports precede the mid-run install; the second fails at collection.

--dry-run performs the same resolution and still exits non-zero when the versions cannot be satisfied together, which is what the test guards against, without installing anything.

This also replaces the conflict assertion, which was inverted:

conflict_occured = "dependency conflicts" in err and "ERROR" not in err

That is only true when pip reports conflicts without an error, so a run failing loudly set it to False. The exit_code == 0 assertion was already doing the real work; it now carries pip's output for diagnosis.

Which issue(s) this PR fixes

Closes #6732

Misc

Verified both directions, so the detection is unchanged rather than merely quieter:

  • clean resolution exits 0, performs the full resolve (Would install … psutil-5.9.8 …), and leaves psutil at 5.9.0 with kserve not importable afterwards
  • pip install --dry-run kserve==0.15.2 psutil==5.9.0 exits 1 with Cannot install kserve==0.15.2 and psutil==5.9.0 because these package versions have conflicting dependencies

The full unit suite now passes twice in a row — 2529 passed, 20 skipped both times, with psutil still 5.9.0 and protobuf still 6.33.6 after the first. Previously the second run produced 31 failures and 8 collection errors. The test also drops from about 13s to under 2s.

This needs a kind/ label, which I cannot add as an outside contributor.

test_install_kserve_with_feast installed kserve into the interpreter
running the suite, with no isolation and no cleanup, so it mutated the
environment every other test was using.

Feast pins psutil==5.9.0 and kserve requires psutil>=5.9.6, so pip could
not leave the installed version alone: it uninstalled psutil before
reinstalling 5.9.8. The unit suite runs pytest -n 8 against one
environment, so any test importing psutil in that window failed,
including every test that shells out to the CLI, since feast.metrics
imports it at module scope. That is how an unrelated docs PR got a red
unit-test-python job on test_3rd_party_providers with ModuleNotFoundError:
No module named 'psutil'. Being timing-dependent, it read as flake.

kserve also pulled protobuf down to 4.25.x, which left the environment
broken for the next run: the installed grpcio-health-checking ships
protobuf 6.x gencode importing google.protobuf.runtime_version. The first
run passed because collection imports precede the mid-run install; the
second failed at collection.

--dry-run performs the same resolution and still exits non-zero when the
versions cannot be satisfied together, which is what the test guards
against, without installing anything. Verified both directions: a clean
resolution exits 0 and leaves psutil at 5.9.0 with kserve not importable,
and 'pip install --dry-run kserve==0.15.2 psutil==5.9.0' exits 1 with
conflicting dependencies.

Also replaces the conflict assertion, which was inverted - it was only
true when pip reported conflicts without an error, so a loud failure set
it to False. The exit code was already doing the real work; it now
carries pip's output for diagnosis.

The full unit suite now passes twice in a row: 2529 passed, 20 skipped
both times, with psutil and protobuf unchanged after the first.

Closes feast-dev#6732

Signed-off-by: Larry Singleton <166439969+larrysingleton007@users.noreply.github.com>
@larrysingleton007
larrysingleton007 requested a review from a team as a code owner August 13, 2026 00:04
@codecov-commenter

codecov-commenter commented Aug 13, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 46.80%. Comparing base (9affee5) to head (ac910bc).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #6733   +/-   ##
=======================================
  Coverage   46.80%   46.80%           
=======================================
  Files         415      415           
  Lines       50397    50397           
  Branches     7215     7215           
=======================================
+ Hits        23589    23590    +1     
  Misses      25155    25155           
+ Partials     1653     1652    -1     
Flag Coverage Δ
go-feature-server 30.58% <ø> (ø)
python-unit 48.13% <ø> (+<0.01%) ⬆️
see 1 file with indirect coverage changes

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9affee5...ac910bc. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unit test installs kserve into the live interpreter, causing nondeterministic failures in unrelated tests

2 participants