test(setup-matrix): give each case its own home directory - #142
Closed
John-David Dalton (jdalton) wants to merge 1 commit into
Closed
test(setup-matrix): give each case its own home directory#142John-David Dalton (jdalton) wants to merge 1 commit into
John-David Dalton (jdalton) wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
Bugbot Autofix is ON. A cloud agent has been kicked off to fix the reported issue.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit a83e84c. Configure here.
run-case.sh isolated only XDG_CACHE_HOME, so every package manager it drives resolved its cache, store and credentials from the caller's real home. Under `setup-matrix.sh run --host` that is the developer's own home directory. Each case now gets HOME inside its per-case WORKDIR, along with the variables that outrank HOME for their tools (CARGO_HOME, GOPATH, GOMODCACHE, GOCACHE, PNPM_HOME, COREPACK_HOME, NUGET_PACKAGES and the remaining XDG dirs). The verification helpers read through those same variables, so they keep resolving without changes.
John-David Dalton (jdalton)
force-pushed
the
fix/isolate-home-in-setup-matrix
branch
from
August 1, 2026 16:43
a83e84c to
930cc3c
Compare
Mikola Lysenko (mikolalysenko)
added a commit
that referenced
this pull request
Aug 5, 2026
#142 + #143 + #144, plus review fixes) (#146) Squashes jdalton's cache-isolation stack — #142 (setup-matrix per-case HOME), #143 (cache_env::isolate() across the Rust integration suites), #144 (hosted production suite) — and folds in the fixes from a 13-agent adversarial review of the stack: - e2e_hosted_production.rs: widen the gem leg's known-defect matcher to the post-depscan#23630 registry signature (compact index 404s as not_built; the /api/v1/dependencies fallback serves an empty 200 body, which dies as "marshal data too short" under bundler 2.x classic Marshal and as "undefined method 'bytes' for nil" under SafeMarshal rubies). This unblocks the hosted-e2e required check, red on every branch since the 2026-08-02 deploy. Verified live: the leg passes against the current production registry. Also isolate the has_command probe (the corepack-download leak the stack closes everywhere else). - e2e_npm.rs: partial COREPACK_HOME/npm_config_cache pin on the binary runner — the macOS global auto-discovery test spawns the CLI, whose yarn/pnpm prefix probes downloaded ~900 corepack files into the real home (full isolate() would defeat the test; same trade-off as global_packages_e2e.rs). - e2e_vendor_gem_build.rs, e2e_vendor_pypi_build.rs: isolate the bundler/ruby/python/uv availability probes so probe and fixture install answer for the same environment. - run-case.sh: export MAVEN_OPTS=-Duser.home so maven actually follows the case home (the JVM derives user.home from passwd, not $HOME — mvn kept writing the real ~/.m2 while the verify helper scanned the empty fake one); preserve rbenv/pyenv/nvm/volta/asdf/sdkman roots and copy ~/.tool-versions so --host runs still launch their toolchains. - cache_env.rs: document why GEM_HOME/GEM_PATH are deliberately not pinned and the --install-dir/BUNDLE_PATH invariant that keeps that safe. Co-authored-by: jdalton <john.david.dalton@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Running the setup matrix on your own machine currently installs packages into your real home directory.
scripts/setup-matrix.sh run --hostdrives real installs across nine ecosystems, and each one writes to the cache, store, or credentials location it finds in$HOME. After a run, your own npm cache, cargo registry, Go module cache, Maven repository and NuGet packages all contain whatever the test cases pulled down.This change gives every case its own home inside the temporary work directory it already creates, so a host run leaves your machine exactly as it found it.
It also makes the matrix more trustworthy. Right now a case can pass because a package was already sitting in your cache from something unrelated — the kind of pass that disappears on a colleague's laptop or a fresh CI runner.
The evidence — the script proves this against itself
run-case.shisolated exactly one thing, the pypi hook's stamp, viaXDG_CACHE_HOME. Nothing pointedHOMEanywhere, so every package manager fell back to the real one.You can see it in the script's own verification helpers, which go looking for the installed artifacts in the caller's home:
Those lookups succeed today. That is the proof the artifacts really are landing there.
The change — one home per case, plus the vars that outrank it
HOMEnow points at$WORKDIR/home, which is already unique per case.Setting
HOMEalone would not have worked. These tools read their own variable first and ignoreHOMEwhen it is set, so each is pointed inside the new home:CARGO_HOMEGOPATH+GOMODCACHEGOCACHEGOPATHalone is not enoughPNPM_HOMECOREPACK_HOMENUGET_PACKAGESXDG_DATA_HOME,XDG_CONFIG_HOME,XDG_STATE_HOMEThe verification helpers needed no changes — they read through these same variables, so they follow the new location automatically.
What I checked — script-level, not a full matrix run
Ran:
bash -non the modified script — passes.WORKDIR— every variable resolves inside it, and the home directory is created.Did not run: the full matrix end to end. It needs Docker images per ecosystem and performs real installs, so this is verified at the script level rather than by a complete run.
CI is unaffected. The
setup-matrixjob runs each case inside a per-ecosystem container, where$HOMEis the container's and thrown away. This problem is specific to host mode.Trade-off: a host run now starts from an empty cache, so the first run of each case re-downloads its dependencies. That cost is also what makes the result reproducible.
Left for a follow-up — the Rust tests have a similar gap
Not touched here, to keep this small:
crates/socket-patch-cli/tests/run real installs during a plaincargo testwithout redirecting caches.e2e_vendor_pypi_build.rspasses an empty environment to pip, while the uv half of that same file sets one up properly.Happy to follow up if you would like these handled the same way.